@mevdragon/vidfarm-devcli 0.2.4 → 0.2.6

package/README.md

@@ -51,7 +51,7 @@ The local CLI runs the same template contract used by the hosted platform, but w
 
 Use `vidfarm session` to print reusable auth headers and a sample `curl` request for the local REST API.
 
-For hosted preview-media uploads, use `vidfarm-devcli presign-preview-media`. It calls the authenticated Vidfarm API with `VIDFARM_API_KEY`, mints a presigned PUT URL under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns the resulting Vidfarm media URL.
+For hosted preview-media uploads, use `vidfarm-devcli presign-preview-media`. It calls the authenticated Vidfarm API with `VIDFARM_API_KEY`, mints a presigned PUT URL under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns a public-read media URL for the uploaded object. Public readability is granted by bucket policy for `developer/*`, not by public write or list access.
 
 ## What The Hosted Platform Expects From Templates
 

package/SKILL.developer.md

@@ -26,6 +26,24 @@ Do not use this skill for:
 - production deploys, release admin, or shared cloud infrastructure
 - asking for AWS, S3, Remotion cloud, or other platform secrets
 
+## Third-Party Publish Rule
+
+Third-party template developers publish template code by pushing git commits to GitHub. That is the only code-distribution step they own.
+
+The third-party developer responsibility is:
+
+- build and validate the template locally
+- commit and push the template folder to GitHub
+- register or update the hosted template source metadata so Vidfarm knows which repo, branch, and template folder to import from
+
+The third-party developer should not:
+
+- run `import-source-prod`
+- use AWS CloudFormation, SSM, EC2, or shared prod credentials
+- treat platform import/activation as part of template authoring
+
+`import-source-prod` and `deploy-template-cycle` are platform-operator commands. They are for internal admin flow, not for external template authors.
+
 For third-party developers, Remotion is local and storage should be accessed through Vidfarm helpers. Do not require cloud Remotion setup or hand-written S3 integration for normal template work.
 
 ## Agent Operating Rule
@@ -101,17 +119,36 @@ Notes:
 - template authoring is TypeScript-first; create and edit template source as `src/template.ts`, not `src/template.js`
 - keep exploratory scripts, one-off test runners, scratch validation files, and temporary job-driving code out of `src/`; place them in a repo-local tmp folder that is a sibling to `src` instead, for example `templates/vidfarm_template_0007/tmp/`
 - when registering a hosted template source, set `template_module_path` to the repo-relative TypeScript entrypoint inside the template folder, for example `templates/vidfarm_template_example/src/template.ts`; the platform build/import flow resolves the compiled `src/template.js` sibling after `npm run build`
+- hosted template registration is git metadata only: repo URL, branch, template entrypoint path, and optionally an explicit commit SHA for a specific import
+- if a template moves folders or switches branches, update the registration metadata; if the template code changed but the location stayed the same, just push a new Git commit
+- the platform determines whether an update exists by comparing the registered branch head against the latest imported release commit
 - do not ask for platform secrets like `ENCRYPTION_SECRET`, `API_KEY_SALT`, `WEBHOOK_SECRET`, admin emails, S3 config, or generic AWS credentials
 - do not ask for `REMOTION_AWS_ACCESS_KEY_ID` or `REMOTION_AWS_SECRET_ACCESS_KEY`
 
-## Template Deploy Cycle
+## Hosted Registration Flow
 
-If the user says "do a template deploy cycle", interpret that as:
+For a third-party developer, the hosted publish handoff is:
 
 1. push the repo changes that contain the target template folder
-2. import that template commit into the hosted platform as a template source release
-3. approve and activate the new release
-4. run the prod deploy script if platform code changed, or reuse the current prod image for a formal restart if the change was template-only
+2. register the source location if it is new, or update the existing registration if the repo, branch, or template path changed
+3. tell the platform operator which branch head or commit SHA should be imported
+
+Registration data should be treated like this:
+
+- `repo_url`: the GitHub repo that contains the template code
+- `branch`: the branch Vidfarm should watch for the latest version, usually `main`
+- `template_module_path`: the repo-relative path to the template TypeScript entrypoint, for example `src/vidfarm_template_funnychat/src/template.ts`
+- `commit_sha`: optional and only needed when you want to import a specific commit instead of the current branch head
+
+Do not tell third-party developers to run AWS-backed operator commands just to publish template code. If the task is only "make my updated template available to Vidfarm", the correct answer is GitHub push plus source registration metadata.
+
+## Template Deploy Cycle
+
+If the user explicitly says "template deploy cycle", interpret that as the internal platform-operator sequence:
+
+1. import that template commit into the hosted platform as a template source release
+2. approve and activate the new release
+3. run the prod deploy script if platform code changed, or reuse the current prod image for a formal restart if the change was template-only
 
 Do not assume "template deploy cycle" means "deploy the entire dirty root repo". If the root worktree has unrelated changes and the release is template-only, prefer reusing the current prod image with:
 
@@ -221,7 +258,7 @@ drafts/
 
 This is the preferred handoff shape for agent-driven template creation. A Vidfarm developer should be able to drag media into `drafts/preview/`, add rough notes to `drafts/source_notes.md`, then tell the agent to create a template from those relative paths.
 
-If the source media is not already present in the repo, upload it into the authenticated developer storage namespace first instead of inventing ad hoc external hosting. Use:
+If the source media is not already present in the repo, upload it into the authenticated developer storage namespace first instead of inventing ad hoc external hosting. An agent can take a developer-provided local file path and run this directly:
 
 ```bash
 npx @mevdragon/vidfarm-devcli presign-preview-media \
@@ -229,7 +266,7 @@ npx @mevdragon/vidfarm-devcli presign-preview-media \
   --directory drafts/preview
 ```
 
-That command calls the hosted Vidfarm API with the user's `VIDFARM_API_KEY`, mints a presigned PUT URL scoped under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns both the `storage_key` and a Vidfarm `preview_media_url`. Agents should prefer that path for hosted preview uploads.
+That command calls the hosted Vidfarm API with the user's `VIDFARM_API_KEY`, mints a presigned PUT URL scoped under `developer/<user_id>/*`, stores each uploaded file under a UUID path segment while preserving the original filename, uploads the file automatically when `--file` is provided, and returns both the `storage_key` and a public-read `preview_media_url`. Public readability for `developer/<user_id>/*` comes from Vidfarm bucket policy, not from public write access, so objects may be read directly by URL but should not be treated as listable or writable without authenticated API access. Agents should prefer that path for hosted preview uploads whenever the input is a local media file path.
 
 ### 2. Start local runtime
 

package/dist/src/app.js

@@ -15,6 +15,7 @@ import { AuthService } from "./services/auth.js";
 import { JobsService } from "./services/jobs.js";
 import { StorageService } from "./services/storage.js";
 import { TemplateSourceService } from "./services/template-sources.js";
+import { TemplateSourceAlreadyRegisteredError } from "./services/template-sources.js";
 const auth = new AuthService();
 const jobs = new JobsService();
 const storage = new StorageService();
@@ -986,11 +987,13 @@ app.post(`${USER_PREFIX}/me/developer/preview-media/presign`, async (c) => {
             publicRead: false,
             expiresIn: 3600
         });
+        const publicUrl = storage.getPublicUrl(storageKey) ?? buildAbsoluteUrl(c, `/template-media?key=${encodeURIComponent(storageKey)}`);
         return c.json({
             file_name: fileName,
             content_type: contentType,
             storage_key: storageKey,
-            preview_media_url: buildAbsoluteUrl(c, `/template-media?key=${encodeURIComponent(storageKey)}`),
+            preview_media_url: publicUrl,
+            public_url: publicUrl,
             upload: {
                 method: upload.method,
                 url: upload.url,
@@ -1063,7 +1066,7 @@ app.post(`${TEMPLATES_PREFIX}/sources`, async (c) => {
             install_command: z.string().min(1).default("npm install"),
             build_command: z.string().min(1).default("npm run build")
         }).parse(await c.req.json());
-        const source = templateSources.registerSource({
+        const registration = await templateSources.registerSource({
             templateId: body.template_id,
             slugId: body.slug_id,
             repoUrl: body.repo_url,
@@ -1073,9 +1076,21 @@ app.post(`${TEMPLATES_PREFIX}/sources`, async (c) => {
             installCommand: body.install_command,
             buildCommand: body.build_command
         });
-        return c.json({ source }, 201);
+        return c.json({
+            source: registration.source,
+            registration_action: registration.action,
+            sync: registration.sync
+        }, registration.action === "created" ? 201 : 200);
     }
     catch (error) {
+        if (error instanceof TemplateSourceAlreadyRegisteredError) {
+            return c.json({
+                error: error.message,
+                registration_status: error.existingSource.status,
+                existing_source: error.existingSource,
+                conflict_field: error.conflictField
+            }, 409);
+        }
         const message = error instanceof Error ? error.message : "Unable to register template source.";
         const status = /already exists/i.test(message) ? 409 : 400;
         return c.json({ error: message }, status);

package/dist/src/cli.js

@@ -235,7 +235,7 @@ async function runImportSourceCommand(argv) {
         import("./registry.js")
     ]);
     const sources = new TemplateSourceService();
-    const source = sources.registerSource({
+    const registration = await sources.registerSource({
         templateId,
         slugId,
         repoUrl,
@@ -246,7 +246,7 @@ async function runImportSourceCommand(argv) {
         buildCommand: parsed.values["build-command"]
     });
     const release = await sources.importRelease({
-        sourceId: source.id,
+        sourceId: registration.source.id,
         commitSha: parsed.values["commit-sha"] ?? null
     });
     let activated = null;
@@ -256,7 +256,9 @@ async function runImportSourceCommand(argv) {
         activated = result.release;
     }
     console.log(JSON.stringify({
-        source,
+        source: registration.source,
+        registration_action: registration.action,
+        sync: registration.sync,
         release,
         activated_release: activated
     }, null, 2));
@@ -542,6 +544,7 @@ async function runPresignPreviewMediaCommand(argv) {
         content_type: payload.content_type,
         storage_key: payload.storage_key,
         preview_media_url: payload.preview_media_url,
+        public_url: payload.public_url ?? payload.preview_media_url,
         upload_result: uploadResult,
         upload: payload.upload,
         curl_upload_example: [

package/dist/src/db.js

@@ -845,6 +845,35 @@ export const database = {
         });
         return this.getTemplateSourceByTemplateId(record.templateId);
     },
+    updateTemplateSource(input) {
+        const current = this.getTemplateSource(input.id);
+        if (!current) {
+            throw new Error("Template source not found.");
+        }
+        db.prepare(`
+      update template_sources
+      set repo_url = @repo_url,
+          branch = @branch,
+          template_module_path = @template_module_path,
+          skill_path = @skill_path,
+          install_command = @install_command,
+          build_command = @build_command,
+          status = @status,
+          updated_at = @updated_at
+      where id = @id
+    `).run({
+            id: input.id,
+            repo_url: input.repoUrl,
+            branch: input.branch,
+            template_module_path: input.templateModulePath,
+            skill_path: input.skillPath,
+            install_command: input.installCommand,
+            build_command: input.buildCommand,
+            status: input.status ?? current.status,
+            updated_at: nowIso()
+        });
+        return this.getTemplateSource(input.id);
+    },
     getTemplateSource(id) {
         const row = db.prepare(`select * from template_sources where id = ?`).get(id);
         return row ? mapTemplateSource(row) : null;
@@ -857,6 +886,15 @@ export const database = {
         const row = db.prepare(`select * from template_sources where slug_id = ?`).get(slugId);
         return row ? mapTemplateSource(row) : null;
     },
+    getTemplateSourceByLocation(repoUrl, branch, templateModulePath) {
+        const row = db.prepare(`
+      select *
+      from template_sources
+      where repo_url = ? and branch = ? and template_module_path = ?
+      limit 1
+    `).get(repoUrl, branch, templateModulePath);
+        return row ? mapTemplateSource(row) : null;
+    },
     listTemplateSources() {
         return db.prepare(`select * from template_sources order by template_id asc`).all().map(mapTemplateSource);
     },
@@ -903,6 +941,16 @@ export const database = {
         const row = db.prepare(`select * from template_releases where source_id = ? and commit_sha = ?`).get(sourceId, commitSha);
         return row ? mapTemplateRelease(row) : null;
     },
+    getLatestTemplateReleaseForSource(sourceId) {
+        const row = db.prepare(`
+      select *
+      from template_releases
+      where source_id = ?
+      order by created_at desc
+      limit 1
+    `).get(sourceId);
+        return row ? mapTemplateRelease(row) : null;
+    },
     listTemplateReleases(templateId) {
         const rows = templateId
             ? db.prepare(`select * from template_releases where template_id = ? order by created_at desc`).all(templateId)

package/dist/src/services/template-sources.js

@@ -10,6 +10,16 @@ import { nowIso } from "../lib/time.js";
 import { loadTemplateFromModule } from "./template-loader.js";
 import { TemplateCertificationService } from "./template-certification.js";
 const execFileAsync = promisify(execFile);
+export class TemplateSourceAlreadyRegisteredError extends Error {
+    existingSource;
+    conflictField;
+    constructor(input) {
+        super(input.message);
+        this.name = "TemplateSourceAlreadyRegisteredError";
+        this.existingSource = input.existingSource;
+        this.conflictField = input.conflictField;
+    }
+}
 export class TemplateSourceService {
     certification = new TemplateCertificationService();
     listSources() {
@@ -18,28 +28,97 @@ export class TemplateSourceService {
     listReleases(templateId) {
         return database.listTemplateReleases(templateId);
     }
-    registerSource(input) {
+    async registerSource(input) {
         deriveTemplateRootDirFromModulePath(input.templateModulePath);
+        const branch = input.branch ?? "production";
         const existingByTemplateId = database.getTemplateSourceByTemplateId(input.templateId);
-        if (existingByTemplateId) {
-            throw new Error("A template with this template_id already exists. Generate a new UUIDv4 and try again.");
+        if (existingByTemplateId && existingByTemplateId.slugId !== input.slugId) {
+            throw new TemplateSourceAlreadyRegisteredError({
+                message: `Template ${input.templateId} is already registered with slug ${existingByTemplateId.slugId}.`,
+                existingSource: existingByTemplateId,
+                conflictField: "template_id"
+            });
         }
         const existingBySlugId = database.getTemplateSourceBySlugId(input.slugId);
-        if (existingBySlugId) {
-            throw new Error(`A template with slug_id ${input.slugId} already exists.`);
-        }
-        return database.createTemplateSource({
-            id: createId("tsrc"),
-            templateId: input.templateId,
-            slugId: input.slugId,
-            repoUrl: input.repoUrl,
-            branch: input.branch ?? "production",
-            templateModulePath: input.templateModulePath,
-            skillPath: input.skillPath ?? defaultSkillPathForTemplateModule(input.templateModulePath),
-            installCommand: input.installCommand ?? "npm install",
-            buildCommand: input.buildCommand ?? "npm run build",
-            status: "active"
-        });
+        if (existingBySlugId && existingBySlugId.templateId !== input.templateId) {
+            throw new TemplateSourceAlreadyRegisteredError({
+                message: `Template slug ${input.slugId} is already registered to template ${existingBySlugId.templateId}.`,
+                existingSource: existingBySlugId,
+                conflictField: "slug_id"
+            });
+        }
+        const existingByLocation = database.getTemplateSourceByLocation(input.repoUrl, branch, input.templateModulePath);
+        if (existingByLocation &&
+            (existingByLocation.templateId !== input.templateId || existingByLocation.slugId !== input.slugId)) {
+            throw new TemplateSourceAlreadyRegisteredError({
+                message: `Template source ${input.repoUrl}#${branch}:${input.templateModulePath} is already registered to ${existingByLocation.templateId}/${existingByLocation.slugId}.`,
+                existingSource: existingByLocation,
+                conflictField: "source_location"
+            });
+        }
+        const existingSource = existingByTemplateId ?? existingBySlugId ?? existingByLocation ?? null;
+        const installCommand = input.installCommand ?? "npm install";
+        const buildCommand = input.buildCommand ?? "npm run build";
+        const skillPath = input.skillPath ?? defaultSkillPathForTemplateModule(input.templateModulePath);
+        if (existingSource) {
+            const nextSource = existingSource.repoUrl !== input.repoUrl ||
+                existingSource.branch !== branch ||
+                existingSource.templateModulePath !== input.templateModulePath ||
+                existingSource.skillPath !== skillPath ||
+                existingSource.installCommand !== installCommand ||
+                existingSource.buildCommand !== buildCommand
+                ? database.updateTemplateSource({
+                    id: existingSource.id,
+                    repoUrl: input.repoUrl,
+                    branch,
+                    templateModulePath: input.templateModulePath,
+                    skillPath,
+                    installCommand,
+                    buildCommand
+                })
+                : existingSource;
+            return {
+                source: nextSource,
+                action: nextSource.updatedAt === existingSource.updatedAt ? "unchanged" : "updated",
+                sync: await this.getSourceSyncStatus(nextSource.id)
+            };
+        }
+        try {
+            const source = database.createTemplateSource({
+                id: createId("tsrc"),
+                templateId: input.templateId,
+                slugId: input.slugId,
+                repoUrl: input.repoUrl,
+                branch,
+                templateModulePath: input.templateModulePath,
+                skillPath,
+                installCommand,
+                buildCommand,
+                status: "active"
+            });
+            return {
+                source,
+                action: "created",
+                sync: await this.getSourceSyncStatus(source.id)
+            };
+        }
+        catch (error) {
+            const existingSource = database.getTemplateSourceByTemplateId(input.templateId) ??
+                database.getTemplateSourceBySlugId(input.slugId) ??
+                database.getTemplateSourceByLocation(input.repoUrl, branch, input.templateModulePath);
+            if (existingSource) {
+                throw new TemplateSourceAlreadyRegisteredError({
+                    message: `Template registration already exists with status ${existingSource.status}.`,
+                    existingSource,
+                    conflictField: existingSource.templateId === input.templateId
+                        ? "template_id"
+                        : existingSource.slugId === input.slugId
+                            ? "slug_id"
+                            : "source_location"
+                });
+            }
+            throw error;
+        }
     }
     async importRelease(input) {
         const source = database.getTemplateSource(input.sourceId);
@@ -120,6 +199,32 @@ export class TemplateSourceService {
             }
         };
     }
+    async getSourceSyncStatus(sourceId) {
+        const source = database.getTemplateSource(sourceId);
+        if (!source) {
+            throw new Error("Template source not found.");
+        }
+        const latestRelease = database.getLatestTemplateReleaseForSource(source.id);
+        try {
+            const headCommitSha = await this.resolveBranchHead(source.repoUrl, source.branch);
+            return {
+                headCommitSha,
+                latestReleaseCommitSha: latestRelease?.commitSha ?? null,
+                importTargetCommitSha: headCommitSha,
+                hasUnimportedUpdate: latestRelease ? latestRelease.commitSha !== headCommitSha : true,
+                headLookupError: null
+            };
+        }
+        catch (error) {
+            return {
+                headCommitSha: null,
+                latestReleaseCommitSha: latestRelease?.commitSha ?? null,
+                importTargetCommitSha: latestRelease?.commitSha ?? null,
+                hasUnimportedUpdate: null,
+                headLookupError: error instanceof Error ? error.message : String(error)
+            };
+        }
+    }
     async resolveBranchHead(repoUrl, branch) {
         const { stdout } = await execFileAsync("git", ["ls-remote", repoUrl, branch], { cwd: process.cwd() });
         const line = stdout.trim().split("\n").find(Boolean);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mevdragon/vidfarm-devcli",
-  "version": "0.2.4",
+  "version": "0.2.6",
   "description": "Developer CLI for running the Vidfarm local template platform.",
   "type": "module",
   "bin": {