@mevdragon/vidfarm-devcli 0.2.1 → 0.2.3

package/PLATFORM_SPEC.md

@@ -1,1039 +0,0 @@
-# Vidfarm Platform Spec
-
-This document defines the initial platform architecture for Vidfarm running on a single Dockerized EC2 host.
-
-The goal is to let in-house developers build new video production templates quickly while the platform centrally owns auth, billing, job orchestration, customer state, and deployment.
-
-For template code distribution in v1.1, the platform should support GitHub-backed in-house template sources, but production activation must still be manual, admin-approved, and pinned to a specific commit SHA.
-
-## Goals
-
-1. Support multiple video production patterns behind one consistent API.
-2. Treat every operation as an async job that immediately returns `job_id`.
-3. Let template developers write normal TypeScript/Node code with normal npm dependencies.
-4. Keep the first production deployment simple enough to run on one EC2 Docker host.
-5. Preserve a clean path to later split heavy workloads into isolated workers or separate services.
-
-## Non-Goals
-
-1. Public marketplace for third-party user-submitted templates.
-2. Multi-region or scale-to-zero serverless deployment in v1.
-3. Full microservice isolation for every template.
-4. Perfect cost attribution in v1. The first target is safe, conservative billing that protects gross margin.
-
-## Core Decision
-
-Vidfarm should run as one shared platform container on EC2.
-
-Templates should be packaged as internal code modules loaded by that platform, not as separately deployed HTTP services by default.
-Platform runtime code should live under `src/*`, while template implementation code should live outside the platform tree under `templates/<template-folder>/*`.
-
-This gives us:
-
-1. One auth and billing boundary.
-2. One job table and one queueing system.
-3. One deployment artifact for the normal case.
-4. Full npm freedom for template developers.
-5. A cleaner developer experience than forcing every template to become its own service.
-
-Templates may still opt into isolated execution later when they have special requirements such as native binaries, unusually high memory use, or independent scaling needs.
-
-## Template Source Of Truth
-
-Template code may live in GitHub, but the platform should not auto-pull floating branch heads into production.
-
-Approved v1.1 release model:
-
-1. developer keeps template code in GitHub
-2. the default import branch is `production`
-3. admin manually reviews the repo and chooses when to import
-4. admin publishes the approved Remotion site bundle if the template needs Remotion
-5. platform resolves the chosen branch head to a commit SHA
-6. platform builds and certifies that exact commit
-7. platform activates a pinned release record for that commit
-8. admin rebuilds and redeploys the production Docker image with the approved release set
-
-Live platform state should therefore point to:
-
-- repo URL
-- branch name
-- exact commit SHA
-- certification result
-- active/inactive release state
-
-It should not point only to a floating branch name.
-
-Release authority should be centralized:
-
-- developers can push source code to GitHub
-- developers cannot directly publish to shared Remotion AWS
-- developers cannot directly promote templates into production Docker
-- shared AWS publish, activation, and production deployment are admin-only steps
-
-## Initial Runtime Choice
-
-### Production Runtime
-
-- Node.js 22
-- TypeScript
-- Hono for the HTTP API
-- SQLite for v1 job state and queue state
-- S3 for customer files and generated artifacts
-- Remotion Lambda for final render workloads where appropriate
-
-### Why Hono
-
-Hono is a good fit for the control plane:
-
-1. Small and fast.
-2. Strong middleware model.
-3. Good TypeScript ergonomics.
-4. Easy to keep the HTTP layer thin while most complexity lives in jobs and template execution.
-
-### Why Not Bun for v1 Runtime
-
-Bun is not forbidden, but it should not be the production baseline for v1.
-
-Reasons:
-
-1. The platform already assumes Node-oriented Docker execution.
-2. AI SDK compatibility and native package behavior are more predictable on Node.
-3. Remotion and adjacent tooling are safer on the Node compatibility path.
-4. The hard part of this system is orchestration correctness, not JavaScript runtime speed.
-
-If desired, Bun can be evaluated later as a local development runner or for specific internal tools.
-
-## Supported Production Patterns
-
-The platform must support all of the following under one model:
-
-1. Pure AI multi-stage production.
-2. Remotion render pipelines.
-3. Hybrid research plus render pipelines.
-4. Animated storytelling pipelines.
-
-The common abstraction is:
-
-1. Customer hits a template endpoint.
-2. Platform validates auth and input.
-3. Platform creates an async job.
-4. Worker executes the requested operation.
-5. Customer polls job state or receives a webhook.
-
-## Architectural Overview
-
-```txt
-Client
-  -> Hono API
-  -> Auth / Billing / Template Registry / Job Creation
-  -> SQLite (jobs, logs, queue, rate-limit state, customer metadata pointers)
-  -> Worker Loop in same container
-  -> External providers (OpenAI, Gemini, OpenRouter, Perplexity, etc.)
-  -> S3 (workspace files, stage artifacts, final outputs)
-  -> Remotion Lambda (when render pipeline requires it)
-```
-
-Initial deployment is one process image with two logical responsibilities:
-
-1. API server
-2. Background worker / dispatcher
-
-These can run in the same container in v1. If needed later, they can be split into separate process types using the same codebase.
-
-## API Principles
-
-All template operations are async-first.
-
-Even if a task could finish quickly, the platform should still prefer job creation so the customer sees one consistent model.
-
-### Base Path
-
-```txt
-/templates/:templateId/*
-```
-
-### Core Endpoints
-
-```txt
-GET    /templates/:templateId
-GET    /templates/:templateId/about/*
-GET    /templates/:templateId/skill
-POST   /templates/:templateId/config
-POST   /templates/:templateId/operations/:operationName
-GET    /templates/:templateId/jobs
-GET    /templates/:templateId/jobs/:jobId
-GET    /templates/:templateId/jobs/:jobId/logs
-POST   /templates/:templateId/jobs/:jobId/cancel
-```
-
-### Request Headers
-
-```txt
-vidfarm-user-id: string
-vidfarm-api-key: string
-```
-
-### Job Creation Request Shape
-
-```json
-{
-  "tracer": "client-generated-string",
-  "payload": {}
-}
-```
-
-### Job Creation Response Shape
-
-```json
-{
-  "job_id": "job_xxx",
-  "tracer": "client-generated-string",
-  "status": "queued"
-}
-```
-
-### Template Metadata Response Shape
-
-```json
-{
-  "id": "4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db",
-  "slug_id": "template_0000",
-  "version": "1.0.0",
-  "title": "Template 0000",
-  "description": "Short-form slideshow pipeline",
-  "viral_dna": "Fast TikTok slideshow hooks with mobile-native pacing.",
-  "preview_media": [
-    "https://api.example.com/templates/4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db/about/preview-01.jpg"
-  ],
-  "link_to_original": "https://www.tiktok.com/@example/video/1234567890",
-  "skill_url": "https://api.example.com/templates/4c7a7e1a-7f35-4f30-9f86-9c8a63c7f2db/skill",
-  "operations": [
-    {
-      "name": "create_slideshow",
-      "description": "Generate slideshow frames.",
-      "providerHint": "openrouter"
-    }
-  ]
-}
-```
-
-`GET /templates/:templateId/about/*` should expose template metadata assets such as preview images or videos. In storage, these assets should use the stable logical prefix `templates/:templateId/about/*`, whether the backing store is local disk or S3.
-
-Template definitions should expose `preview_media` as absolute HTTPS URLs. When the backing store is S3, those entries should point directly at the S3 object for assets stored under `templates/:templateId/about/*`.
-
-### Job List Filtering
-
-`GET /templates/:templateId/jobs` should support:
-
-- `tracer`
-- `start_time`
-- `end_time`
-- `limit`
-
-`GET /templates/:templateId/jobs/:jobId/logs` should use the same time window language:
-
-- `start_time`
-- `end_time`
-- `limit`
-
-`logs_from` is deprecated and should not be used in the standard.
-
-`GET /templates/:templateId` is the template "about" response and must also return:
-
-- `slug_id: string`
-- `title: string`
-- `viral_dna: string`
-- `preview_media: string[]`
-- `link_to_original: string`
-
-In the template definition standard, this response `title` and `description` should be sourced from `template.about.title` and `template.about.description`, not top-level template fields.
-
-## Template Model
-
-Templates are normal TypeScript packages with unrestricted internal structure and should live under a repo-level `templates/` directory, for example `templates/template_0000/*`.
-
-They may:
-
-1. Import npm libraries.
-2. Define helper modules.
-3. Bundle prompt files.
-4. Include Remotion compositions.
-5. Call provider SDKs.
-6. Run arbitrary internal orchestration logic.
-
-The framework should not force templates into a single-file callback model.
-
-### Template Contract
-
-The external API surface should be defined as operations, not just raw stage names.
-
-Suggested shape:
-
-```ts
-export const myTemplate = defineTemplate({
-  id: "123e4567-e89b-42d3-a456-426614174000",
-  slugId: "ugc_voiceover_v1",
-  version: "1.0.0",
-  about: {
-    title: "UGC Voiceover V1",
-    description: "Short-form UGC voiceover pipeline",
-    viral_dna: "Fast hooks, native pacing, and repeatable creator-style framing.",
-    preview_media: ["https://cdn.example.com/templates/ugc-voiceover-v1/about/preview-01.mp4"],
-    link_to_original: "https://www.tiktok.com/@example/video/1234567890"
-  },
-  configSchema: z.object({
-    defaultProvider: z.enum(["openai", "gemini", "openrouter", "perplexity"]).default("openai")
-  }),
-
-  operations: {
-    scaffold: {
-      description: "Generate a script scaffold.",
-      inputSchema: z.object({
-        topic: z.string()
-      }),
-      workflow: "scaffoldWorkflow",
-      providerHint: "openai",
-      webhookSupport: true
-    },
-    render: {
-      description: "Submit final render work.",
-      inputSchema: z.object({
-        storyboardId: z.string()
-      }),
-      workflow: "renderWorkflow",
-      webhookSupport: true
-    }
-  },
-
-  jobs: {
-    async scaffoldWorkflow(ctx, input) {
-      return {
-        progress: 1,
-        output: {}
-      };
-    },
-    async renderWorkflow(ctx, input) {
-      return {
-        progress: 1,
-        output: {}
-      };
-    }
-  }
-});
-```
-
-### Why This Contract
-
-This separates:
-
-1. Public API entrypoints.
-2. Internal workflow implementation.
-3. Template metadata and validation.
-
-It gives template developers full control over their workflow logic while keeping the platform contract stable.
-
-## Template Execution Context
-
-Each operation or job should receive a framework-owned context object.
-
-Suggested capabilities:
-
-```ts
-interface TemplateJobContext {
-  env: "development" | "production";
-  customer: CustomerContext;
-  templateConfig: Record<string, unknown>;
-  logger: {
-    debug(message: string, metadata?: Record<string, unknown>): void;
-    info(message: string, metadata?: Record<string, unknown>): void;
-    warn(message: string, metadata?: Record<string, unknown>): void;
-    error(message: string, metadata?: Record<string, unknown>): void;
-    progress(progress: number, message: string, metadata?: Record<string, unknown>): void;
-  };
-  jobs: {
-    enqueueChild(input: {
-      operationName: string;
-      workflowName: string;
-      payload: Record<string, unknown>;
-      providerHint?: ProviderType;
-    }): Promise<{ jobId: string }>;
-  };
-  storage: {
-    putJson(key: string, value: unknown): Promise<{ key: string; url: string | null }>;
-    putText(key: string, value: string, contentType?: string): Promise<{ key: string; url: string | null }>;
-    putBuffer(
-      key: string,
-      value: Uint8Array,
-      options?: { contentType?: string; kind?: string; metadata?: Record<string, unknown> }
-    ): Promise<{ key: string; url: string | null }>;
-    getPublicUrl(key: string): string | null;
-  };
-  billing: {
-    record(input: {
-      type: "ai_generation" | "render" | "storage_write" | "cpu_estimate";
-      costUsd: number;
-      chargeUsd?: number;
-      metadata?: Record<string, unknown>;
-    }): Promise<void>;
-  };
-  providers: {
-    generateText(input: {
-      provider: ProviderType;
-      model: string;
-      prompt: string;
-      temperature?: number;
-    }): Promise<{ text: string }>;
-    generateImage(input: {
-      provider: ProviderType;
-      model: string;
-      prompt: string;
-      size?: string;
-    }): Promise<{ bytes: Uint8Array; contentType: string; revisedPrompt: string | null }>;
-    analyzeImageLayout(input: {
-      provider: ProviderType;
-      model: string;
-      imageUrl: string;
-      overlayText: string;
-    }): Promise<{
-      zone: "top" | "center" | "bottom";
-      align: "left" | "center" | "right";
-      maxWidthPercent: number;
-      justification: string;
-    }>;
-  };
-  remotion: {
-    render(input: {
-      compositionId: string;
-      serveUrl?: string;
-      entryPoint?: string;
-      outputKey?: string;
-      inputProps: Record<string, unknown>;
-    }): Promise<{ renderId: string; outputUrl: string | null; metadata: Record<string, unknown> }>;
-  };
-}
-```
-
-Framework-owned context capabilities should include:
-
-1. Resolving customer AI keys safely.
-2. Writing artifacts through a stable storage prefix.
-3. Enqueuing child jobs.
-4. Emitting logs and progress.
-5. Recording billable events.
-6. Submitting downstream renders through a Remotion adapter.
-7. Calling provider adapters through centralized rate-limit enforcement.
-
-## Environment Behavior
-
-The platform must clearly distinguish development from production.
-
-### Development
-
-Developer-owned API keys from local `.env` are allowed.
-
-This is for:
-
-1. Local testing.
-2. Template development.
-3. Dry-running internal workflows before deployment.
-
-Template authors should also be able to run the same certification harness locally through the developer CLI before admin review.
-
-### Production
-
-The platform must use customer-owned provider keys stored in the customer profile when the template requests external AI inference on behalf of that customer.
-
-Platform-controlled keys may still exist for:
-
-1. Platform-level fallback behavior.
-2. Internal moderation or diagnostics.
-3. Emergency operations.
-
-But customer-billed workloads should default to customer-owned keys when available.
-
-Admin-only template source import and activation are allowed in production. Developer template changes should not go live until an admin explicitly imports and activates a pinned release.
-
-## Developer And Admin Auth Model
-
-The platform’s API auth remains customer-style `vidfarm-user-id` plus `vidfarm-api-key`, but the platform must also support an admin allowlist for template-source management endpoints.
-
-Minimum v1.1 rule:
-
-1. all normal template execution uses standard platform auth
-2. template source registration, import, and activation endpoints are admin-only
-3. admin authorization may begin as an allowlist of trusted emails
-
-This keeps v1 simple while still distinguishing runtime customers from internal release operators.
-
-## Customer Profile Model
-
-Each customer profile should support:
-
-1. Multiple provider API keys.
-2. Multiple keys per provider.
-3. Workspace file storage references.
-4. Webhook destinations.
-5. Billing preferences and limits.
-
-Suggested provider key record:
-
-```ts
-interface CustomerProviderKey {
-  id: string;
-  provider: "openai" | "gemini" | "openrouter" | "perplexity";
-  encryptedSecret: string;
-  label?: string;
-  status: "active" | "paused" | "rate_limited" | "invalid";
-  lastUsedAt?: string;
-  cooldownUntil?: string;
-}
-```
-
-Customers may store multiple keys for the same provider. The platform should treat those keys as a small pooled resource that jobs must acquire before making outbound AI requests.
-
-## Queueing and Async Jobs
-
-The platform is async-native.
-
-Every operation should create a job record and return immediately.
-
-### Job State
-
-Suggested states:
-
-```txt
-queued
-running
-waiting_for_child
-waiting_for_human
-succeeded
-failed
-cancelled
-```
-
-### Job Data
-
-Each job record should track:
-
-1. `job_id`
-2. `template_id`
-3. `operation_name`
-4. `tracer`
-5. `status`
-6. `payload`
-7. `result`
-8. `error`
-9. `progress`
-10. `webhook_url`
-11. `parent_job_id`
-12. `customer_id`
-13. `reservation_id` or billing reference
-14. timestamps
-
-### Logs
-
-Logs must be stored as structured job events, not just raw text.
-
-Each event should support:
-
-1. timestamp
-2. level
-3. message
-4. machine-readable metadata
-5. progress update
-6. artifact references
-
-This lets the client render a live job timeline later.
-
-## SQLite-Backed AI Key Queue
-
-SQLite is not only the v1 job store. It is also the coordination layer for customer AI API key usage.
-
-The intended model is:
-
-1. A job becomes runnable.
-2. The worker identifies which provider and model the next step requires.
-3. The worker attempts to lease one eligible customer key from SQLite.
-4. If a lease is granted, the worker performs the outbound API call.
-5. The worker records usage, updates cooldown state if needed, and releases the lease.
-
-This gives the platform a lightweight queue for AI key access without needing Redis, SQS, or a separate lock service.
-
-### Why SQLite Is Acceptable in v1
-
-This is a reasonable design if all of the following remain true:
-
-1. One EC2 host is the active source of truth.
-2. The platform runs a moderate number of worker loops.
-3. SQLite is configured in WAL mode.
-4. Lease acquisition is done transactionally.
-5. Jobs are retry-safe and can be rescheduled when no key is available.
-
-### Core Idea
-
-The AI key queue is represented by a combination of:
-
-1. Customer provider key records.
-2. Active key lease records.
-3. Key usage and error events.
-4. Cooldown timestamps after rate-limit responses.
-
-There is no separate message broker for API key access. Eligibility is derived from database state at lease time.
-
-### Suggested Tables
-
-```sql
-create table customer_provider_keys (
-  id text primary key,
-  customer_id text not null,
-  provider text not null,
-  label text,
-  encrypted_secret text not null,
-  status text not null,
-  weight integer not null default 1,
-  last_used_at text,
-  cooldown_until text,
-  disabled_reason text,
-  created_at text not null,
-  updated_at text not null
-);
-
-create table provider_key_leases (
-  key_id text primary key,
-  lease_token text not null,
-  worker_id text not null,
-  job_id text not null,
-  leased_at text not null,
-  expires_at text not null
-);
-
-create table provider_key_usage_events (
-  id text primary key,
-  key_id text not null,
-  job_id text not null,
-  provider text not null,
-  model text,
-  event_type text not null,
-  input_tokens integer,
-  output_tokens integer,
-  cost_usd real,
-  created_at text not null
-);
-```
-
-Optional model capability table:
-
-```sql
-create table provider_key_capabilities (
-  key_id text not null,
-  model text not null,
-  primary key (key_id, model)
-);
-```
-
-### Lease Acquisition
-
-Workers must acquire a key lease before making any outbound provider request on behalf of a customer.
-
-Lease acquisition should happen inside a transaction using `BEGIN IMMEDIATE`.
-
-The query should exclude keys that are:
-
-1. Not active.
-2. In cooldown.
-3. Already leased and whose lease has not expired.
-4. Incompatible with the requested provider or model.
-
-Preferred selection order in v1:
-
-1. Least recently used eligible key.
-2. Higher weight first when weights differ.
-
-Illustrative flow:
-
-```txt
-BEGIN IMMEDIATE
-1. Select one eligible key
-2. Insert active lease row
-3. Commit
-```
-
-Illustrative query shape:
-
-```sql
-select k.id
-from customer_provider_keys k
-left join provider_key_leases l
-  on l.key_id = k.id
-  and l.expires_at > datetime('now')
-where k.customer_id = ?
-  and k.provider = ?
-  and k.status = 'active'
-  and (k.cooldown_until is null or k.cooldown_until <= datetime('now'))
-  and l.key_id is null
-order by k.last_used_at asc nulls first, k.weight desc
-limit 1;
-```
-
-If a key is found, the worker inserts a lease row with a short expiry such as 30 to 90 seconds.
-
-If no key is found, the worker must not busy-loop. It should reschedule the job for a future run.
-
-### Lease Semantics
-
-Lease rows should contain:
-
-1. `key_id`
-2. `lease_token`
-3. `worker_id`
-4. `job_id`
-5. `leased_at`
-6. `expires_at`
-
-The lease token should be required for release or extension so one worker cannot accidentally release another worker's lease.
-
-### Lease Expiry and Recovery
-
-If a worker crashes, its lease should naturally expire and the key should become eligible again.
-
-For long-running requests, the platform may optionally support lease extension heartbeats. This is useful when the provider call or downstream processing can exceed the default lease duration.
-
-### Success Path
-
-After a successful provider call, the worker should:
-
-1. Record a usage event.
-2. Update `last_used_at`.
-3. Clear any temporary rate-limit status when appropriate.
-4. Release the lease.
-
-### Rate-Limit Path
-
-If the provider returns a rate-limit response, the worker should:
-
-1. Record a `rate_limit` usage event.
-2. Put the key into cooldown by setting `cooldown_until`.
-3. Release the lease.
-4. Reschedule the job.
-
-Cooldown duration may initially be determined by:
-
-1. Provider response headers if available.
-2. Provider-specific backoff policy.
-3. Conservative defaults when headers are absent.
-
-### Auth Failure Path
-
-If the provider reports invalid credentials, the worker should:
-
-1. Record an `auth_error` usage event.
-2. Mark the key `invalid`.
-3. Release the lease.
-4. Retry with another key if one exists.
-5. Fail the job clearly if no valid key remains.
-
-### Scheduler Behavior
-
-The scheduler should treat jobs as runnable only when both are true:
-
-1. The job itself is ready to run.
-2. A compatible provider key can likely be leased now or soon.
-
-Recommended loop:
-
-1. Fetch queued jobs ordered by `run_after`.
-2. Attempt key lease acquisition for the next provider-dependent step.
-3. If lease succeeds, run the step and mark job `running`.
-4. If lease fails, move `run_after` forward instead of spinning.
-5. Retry later.
-
-This is what makes the AI key queue effectively a SQLite-backed coordination system rather than a separate infrastructure dependency.
-
-### Observability
-
-The platform should emit logs and metrics for:
-
-1. Lease acquisition success rate.
-2. Lease wait time.
-3. Key cooldown frequency by provider.
-4. Key invalidation frequency.
-5. Job deferrals caused by unavailable keys.
-
-These signals will tell us when SQLite remains sufficient and when key coordination needs a stronger backend.
-
-## Rate Limiting and Provider Routing
-
-Customer keys are not interchangeable infinite resources.
-
-The platform must route AI calls through a provider layer that understands:
-
-1. Provider type.
-2. Model name.
-3. Key-level rate limits.
-4. Backoff behavior.
-5. Retry policy.
-6. Temporary key disablement after provider errors.
-
-### Initial Approach
-
-Use SQLite-backed leasing for queue and key selection in v1.
-
-This is acceptable if:
-
-1. One EC2 host is the source of truth.
-2. SQLite is configured in WAL mode.
-3. Concurrency expectations remain moderate.
-4. Jobs are idempotent enough to survive retries.
-
-### Future Upgrade Path
-
-If platform concurrency or reliability needs outgrow SQLite, move the job and rate-limit state to Postgres before adopting many-worker horizontal scale.
-
-## Billing Model
-
-The platform owns billing enforcement, but templates should emit billing events through framework APIs.
-
-Templates should not hand-roll their own pricing logic in arbitrary ways.
-
-### Billing Principle
-
-Bill conservatively enough to avoid cloud-cost loss.
-
-Current target:
-
-```txt
-customer_charge_usd ~= platform_cost_usd * 2
-```
-
-This is a margin buffer, not a final finance system.
-
-### Billing Event Types
-
-The framework should support at least:
-
-1. External AI token usage.
-2. Remotion render usage.
-3. EC2 / CPU / memory approximation.
-4. Storage writes.
-5. Data egress or expensive file processing when relevant.
-
-### Billing API for Templates
-
-Templates should call framework helpers like:
-
-```ts
-await ctx.billing.record({
-  type: "ai_generation",
-  provider: "openai",
-  model: "gpt-4.1",
-  estimatedCostUsd: 0.024,
-  metadata: {},
-});
-```
-
-The framework should translate these events into customer-facing charges.
-
-## Webhooks
-
-Every job may include an optional webhook destination.
-
-The platform should emit webhook events for:
-
-1. `job.queued`
-2. `job.running`
-3. `job.progress`
-4. `job.succeeded`
-5. `job.failed`
-6. `job.cancelled`
-
-Webhook delivery should be:
-
-1. Signed
-2. Retried with backoff
-3. Persisted as delivery attempts
-
-## File and Artifact Storage
-
-S3 is the system of record for customer-uploaded files and large generated outputs.
-
-### Customer Workspace Convention
-
-Suggested logical prefix:
-
-```txt
-s3://bucket/customers/:customerId/workspace/...
-```
-
-### Job Artifact Convention
-
-Suggested logical prefix:
-
-```txt
-s3://bucket/templates/:templateId/users/:userId/jobs/:jobId/...
-```
-
-Artifacts may include:
-
-1. Prompt snapshots
-2. Storyboards
-3. Preview images
-4. Audio assets
-5. Subtitle files
-6. Render manifests
-7. Final video outputs
-
-This prefix is for template-generated outputs and intermediate artifacts. Keep it stable across storage backends so local development mirrors production object layout.
-
-### Template Metadata Convention
-
-Suggested logical prefix:
-
-```txt
-s3://bucket/templates/:templateId/about/...
-```
-
-This prefix is for framework-owned template metadata assets such as preview images, preview videos, and other public "about" media referenced by `GET /templates/:templateId`.
-
-## Remotion Integration
-
-Remotion should be treated as a specialized downstream execution path, not the center of the platform.
-
-Templates can invoke Remotion through a framework adapter.
-
-Suggested flow:
-
-1. Template job prepares structured render input.
-2. Template writes required assets through framework storage.
-3. Template calls `ctx.remotion.render(...)`.
-4. The adapter renders locally or via Lambda depending on environment config.
-5. Final artifact is attached back to the parent job result.
-
-This keeps Remotion as one implementation detail among several, rather than forcing the platform to be Remotion-first.
-
-## Isolation Policy
-
-Default mode is shared in-process execution inside the main platform runtime.
-
-### Shared Execution Is Correct By Default
-
-Use shared execution when the template:
-
-1. Uses standard Node dependencies.
-2. Fits within normal memory and CPU budgets.
-3. Does not need a custom OS image.
-4. Can safely coexist with other templates.
-
-### Isolated Execution Is an Escape Hatch
-
-Allow a template to declare isolated execution later if it needs:
-
-1. Heavy FFmpeg or native binary workloads.
-2. Custom Chromium or system library requirements.
-3. A stricter security or resource boundary.
-4. Independent scaling or scheduling.
-
-Even then, the main platform should still own:
-
-1. Auth
-2. Billing
-3. Job creation
-4. Customer state
-5. Webhook delivery
-
-## Suggested Repository Shape
-
-```txt
-/src
-/templates/template_0000
-/templates/template_0001
-/AWS_REMOTION_HANDOFF.md
-/PLATFORM_SPEC.md
-/SKILL.director.md
-/SKILL.developer.md
-```
-
-## Suggested Internal Components
-
-### Platform API
-
-Responsibilities:
-
-1. Request validation
-2. Auth
-3. Template lookup
-4. Config updates
-5. Job creation
-6. Job status reads
-7. Webhook registration
-
-### Worker / Dispatcher
-
-Responsibilities:
-
-1. Pull queued jobs
-2. Acquire provider key leases
-3. Execute template jobs
-4. Persist logs and artifacts
-5. Update billing
-6. Deliver completion webhooks
-
-### Template Registry
-
-Responsibilities:
-
-1. Register approved in-house templates
-2. Expose metadata
-3. Resolve operations and jobs
-4. Enforce version compatibility
-
-## Security Notes
-
-1. Customer provider keys must be encrypted at rest.
-2. API keys must be hash-verified, not stored in plaintext.
-3. Template code is trusted internal code in v1, not untrusted tenant code.
-4. Webhook signatures must be mandatory.
-5. Customer file access must always be scoped by customer identity.
-6. Template source imports must pin an exact commit SHA before activation.
-7. Templates must include a `SKILL.md` file so customer AI agents have a framework-owned usage contract.
-
-## Operational Notes for v1
-
-1. Run one Docker image on one EC2 host.
-2. Keep API and worker in the same deployable unit initially.
-3. Use SQLite in WAL mode.
-4. Back up SQLite and treat it as transitional infrastructure.
-5. Store all large artifacts in S3, not on container disk.
-6. Assume template code is centrally reviewed before deployment.
-7. Not every template requires Remotion. Remotion validation should only apply to templates that actually call the Remotion adapter.
-
-## Template Certification Minimum
-
-A template must not be activatable unless all of the following pass:
-
-1. template metadata contract is valid
-2. operation-to-workflow references are valid
-3. a template-local `SKILL.md` file exists
-4. every operation defines a smoke-test payload
-5. the smoke-test harness passes
-6. the release is associated with a pinned commit SHA
-
-## Known Limits of v1
-
-1. SQLite is a reasonable starting point but not the final queueing backend for large-scale concurrency.
-2. Shared in-process template execution is simpler operationally but weaker as a hard isolation boundary.
-3. EC2 cost attribution will begin as approximation plus provider-cost tracking, not perfect real-time infrastructure metering.
-
-## Recommendation Summary
-
-The recommended initial Vidfarm platform is:
-
-1. One Node.js 22 Docker container on EC2.
-2. Hono as the HTTP API layer.
-3. SQLite as the initial jobs and rate-limit store.
-4. S3 for workspace and artifact storage.
-5. Remotion Lambda as a downstream rendering path.
-6. Templates implemented as normal internal TypeScript packages with full npm access.
-7. Public API defined as async operations that enqueue jobs.
-8. Optional isolated template execution added later only when justified by concrete workload needs.
-
-This is the simplest architecture that matches the product goals without prematurely turning each template into a separate service.