@effing/ffs 0.6.1 → 0.7.1

package/README.md

@@ -12,7 +12,7 @@ Takes an `EffieData` composition and renders it to an MP4 video using FFmpeg. Us
 npm install @effing/ffs
 ```
 
-FFmpeg is bundled via `ffmpeg-static` — no system installation required.
+FFmpeg is bundled via `@effing/ffmpeg` — no system installation required.
 
 ## Quick Start
 
@@ -44,17 +44,21 @@ npx @effing/ffs
 FFS_PORT=8080 npx @effing/ffs
 ```
 
-Rendering is a two-step process with the HTTP server: first obtain a stream URL, then stream that URL to get the video.
+Rendering is a two-step process: POST to create a job, then connect to the SSE progress stream to track warmup and rendering. The video URL is revealed in the `ready` event.
 
 ```bash
-# 1. Obtain a stream URL
+# 1. Create a render job
 curl -X POST http://localhost:2000/render \
   -H "Content-Type: application/json" \
-  -d @composition.json
-# Returns: { "id": "...", "url": "http://localhost:2000/render/123e4567-e89b-12d3-a456-426614174000" }
+  -d '{"effie": ...}'
+# Returns: { "id": "...", "progressUrl": "http://localhost:2000/render/.../progress" }
 
-# 2. Stream the URL to get the video
-curl http://localhost:2000/render/123e4567-e89b-12d3-a456-426614174000 -o output.mp4
+# 2. Connect to SSE progress stream (or use EventSource in browser)
+curl http://localhost:2000/render/.../progress
+# SSE events: warmup:start, warmup:progress, warmup:complete, ready (with videoUrl)
+
+# 3. Fetch the video
+curl http://localhost:2000/render/.../video -o output.mp4
 ```
 
 The server uses an internal HTTP proxy for video/audio URLs to ensure reliable DNS resolution in containerized environments (e.g., Alpine Linux). This is why you might see another server running on a random port.
@@ -73,25 +77,13 @@ The server uses an internal HTTP proxy for video/audio URLs to ensure reliable D
 | `FFS_TRANSIENT_STORE_ACCESS_KEY` | S3 access key ID                                     |
 | `FFS_TRANSIENT_STORE_SECRET_KEY` | S3 secret access key                                 |
 | `FFS_TRANSIENT_STORE_LOCAL_DIR`  | Local storage directory (when not using S3)          |
-| `FFS_SOURCE_CACHE_TTL_MS`        | TTL for cached sources in ms (default: 60 min)       |
-| `FFS_JOB_DATA_TTL_MS`            | TTL for job data in ms (default: 8 hours)            |
+| `FFS_TRANSIENT_STORE_TTL_MS`     | TTL for all transient data in ms (default: 60 min)   |
 | `FFS_WARMUP_CONCURRENCY`         | Concurrent source fetches during warmup (default: 4) |
 
 When `FFS_TRANSIENT_STORE_BUCKET` is not set, FFS uses the local filesystem for storage (default: system temp directory). Local files are automatically cleaned up after the TTL expires.
 
 For S3 storage, the TTL is set as the `Expires` header on objects. Note that this is metadata only. To enable automatic deletion, configure [S3 lifecycle rules](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html) on your bucket to delete expired objects.
 
-## Concepts
-
-### EffieRenderer
-
-The main class that orchestrates video rendering:
-
-1. **Builds FFmpeg command** — Constructs complex filter graphs for overlays, transitions, effects
-2. **Fetches sources** — Downloads images, animations, and audio from URLs
-3. **Processes layers** — Applies motion, effects, and timing to each layer
-4. **Outputs video** — Streams H.264/AAC MP4 to stdout or file
-
 ## API Overview
 
 ### EffieRenderer
@@ -141,20 +133,15 @@ When running as an HTTP server, FFS provides endpoints for rendering, cache warm
 
 ### `POST /render`
 
-Creates a render job and returns a stream URL to execute it. Supports two request formats:
-
-**Raw EffieData**: Body is raw EffieData, options in query params.
+Creates a render job that includes warmup and render phases. Supports optional cache purging.
 
-| Query Param | Effect                    |
-| ----------- | ------------------------- |
-| `scale`     | Scale factor (default: 1) |
-
-**Wrapped format**: Body contains `effie` plus options.
+**Request:**
 
 ```typescript
 type RenderOptions = {
   effie: EffieData | string; // EffieData object or URL to fetch from
   scale?: number; // Scale factor (default: 1)
+  purge?: boolean; // Purge cached sources before warmup
   upload?: {
     videoUrl: string; // Pre-signed URL to upload rendered video
     coverUrl?: string; // Pre-signed URL to upload cover image
@@ -162,70 +149,84 @@ type RenderOptions = {
 };
 ```
 
-| Option         | Effect                                         |
-| -------------- | ---------------------------------------------- |
-| `effie` as URL | Fetches EffieData from the URL before storing  |
-| `upload`       | GET will upload and stream SSE progress events |
-
 **Response:**
 
 ```json
 {
   "id": "550e8400-e29b-41d4-a716-446655440000",
-  "url": "http://localhost:2000/render/550e8400-e29b-41d4-a716-446655440000"
+  "progressUrl": "http://localhost:2000/render/550e8400-e29b-41d4-a716-446655440000/progress"
 }
 ```
 
-### `GET /render/:id`
+### `GET /render/:id/progress`
 
-Executes the render job. Behavior depends on whether `upload` was specified:
+Streams warmup and render progress via SSE. All warmup events are prefixed with `warmup:`, render events with `render:`.
 
-**Without upload** — Streams the MP4 video directly:
+**Events:**
 
-```bash
-curl http://localhost:2000/render/550e8400-... -o output.mp4
-```
+| Event                | Phase  | Data                                                              |
+| -------------------- | ------ | ----------------------------------------------------------------- |
+| `purge:complete`     | purge  | `{ "purged": 3, "total": 5 }`                                     |
+| `warmup:start`       | warmup | `{ "total": 5 }`                                                  |
+| `warmup:progress`    | warmup | `{ "url": "...", "status": "hit"\|"cached"\|"error", ... }`       |
+| `warmup:downloading` | warmup | `{ "url": "...", "status": "downloading", "bytesReceived": ... }` |
+| `warmup:summary`     | warmup | `{ "cached": 5, "failed": 0, "skipped": 0, "total": 5 }`          |
+| `warmup:complete`    | warmup | `{ "status": "ready" }`                                           |
+| `render:started`     | render | `{ "status": "rendering" }`                                       |
+| `keepalive`          | both   | `{ "phase": "warmup" }` or `{ "phase": "render" }`                |
+| `render:complete`    | render | `{ "status": "uploaded", "timings": {...} }` (upload mode)        |
+| `ready`              | -      | `{ "videoUrl": "..." }` (non-upload mode)                         |
+| `complete`           | -      | `{ "status": "done" }` (upload mode)                              |
+| `error`              | any    | `{ "phase": "warmup"\|"render", "message": "..." }`               |
 
-**With upload** — Streams progress via Server-Sent Events (SSE) while uploading:
+**Without upload** — The `ready` event provides a `videoUrl` pointing to `/render/:id/video`. The actual rendering happens when you fetch that URL:
 
-| Event       | Data                                                       |
-| ----------- | ---------------------------------------------------------- |
-| `started`   | `{ "status": "rendering" }`                                |
-| `keepalive` | `{ "status": "rendering" }` or `{ "status": "uploading" }` |
-| `complete`  | `{ "status": "uploaded", "timings": {...} }`               |
-| `error`     | `{ "message": "..." }`                                     |
+```typescript
+const events = new EventSource(progressUrl);
+events.addEventListener("ready", (e) => {
+  const { videoUrl } = JSON.parse(e.data);
+  // Fetch videoUrl to stream the rendered video
+  events.close();
+});
+```
 
-**Example:**
+**With upload** — Uploads directly and streams progress:
 
 ```typescript
-// Create render job
-const { url } = await fetch("/render", {
-  method: "POST",
-  headers: { "Content-Type": "application/json" },
-  body: JSON.stringify({ effie: effieData, scale: 0.5 }),
-}).then((r) => r.json());
+const events = new EventSource(progressUrl);
+events.addEventListener("render:complete", (e) => {
+  const { timings } = JSON.parse(e.data);
+  console.log("Uploaded!", timings);
+});
+events.addEventListener("complete", () => {
+  events.close();
+});
+```
 
-// Stream video directly
-const videoResponse = await fetch(url);
-const videoBlob = await videoResponse.blob();
+### `GET /render/:id/video`
+
+Streams the rendered MP4 video (non-upload mode only). Returns 404 until the warmup phase completes and the video sub-job is created.
+
+```bash
+curl http://localhost:2000/render/550e8400-.../video -o output.mp4
 ```
 
 ### `POST /warmup`
 
-Creates a warmup job for pre-fetching and caching the sources from an Effie composition. Use this to avoid render timeouts when sources (especially annies) take a long time to generate.
+Creates a standalone warmup job for pre-fetching and caching the sources from an Effie composition.
 
-**Request:** Same format as `/render` (raw or wrapped EffieData with `effie` field).
+**Request:** Same format as `/render` (wrapped EffieData with `effie` field).
 
 **Response:**
 
 ```json
 {
   "id": "550e8400-e29b-41d4-a716-446655440000",
-  "url": "http://localhost:2000/warmup/550e8400-e29b-41d4-a716-446655440000"
+  "progressUrl": "http://localhost:2000/warmup/550e8400-e29b-41d4-a716-446655440000/progress"
 }
 ```
 
-### `GET /warmup/:id`
+### `GET /warmup/:id/progress`
 
 Runs the cache warmup job and streams the progress via Server-Sent Events (SSE). Connect with `EventSource` for real-time updates.
 
@@ -244,14 +245,14 @@ Runs the cache warmup job and streams the progress via Server-Sent Events (SSE).
 
 ```typescript
 // Create warmup job
-const { url } = await fetch("/warmup", {
+const { progressUrl } = await fetch("/warmup", {
   method: "POST",
   headers: { "Content-Type": "application/json" },
   body: JSON.stringify({ effie: effieData }),
 }).then((r) => r.json());
 
-// Stream progress (url is a full URL)
-const events = new EventSource(url);
+// Stream progress
+const events = new EventSource(progressUrl);
 events.addEventListener("complete", () => {
   events.close();
   // Now safe to call /render
@@ -262,7 +263,7 @@ events.addEventListener("complete", () => {
 
 Purges cached sources for a given Effie composition.
 
-**Request:** Same format as `/render` (raw or wrapped EffieData with `effie` field).
+**Request:** Same format as `/render` (wrapped EffieData with `effie` field).
 
 **Response:**
 
@@ -270,73 +271,6 @@ Purges cached sources for a given Effie composition.
 { "purged": 3, "total": 5 }
 ```
 
-### `POST /warmup-and-render`
-
-Creates a combined warmup and render job that runs both phases in a single SSE stream. This is useful when you want to warmup sources and render in one request.
-
-**Request:**
-
-```typescript
-type WarmupAndRenderOptions = {
-  effie: EffieData | string; // EffieData object or URL to fetch from
-  scale?: number; // Scale factor (default: 1)
-  upload?: {
-    videoUrl: string; // Pre-signed URL to upload rendered video
-    coverUrl?: string; // Pre-signed URL to upload cover image
-  };
-};
-```
-
-**Response:**
-
-```json
-{
-  "id": "550e8400-e29b-41d4-a716-446655440000",
-  "url": "http://localhost:2000/warmup-and-render/550e8400-e29b-41d4-a716-446655440000"
-}
-```
-
-### `GET /warmup-and-render/:id`
-
-Executes the combined warmup and render job, streaming progress via SSE. All events are prefixed with `warmup:` or `render:` to indicate the phase.
-
-**Events:**
-
-| Event                | Phase  | Data                                                              |
-| -------------------- | ------ | ----------------------------------------------------------------- |
-| `warmup:start`       | warmup | `{ "total": 5 }`                                                  |
-| `warmup:progress`    | warmup | `{ "url": "...", "status": "hit"\|"cached"\|"error", ... }`       |
-| `warmup:downloading` | warmup | `{ "url": "...", "status": "downloading", "bytesReceived": ... }` |
-| `warmup:summary`     | warmup | `{ "cached": 5, "failed": 0, "skipped": 0, "total": 5 }`          |
-| `warmup:complete`    | warmup | `{ "status": "ready" }`                                           |
-| `render:started`     | render | `{ "status": "rendering" }`                                       |
-| `keepalive`          | both   | `{ "phase": "warmup" }` or `{ "phase": "render" }`                |
-| `render:complete`    | render | `{ "status": "uploaded", "timings": {...} }` (upload mode)        |
-| `complete`           | -      | `{ "status": "ready", "videoUrl": "..." }` (non-upload mode)      |
-| `error`              | any    | `{ "phase": "warmup"\|"render", "message": "..." }`               |
-
-**Without upload** — Returns a `videoUrl` pointing to `/render/:id` for streaming:
-
-```typescript
-const events = new EventSource(url);
-events.addEventListener("complete", (e) => {
-  const { videoUrl } = JSON.parse(e.data);
-  // Fetch videoUrl to stream the rendered video
-  events.close();
-});
-```
-
-**With upload** — Uploads directly and streams progress:
-
-```typescript
-const events = new EventSource(url);
-events.addEventListener("render:complete", (e) => {
-  const { timings } = JSON.parse(e.data);
-  console.log("Uploaded!", timings);
-  events.close();
-});
-```
-
 ## Backend Separation
 
 FFS supports running warmup and render on separate backends via resolver callbacks.
@@ -369,8 +303,9 @@ const ctx = await createServerContext({
 });
 ```
 
-The render resolver receives the effie data; the warmup resolver receives the source list.
-Both receive optional metadata (passed via handler options). Return `null` to handle locally.
+The `warmupBackendResolver` determines where warmup _work_ happens — used by `/warmup/:id/progress` and the warmup phase within `/render/:id/progress`. The `renderBackendResolver` determines where video rendering _work_ happens — used by `/render/:id/video` and the render+upload phase in upload mode.
+
+Both resolvers receive optional metadata (passed via handler options). Return `null` to handle locally.
 
 ### Job metadata
 
@@ -429,33 +364,36 @@ const finalStream = await joinRenderer.render();
 
 ```typescript
 // Create render job
-const { url } = await fetch("http://localhost:2000/render", {
+const { progressUrl } = await fetch("http://localhost:2000/render", {
   method: "POST",
   headers: { "Content-Type": "application/json" },
   body: JSON.stringify({ effie: effieData, scale: 0.5 }),
 }).then((r) => r.json());
 
-// Stream the video
-const video = await fetch(url).then((r) => r.blob());
+// Connect to SSE progress
+const events = new EventSource(progressUrl);
+events.addEventListener("ready", (e) => {
+  const { videoUrl } = JSON.parse(e.data);
+  // Fetch the video (rendering happens on-demand)
+  const video = await fetch(videoUrl).then((r) => r.blob());
+  events.close();
+});
 ```
 
-**Fetch from URL and render:**
+**Render with cache purge:**
 
 ```typescript
-const { url } = await fetch("http://localhost:2000/render", {
+const { progressUrl } = await fetch("http://localhost:2000/render", {
   method: "POST",
   headers: { "Content-Type": "application/json" },
-  body: JSON.stringify({
-    effie: "https://example.com/composition.json",
-    scale: 0.5,
-  }),
+  body: JSON.stringify({ effie: effieData, scale: 0.5, purge: true }),
 }).then((r) => r.json());
 ```
 
 **Render and upload to S3 (SSE progress):**
 
 ```typescript
-const { url } = await fetch("http://localhost:2000/render", {
+const { progressUrl } = await fetch("http://localhost:2000/render", {
   method: "POST",
   headers: { "Content-Type": "application/json" },
   body: JSON.stringify({
@@ -468,38 +406,9 @@ const { url } = await fetch("http://localhost:2000/render", {
 }).then((r) => r.json());
 
 // Connect to SSE for progress
-const events = new EventSource(url);
-events.addEventListener("complete", (e) => {
-  const { timings } = JSON.parse(e.data);
-  console.log("Uploaded!", timings);
-  events.close();
-});
-```
-
-**Warmup and render in one stream:**
-
-```typescript
-const { url } = await fetch("http://localhost:2000/warmup-and-render", {
-  method: "POST",
-  headers: { "Content-Type": "application/json" },
-  body: JSON.stringify({ effie: effieData, scale: 0.5 }),
-}).then((r) => r.json());
-
-// Connect to SSE for combined progress
-const events = new EventSource(url);
-
-events.addEventListener("warmup:progress", (e) => {
-  const { url, status, cached, total } = JSON.parse(e.data);
-  console.log(`Warmup: ${cached}/${total} - ${url} ${status}`);
-});
-
-events.addEventListener("render:started", () => {
-  console.log("Rendering started...");
-});
-
-events.addEventListener("complete", (e) => {
-  const { videoUrl } = JSON.parse(e.data);
-  console.log("Ready! Video at:", videoUrl);
+const events = new EventSource(progressUrl);
+events.addEventListener("complete", () => {
+  console.log("Done!");
   events.close();
 });
 ```

package/dist/{chunk-PERB3C4S.js → chunk-AUH73KG2.js}

@@ -15,14 +15,12 @@ import { pipeline } from "stream/promises";
 import path from "path";
 import os from "os";
 import crypto from "crypto";
-var DEFAULT_SOURCE_TTL_MS = 60 * 60 * 1e3;
-var DEFAULT_JOB_DATA_TTL_MS = 8 * 60 * 60 * 1e3;
+var DEFAULT_TTL_MS = 60 * 60 * 1e3;
 var S3TransientStore = class {
   client;
   bucket;
   prefix;
-  sourceTtlMs;
-  jobDataTtlMs;
+  ttlMs;
   constructor(options) {
     this.client = new S3Client({
       endpoint: options.endpoint,
@@ -35,8 +33,7 @@ var S3TransientStore = class {
     });
     this.bucket = options.bucket;
     this.prefix = options.prefix ?? "";
-    this.sourceTtlMs = options.sourceTtlMs ?? DEFAULT_SOURCE_TTL_MS;
-    this.jobDataTtlMs = options.jobDataTtlMs ?? DEFAULT_JOB_DATA_TTL_MS;
+    this.ttlMs = options.ttlMs ?? DEFAULT_TTL_MS;
   }
   getExpires(ttlMs) {
     return new Date(Date.now() + ttlMs);
@@ -51,7 +48,7 @@ var S3TransientStore = class {
         Bucket: this.bucket,
         Key: this.getFullKey(key),
         Body: stream,
-        Expires: this.getExpires(ttlMs ?? this.sourceTtlMs)
+        Expires: this.getExpires(ttlMs ?? this.ttlMs)
       }
     });
     await upload.done();
@@ -119,7 +116,7 @@ var S3TransientStore = class {
         Key: this.getFullKey(key),
         Body: JSON.stringify(data),
         ContentType: "application/json",
-        Expires: this.getExpires(ttlMs ?? this.jobDataTtlMs)
+        Expires: this.getExpires(ttlMs ?? this.ttlMs)
       })
     );
   }
@@ -149,15 +146,10 @@ var LocalTransientStore = class {
   baseDir;
   initialized = false;
   cleanupInterval;
-  sourceTtlMs;
-  jobDataTtlMs;
-  /** For cleanup, use the longer of the two TTLs */
-  maxTtlMs;
+  ttlMs;
   constructor(options) {
     this.baseDir = options?.baseDir ?? path.join(os.tmpdir(), "ffs-transient");
-    this.sourceTtlMs = options?.sourceTtlMs ?? DEFAULT_SOURCE_TTL_MS;
-    this.jobDataTtlMs = options?.jobDataTtlMs ?? DEFAULT_JOB_DATA_TTL_MS;
-    this.maxTtlMs = Math.max(this.sourceTtlMs, this.jobDataTtlMs);
+    this.ttlMs = options?.ttlMs ?? DEFAULT_TTL_MS;
     this.cleanupInterval = setInterval(() => {
       this.cleanupExpired().catch(console.error);
     }, 3e5);
@@ -188,7 +180,7 @@ var LocalTransientStore = class {
       } else if (entry.isFile()) {
         try {
           const stat = await fs.stat(fullPath);
-          if (now - stat.mtimeMs > this.maxTtlMs) {
+          if (now - stat.mtimeMs > this.ttlMs) {
             await fs.rm(fullPath, { force: true });
           }
         } catch {
@@ -272,8 +264,7 @@ var LocalTransientStore = class {
   }
 };
 function createTransientStore() {
-  const sourceTtlMs = process.env.FFS_SOURCE_CACHE_TTL_MS ? parseInt(process.env.FFS_SOURCE_CACHE_TTL_MS, 10) : DEFAULT_SOURCE_TTL_MS;
-  const jobDataTtlMs = process.env.FFS_JOB_DATA_TTL_MS ? parseInt(process.env.FFS_JOB_DATA_TTL_MS, 10) : DEFAULT_JOB_DATA_TTL_MS;
+  const ttlMs = process.env.FFS_TRANSIENT_STORE_TTL_MS ? parseInt(process.env.FFS_TRANSIENT_STORE_TTL_MS, 10) : DEFAULT_TTL_MS;
   if (process.env.FFS_TRANSIENT_STORE_BUCKET) {
     return new S3TransientStore({
       endpoint: process.env.FFS_TRANSIENT_STORE_ENDPOINT,
@@ -282,14 +273,12 @@ function createTransientStore() {
       prefix: process.env.FFS_TRANSIENT_STORE_PREFIX,
       accessKeyId: process.env.FFS_TRANSIENT_STORE_ACCESS_KEY,
       secretAccessKey: process.env.FFS_TRANSIENT_STORE_SECRET_KEY,
-      sourceTtlMs,
-      jobDataTtlMs
+      ttlMs
     });
   }
   return new LocalTransientStore({
     baseDir: process.env.FFS_TRANSIENT_STORE_LOCAL_DIR,
-    sourceTtlMs,
-    jobDataTtlMs
+    ttlMs
   });
 }
 function hashUrl(url) {
@@ -304,14 +293,14 @@ function warmupJobStoreKey(jobId) {
 function renderJobStoreKey(jobId) {
   return `jobs/render/${jobId}.json`;
 }
-function warmupAndRenderJobStoreKey(jobId) {
-  return `jobs/warmup-and-render/${jobId}.json`;
+function videoJobStoreKey(jobId) {
+  return `jobs/video/${jobId}.json`;
 }
 var storeKeys = {
   source: sourceStoreKey,
   warmupJob: warmupJobStoreKey,
   renderJob: renderJobStoreKey,
-  warmupAndRenderJob: warmupAndRenderJobStoreKey
+  videoJob: videoJobStoreKey
 };
 
 // src/fetch.ts

package/dist/{chunk-O7Z6DV2I.js → chunk-CSKH34HX.js}

@@ -1,7 +1,7 @@
 import {
   ffsFetch,
   storeKeys
-} from "./chunk-4N2GLGC5.js";
+} from "./chunk-UN34ESVZ.js";
 
 // src/render.ts
 import { Readable } from "stream";
@@ -936,4 +936,4 @@ export {
   FFmpegRunner,
   EffieRenderer
 };
-//# sourceMappingURL=chunk-O7Z6DV2I.js.map
+//# sourceMappingURL=chunk-CSKH34HX.js.map