@replayio-app-building/netlify-recorder 0.16.0 → 0.17.0

package/README.md

@@ -1,6 +1,6 @@
 # @replayio-app-building/netlify-recorder
 
-Capture and replay Netlify function executions as [Replay](https://replay.io) recordings. This package intercepts outbound network calls and environment variable reads during handler execution, stores the captured data as a blob, and can later reproduce the exact execution as a Replay recording for debugging and analysis.
+Capture and replay Netlify function executions as [Replay](https://replay.io) recordings. This package intercepts outbound network calls and environment variable reads during handler execution, stores the captured data via the Netlify Recorder service, and can later reproduce the exact execution as a Replay recording for debugging and analysis.
 
 ## Installation
 
@@ -8,23 +8,11 @@ Capture and replay Netlify function executions as [Replay](https://replay.io) re
 npm install @replayio-app-building/netlify-recorder
 ```
 
-## Integration Options
-
-There are two ways to use this package:
-
-- **Option A: Use the Netlify Recorder service (recommended)** — The service handles blob storage, request tracking, and recording creation. Your app just wraps its handlers and calls `remoteCallbacks()`. No database or container infrastructure needed.
-
-- **Option B: Self-hosted** — You manage your own blob storage, database tables, and recording containers. Full control, but requires more setup.
-
----
-
-## Option A: Using the Netlify Recorder Service
-
-The Netlify Recorder app (`https://netlify-recorder-bm4wmw.netlify.app`) provides a hosted service that stores captured request data, manages a pool of recording containers, and creates Replay recordings on demand. Your app needs zero database or infrastructure setup.
+## Setup
 
 ### 1. Set required environment variables
 
-`finishRequest` needs to know which repository, branch, and commit your deployed code belongs to. Set these three environment variables on your Netlify site:
+`finishRequest` needs to know which repository, branch, and commit your deployed code belongs to. Set these environment variables on your Netlify site:
 
 | Variable | Description | How to set |
 |---|---|---|
@@ -115,7 +103,7 @@ export default createRecordingRequestHandler(
 
 ### 3. Create recordings
 
-When you want to turn a captured request into a Replay recording, POST to the service's `create-recording` endpoint with the request ID. If the request was created with a secret, you must include it:
+When you want to turn a captured request into a Replay recording, POST to the Netlify Recorder service's `create-recording` endpoint with the request ID. If the request was created with a secret, you must include it:
 
 ```typescript
 const response = await fetch(
@@ -132,7 +120,7 @@ const response = await fetch(
 );
 ```
 
-The service looks up the stored blob data, dispatches the work to a pool container, and creates the recording.
+The service looks up the stored blob data, dispatches the work to a recording container, and creates the recording.
 
 If `webhookUrl` is provided, the service will POST the result to that URL when the recording completes (or fails):
 
@@ -225,7 +213,7 @@ Requests created without a secret remain accessible without one (backward compat
 
 Store your secret as a `NETLIFY_RECORDER_SECRET` environment variable. To keep it secure:
 
-1. **Netlify site environment:** Add `NETLIFY_RECORDER_SECRET` in your Netlify site's environment variables (Site settings → Environment variables). This makes it available to all deployed functions.
+1. **Netlify site environment:** Add `NETLIFY_RECORDER_SECRET` in your Netlify site's environment variables (Site settings > Environment variables). This makes it available to all deployed functions.
 
 2. **Branch-level secret** (for CI/development): If you're using the Netlify Recorder agent infrastructure, store the secret as a branch secret so deploy scripts and background agents can access it:
 
@@ -239,158 +227,6 @@ Use a strong random string (e.g. `openssl rand -base64 32`) and rotate it if com
 
 ---
 
-## Option B: Self-Hosted
-
-If you need full control, you can manage your own blob storage, database, and recording containers. This requires more setup but gives you complete ownership of the data pipeline.
-
-### 1. Set required environment variables
-
-Same as Option A — you must set `REPLAY_REPOSITORY_URL`, `COMMIT_SHA`, and `BRANCH_NAME` on your Netlify site. See the table in Option A, Step 1 above.
-
-### 2. Wrap your Netlify function
-
-Use `createRecordingRequestHandler` with custom callbacks that write to your own storage and database:
-
-```typescript
-import { createRecordingRequestHandler } from "@replayio-app-building/netlify-recorder";
-
-const handler = createRecordingRequestHandler(
-  async (event) => {
-    const result = await myBusinessLogic();
-
-    return {
-      statusCode: 200,
-      headers: { "Content-Type": "application/json" },
-      body: JSON.stringify(result),
-    };
-  },
-  {
-    secret: process.env.NETLIFY_RECORDER_SECRET,
-    callbacks: {
-      uploadBlob: async (data) => {
-        // Upload the JSON string to your blob storage (S3, R2, etc.)
-        const res = await fetch("https://storage.example.com/upload", {
-          method: "PUT",
-          body: data,
-        });
-        const { url } = await res.json();
-        return url;
-      },
-      storeRequestData: async ({ blobUrl, commitSha, branchName, repositoryUrl, handlerPath, secret }) => {
-        const [row] = await sql`
-          INSERT INTO requests (blob_url, commit_sha, branch_name, repository_url, handler_path, secret, status)
-          VALUES (${blobUrl}, ${commitSha}, ${branchName}, ${repositoryUrl}, ${handlerPath}, ${secret}, 'captured')
-          RETURNING id
-        `;
-        return row.id;
-      },
-    },
-  }
-);
-
-export { handler };
-```
-
-### 3. Create a requests database table
-
-```sql
-CREATE TABLE IF NOT EXISTS requests (
-  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-  blob_url TEXT,
-  commit_sha TEXT,
-  branch_name TEXT,
-  repository_url TEXT,
-  handler_path TEXT,
-  secret TEXT,
-  recording_id TEXT,
-  status TEXT NOT NULL DEFAULT 'captured'
-    CHECK (status IN ('captured', 'processing', 'recorded', 'failed')),
-  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
-);
-
-CREATE INDEX IF NOT EXISTS idx_requests_secret ON requests (secret) WHERE secret IS NOT NULL;
-```
-
-### 4. Create a background function to produce recordings
-
-```typescript
-import { ensureRequestRecording } from "@replayio-app-building/netlify-recorder";
-import type { Handler } from "@netlify/functions";
-
-const handler: Handler = async (event) => {
-  const { requestId } = JSON.parse(event.body ?? "{}");
-
-  const recordingId = await ensureRequestRecording(requestId, {
-    repositoryUrl: process.env.APP_REPOSITORY_URL!,
-    lookupRequest: async (id) => {
-      const [row] = await sql`
-        SELECT blob_url, commit_sha, branch_name, handler_path
-        FROM requests WHERE id = ${id}
-      `;
-      return {
-        blobUrl: row.blob_url,
-        commitSha: row.commit_sha,
-        branchName: row.branch_name ?? "main",
-        handlerPath: row.handler_path,
-      };
-    },
-    updateStatus: async (id, status, recordingId) => {
-      await sql`
-        UPDATE requests
-        SET status = ${status},
-            recording_id = ${recordingId ?? null},
-            updated_at = NOW()
-        WHERE id = ${id}
-      `;
-    },
-  });
-
-  return {
-    statusCode: 200,
-    body: JSON.stringify({ recordingId }),
-  };
-};
-
-export { handler };
-```
-
-### 5. Create a container script
-
-This script runs inside the recording container under `replay-node`:
-
-```typescript
-// scripts/create-request-recording.ts
-import { createRequestRecording } from "@replayio-app-building/netlify-recorder";
-
-const args = process.argv.slice(2);
-const blobUrl = args[args.indexOf("--blob-url") + 1]!;
-const handlerPath = args[args.indexOf("--handler-path") + 1]!;
-
-await createRequestRecording(blobUrl, handlerPath, {
-  method: "POST",
-  url: handlerPath,
-  headers: {},
-});
-```
-
-### Required infrastructure
-
-Self-hosted recording requires these environment variables:
-
-| Variable | Description |
-|---|---|
-| `INFISICAL_CLIENT_ID` | Infisical service account client ID |
-| `INFISICAL_CLIENT_SECRET` | Infisical service account client secret |
-| `INFISICAL_PROJECT_ID` | Infisical project ID |
-| `INFISICAL_ENVIRONMENT` | Infisical environment (e.g. `production`) |
-| `FLY_API_TOKEN` | Fly.io API token for container management |
-| `FLY_APP_NAME` | Fly.io app name for container deployment |
-| `APP_REPOSITORY_URL` | Git repository URL for container cloning |
-| `RECORD_REPLAY_API_KEY` | Replay API key for recording upload |
-
----
-
 ## Audit Log Support
 
 The package automatically tracks database mutations (INSERT, UPDATE, DELETE) in an `audit_log` table and links each change to the Replay request that caused it. When your handler is wrapped with `createRecordingRequestHandler`, all Neon SQL queries are automatically tagged with the request ID — no changes to your SQL code required.
@@ -484,7 +320,7 @@ When `context.waitUntil` is not available (v1 handlers or missing context), the
 
 **Parameters:**
 - `handler` — Your async handler function `(event, context?) => Promise<{ statusCode, headers?, body? }>`
-- `options.callbacks` — Either `remoteCallbacks(serviceUrl)` or custom `{ uploadBlob, storeRequestData }` callbacks
+- `options.callbacks` — `remoteCallbacks(serviceUrl)` for sending captured data to the Netlify Recorder service
 - `options.handlerPath` — Path to the handler file (used for recording metadata)
 - `options.commitSha` — Override `COMMIT_SHA` env var
 - `options.branchName` — Override `BRANCH_NAME` env var
@@ -520,7 +356,7 @@ Logs a `console.warn` when the total duration exceeds 2 seconds or when individu
 
 **Parameters:**
 - `requestContext` — The context returned by `startRequest`
-- `callbacks` — Either `remoteCallbacks(serviceUrl)` or custom `{ uploadBlob, storeRequestData }` callbacks
+- `callbacks` — `remoteCallbacks(serviceUrl)` for sending captured data to the Netlify Recorder service
 - `response` — The handler's response object (`{ statusCode, headers?, body? }`)
 - `options.handlerPath` — Path to the handler file (used for recording metadata)
 - `options.commitSha` — Override `COMMIT_SHA` env var
@@ -531,30 +367,11 @@ Logs a `console.warn` when the total duration exceeds 2 seconds or when individu
 
 ### `remoteCallbacks(serviceUrl): FinishRequestCallbacks`
 
-Creates callbacks for `finishRequest` that send captured data to the Netlify Recorder service. The service handles blob storage and request tracking — no local database needed.
+Creates callbacks for `finishRequest` that send captured data to the Netlify Recorder service. The service handles blob storage and request tracking.
 
 **Parameters:**
 - `serviceUrl` — Base URL of the Netlify Recorder service (e.g. `"https://netlify-recorder-bm4wmw.netlify.app"`)
 
-### `ensureRequestRecording(requestId, options): Promise<string>`
-
-Spawns a container via `@replayio/app-building` to create a Replay recording from captured request data. Returns the recording ID. Only needed for self-hosted setups (Option B).
-
-**Parameters:**
-- `requestId` — The request to create a recording for
-- `options.repositoryUrl` — Git repository URL for the container to clone
-- `options.lookupRequest(id)` — Fetches `{ blobUrl, commitSha, branchName, handlerPath }` from the database
-- `options.updateStatus(id, status, recordingId?)` — Updates the request status in the database
-
-### `createRequestRecording(blobUrl, handlerPath, requestInfo): Promise<void>`
-
-Called inside a container running under `replay-node`. Downloads the captured data blob, installs replay-mode interceptors (which return pre-recorded responses instead of making real calls), and executes the original handler so replay-node can record the execution. Only needed for self-hosted setups (Option B).
-
-**Parameters:**
-- `blobUrl` — URL to the captured data blob
-- `handlerPath` — Path to the handler module to execute
-- `requestInfo` — The original request info to replay
-
 ### `databaseAuditEnsureLogTable(sql): Promise<void>`
 
 Creates the `audit_log` table, its indexes, and a reusable PL/pgSQL trigger function (`audit_trigger_function`). Call once during schema initialization.
@@ -579,8 +396,6 @@ Returns all rows from the `audit_log` table, ordered by `performed_at` DESC.
 
 ## Environment Variables
 
-### Required for all setups (read by `finishRequest`)
-
 These must be set on your Netlify site. Your deploy script should resolve them from git and push them to the Netlify environment before deploying. `finishRequest` will throw an error if any are missing.
 
 | Variable | Description | How to resolve |
@@ -590,15 +405,8 @@ These must be set on your Netlify site. Your deploy script should resolve them f
 | `REPLAY_REPOSITORY_URL` | Git repository URL (no embedded credentials) | `git remote get-url origin` (strip tokens) |
 | `NETLIFY_RECORDER_SECRET` | Secret for access control (strongly recommended) | `openssl rand -base64 32` — store in Netlify site env vars |
 
-### Required for self-hosted recording (Option B)
-
-| Variable | Description |
-|---|---|
-| `RECORD_REPLAY_API_KEY` | Replay API key for uploading recordings |
-| `APP_REPOSITORY_URL` | Git repository URL for container cloning |
-
 ## How It Works
 
-1. **Capture phase** (`createRecordingRequestHandler` or `startRequest` / `finishRequest`): When a Netlify function handles a request, the recording layer patches `globalThis.fetch` and `process.env` with Proxies that record every outbound network call and environment variable read. When the handler completes, the originals are restored, the captured data is serialized to JSON, and sent to either the remote service (via `remoteCallbacks`) or your own storage (via custom callbacks).
+1. **Capture phase** (`createRecordingRequestHandler` or `startRequest` / `finishRequest`): When a Netlify function handles a request, the recording layer patches `globalThis.fetch` and `process.env` with Proxies that record every outbound network call and environment variable read. When the handler completes, the originals are restored, the captured data is serialized to JSON, and sent to the Netlify Recorder service via `remoteCallbacks`.
 
-2. **Recording phase**: The captured blob is sent to a recording container (either via the Netlify Recorder service or self-hosted). Inside the container, `createRequestRecording` downloads the blob, installs replay-mode interceptors that return the pre-recorded responses, and re-executes the handler under `replay-node`. Since replay-node records all execution, this produces a Replay recording of the exact same handler execution.
+2. **Recording phase**: The Netlify Recorder service dispatches the captured data to a recording container. Inside the container, the captured blob is downloaded, replay-mode interceptors return the pre-recorded responses instead of making real calls, and the original handler is re-executed under `replay-node`. Since replay-node records all execution, this produces a Replay recording of the exact same handler execution.

package/dist/index.d.ts

@@ -93,24 +93,66 @@ interface BlobData {
     handlerResponse?: HandlerResponse$1;
 }
 interface FinishRequestCallbacks {
+    /** Uploads serialized captured data and returns the blob URL. */
+    uploadBlob: (data: string) => Promise<string>;
     /**
-     * Stores the captured request data and returns the request ID.
+     * Stores request metadata in the database and returns the request ID.
      *
      * When `requestId` is provided (from `createRecordingRequestHandler`'s
      * `waitUntil` flow), the callback should use it as the row ID so the
-     * client-facing header and the stored record match. When omitted, the
-     * callback generates its own ID.
+     * client-facing header and the stored record match.  When omitted, the
+     * callback generates its own ID (backward-compatible).
      */
-    storeRequest: (data: {
-        blobData: string;
+    storeRequestData: (data: {
+        blobUrl: string;
         commitSha: string;
         branchName: string;
         repositoryUrl: string;
         handlerPath: string;
         /** Pre-generated request ID. Use as the row ID when provided. */
         requestId?: string;
+        /** Optional secret that restricts access to this request. */
+        secret?: string;
     }) => Promise<string>;
 }
+/**
+ * Infrastructure credentials required to start a recording container.
+ *
+ * The container is started on Fly.io via the `@replayio/app-building` package.
+ * It requires Infisical credentials (for secrets management inside the
+ * container) and a Fly.io token + app name.
+ *
+ * These must be set as environment variables on the Netlify site:
+ *   INFISICAL_CLIENT_ID, INFISICAL_CLIENT_SECRET,
+ *   INFISICAL_PROJECT_ID, INFISICAL_ENVIRONMENT,
+ *   FLY_API_TOKEN, FLY_APP_NAME
+ */
+interface ContainerInfraConfig {
+    infisicalClientId: string;
+    infisicalClientSecret: string;
+    infisicalProjectId: string;
+    infisicalEnvironment: string;
+    flyToken: string;
+    flyApp: string;
+}
+interface EnsureRecordingOptions {
+    repositoryUrl: string;
+    /** Infrastructure credentials for starting the recording container. */
+    infraConfig?: ContainerInfraConfig;
+    /** Webhook URL the container can POST log entries to (optional). */
+    webhookUrl?: string;
+    /** Looks up request metadata by ID. */
+    lookupRequest: (requestId: string) => Promise<{
+        blobUrl: string;
+        commitSha: string;
+        branchName: string;
+        handlerPath: string;
+    }>;
+    /** Updates the request status (and optionally recording ID) in the database. */
+    updateStatus: (requestId: string, status: string, recordingId?: string) => Promise<void>;
+    /** Optional callback for the caller to emit structured log entries. */
+    onLog?: (level: "info" | "warn" | "error", message: string) => Promise<void>;
+}
 
 /**
  * Called at the beginning of a Netlify handler execution.
@@ -170,16 +212,32 @@ interface FinishRequestOptions {
     repositoryUrl?: string;
     /**
      * Pre-generated request ID. When provided, this ID is passed to the
-     * `storeRequest` callback so the stored row matches the ID already
+     * `storeRequestData` callback so the stored row matches the ID already
      * sent to the client in the `X-Replay-Request-Id` header.
+     *
+     * Used by `createRecordingRequestHandler` in the `waitUntil` flow where
+     * the response is returned before `finishRequest` runs.
      */
     requestId?: string;
+    /**
+     * Optional secret string. When set, the stored request is only
+     * accessible via API calls that provide the same secret value.
+     */
+    secret?: string;
 }
 /**
  * Called at the end of the handler execution.
  * Restores original globals, serializes all captured data,
- * stores the request via the provided callback, and sets the
- * X-Replay-Request-Id header.
+ * uploads it as a JSON blob via the provided callback,
+ * stores the request metadata, and sets the X-Replay-Request-Id header.
+ *
+ * **Important:** The returned response includes the `X-Replay-Request-Id`
+ * header. You must send the returned response to the client — not the
+ * original response object you passed in.
+ *
+ * Logs a warning to `console.warn` when the total finishRequest time or
+ * individual callback steps exceed their thresholds, to help diagnose
+ * slow blob uploads or database writes.
  */
 declare function finishRequest(requestContext: RequestContext, callbacks: FinishRequestCallbacks, response: FullHandlerResponse, options?: FinishRequestOptions): Promise<FullHandlerResponse>;
 
@@ -196,15 +254,80 @@ interface CreateRecordingRequestHandlerOptions extends FinishRequestOptions {
  *
  * Automatically calls `startRequest` before the handler and `finishRequest`
  * after, capturing all outbound network calls and environment variable reads.
- * The captured data is stored via the provided callbacks.
+ * On error, interceptors are cleaned up and the error is re-thrown.
  *
  * **Response timing:** When the Netlify Functions v2 `context` object is
  * available (with `waitUntil`), the response is returned to the client
  * **immediately** with a pre-generated `X-Replay-Request-Id` header. The
- * data storage continues in the background via `context.waitUntil()`.
+ * blob upload and metadata storage continue in the background via
+ * `context.waitUntil()`. This avoids adding latency to the client response.
+ *
+ * When `context.waitUntil` is not available (v1 handlers or missing context),
+ * the wrapper falls back to awaiting `finishRequest` before returning.
+ *
+ * For v2 handlers the request body is read from a clone internally — your
+ * handler still receives the original request with an unconsumed body.
+ *
+ * @example v1 handler (NetlifyEvent)
+ * ```typescript
+ * import { createRecordingRequestHandler, remoteCallbacks } from "@replayio-app-building/netlify-recorder";
+ *
+ * const handler = createRecordingRequestHandler(
+ *   async (event) => {
+ *     const result = await myBusinessLogic(event.body);
+ *     return {
+ *       statusCode: 200,
+ *       headers: { "Content-Type": "application/json" },
+ *       body: JSON.stringify(result),
+ *     };
+ *   },
+ *   {
+ *     callbacks: remoteCallbacks("https://netlify-recorder-bm4wmw.netlify.app"),
+ *     handlerPath: "netlify/functions/my-handler",
+ *   }
+ * );
+ *
+ * export { handler };
+ * ```
+ *
+ * @example v2 handler (Web API Request)
+ * ```typescript
+ * import { createRecordingRequestHandler, remoteCallbacks } from "@replayio-app-building/netlify-recorder";
+ *
+ * export default createRecordingRequestHandler(
+ *   async (req) => {
+ *     // Body is still available — startRequest reads from a clone
+ *     const body = await (req as Request).json();
+ *     const result = await myBusinessLogic(body);
+ *     return {
+ *       statusCode: 200,
+ *       headers: { "Content-Type": "application/json" },
+ *       body: JSON.stringify(result),
+ *     };
+ *   },
+ *   {
+ *     callbacks: remoteCallbacks("https://netlify-recorder-bm4wmw.netlify.app"),
+ *     handlerPath: "netlify/functions/my-handler",
+ *   }
+ * );
+ * ```
  */
 declare function createRecordingRequestHandler(handler: (event: NetlifyEvent | NetlifyV2Request, context?: unknown) => Promise<HandlerResponse>, options: CreateRecordingRequestHandlerOptions): (event: NetlifyEvent | NetlifyV2Request, context?: unknown) => Promise<HandlerResponse>;
 
+/**
+ * Creates `FinishRequestCallbacks` that send captured data to a remote
+ * Netlify Recorder service. This removes the need for the consuming app
+ * to set up its own blob storage or database tables.
+ *
+ * The callbacks upload blob data and store request metadata via the
+ * service's `/api/store-request` endpoint, which handles UploadThing
+ * storage and database insertion.
+ *
+ * @param serviceUrl - Base URL of the Netlify Recorder service
+ *   (e.g. "https://netlify-recorder-bm4wmw.netlify.app")
+ */
+declare function remoteCallbacks(serviceUrl: string): FinishRequestCallbacks;
+
 /**
  * Redacts sensitive environment variable values from blob data.
  *
@@ -225,6 +348,28 @@ declare function createRecordingRequestHandler(handler: (event: NetlifyEvent | N
  */
 declare function redactBlobData(blobData: BlobData): BlobData;
 
+/**
+ * Called by a background function to convert a request ID into a Replay recording ID.
+ *
+ * The function:
+ *  1. Looks up request metadata (blob URL, commit, handler path).
+ *  2. Delegates to `spawnRecordingContainer` which starts a detached Fly.io
+ *     container, runs the recording script under replay-node, and uploads
+ *     the resulting recording.
+ *  3. Updates the request status with the recording ID.
+ *
+ * **Required infrastructure:** Infisical credentials and a Fly.io token/app.
+ * See `ContainerInfraConfig` in types.ts for details. When these are not
+ * configured the function fails with an actionable error message listing
+ * the missing environment variables.
+ */
+declare function ensureRequestRecording(requestId: string, options: EnsureRecordingOptions): Promise<string>;
+/**
+ * Reads infrastructure config from environment variables.
+ * Returns undefined if any required variable is missing.
+ */
+declare function readInfraConfigFromEnv(): ContainerInfraConfig | undefined;
+
 interface RecordingResult {
     /** Whether a response mismatch was detected between capture and replay. */
     responseMismatch: boolean;
@@ -249,49 +394,39 @@ interface RecordingResult {
  */
 declare function createRequestRecording(blobUrlOrData: string | BlobData, handlerPath: string, requestInfo: RequestInfo): Promise<RecordingResult>;
 
-type SqlFunction$1 = (...args: any[]) => Promise<any[]>;
-interface BackendRequest {
-    id: string;
-    blob_data: string;
-    handler_path: string;
-    commit_sha: string;
-    branch_name: string;
-    repository_url: string | null;
-    status: string;
-    recording_id: string | null;
-    error_message: string | null;
-    created_at: string;
-    updated_at: string;
-}
 /**
- * Creates the `backend_requests` table. Call during schema initialization.
- *
- * Each package client stores captured request data in its own database
- * using this table. The blob data (captured network calls, env reads, etc.)
- * is stored directly in the `blob_data` column rather than in external
- * blob storage.
+ * Options for spawning a recording container from a blob URL.
+ * This is the core building block — it knows nothing about request IDs or databases.
  */
-declare function backendRequestsEnsureTable(sql: SqlFunction$1): Promise<void>;
-declare function backendRequestsInsert(sql: SqlFunction$1, data: {
-    id?: string;
-    blobData: string;
+interface SpawnRecordingContainerOptions {
+    /** URL (or data: URI) of the captured request blob JSON. */
+    blobUrl: string;
+    /** Handler file path relative to the app root (e.g. "netlify/functions/generate-haiku"). */
     handlerPath: string;
+    /** Git commit SHA to check out inside the container. */
     commitSha: string;
+    /** Git branch to clone. */
     branchName: string;
-    repositoryUrl?: string | null;
-}): Promise<string>;
-declare function backendRequestsGet(sql: SqlFunction$1, id: string): Promise<BackendRequest | null>;
-declare function backendRequestsGetBlobData(sql: SqlFunction$1, id: string): Promise<string | null>;
-declare function backendRequestsList(sql: SqlFunction$1, filters?: {
-    status?: string;
-    limit?: number;
-}): Promise<BackendRequest[]>;
-declare function backendRequestsUpdateStatus(sql: SqlFunction$1, id: string, status: string, recordingId?: string, errorMessage?: string): Promise<void>;
+    /** Git repository URL for the app. */
+    repositoryUrl: string;
+    /** Infrastructure credentials for Fly.io + Infisical. */
+    infraConfig: ContainerInfraConfig;
+    /** Optional webhook URL the container can POST log events to. */
+    logWebhookUrl?: string;
+    /** Optional callback for structured log entries. */
+    onLog?: (level: "info" | "warn" | "error", message: string) => Promise<void>;
+}
 /**
- * Convenience helper: creates `FinishRequestCallbacks` that store
- * captured request data directly in the `backend_requests` table.
+ * Spawns a detached Fly.io container that:
+ *  1. Clones the app repo at the correct branch
+ *  2. Checks out the exact commit
+ *  3. Runs `scripts/create-request-recording.ts` under replay-node
+ *  4. Uploads the resulting recording
+ *  5. Outputs the recording ID
+ *
+ * Returns the recording ID on success, or throws on failure.
  */
-declare function databaseCallbacks(sql: SqlFunction$1): FinishRequestCallbacks;
+declare function spawnRecordingContainer(options: SpawnRecordingContainerOptions): Promise<string>;
 
 type SqlFunction = (...args: any[]) => Promise<any[]>;
 /**
@@ -316,4 +451,4 @@ declare function databaseAuditDumpLogTable(sql: SqlFunction): Promise<Record<str
 
 declare function getCurrentRequestId(): string | null;
 
-export { type BackendRequest, type BlobData, type CapturedData, type CreateRecordingRequestHandlerOptions, type EnvRead, type FinishRequestCallbacks, type FinishRequestOptions, type HandlerResponse$1 as HandlerResponse, type NetlifyEvent, type NetlifyV2Request, type NetworkCall, type RecordingResult, type RequestContext, type RequestInfo, backendRequestsEnsureTable, backendRequestsGet, backendRequestsGetBlobData, backendRequestsInsert, backendRequestsList, backendRequestsUpdateStatus, createRecordingRequestHandler, createRequestRecording, databaseAuditDumpLogTable, databaseAuditEnsureLogTable, databaseAuditMonitorTable, databaseCallbacks, finishRequest, getCurrentRequestId, redactBlobData, startRequest };
+export { type BlobData, type CapturedData, type ContainerInfraConfig, type CreateRecordingRequestHandlerOptions, type EnsureRecordingOptions, type EnvRead, type FinishRequestCallbacks, type FinishRequestOptions, type HandlerResponse$1 as HandlerResponse, type NetlifyEvent, type NetlifyV2Request, type NetworkCall, type RecordingResult, type RequestContext, type RequestInfo, type SpawnRecordingContainerOptions, createRecordingRequestHandler, createRequestRecording, databaseAuditDumpLogTable, databaseAuditEnsureLogTable, databaseAuditMonitorTable, ensureRequestRecording, finishRequest, getCurrentRequestId, readInfraConfigFromEnv, redactBlobData, remoteCallbacks, spawnRecordingContainer, startRequest };

package/dist/index.js

@@ -490,6 +490,7 @@ function redactBlobData(blobData) {
 
 // src/finishRequest.ts
 var SLOW_THRESHOLD_MS = 2e3;
+var SLOW_STEP_THRESHOLD_MS = 1e3;
 async function finishRequest(requestContext, callbacks, response, options) {
   const finishStart = Date.now();
   requestContext.cleanup();
@@ -534,20 +535,34 @@ async function finishRequest(requestContext, callbacks, response, options) {
   };
   const blobData = redactBlobData(rawBlobData);
   const blobContent = JSON.stringify(blobData);
+  const uploadStart = Date.now();
+  const blobUrl = await callbacks.uploadBlob(blobContent);
+  const uploadDuration = Date.now() - uploadStart;
+  if (uploadDuration > SLOW_STEP_THRESHOLD_MS) {
+    console.warn(
+      `netlify-recorder: uploadBlob took ${uploadDuration}ms (handler: ${handlerPath})`
+    );
+  }
   const storeStart = Date.now();
-  const storedRequestId = await callbacks.storeRequest({
-    blobData: blobContent,
+  const storedRequestId = await callbacks.storeRequestData({
+    blobUrl,
     commitSha,
     branchName,
     repositoryUrl,
     handlerPath,
-    requestId: options?.requestId
+    requestId: options?.requestId,
+    secret: options?.secret
   });
   const storeDuration = Date.now() - storeStart;
+  if (storeDuration > SLOW_STEP_THRESHOLD_MS) {
+    console.warn(
+      `netlify-recorder: storeRequestData took ${storeDuration}ms (handler: ${handlerPath})`
+    );
+  }
   const totalDuration = Date.now() - finishStart;
   if (totalDuration > SLOW_THRESHOLD_MS) {
     console.warn(
-      `netlify-recorder: finishRequest took ${totalDuration}ms total (store: ${storeDuration}ms, handler: ${handlerPath})`
+      `netlify-recorder: finishRequest took ${totalDuration}ms total (upload: ${uploadDuration}ms, store: ${storeDuration}ms, handler: ${handlerPath})`
     );
   }
   return {
@@ -603,6 +618,284 @@ function createRecordingRequestHandler(handler, options) {
   };
 }
 
+// src/remoteCallbacks.ts
+function remoteCallbacks(serviceUrl) {
+  const base = serviceUrl.replace(/\/+$/, "");
+  let pendingBlobData;
+  return {
+    uploadBlob: async (data) => {
+      pendingBlobData = data;
+      return "__pending__";
+    },
+    storeRequestData: async (metadata) => {
+      const blobData = pendingBlobData;
+      pendingBlobData = void 0;
+      if (!blobData) {
+        throw new Error(
+          "remoteCallbacks: uploadBlob must be called before storeRequestData"
+        );
+      }
+      const res = await fetch(
+        `${base}/api/store-request`,
+        {
+          method: "POST",
+          headers: { "Content-Type": "application/json" },
+          body: JSON.stringify({
+            blobData,
+            handlerPath: metadata.handlerPath,
+            commitSha: metadata.commitSha,
+            branchName: metadata.branchName,
+            repositoryUrl: metadata.repositoryUrl,
+            requestId: metadata.requestId,
+            secret: metadata.secret
+          })
+        }
+      );
+      if (!res.ok) {
+        const errBody = await res.text().catch(() => "(unreadable)");
+        throw new Error(
+          `Netlify Recorder store-request failed: ${res.status} ${errBody}`
+        );
+      }
+      const result = await res.json();
+      return result.requestId;
+    }
+  };
+}
+
+// src/spawnRecordingContainer.ts
+async function spawnRecordingContainer(options) {
+  const {
+    blobUrl,
+    handlerPath,
+    commitSha,
+    branchName,
+    repositoryUrl,
+    infraConfig,
+    logWebhookUrl,
+    onLog
+  } = options;
+  const emit = async (level, message) => {
+    if (onLog) {
+      try {
+        await onLog(level, message);
+      } catch {
+      }
+    }
+  };
+  await emit("info", "Logging in to Infisical");
+  const {
+    infisicalLogin,
+    startContainer,
+    FileContainerRegistry,
+    httpGet,
+    httpOptsFor
+  } = await import("@replayio/app-building");
+  const infisicalToken = await infisicalLogin(
+    infraConfig.infisicalClientId,
+    infraConfig.infisicalClientSecret
+  );
+  const infisicalConfig = {
+    token: infisicalToken,
+    projectId: infraConfig.infisicalProjectId,
+    environment: infraConfig.infisicalEnvironment
+  };
+  const registry = new FileContainerRegistry("/tmp/netlify-recorder-containers.json");
+  const initialPrompt = [
+    `IMPORTANT: Follow these steps EXACTLY. Run each command as shown. Print ALL output.`,
+    `Do NOT explore the codebase, read AGENTS.md, or deviate from these steps.`,
+    `Do NOT attempt to debug, fix, shim, or work around ANY errors. If a command fails,`,
+    `print the full error output and move on to the next step. Errors during handler replay`,
+    `(like "No more recorded network calls" or DB errors) are EXPECTED and harmless.`,
+    ``,
+    `=== Step 1: Install dependencies ===`,
+    `cd /repo/apps/netlify-recorder && npm install 2>&1`,
+    ``,
+    `=== Step 2: Checkout the exact commit ===`,
+    `git fetch origin ${commitSha} 2>&1 || git fetch --all 2>&1`,
+    `git checkout ${commitSha} 2>&1`,
+    ``,
+    `=== Step 3: Verify recording script exists ===`,
+    `ls -la /repo/apps/netlify-recorder/scripts/create-request-recording.ts`,
+    `If the file does NOT exist, print "ERROR: create-request-recording.ts not found" and STOP.`,
+    ``,
+    `=== Step 4: Pre-compile for replay-node (Node v16) ===`,
+    `replay-node is Node v16 \u2014 it cannot run TypeScript or use modern APIs directly.`,
+    `You MUST compile everything with esbuild first. Run these commands exactly:`,
+    ``,
+    `# Install undici for web API polyfills (fetch, Headers, Response):`,
+    `cd /repo/apps/netlify-recorder && npm install undici@5 2>&1`,
+    ``,
+    `# Create the polyfill loader:`,
+    `cat > /tmp/web-polyfill.cjs << 'POLYFILL'`,
+    `try {`,
+    `  var u = require("undici");`,
+    `  if (!globalThis.fetch) globalThis.fetch = u.fetch;`,
+    `  if (!globalThis.Headers) globalThis.Headers = u.Headers;`,
+    `  if (!globalThis.Response) globalThis.Response = u.Response;`,
+    `  if (!globalThis.Request) globalThis.Request = u.Request;`,
+    `} catch(e) { console.error("polyfill warning:", e.message); }`,
+    `POLYFILL`,
+    ``,
+    `# Compile the recording script (bundles all local TS dependencies):`,
+    `npx esbuild scripts/create-request-recording.ts \\`,
+    `  --bundle --platform=node --target=node16 --format=cjs \\`,
+    `  --outfile=/tmp/create-recording.cjs 2>&1`,
+    ``,
+    `# Compile the handler (bundles everything including node_modules):`,
+    `npx esbuild ${handlerPath}.ts \\`,
+    `  --bundle --platform=node --target=node16 --format=cjs \\`,
+    `  --outfile=/tmp/handler.cjs 2>&1`,
+    ``,
+    `=== Step 5: Run under replay-node ===`,
+    `This MUST use replay-node so the execution is recorded. Run exactly:`,
+    ``,
+    `cd /repo/apps/netlify-recorder && npx @replayio/node \\`,
+    `  -r /tmp/web-polyfill.cjs /tmp/create-recording.cjs \\`,
+    `  --blob-url '${blobUrl}' \\`,
+    `  --handler-path '/tmp/handler.cjs' 2>&1`,
+    ``,
+    `The output will show captured data being replayed. Errors like "No more recorded`,
+    `network calls" or "DATABASE_URL" errors are EXPECTED \u2014 they come from post-handler`,
+    `DB operations that were not in the original blob. Do NOT try to fix these.`,
+    ``,
+    `=== Step 6: Upload the recording ===`,
+    `exec-secrets RECORD_REPLAY_API_KEY -- npx replayio upload --all 2>&1`,
+    ``,
+    `Find the recording ID (UUID) in the upload output and print:`,
+    `  recording: <recording-id>`,
+    ``,
+    `Then output <DONE>.`
+  ].join("\n");
+  await emit("info", "Starting detached container on Fly.io");
+  const state = await startContainer(
+    {
+      infisical: infisicalConfig,
+      registry,
+      flyToken: infraConfig.flyToken,
+      flyApp: infraConfig.flyApp,
+      detached: true,
+      initialPrompt,
+      webhookUrl: logWebhookUrl
+    },
+    {
+      repoUrl: repositoryUrl,
+      cloneBranch: branchName
+    }
+  );
+  await emit("info", `Container started: ${state.containerName} at ${state.baseUrl}`);
+  const maxWaitMs = 10 * 60 * 1e3;
+  const pollIntervalMs = 1e4;
+  const deadline = Date.now() + maxWaitMs;
+  let containerDone = false;
+  while (Date.now() < deadline) {
+    try {
+      const status = await httpGet(`${state.baseUrl}/status`, httpOptsFor(state));
+      if (status?.state === "stopped" || status?.state === "stopping") {
+        containerDone = true;
+        break;
+      }
+    } catch {
+      containerDone = true;
+      break;
+    }
+    await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
+  }
+  if (!containerDone) {
+    await emit("warn", "Container did not finish within 10 minutes");
+  }
+  let recordingId = null;
+  try {
+    const logs = await httpGet(`${state.baseUrl}/logs?offset=0`, httpOptsFor(state));
+    if (typeof logs === "string") {
+      const match = logs.match(/recording[:\s]+([a-f0-9-]{36})/i);
+      if (match?.[1]) {
+        recordingId = match[1];
+      }
+    }
+  } catch {
+    await emit("warn", "Could not read container logs after exit");
+  }
+  if (!recordingId) {
+    await emit("error", "Container completed but no recording ID was found in output");
+    throw new Error("Recording creation failed: no recording ID returned from container");
+  }
+  await emit("info", `Container completed \u2014 recording ID: ${recordingId}`);
+  return recordingId;
+}
+
+// src/ensureRequestRecording.ts
+async function ensureRequestRecording(requestId, options) {
+  const { repositoryUrl, infraConfig, webhookUrl, lookupRequest, updateStatus, onLog } = options;
+  const emit = async (level, message) => {
+    if (onLog) {
+      try {
+        await onLog(level, message);
+      } catch {
+      }
+    }
+  };
+  await updateStatus(requestId, "processing");
+  try {
+    if (!infraConfig) {
+      const missing = getMissingInfraVars();
+      throw new Error(
+        `Container infrastructure not configured. Missing environment variables: ${missing.join(", ")}. These must be set on the Netlify site for recording creation to work.`
+      );
+    }
+    const requestData = await lookupRequest(requestId);
+    await emit("info", `Request data retrieved \u2014 handler: ${requestData.handlerPath}, branch: ${requestData.branchName}, commit: ${requestData.commitSha}`);
+    const recordingId = await spawnRecordingContainer({
+      blobUrl: requestData.blobUrl,
+      handlerPath: requestData.handlerPath,
+      commitSha: requestData.commitSha,
+      branchName: requestData.branchName,
+      repositoryUrl,
+      infraConfig,
+      logWebhookUrl: webhookUrl,
+      onLog
+    });
+    await emit("info", `Recording created successfully: ${recordingId}`);
+    await updateStatus(requestId, "recorded", recordingId);
+    return recordingId;
+  } catch (err) {
+    const message = err instanceof Error ? err.message : String(err);
+    await emit("error", `Recording creation failed: ${message}`);
+    await updateStatus(requestId, "failed");
+    throw err;
+  }
+}
+function getMissingInfraVars() {
+  const required = [
+    "INFISICAL_CLIENT_ID",
+    "INFISICAL_CLIENT_SECRET",
+    "INFISICAL_PROJECT_ID",
+    "INFISICAL_ENVIRONMENT",
+    "FLY_API_TOKEN",
+    "FLY_APP_NAME"
+  ];
+  return required.filter((name) => !process.env[name]);
+}
+function readInfraConfigFromEnv() {
+  const clientId = process.env.INFISICAL_CLIENT_ID;
+  const clientSecret = process.env.INFISICAL_CLIENT_SECRET;
+  const projectId = process.env.INFISICAL_PROJECT_ID;
+  const environment = process.env.INFISICAL_ENVIRONMENT;
+  const flyToken = process.env.FLY_API_TOKEN;
+  const flyApp = process.env.FLY_APP_NAME;
+  if (!clientId || !clientSecret || !projectId || !environment || !flyToken || !flyApp) {
+    return void 0;
+  }
+  return {
+    infisicalClientId: clientId,
+    infisicalClientSecret: clientSecret,
+    infisicalProjectId: projectId,
+    infisicalEnvironment: environment,
+    flyToken,
+    flyApp
+  };
+}
+
 // src/createRequestRecording.ts
 async function createRequestRecording(blobUrlOrData, handlerPath, requestInfo) {
   let blobData;
@@ -910,131 +1203,6 @@ async function createRequestRecording(blobUrlOrData, handlerPath, requestInfo) {
   return result;
 }
 
-// src/backendRequests.ts
-async function backendRequestsEnsureTable(sql) {
-  await sql`
-    CREATE TABLE IF NOT EXISTS backend_requests (
-      id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-      blob_data TEXT NOT NULL,
-      handler_path TEXT NOT NULL,
-      commit_sha TEXT NOT NULL,
-      branch_name TEXT NOT NULL DEFAULT 'main',
-      repository_url TEXT,
-      status TEXT NOT NULL DEFAULT 'captured'
-        CHECK (status IN ('captured', 'queued', 'processing', 'recorded', 'failed')),
-      recording_id TEXT,
-      error_message TEXT,
-      created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-      updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
-    )
-  `;
-  await sql`
-    CREATE INDEX IF NOT EXISTS idx_backend_requests_status ON backend_requests (status)
-  `;
-  await sql`
-    CREATE INDEX IF NOT EXISTS idx_backend_requests_created_at ON backend_requests (created_at DESC)
-  `;
-}
-async function backendRequestsInsert(sql, data) {
-  if (data.id) {
-    await sql`
-      INSERT INTO backend_requests (id, blob_data, handler_path, commit_sha, branch_name, repository_url)
-      VALUES (
-        ${data.id}::uuid,
-        ${data.blobData},
-        ${data.handlerPath},
-        ${data.commitSha},
-        ${data.branchName},
-        ${data.repositoryUrl ?? null}
-      )
-    `;
-    return data.id;
-  }
-  const rows = await sql`
-    INSERT INTO backend_requests (blob_data, handler_path, commit_sha, branch_name, repository_url)
-    VALUES (
-      ${data.blobData},
-      ${data.handlerPath},
-      ${data.commitSha},
-      ${data.branchName},
-      ${data.repositoryUrl ?? null}
-    )
-    RETURNING id
-  `;
-  return rows[0]?.id ?? "";
-}
-async function backendRequestsGet(sql, id) {
-  const rows = await sql`
-    SELECT id, blob_data, handler_path, commit_sha, branch_name, repository_url,
-           status, recording_id, error_message, created_at, updated_at
-    FROM backend_requests WHERE id = ${id}
-  `;
-  return rows[0] ?? null;
-}
-async function backendRequestsGetBlobData(sql, id) {
-  const rows = await sql`
-    SELECT blob_data FROM backend_requests WHERE id = ${id}
-  `;
-  return rows[0]?.blob_data ?? null;
-}
-async function backendRequestsList(sql, filters) {
-  const limit = filters?.limit ?? 50;
-  if (filters?.status) {
-    const rows2 = await sql`
-      SELECT id, blob_data, handler_path, commit_sha, branch_name, repository_url,
-             status, recording_id, error_message, created_at, updated_at
-      FROM backend_requests
-      WHERE status = ${filters.status}
-      ORDER BY created_at DESC
-      LIMIT ${limit}
-    `;
-    return rows2;
-  }
-  const rows = await sql`
-    SELECT id, blob_data, handler_path, commit_sha, branch_name, repository_url,
-           status, recording_id, error_message, created_at, updated_at
-    FROM backend_requests
-    ORDER BY created_at DESC
-    LIMIT ${limit}
-  `;
-  return rows;
-}
-async function backendRequestsUpdateStatus(sql, id, status, recordingId, errorMessage) {
-  if (recordingId) {
-    await sql`
-      UPDATE backend_requests
-      SET status = ${status}, recording_id = ${recordingId}, updated_at = NOW()
-      WHERE id = ${id}
-    `;
-  } else if (errorMessage) {
-    await sql`
-      UPDATE backend_requests
-      SET status = ${status}, error_message = ${errorMessage}, updated_at = NOW()
-      WHERE id = ${id}
-    `;
-  } else {
-    await sql`
-      UPDATE backend_requests
-      SET status = ${status}, updated_at = NOW()
-      WHERE id = ${id}
-    `;
-  }
-}
-function databaseCallbacks(sql) {
-  return {
-    storeRequest: async (data) => {
-      return backendRequestsInsert(sql, {
-        id: data.requestId,
-        blobData: data.blobData,
-        handlerPath: data.handlerPath,
-        commitSha: data.commitSha,
-        branchName: data.branchName,
-        repositoryUrl: data.repositoryUrl
-      });
-    }
-  };
-}
-
 // src/databaseAudit.ts
 async function databaseAuditEnsureLogTable(sql) {
   await sql`
@@ -1115,20 +1283,17 @@ async function databaseAuditDumpLogTable(sql) {
   return rows;
 }
 export {
-  backendRequestsEnsureTable,
-  backendRequestsGet,
-  backendRequestsGetBlobData,
-  backendRequestsInsert,
-  backendRequestsList,
-  backendRequestsUpdateStatus,
   createRecordingRequestHandler,
   createRequestRecording,
   databaseAuditDumpLogTable,
   databaseAuditEnsureLogTable,
   databaseAuditMonitorTable,
-  databaseCallbacks,
+  ensureRequestRecording,
   finishRequest,
   getCurrentRequestId,
+  readInfraConfigFromEnv,
   redactBlobData,
+  remoteCallbacks,
+  spawnRecordingContainer,
   startRequest
 };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@replayio-app-building/netlify-recorder",
-  "version": "0.16.0",
+  "version": "0.17.0",
   "description": "Capture and replay Netlify function executions as Replay recordings",
   "type": "module",
   "exports": {