@replayio-app-building/netlify-recorder 0.16.0 → 0.18.0

package/README.md

@@ -1,6 +1,6 @@
 # @replayio-app-building/netlify-recorder
 
-Capture and replay Netlify function executions as [Replay](https://replay.io) recordings. This package intercepts outbound network calls and environment variable reads during handler execution, stores the captured data as a blob, and can later reproduce the exact execution as a Replay recording for debugging and analysis.
+Capture and replay Netlify function executions as [Replay](https://replay.io) recordings. This package intercepts outbound network calls and environment variable reads during handler execution, stores the captured data in your app's own database, and can later reproduce the exact execution as a Replay recording for debugging and analysis.
 
 ## Installation
 
@@ -8,32 +8,31 @@ Capture and replay Netlify function executions as [Replay](https://replay.io) re
 npm install @replayio-app-building/netlify-recorder
 ```
 
-## Integration Options
+## Setup
 
-There are two ways to use this package:
+### 1. Create the backend_requests table
 
-- **Option A: Use the Netlify Recorder service (recommended)** — The service handles blob storage, request tracking, and recording creation. Your app just wraps its handlers and calls `remoteCallbacks()`. No database or container infrastructure needed.
+The package stores captured request data directly in your app's database. Call `backendRequestsEnsureTable` once during schema initialization to create the `backend_requests` table:
 
-- **Option B: Self-hosted** — You manage your own blob storage, database tables, and recording containers. Full control, but requires more setup.
-
----
+```typescript
+import { backendRequestsEnsureTable } from "@replayio-app-building/netlify-recorder";
 
-## Option A: Using the Netlify Recorder Service
+await backendRequestsEnsureTable(sql);
+```
 
-The Netlify Recorder app (`https://netlify-recorder-bm4wmw.netlify.app`) provides a hosted service that stores captured request data, manages a pool of recording containers, and creates Replay recordings on demand. Your app needs zero database or infrastructure setup.
+This creates a table with columns for the serialized blob data, git metadata (commit SHA, branch, repository URL), handler path, recording status, and timestamps.
 
-### 1. Set required environment variables
+### 2. Set required environment variables
 
-`finishRequest` needs to know which repository, branch, and commit your deployed code belongs to. Set these three environment variables on your Netlify site:
+`finishRequest` needs to know which repository, branch, and commit your deployed code belongs to. Set these environment variables on your Netlify site:
 
 | Variable | Description | How to set |
 |---|---|---|
 | `REPLAY_REPOSITORY_URL` | Your app's git repository URL (e.g. `https://github.com/org/repo.git`) | Set in your deploy script or Netlify site settings |
 | `COMMIT_SHA` | The git commit hash of the deployed code | Set in your deploy script via `git rev-parse HEAD` |
 | `BRANCH_NAME` | The git branch of the deployed code | Set in your deploy script via `git rev-parse --abbrev-ref HEAD` |
-| `NETLIFY_RECORDER_SECRET` | Secret string for access control — restricts who can view and act on your captured requests | Set in Netlify site environment variables or via `set-branch-secret` |
 
-The first three are **required** — `finishRequest` will throw an error if any are missing. `NETLIFY_RECORDER_SECRET` is strongly recommended to prevent other apps from accessing your captured request data. Your deploy script should resolve the git values and set them on the Netlify site before deploying. Example:
+All three are **required** — `finishRequest` will throw an error if any are missing. Your deploy script should resolve the git values and set them on the Netlify site before deploying. Example:
 
 ```typescript
 // In your deploy script:
@@ -45,19 +44,20 @@ const repositoryUrl = execSync("git remote get-url origin", { encoding: "utf-8"
 // Set these on your Netlify site via the Netlify API or CLI
 ```
 
-### 2. Wrap your Netlify function
+### 3. Wrap your Netlify function
 
-Use `createRecordingRequestHandler` with `remoteCallbacks()` to wrap your handler with automatic request capture. Set `secret` to restrict access to captured requests — only API calls providing the same secret can view or act on them.
+Use `createRecordingRequestHandler` with `databaseCallbacks(sql)` to wrap your handler with automatic request capture. The captured data is stored directly in the `backend_requests` table in your database.
 
 **v1 handler** (Netlify Functions v1 — `event` with `httpMethod`, `path`, etc.):
 
 ```typescript
 import {
   createRecordingRequestHandler,
-  remoteCallbacks,
+  databaseCallbacks,
 } from "@replayio-app-building/netlify-recorder";
+import { neon } from "@neondatabase/serverless";
 
-const RECORDER_URL = "https://netlify-recorder-bm4wmw.netlify.app";
+const sql = neon(process.env.DATABASE_URL!);
 
 const handler = createRecordingRequestHandler(
   async (event) => {
@@ -70,9 +70,8 @@ const handler = createRecordingRequestHandler(
     };
   },
   {
-    callbacks: remoteCallbacks(RECORDER_URL),
+    callbacks: databaseCallbacks(sql),
     handlerPath: "netlify/functions/my-handler",
-    secret: process.env.NETLIFY_RECORDER_SECRET,
   }
 );
 
@@ -84,10 +83,11 @@ export { handler };
 ```typescript
 import {
   createRecordingRequestHandler,
-  remoteCallbacks,
+  databaseCallbacks,
 } from "@replayio-app-building/netlify-recorder";
+import { neon } from "@neondatabase/serverless";
 
-const RECORDER_URL = "https://netlify-recorder-bm4wmw.netlify.app";
+const sql = neon(process.env.DATABASE_URL!);
 
 // The wrapper reads the body from a clone — you can still read req.json() etc.
 export default createRecordingRequestHandler(
@@ -102,37 +102,57 @@ export default createRecordingRequestHandler(
     };
   },
   {
-    callbacks: remoteCallbacks(RECORDER_URL),
+    callbacks: databaseCallbacks(sql),
     handlerPath: "netlify/functions/my-handler",
-    secret: process.env.NETLIFY_RECORDER_SECRET,
   }
 );
 ```
 
-`createRecordingRequestHandler` automatically captures all outbound network calls and environment variable reads during your handler's execution, then sends the captured data to the Netlify Recorder service. The response includes an `X-Replay-Request-Id` header with the request ID managed by the service.
+`createRecordingRequestHandler` automatically captures all outbound network calls and environment variable reads during your handler's execution, then stores the captured data in the `backend_requests` table. The response includes an `X-Replay-Request-Id` header with the ID of the stored request.
 
 > **Note:** Always use the response returned by the wrapper (or `finishRequest`), not your original response object. The wrapper adds the `X-Replay-Request-Id` header to the response it returns.
 
-### 3. Create recordings
+### 4. Create recordings via the Netlify Recorder service
+
+When you want to turn a captured request into a Replay recording, POST to the Netlify Recorder service's `create-recording` endpoint. You need to provide a `blobDataUrl` — a publicly accessible URL where the recording container can fetch the captured blob data.
 
-When you want to turn a captured request into a Replay recording, POST to the service's `create-recording` endpoint with the request ID. If the request was created with a secret, you must include it:
+If your app exposes the blob data via an endpoint (e.g. using `backendRequestsGetBlobData`), construct the URL and pass it:
 
 ```typescript
+const RECORDER_URL = "https://netlify-recorder-bm4wmw.netlify.app";
+const blobDataUrl = `https://your-app.netlify.app/api/get-backend-request-blob?requestId=${requestId}`;
+
 const response = await fetch(
   `${RECORDER_URL}/api/create-recording`,
   {
     method: "POST",
     headers: { "Content-Type": "application/json" },
     body: JSON.stringify({
-      requestId,
-      secret,
+      blobDataUrl,
+      handlerPath: "netlify/functions/my-handler",
+      commitSha: "abc123",
+      branchName: "main",
+      repositoryUrl: "https://github.com/org/repo.git", // optional
       webhookUrl: "https://your-app.netlify.app/api/on-recording-complete", // optional
     }),
   }
 );
+
+const { requestId: serviceRequestId } = await response.json();
 ```
 
-The service looks up the stored blob data, dispatches the work to a pool container, and creates the recording.
+The `create-recording` endpoint accepts:
+
+| Parameter | Required | Description |
+|---|---|---|
+| `blobDataUrl` | Yes | URL where the recording container can fetch the captured blob JSON |
+| `handlerPath` | Yes | Path to the handler file (e.g. `netlify/functions/my-handler`) |
+| `commitSha` | Yes | Git commit SHA of the deployed code |
+| `branchName` | Yes | Git branch of the deployed code |
+| `repositoryUrl` | No | Git repository URL for the container to clone |
+| `webhookUrl` | No | URL to POST the result to when the recording completes or fails |
+
+The service dispatches the work to a recording container, which fetches the blob data from `blobDataUrl`, replays the handler execution under `replay-node`, and produces a Replay recording.
 
 If `webhookUrl` is provided, the service will POST the result to that URL when the recording completes (or fails):
 
@@ -146,249 +166,32 @@ On failure:
 { "status": "failed", "error": "Error message" }
 ```
 
-### 4. Check recording status
-
-You can poll the recording status at any time. If the request was created with a secret, you must include it:
-
-```typescript
-const res = await fetch(
-  `${RECORDER_URL}/api/get-request?requestId=${requestId}&secret=${secret}`
-);
-const { status, recordingId } = await res.json();
-// status: "captured" | "processing" | "recorded" | "failed"
-// recordingId: string | null
-```
-
-Requests created with a secret return 403 if the secret is missing or incorrect.
-
-### 5. Access control with secrets
-
-You can restrict access to captured requests by setting a `secret` string. When a secret is set, the request is only accessible via API calls that provide the same secret value. This lets each app isolate its requests from other apps sharing the same Netlify Recorder service.
-
-#### Setting a secret
-
-Pass `secret` in the options when wrapping your handler:
-
-```typescript
-export default createRecordingRequestHandler(
-  async (req) => {
-    const result = await myBusinessLogic();
-    return { statusCode: 200, body: JSON.stringify(result) };
-  },
-  {
-    callbacks: remoteCallbacks(RECORDER_URL),
-    handlerPath: "netlify/functions/my-handler",
-    secret: process.env.NETLIFY_RECORDER_SECRET,
-  }
-);
-```
-
-#### Listing requests by secret
-
-Use the `requests` endpoint with a `secret` parameter to retrieve all requests associated with your secret, with optional filtering by status, handler path, and time range:
-
-```typescript
-const res = await fetch(
-  `${RECORDER_URL}/api/requests?secret=${secret}&status=recorded&handlerPath=netlify/functions/my-handler&after=2025-01-01T00:00:00Z&before=2025-12-31T23:59:59Z`
-);
-const { rows, total, page, limit } = await res.json();
-```
-
-Query parameters:
-
-| Parameter | Description |
-|---|---|
-| `secret` | **(required)** The secret string used when creating the requests |
-| `id` | Search by request ID prefix |
-| `status` | Filter by status: `captured`, `queued`, `processing`, `recorded`, `failed`, `all` |
-| `handlerPath` | Filter by handler path (exact match) |
-| `after` | Only include requests created at or after this ISO timestamp |
-| `before` | Only include requests created at or before this ISO timestamp |
-| `page` | Page number (default 1) |
-| `limit` | Page size (default 20, max 100) |
-
-Without a `secret` parameter, only requests created without a secret are returned.
-
-#### Checking request status with a secret
-
-When a request was created with a secret, you must include the secret when polling its status:
-
-```typescript
-const res = await fetch(
-  `${RECORDER_URL}/api/get-request?requestId=${requestId}&secret=${secret}`
-);
-```
-
-Requests created without a secret remain accessible without one (backward compatible).
-
-#### Managing your secret
-
-Store your secret as a `NETLIFY_RECORDER_SECRET` environment variable. To keep it secure:
-
-1. **Netlify site environment:** Add `NETLIFY_RECORDER_SECRET` in your Netlify site's environment variables (Site settings → Environment variables). This makes it available to all deployed functions.
-
-2. **Branch-level secret** (for CI/development): If you're using the Netlify Recorder agent infrastructure, store the secret as a branch secret so deploy scripts and background agents can access it:
-
-   ```bash
-   set-branch-secret NETLIFY_RECORDER_SECRET "your-secret-value"
-   ```
-
-3. **Local development:** Add `NETLIFY_RECORDER_SECRET` to your `.env` file (make sure `.env` is in `.gitignore`).
-
-Use a strong random string (e.g. `openssl rand -base64 32`) and rotate it if compromised. All requests created under the old secret remain accessible only with the old secret value — there is no migration mechanism, so plan rotations during low-traffic windows.
-
----
-
-## Option B: Self-Hosted
-
-If you need full control, you can manage your own blob storage, database, and recording containers. This requires more setup but gives you complete ownership of the data pipeline.
-
-### 1. Set required environment variables
-
-Same as Option A — you must set `REPLAY_REPOSITORY_URL`, `COMMIT_SHA`, and `BRANCH_NAME` on your Netlify site. See the table in Option A, Step 1 above.
-
-### 2. Wrap your Netlify function
-
-Use `createRecordingRequestHandler` with custom callbacks that write to your own storage and database:
-
-```typescript
-import { createRecordingRequestHandler } from "@replayio-app-building/netlify-recorder";
-
-const handler = createRecordingRequestHandler(
-  async (event) => {
-    const result = await myBusinessLogic();
-
-    return {
-      statusCode: 200,
-      headers: { "Content-Type": "application/json" },
-      body: JSON.stringify(result),
-    };
-  },
-  {
-    secret: process.env.NETLIFY_RECORDER_SECRET,
-    callbacks: {
-      uploadBlob: async (data) => {
-        // Upload the JSON string to your blob storage (S3, R2, etc.)
-        const res = await fetch("https://storage.example.com/upload", {
-          method: "PUT",
-          body: data,
-        });
-        const { url } = await res.json();
-        return url;
-      },
-      storeRequestData: async ({ blobUrl, commitSha, branchName, repositoryUrl, handlerPath, secret }) => {
-        const [row] = await sql`
-          INSERT INTO requests (blob_url, commit_sha, branch_name, repository_url, handler_path, secret, status)
-          VALUES (${blobUrl}, ${commitSha}, ${branchName}, ${repositoryUrl}, ${handlerPath}, ${secret}, 'captured')
-          RETURNING id
-        `;
-        return row.id;
-      },
-    },
-  }
-);
-
-export { handler };
-```
-
-### 3. Create a requests database table
-
-```sql
-CREATE TABLE IF NOT EXISTS requests (
-  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-  blob_url TEXT,
-  commit_sha TEXT,
-  branch_name TEXT,
-  repository_url TEXT,
-  handler_path TEXT,
-  secret TEXT,
-  recording_id TEXT,
-  status TEXT NOT NULL DEFAULT 'captured'
-    CHECK (status IN ('captured', 'processing', 'recorded', 'failed')),
-  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
-);
-
-CREATE INDEX IF NOT EXISTS idx_requests_secret ON requests (secret) WHERE secret IS NOT NULL;
-```
+### 5. Manage stored requests
 
-### 4. Create a background function to produce recordings
+Use the `backendRequests*` helpers to query and manage captured requests in your database:
 
 ```typescript
-import { ensureRequestRecording } from "@replayio-app-building/netlify-recorder";
-import type { Handler } from "@netlify/functions";
-
-const handler: Handler = async (event) => {
-  const { requestId } = JSON.parse(event.body ?? "{}");
-
-  const recordingId = await ensureRequestRecording(requestId, {
-    repositoryUrl: process.env.APP_REPOSITORY_URL!,
-    lookupRequest: async (id) => {
-      const [row] = await sql`
-        SELECT blob_url, commit_sha, branch_name, handler_path
-        FROM requests WHERE id = ${id}
-      `;
-      return {
-        blobUrl: row.blob_url,
-        commitSha: row.commit_sha,
-        branchName: row.branch_name ?? "main",
-        handlerPath: row.handler_path,
-      };
-    },
-    updateStatus: async (id, status, recordingId) => {
-      await sql`
-        UPDATE requests
-        SET status = ${status},
-            recording_id = ${recordingId ?? null},
-            updated_at = NOW()
-        WHERE id = ${id}
-      `;
-    },
-  });
-
-  return {
-    statusCode: 200,
-    body: JSON.stringify({ recordingId }),
-  };
-};
+import {
+  backendRequestsGet,
+  backendRequestsList,
+  backendRequestsUpdateStatus,
+} from "@replayio-app-building/netlify-recorder";
 
-export { handler };
-```
+// Get a single request by ID
+const request = await backendRequestsGet(sql, requestId);
+// request.status: "captured" | "queued" | "processing" | "recorded" | "failed"
+// request.recording_id: string | null
 
-### 5. Create a container script
+// List requests with optional filters
+const requests = await backendRequestsList(sql, { status: "captured", limit: 20 });
 
-This script runs inside the recording container under `replay-node`:
+// Update status after recording completes
+await backendRequestsUpdateStatus(sql, requestId, "recorded", recordingId);
 
-```typescript
-// scripts/create-request-recording.ts
-import { createRequestRecording } from "@replayio-app-building/netlify-recorder";
-
-const args = process.argv.slice(2);
-const blobUrl = args[args.indexOf("--blob-url") + 1]!;
-const handlerPath = args[args.indexOf("--handler-path") + 1]!;
-
-await createRequestRecording(blobUrl, handlerPath, {
-  method: "POST",
-  url: handlerPath,
-  headers: {},
-});
+// Update status on failure
+await backendRequestsUpdateStatus(sql, requestId, "failed", undefined, "Error message");
 ```
 
-### Required infrastructure
-
-Self-hosted recording requires these environment variables:
-
-| Variable | Description |
-|---|---|
-| `INFISICAL_CLIENT_ID` | Infisical service account client ID |
-| `INFISICAL_CLIENT_SECRET` | Infisical service account client secret |
-| `INFISICAL_PROJECT_ID` | Infisical project ID |
-| `INFISICAL_ENVIRONMENT` | Infisical environment (e.g. `production`) |
-| `FLY_API_TOKEN` | Fly.io API token for container management |
-| `FLY_APP_NAME` | Fly.io app name for container deployment |
-| `APP_REPOSITORY_URL` | Git repository URL for container cloning |
-| `RECORD_REPLAY_API_KEY` | Replay API key for recording upload |
-
 ---
 
 ## Audit Log Support
@@ -425,7 +228,7 @@ No special SQL wrapper is needed. Any Neon SQL query inside a handler wrapped wi
 ```typescript
 import {
   createRecordingRequestHandler,
-  remoteCallbacks,
+  databaseCallbacks,
 } from "@replayio-app-building/netlify-recorder";
 
 export default createRecordingRequestHandler(
@@ -435,9 +238,8 @@ export default createRecordingRequestHandler(
     return { statusCode: 200, body: "OK" };
   },
   {
-    callbacks: remoteCallbacks(RECORDER_URL),
+    callbacks: databaseCallbacks(sql),
     handlerPath: "netlify/functions/create-order",
-    secret: process.env.NETLIFY_RECORDER_SECRET,
   }
 );
 ```
@@ -478,22 +280,21 @@ The network interceptor detects Neon SQL HTTP requests (which use `fetch` intern
 
 Wraps a Netlify handler function with automatic request recording. This is the recommended way to integrate — it handles `startRequest`/`finishRequest` and error cleanup internally.
 
-**Response timing:** When the Netlify Functions v2 `context` object is available (with `waitUntil`), the response is returned to the client **immediately** with a pre-generated `X-Replay-Request-Id` header. The blob upload and metadata storage continue in the background via `context.waitUntil()`, adding zero latency to the client response.
+**Response timing:** When the Netlify Functions v2 `context` object is available (with `waitUntil`), the response is returned to the client **immediately** with a pre-generated `X-Replay-Request-Id` header. The data storage continues in the background via `context.waitUntil()`, adding zero latency to the client response.
 
 When `context.waitUntil` is not available (v1 handlers or missing context), the wrapper falls back to awaiting `finishRequest` before returning.
 
 **Parameters:**
 - `handler` — Your async handler function `(event, context?) => Promise<{ statusCode, headers?, body? }>`
-- `options.callbacks` — Either `remoteCallbacks(serviceUrl)` or custom `{ uploadBlob, storeRequestData }` callbacks
+- `options.callbacks` — `databaseCallbacks(sql)` to store captured data in the `backend_requests` table
 - `options.handlerPath` — Path to the handler file (used for recording metadata)
 - `options.commitSha` — Override `COMMIT_SHA` env var
 - `options.branchName` — Override `BRANCH_NAME` env var
 - `options.repositoryUrl` — Override `REPLAY_REPOSITORY_URL` env var
-- `options.secret` — Optional secret string. When set, the stored request is only accessible via API calls that provide the same secret value.
 
 **Returns:** A wrapped handler function with the same signature.
 
-**Callbacks note:** When using the `waitUntil` flow, `storeRequestData` receives a `requestId` field in its data parameter. Callbacks should use this as the row ID so the stored record matches the ID already sent to the client.
+**Callbacks note:** When using the `waitUntil` flow, `storeRequest` receives a `requestId` field in its data parameter. Callbacks should use this as the row ID so the stored record matches the ID already sent to the client.
 
 ### `startRequest(event): RequestContext`
 
@@ -510,51 +311,144 @@ For v2 Request inputs, the body is read from a **clone** — the original reques
 
 ### `finishRequest(requestContext, callbacks, response, options?): Promise<HandlerResponse>`
 
-Lower-level API. Finalizes the request capture. Restores original `fetch` and `process.env`, serializes the captured data, uploads it via the callbacks, and returns the response with `X-Replay-Request-Id` header set.
+Lower-level API. Finalizes the request capture. Restores original `fetch` and `process.env`, serializes the captured data, stores it via the callback, and returns the response with `X-Replay-Request-Id` header set.
 
 **Important:** You must send the returned response to the client — it contains the `X-Replay-Request-Id` header.
 
-Logs a `console.warn` when the total duration exceeds 2 seconds or when individual callback steps (upload, store) exceed 1 second, to help diagnose slow operations.
+Logs a `console.warn` when the total duration exceeds 2 seconds to help diagnose slow operations.
 
 **Requires** the following environment variables (or equivalent options overrides): `COMMIT_SHA`, `BRANCH_NAME`, `REPLAY_REPOSITORY_URL`. Throws if any are missing.
 
 **Parameters:**
 - `requestContext` — The context returned by `startRequest`
-- `callbacks` — Either `remoteCallbacks(serviceUrl)` or custom `{ uploadBlob, storeRequestData }` callbacks
+- `callbacks` — `databaseCallbacks(sql)` or a custom `{ storeRequest }` callback
 - `response` — The handler's response object (`{ statusCode, headers?, body? }`)
 - `options.handlerPath` — Path to the handler file (used for recording metadata)
 - `options.commitSha` — Override `COMMIT_SHA` env var
 - `options.branchName` — Override `BRANCH_NAME` env var
 - `options.repositoryUrl` — Override `REPLAY_REPOSITORY_URL` env var
 - `options.requestId` — Pre-generated request ID (used by `createRecordingRequestHandler` in the `waitUntil` flow)
-- `options.secret` — Optional secret string. When set, the stored request is only accessible via API calls that provide the same secret value.
 
-### `remoteCallbacks(serviceUrl): FinishRequestCallbacks`
+### `databaseCallbacks(sql): FinishRequestCallbacks`
+
+Creates a `FinishRequestCallbacks` object that stores captured request data directly in the `backend_requests` table. This is the standard way to provide callbacks to `createRecordingRequestHandler` or `finishRequest`.
+
+**Parameters:**
+- `sql` — A Neon SQL tagged-template function
+
+**Returns:** An object with a `storeRequest` method that inserts rows into `backend_requests`.
+
+### `backendRequestsEnsureTable(sql): Promise<void>`
+
+Creates the `backend_requests` table and its indexes. Call once during schema initialization.
+
+The table schema:
+
+| Column | Type | Description |
+|---|---|---|
+| `id` | UUID (PK) | Auto-generated request ID |
+| `blob_data` | TEXT | Serialized JSON of captured request data |
+| `handler_path` | TEXT | Path to the handler file |
+| `commit_sha` | TEXT | Git commit SHA |
+| `branch_name` | TEXT | Git branch name (default: `'main'`) |
+| `repository_url` | TEXT | Git repository URL (nullable) |
+| `status` | TEXT | `'captured'`, `'queued'`, `'processing'`, `'recorded'`, or `'failed'` |
+| `recording_id` | TEXT | Replay recording ID (set when status is `'recorded'`) |
+| `error_message` | TEXT | Error details (set when status is `'failed'`) |
+| `created_at` | TIMESTAMPTZ | Row creation time |
+| `updated_at` | TIMESTAMPTZ | Last update time |
+
+**Parameters:**
+- `sql` — A Neon SQL tagged-template function
+
+### `backendRequestsInsert(sql, data): Promise<string>`
+
+Inserts a new row into `backend_requests` and returns the generated (or provided) ID.
+
+**Parameters:**
+- `sql` — A Neon SQL tagged-template function
+- `data.blobData` — Serialized blob JSON string
+- `data.handlerPath` — Handler file path
+- `data.commitSha` — Git commit SHA
+- `data.branchName` — Git branch name
+- `data.repositoryUrl` — Git repository URL (optional)
+- `data.id` — Pre-generated UUID (optional; auto-generated if omitted)
+
+### `backendRequestsGet(sql, id): Promise<BackendRequest | null>`
+
+Retrieves a single request by ID, or `null` if not found.
+
+**Parameters:**
+- `sql` — A Neon SQL tagged-template function
+- `id` — The request UUID
+
+### `backendRequestsGetBlobData(sql, id): Promise<string | null>`
+
+Retrieves only the `blob_data` column for a request, or `null` if not found. Use this to serve blob data to the recording container without fetching the full row.
+
+**Parameters:**
+- `sql` — A Neon SQL tagged-template function
+- `id` — The request UUID
+
+### `backendRequestsList(sql, filters?): Promise<BackendRequest[]>`
 
-Creates callbacks for `finishRequest` that send captured data to the Netlify Recorder service. The service handles blob storage and request tracking — no local database needed.
+Lists requests ordered by `created_at` DESC, with optional filters.
 
 **Parameters:**
-- `serviceUrl` — Base URL of the Netlify Recorder service (e.g. `"https://netlify-recorder-bm4wmw.netlify.app"`)
+- `sql` — A Neon SQL tagged-template function
+- `filters.status` — Filter by status (e.g. `"captured"`, `"recorded"`)
+- `filters.limit` — Maximum rows to return (default: 50)
 
-### `ensureRequestRecording(requestId, options): Promise<string>`
+### `backendRequestsUpdateStatus(sql, id, status, recordingId?, errorMessage?): Promise<void>`
 
-Spawns a container via `@replayio/app-building` to create a Replay recording from captured request data. Returns the recording ID. Only needed for self-hosted setups (Option B).
+Updates the status of a request. Optionally sets `recording_id` (on success) or `error_message` (on failure).
 
 **Parameters:**
-- `requestId` — The request to create a recording for
-- `options.repositoryUrl` — Git repository URL for the container to clone
-- `options.lookupRequest(id)` — Fetches `{ blobUrl, commitSha, branchName, handlerPath }` from the database
-- `options.updateStatus(id, status, recordingId?)` — Updates the request status in the database
+- `sql` — A Neon SQL tagged-template function
+- `id` — The request UUID
+- `status` — New status string
+- `recordingId` — Replay recording ID (optional, for `"recorded"` status)
+- `errorMessage` — Error details (optional, for `"failed"` status)
 
-### `createRequestRecording(blobUrl, handlerPath, requestInfo): Promise<void>`
+### `createRequestRecording(blobUrlOrData, handlerPath, requestInfo): Promise<RecordingResult>`
 
-Called inside a container running under `replay-node`. Downloads the captured data blob, installs replay-mode interceptors (which return pre-recorded responses instead of making real calls), and executes the original handler so replay-node can record the execution. Only needed for self-hosted setups (Option B).
+Called inside a recording container running under `replay-node`. Downloads the captured data blob (or accepts pre-parsed `BlobData`), installs replay-mode interceptors that return pre-recorded responses instead of making real calls, and executes the original handler so `replay-node` can record the execution.
+
+Returns a `RecordingResult` with mismatch detection:
+
+| Field | Type | Description |
+|---|---|---|
+| `responseMismatch` | boolean | Whether the replay response differs from the captured response |
+| `mismatchDetails` | string? | Description of the mismatch |
+| `replayResponse` | HandlerResponse? | The response produced during replay |
+| `capturedResponse` | HandlerResponse? | The original captured response |
+| `unconsumedNetworkCalls` | boolean | Whether some recorded network calls were not replayed |
+| `unconsumedNetworkDetails` | string? | Details about unconsumed calls |
 
 **Parameters:**
-- `blobUrl` — URL to the captured data blob
+- `blobUrlOrData` — URL to the captured data blob, or pre-parsed `BlobData` object
 - `handlerPath` — Path to the handler module to execute
 - `requestInfo` — The original request info to replay
 
+### `getCurrentRequestId(): string | null`
+
+Returns the request ID for the currently executing handler, or `null` if no handler is active. Useful for correlating logs or database operations with the current request.
+
+### `redactBlobData(data): BlobData`
+
+Redacts sensitive environment variable values from captured blob data before storage. Applied automatically by `finishRequest` — you only need to call this directly if using the lower-level APIs.
+
+Redaction rules:
+- Allow-listed keys (standard system/runtime variables like `NODE_ENV`, `PATH`, `COMMIT_SHA`) are never redacted
+- Values that are `undefined` or 8 characters or shorter are kept as-is
+- All other values are replaced with `*` repeated to the same length
+- Redacted values are also scrubbed from all other string fields in the blob (request headers, network call bodies, etc.) to prevent leakage through embedded values
+
+**Parameters:**
+- `data` — A `BlobData` object
+
+**Returns:** A new `BlobData` object with sensitive values masked.
+
 ### `databaseAuditEnsureLogTable(sql): Promise<void>`
 
 Creates the `audit_log` table, its indexes, and a reusable PL/pgSQL trigger function (`audit_trigger_function`). Call once during schema initialization.
@@ -579,8 +473,6 @@ Returns all rows from the `audit_log` table, ordered by `performed_at` DESC.
 
 ## Environment Variables
 
-### Required for all setups (read by `finishRequest`)
-
 These must be set on your Netlify site. Your deploy script should resolve them from git and push them to the Netlify environment before deploying. `finishRequest` will throw an error if any are missing.
 
 | Variable | Description | How to resolve |
@@ -588,17 +480,9 @@ These must be set on your Netlify site. Your deploy script should resolve them f
 | `COMMIT_SHA` | Git commit hash of the deployed code | `git rev-parse HEAD` |
 | `BRANCH_NAME` | Git branch of the deployed code | `git rev-parse --abbrev-ref HEAD` |
 | `REPLAY_REPOSITORY_URL` | Git repository URL (no embedded credentials) | `git remote get-url origin` (strip tokens) |
-| `NETLIFY_RECORDER_SECRET` | Secret for access control (strongly recommended) | `openssl rand -base64 32` — store in Netlify site env vars |
-
-### Required for self-hosted recording (Option B)
-
-| Variable | Description |
-|---|---|
-| `RECORD_REPLAY_API_KEY` | Replay API key for uploading recordings |
-| `APP_REPOSITORY_URL` | Git repository URL for container cloning |
 
 ## How It Works
 
-1. **Capture phase** (`createRecordingRequestHandler` or `startRequest` / `finishRequest`): When a Netlify function handles a request, the recording layer patches `globalThis.fetch` and `process.env` with Proxies that record every outbound network call and environment variable read. When the handler completes, the originals are restored, the captured data is serialized to JSON, and sent to either the remote service (via `remoteCallbacks`) or your own storage (via custom callbacks).
+1. **Capture phase** (`createRecordingRequestHandler` or `startRequest` / `finishRequest`): When a Netlify function handles a request, the recording layer patches `globalThis.fetch` and `process.env` with Proxies that record every outbound network call and environment variable read. Sensitive environment variable values are automatically redacted. When the handler completes, the originals are restored, the captured data is serialized to JSON, and stored in the `backend_requests` table in your database via `databaseCallbacks`.
 
-2. **Recording phase**: The captured blob is sent to a recording container (either via the Netlify Recorder service or self-hosted). Inside the container, `createRequestRecording` downloads the blob, installs replay-mode interceptors that return the pre-recorded responses, and re-executes the handler under `replay-node`. Since replay-node records all execution, this produces a Replay recording of the exact same handler execution.
+2. **Recording phase**: To create a Replay recording, POST to the Netlify Recorder service's `create-recording` endpoint with a `blobDataUrl` pointing to the captured data. The service dispatches the work to a recording container, which fetches the blob data from the URL, installs replay-mode interceptors that return pre-recorded responses instead of making real calls, and re-executes the handler under `replay-node`. Since `replay-node` records all execution, this produces a Replay recording of the exact same handler execution. The recording result includes mismatch detection — the service compares the replay response against the originally captured response to flag any divergence.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@replayio-app-building/netlify-recorder",
-  "version": "0.16.0",
+  "version": "0.18.0",
   "description": "Capture and replay Netlify function executions as Replay recordings",
   "type": "module",
   "exports": {