@replayio-app-building/netlify-recorder 0.25.0 → 0.27.0

package/README.md

@@ -1,6 +1,6 @@
 # @replayio-app-building/netlify-recorder
 
-Capture and replay Netlify function executions as [Replay](https://replay.io) recordings. This package intercepts outbound network calls and environment variable reads during handler execution, stores the captured data in your app's own database, and can later reproduce the exact execution as a Replay recording for debugging and analysis.
+Capture and replay Netlify function executions as [Replay](https://replay.io) recordings. This package intercepts outbound network calls and environment variable reads during handler execution, uploads the captured data to UploadThing, stores the URL in your app's database, and can later reproduce the exact execution as a Replay recording for debugging and analysis.
 
 ## Installation
 
@@ -20,19 +20,20 @@ import { backendRequestsEnsureTable } from "@replayio-app-building/netlify-recor
 await backendRequestsEnsureTable(sql);
 ```
 
-This creates a table with columns for the serialized blob data, git metadata (commit SHA, branch, repository URL), handler path, recording status, and timestamps.
+This creates a table with columns for the blob data URL (UploadThing), git metadata (commit SHA, branch, repository URL), handler path, recording status, and timestamps.
 
 ### 2. Set required environment variables
 
-`finishRequest` needs to know which repository, branch, and commit your deployed code belongs to. Set these environment variables on your Netlify site:
+Set these environment variables on your Netlify site:
 
 | Variable | Description | How to set |
 |---|---|---|
 | `REPLAY_REPOSITORY_URL` | Your app's git repository URL (e.g. `https://github.com/org/repo.git`) | Set in your deploy script or Netlify site settings |
 | `COMMIT_SHA` | The git commit hash of the deployed code | Set in your deploy script via `git rev-parse HEAD` |
 | `BRANCH_NAME` | The git branch of the deployed code | Set in your deploy script via `git rev-parse --abbrev-ref HEAD` |
+| `UPLOADTHING_TOKEN` | UploadThing API token for blob data storage | Set in Netlify site settings |
 
-All three are **required** — `finishRequest` will throw an error if any are missing. Your deploy script should resolve the git values and set them on the Netlify site before deploying. Example:
+The first three are **required** by `finishRequest` (it will throw if missing). `UPLOADTHING_TOKEN` is required by `databaseCallbacks` to upload captured blob data. Your deploy script should resolve the git values and set them on the Netlify site before deploying. Example:
 
 ```typescript
 // In your deploy script:
@@ -46,7 +47,7 @@ const repositoryUrl = execSync("git remote get-url origin", { encoding: "utf-8"
 
 ### 3. Wrap your Netlify function
 
-Use `createRecordingRequestHandler` with `databaseCallbacks(sql)` to wrap your handler with automatic request capture. The captured data is stored directly in the `backend_requests` table in your database.
+Use `createRecordingRequestHandler` with `databaseCallbacks(sql)` to wrap your handler with automatic request capture. The captured data is uploaded to UploadThing and the URL is stored in the `backend_requests` table.
 
 **v1 handler** (Netlify Functions v1 — `event` with `httpMethod`, `path`, etc.):
 
@@ -108,13 +109,13 @@ export default createRecordingRequestHandler(
 );
 ```
 
-`createRecordingRequestHandler` automatically captures all outbound network calls and environment variable reads during your handler's execution, then stores the captured data in the `backend_requests` table. The response includes an `X-Replay-Request-Id` header with the ID of the stored request.
+`createRecordingRequestHandler` automatically captures all outbound network calls and environment variable reads during your handler's execution, uploads the captured data to UploadThing, and stores the URL in the `backend_requests` table. The response includes an `X-Replay-Request-Id` header with the ID of the stored request.
 
 > **Note:** Always use the response returned by the wrapper (or `finishRequest`), not your original response object. The wrapper adds the `X-Replay-Request-Id` header to the response it returns.
 
 ### 4. Create recordings via the Netlify Recorder service
 
-Use `ensureRequestRecording` to turn a captured request into a Replay recording. It checks the `backend_requests` table first — if a recording already exists, it returns the recording ID immediately without calling the service. Otherwise it dispatches the work to the Netlify Recorder service and updates the row status to `"queued"`.
+Use `ensureRequestRecording` to turn a captured request into a Replay recording. It checks the `backend_requests` table first — if a recording already exists, it returns the recording ID immediately without calling the service. Otherwise it passes the stored blob data URL to the Netlify Recorder service and updates the row status to `"queued"`.
 
 ```typescript
 import { ensureRequestRecording } from "@replayio-app-building/netlify-recorder";
@@ -137,16 +138,6 @@ The function is idempotent — calling it multiple times for the same request is
 - **`status: "queued"` or `"processing"`** — returns `null` without re-queuing
 - **`status: "captured"` or `"failed"`** — calls the service, updates status to `"queued"`, returns `null`
 
-The `blobDataUrl` must be a direct URL to the blob JSON (e.g. an UploadThing URL). The recording container fetches this URL directly — no callback to the recorder service is involved:
-
-```typescript
-const recordingId = await ensureRequestRecording(sql, requestId, {
-  recorderUrl: RECORDER_URL,
-  blobDataUrl: `https://utfs.io/f/${blobFileKey}`, // direct URL to stored blob data
-  webhookUrl: "https://your-app.netlify.app/api/on-recording-complete", // optional
-});
-```
-
 When `webhookUrl` is provided, the service POSTs the result when the recording completes:
 
 On success:
@@ -324,12 +315,14 @@ Logs a `console.warn` when the total duration exceeds 2 seconds to help diagnose
 
 ### `databaseCallbacks(sql): FinishRequestCallbacks`
 
-Creates a `FinishRequestCallbacks` object that stores captured request data directly in the `backend_requests` table. This is the standard way to provide callbacks to `createRecordingRequestHandler` or `finishRequest`.
+Creates a `FinishRequestCallbacks` object that uploads captured request data to UploadThing and stores the URL in the `backend_requests` table. This is the standard way to provide callbacks to `createRecordingRequestHandler` or `finishRequest`.
+
+Requires `UPLOADTHING_TOKEN` environment variable and the `uploadthing` package.
 
 **Parameters:**
 - `sql` — A Neon SQL tagged-template function
 
-**Returns:** An object with a `storeRequest` method that inserts rows into `backend_requests`.
+**Returns:** An object with a `storeRequest` method that uploads blob data and inserts rows into `backend_requests`.
 
 ### `backendRequestsEnsureTable(sql): Promise<void>`
 
@@ -340,7 +333,7 @@ The table schema:
 | Column | Type | Description |
 |---|---|---|
 | `id` | UUID (PK) | Auto-generated request ID |
-| `blob_data` | TEXT | Serialized JSON of captured request data |
+| `blob_data_url` | TEXT | URL to the uploaded blob JSON (UploadThing) |
 | `handler_path` | TEXT | Path to the handler file |
 | `commit_sha` | TEXT | Git commit SHA |
 | `branch_name` | TEXT | Git branch name (default: `'main'`) |
@@ -360,7 +353,7 @@ Inserts a new row into `backend_requests` and returns the generated (or provided
 
 **Parameters:**
 - `sql` — A Neon SQL tagged-template function
-- `data.blobData` — Serialized blob JSON string
+- `data.blobDataUrl` — URL to the uploaded blob JSON (e.g. UploadThing URL)
 - `data.handlerPath` — Handler file path
 - `data.commitSha` — Git commit SHA
 - `data.branchName` — Git branch name
@@ -375,9 +368,9 @@ Retrieves a single request by ID, or `null` if not found.
 - `sql` — A Neon SQL tagged-template function
 - `id` — The request UUID
 
-### `backendRequestsGetBlobData(sql, id): Promise<string | null>`
+### `backendRequestsGetBlobUrl(sql, id): Promise<string | null>`
 
-Retrieves only the `blob_data` column for a request, or `null` if not found. Use this to serve blob data to the recording container without fetching the full row.
+Retrieves only the `blob_data_url` column for a request, or `null` if not found.
 
 **Parameters:**
 - `sql` — A Neon SQL tagged-template function
@@ -411,7 +404,6 @@ Ensures a Replay recording exists (or is being created) for a backend request. L
 - `sql` — A Neon SQL tagged-template function
 - `requestId` — The backend request UUID
 - `options.recorderUrl` — Base URL of the Netlify Recorder service
-- `options.blobDataUrl` — Direct URL where the recording container fetches the blob JSON (e.g. an UploadThing URL)
 - `options.webhookUrl` — URL to POST the result to when the recording completes or fails
 
 **Returns:** The recording ID (`string`) if the request is already recorded, or `null` if the recording was queued or is in progress.

package/dist/index.d.ts

@@ -234,7 +234,7 @@ declare function createRequestRecording(blobUrlOrData: string | BlobData, handle
 type SqlFunction$1 = (...args: any[]) => Promise<any[]>;
 interface BackendRequest {
     id: string;
-    blob_data: string;
+    blob_data_url: string;
     handler_path: string;
     commit_sha: string;
     branch_name: string;
@@ -250,39 +250,35 @@ interface BackendRequest {
  *
  * Each package client stores captured request data in its own database
  * using this table. The blob data (captured network calls, env reads, etc.)
- * is stored directly in the `blob_data` column rather than in external
- * blob storage.
+ * is uploaded to UploadThing at capture time and only the URL is stored.
  */
 declare function backendRequestsEnsureTable(sql: SqlFunction$1): Promise<void>;
 declare function backendRequestsInsert(sql: SqlFunction$1, data: {
     id?: string;
-    blobData: string;
+    blobDataUrl: string;
     handlerPath: string;
     commitSha: string;
     branchName: string;
     repositoryUrl?: string | null;
 }): Promise<string>;
 declare function backendRequestsGet(sql: SqlFunction$1, id: string): Promise<BackendRequest | null>;
-declare function backendRequestsGetBlobData(sql: SqlFunction$1, id: string): Promise<string | null>;
+declare function backendRequestsGetBlobUrl(sql: SqlFunction$1, id: string): Promise<string | null>;
 declare function backendRequestsList(sql: SqlFunction$1, filters?: {
     status?: string;
     limit?: number;
 }): Promise<BackendRequest[]>;
 declare function backendRequestsUpdateStatus(sql: SqlFunction$1, id: string, status: string, recordingId?: string, errorMessage?: string): Promise<void>;
 /**
- * Convenience helper: creates `FinishRequestCallbacks` that store
- * captured request data directly in the `backend_requests` table.
+ * Convenience helper: creates `FinishRequestCallbacks` that upload
+ * captured request data to UploadThing and store the URL in the
+ * `backend_requests` table.
+ *
+ * Requires `UPLOADTHING_TOKEN` environment variable and the `uploadthing` package.
  */
 declare function databaseCallbacks(sql: SqlFunction$1): FinishRequestCallbacks;
 interface EnsureRequestRecordingOptions {
     /** Base URL of the Netlify Recorder service (e.g. "https://netlify-recorder-bm4wmw.netlify.app"). */
     recorderUrl: string;
-    /**
-     * Full URL where the recording container can fetch the blob JSON for this request.
-     * Must be a direct URL to the blob data (e.g. an UploadThing URL). The container
-     * fetches this URL directly — no callback to the recorder service is involved.
-     */
-    blobDataUrl: string;
     /** URL to POST the result to when the recording completes or fails. */
     webhookUrl?: string;
 }
@@ -292,8 +288,9 @@ interface EnsureRequestRecordingOptions {
  * 1. Looks up the request in `backend_requests`.
  * 2. If a recording already exists (`status === "recorded"`), returns the recording ID immediately.
  * 3. If the request is already queued or processing, returns `null` without re-queuing.
- * 4. Otherwise, calls the Netlify Recorder service's `create-recording` endpoint,
- *    updates the row to `"queued"`, and returns `null`.
+ * 4. Otherwise, calls the Netlify Recorder service's `create-recording` endpoint
+ *    with the blob data URL from the stored request, updates the row to `"queued"`,
+ *    and returns `null`.
  *
  * This function is idempotent — calling it multiple times for the same request
  * is safe. Once the recording completes, subsequent calls return the recording ID.
@@ -323,4 +320,4 @@ declare function databaseAuditDumpLogTable(sql: SqlFunction): Promise<Record<str
 
 declare function getCurrentRequestId(): string | null;
 
-export { type BackendRequest, type BlobData, type CapturedData, type CreateRecordingRequestHandlerOptions, type EnsureRequestRecordingOptions, type EnvRead, type FinishRequestCallbacks, type FinishRequestOptions, type HandlerResponse$1 as HandlerResponse, type NetlifyEvent, type NetlifyV2Request, type NetworkCall, type RecordingResult, type RequestContext, type RequestInfo, backendRequestsEnsureTable, backendRequestsGet, backendRequestsGetBlobData, backendRequestsInsert, backendRequestsList, backendRequestsUpdateStatus, createRecordingRequestHandler, createRequestRecording, databaseAuditDumpLogTable, databaseAuditEnsureLogTable, databaseAuditMonitorTable, databaseCallbacks, ensureRequestRecording, finishRequest, getCurrentRequestId, redactBlobData, startRequest };
+export { type BackendRequest, type BlobData, type CapturedData, type CreateRecordingRequestHandlerOptions, type EnsureRequestRecordingOptions, type EnvRead, type FinishRequestCallbacks, type FinishRequestOptions, type HandlerResponse$1 as HandlerResponse, type NetlifyEvent, type NetlifyV2Request, type NetworkCall, type RecordingResult, type RequestContext, type RequestInfo, backendRequestsEnsureTable, backendRequestsGet, backendRequestsGetBlobUrl, backendRequestsInsert, backendRequestsList, backendRequestsUpdateStatus, createRecordingRequestHandler, createRequestRecording, databaseAuditDumpLogTable, databaseAuditEnsureLogTable, databaseAuditMonitorTable, databaseCallbacks, ensureRequestRecording, finishRequest, getCurrentRequestId, redactBlobData, startRequest };

package/dist/index.js

@@ -563,10 +563,10 @@ async function finishRequest(requestContext, callbacks, response, options) {
 }
 
 // src/createRecordingRequestHandler.ts
-import crypto from "crypto";
+import crypto2 from "crypto";
 function createRecordingRequestHandler(handler, options) {
   return async (event, context) => {
-    const requestId = crypto.randomUUID();
+    const requestId = crypto2.randomUUID();
     setCurrentRequestId(requestId);
     const reqContext = startRequest(event);
     let response;
@@ -944,7 +944,7 @@ async function backendRequestsEnsureTable(sql) {
   await sql`
     CREATE TABLE IF NOT EXISTS backend_requests (
       id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-      blob_data TEXT NOT NULL,
+      blob_data_url TEXT NOT NULL,
       handler_path TEXT NOT NULL,
       commit_sha TEXT NOT NULL,
       branch_name TEXT NOT NULL DEFAULT 'main',
@@ -967,10 +967,10 @@ async function backendRequestsEnsureTable(sql) {
 async function backendRequestsInsert(sql, data) {
   if (data.id) {
     await sql`
-      INSERT INTO backend_requests (id, blob_data, handler_path, commit_sha, branch_name, repository_url)
+      INSERT INTO backend_requests (id, blob_data_url, handler_path, commit_sha, branch_name, repository_url)
       VALUES (
         ${data.id}::uuid,
-        ${data.blobData},
+        ${data.blobDataUrl},
         ${data.handlerPath},
         ${data.commitSha},
         ${data.branchName},
@@ -980,9 +980,9 @@ async function backendRequestsInsert(sql, data) {
     return data.id;
   }
   const rows = await sql`
-    INSERT INTO backend_requests (blob_data, handler_path, commit_sha, branch_name, repository_url)
+    INSERT INTO backend_requests (blob_data_url, handler_path, commit_sha, branch_name, repository_url)
     VALUES (
-      ${data.blobData},
+      ${data.blobDataUrl},
       ${data.handlerPath},
       ${data.commitSha},
       ${data.branchName},
@@ -994,23 +994,23 @@ async function backendRequestsInsert(sql, data) {
 }
 async function backendRequestsGet(sql, id) {
   const rows = await sql`
-    SELECT id, blob_data, handler_path, commit_sha, branch_name, repository_url,
+    SELECT id, blob_data_url, handler_path, commit_sha, branch_name, repository_url,
            status, recording_id, error_message, created_at, updated_at
     FROM backend_requests WHERE id = ${id}
   `;
   return rows[0] ?? null;
 }
-async function backendRequestsGetBlobData(sql, id) {
+async function backendRequestsGetBlobUrl(sql, id) {
   const rows = await sql`
-    SELECT blob_data FROM backend_requests WHERE id = ${id}
+    SELECT blob_data_url FROM backend_requests WHERE id = ${id}
   `;
-  return rows[0]?.blob_data ?? null;
+  return rows[0]?.blob_data_url ?? null;
 }
 async function backendRequestsList(sql, filters) {
   const limit = filters?.limit ?? 50;
   if (filters?.status) {
     const rows2 = await sql`
-      SELECT id, blob_data, handler_path, commit_sha, branch_name, repository_url,
+      SELECT id, blob_data_url, handler_path, commit_sha, branch_name, repository_url,
              status, recording_id, error_message, created_at, updated_at
       FROM backend_requests
       WHERE status = ${filters.status}
@@ -1020,7 +1020,7 @@ async function backendRequestsList(sql, filters) {
     return rows2;
   }
   const rows = await sql`
-    SELECT id, blob_data, handler_path, commit_sha, branch_name, repository_url,
+    SELECT id, blob_data_url, handler_path, commit_sha, branch_name, repository_url,
            status, recording_id, error_message, created_at, updated_at
     FROM backend_requests
     ORDER BY created_at DESC
@@ -1049,12 +1049,42 @@ async function backendRequestsUpdateStatus(sql, id, status, recordingId, errorMe
     `;
   }
 }
+async function uploadBlobData(blobData, requestId) {
+  let UTApi;
+  try {
+    ({ UTApi } = await Function('return import("uploadthing/server")')());
+  } catch {
+    throw new Error(
+      "netlify-recorder requires the 'uploadthing' package. Install it with: npm install uploadthing"
+    );
+  }
+  const token = process.env.UPLOADTHING_TOKEN;
+  if (!token) {
+    throw new Error(
+      "netlify-recorder: UPLOADTHING_TOKEN environment variable is required to upload blob data"
+    );
+  }
+  const utapi = new UTApi({ token });
+  const file = new File(
+    [blobData],
+    `blob-${requestId}.json`,
+    { type: "application/json" }
+  );
+  const result = await utapi.uploadFiles(file);
+  if (result.error || !result.data?.ufsUrl) {
+    throw new Error(
+      `Failed to upload blob data for request ${requestId}: ${JSON.stringify(result.error ?? "no URL returned")}`
+    );
+  }
+  return result.data.ufsUrl;
+}
 function databaseCallbacks(sql) {
   return {
     storeRequest: async (data) => {
+      const blobDataUrl = await uploadBlobData(data.blobData, data.requestId ?? crypto.randomUUID());
       return backendRequestsInsert(sql, {
         id: data.requestId,
-        blobData: data.blobData,
+        blobDataUrl,
         handlerPath: data.handlerPath,
         commitSha: data.commitSha,
         branchName: data.branchName,
@@ -1079,7 +1109,7 @@ async function ensureRequestRecording(sql, requestId, options) {
     method: "POST",
     headers: { "Content-Type": "application/json" },
     body: JSON.stringify({
-      blobDataUrl: options.blobDataUrl,
+      blobDataUrl: request.blob_data_url,
       handlerPath: request.handler_path,
       commitSha: request.commit_sha,
       branchName: request.branch_name,
@@ -1190,7 +1220,7 @@ async function databaseAuditDumpLogTable(sql) {
 export {
   backendRequestsEnsureTable,
   backendRequestsGet,
-  backendRequestsGetBlobData,
+  backendRequestsGetBlobUrl,
   backendRequestsInsert,
   backendRequestsList,
   backendRequestsUpdateStatus,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@replayio-app-building/netlify-recorder",
-  "version": "0.25.0",
+  "version": "0.27.0",
   "description": "Capture and replay Netlify function executions as Replay recordings",
   "type": "module",
   "exports": {
@@ -18,11 +18,15 @@
     "prepublishOnly": "tsup"
   },
   "peerDependencies": {
-    "@replayio/app-building": ">=1.0.0"
+    "@replayio/app-building": ">=1.0.0",
+    "uploadthing": ">=7.0.0"
   },
   "peerDependenciesMeta": {
     "@replayio/app-building": {
       "optional": true
+    },
+    "uploadthing": {
+      "optional": true
     }
   },
   "devDependencies": {