@expresscsv/sdk 0.1.21 → 0.1.23

package/README.md

@@ -3,7 +3,7 @@
 [![npm version](https://img.shields.io/npm/v/@expresscsv/sdk.svg)](https://www.npmjs.com/package/@expresscsv/sdk)
 ![license](https://img.shields.io/npm/l/@expresscsv/sdk.svg)
 
-A TypeScript SDK for embedding the [ExpressCSV](https://expresscsv.com) CSV import widget into any web application. Define a schema, open the widget, and receive validated, typed data in chunks.
+A TypeScript SDK for embedding the [ExpressCSV](https://expresscsv.com) CSV importer into any web application. Define a schema, open the importer, and receive validated, typed data in chunks.
 
 If you want to define schemas in shared or backend code without any frontend dependencies, use [`@expresscsv/schemas`](https://www.npmjs.com/package/@expresscsv/schemas) for the schema definition itself, then pass that schema into `@expresscsv/sdk`.
 
@@ -35,223 +35,93 @@ const schema = x.row({
 
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
   title: "Import Users",
 });
 
-// Open the widget and process data in chunks
+// Open the importer and process data in chunks
 importer.open({
   onData: (chunk, next) => {
     console.log(`Chunk ${chunk.currentChunkIndex + 1}/${chunk.totalChunks}`);
+    console.log("Session:", chunk.sessionId);
+    console.log("Idempotency key:", chunk.chunkIdempotencyKey);
     console.log("Records:", chunk.records);
     // Process your validated, typed records here
     next();
   },
-  onComplete: () => {
-    console.log("All chunks processed successfully");
+  onComplete: ({ sessionId }) => {
+    console.log("All chunks processed successfully for", sessionId);
   },
-  onError: (error) => {
-    console.error("Import failed:", error);
+  onError: (error, { sessionId }) => {
+    console.error("Import failed for", sessionId, error);
   },
 });
 ```
 
 If your schema is assembled dynamically at runtime, still use `x.row(...)`. Just note that dynamic schema assembly can widen TypeScript inference, so use it intentionally when you need runtime-driven columns.
 
-Your `publishableKey` is available from the [ExpressCSV dashboard](https://expresscsv.com). Two key types are available: **production** keys for live usage, and **dev/testing** keys that provide unlimited test imports.
+Have your backend keep the ExpressCSV secret key, call the ExpressCSV session-creation endpoint, and return a short-lived importer session token to the browser via `getSessionToken`. The SDK opens the importer immediately, then completes the initial session bootstrap in the background.
 
-## Webhook Delivery
+## Delivery
 
-Deliver imported data directly to your backend via webhook instead of (or in addition to) a local callback:
-
-```typescript
-importer.open({
-  webhook: {
-    url: "https://api.example.com/webhooks/csv-import",
-    method: "POST",
-    headers: {
-      Authorization: "Bearer your-api-token",
-    },
-    metadata: {
-      source: "web-app",
-      userId: "user-123",
-    },
-  },
-  onComplete: () => {
-    console.log("Webhook delivery initiated");
-  },
-  onError: (error) => {
-    console.error("Delivery error:", error);
-  },
-});
-```
-
-### Webhook Payload Structure
-
-Each chunk is delivered as a JSON `POST` (or whichever method you configured) to your endpoint. The request body has this shape:
-
-```typescript
-interface WebhookPayload {
-  /** Imported records for this chunk, matching your schema */
-  records: Record<string, unknown>[];
-  /** 0-based index of the current chunk */
-  chunkIndex: number;
-  /** Total number of chunks in this delivery */
-  totalChunks: number;
-  /** Total number of records across all chunks */
-  totalRecords: number;
-  /** Present only if you passed `metadata` in `WebhookConfig` */
-  metadata?: Record<string, unknown>;
-  /** Delivery context added by ExpressCSV */
-  delivery: {
-    publishableKey: string;
-    environmentName: string;
-    environmentType: string;
-    teamSlug: string;
-    importIdentifier: string;
-    deliveryId: string;
-    timestamp: string; // ISO 8601
-  };
-}
-```
-
-When you already have a schema, prefer `InferWebhookPayload<typeof schema>` from `@expresscsv/schemas` instead of rewriting this object shape by hand.
-
-Example payload:
-
-```json
-{
-  "records": [
-    { "name": "Alice Johnson", "email": "alice@example.com", "age": 32 },
-    { "name": "Bob Smith", "email": "bob@example.com", "age": 45 }
-  ],
-  "chunkIndex": 0,
-  "totalChunks": 3,
-  "totalRecords": 2500,
-  "metadata": {
-    "source": "web-app",
-    "userId": "user-123"
-  },
-  "delivery": {
-    "publishableKey": "pk_live_abc123",
-    "environmentName": "Production",
-    "environmentType": "production",
-    "teamSlug": "my-team",
-    "importIdentifier": "user-import",
-    "deliveryId": "del_abc123",
-    "timestamp": "2026-03-02T14:30:00.000Z"
-  }
-}
-```
-
-The request includes a `Content-Type: application/json` header plus any custom `headers` you specified in `WebhookConfig`.
-
-### Handling Webhooks on Your Server
-
-Your endpoint should return a **2xx** status code to acknowledge each chunk. Non-2xx responses trigger the following retry behaviour:
-
-- **5xx** and **429** responses are retried automatically (up to 5 attempts per chunk).
-- **4xx** responses (except 429) are treated as permanent failures and are **not** retried.
-
-Chunks are delivered **serially** — the next chunk is only sent after the previous one succeeds.
-
-If your schema is defined in shared or backend code with `@expresscsv/schemas`, you can reuse the built-in webhook payload helper when handling deliveries:
-
-```typescript
-import express from "express";
-import type { InferWebhookPayload } from "@expresscsv/schemas";
-import { employeeSchema } from "../shared/employee-schema";
-
-const app = express();
-app.use(express.json());
-
-app.post("/webhooks/csv-import", async (req, res) => {
-  const payload = req.body as InferWebhookPayload<typeof employeeSchema>;
-
-  await db.insertMany("users", payload.records);
-
-  res.status(200).json({ ok: true });
-});
-```
-
-Below is a minimal Express.js example:
-
-```typescript
-import express from "express";
-
-const app = express();
-app.use(express.json());
-
-app.post("/webhooks/csv-import", async (req, res) => {
-  const token = req.headers.authorization;
-  if (token !== "Bearer your-api-token") {
-    return res.status(401).json({ error: "Unauthorized" });
-  }
-
-  const { records, chunkIndex, totalChunks, totalRecords, metadata, delivery } =
-    req.body;
-
-  try {
-    // Process the records — e.g. insert into your database
-    await db.insertMany("users", records);
-
-    console.log(
-      `Chunk ${chunkIndex + 1}/${totalChunks} processed ` +
-        `(${records.length} records, delivery ${delivery.deliveryId})`
-    );
-
-    res.status(200).json({ success: true });
-  } catch (error) {
-    // Return 500 so ExpressCSV retries this chunk
-    console.error("Failed to process chunk:", error);
-    res.status(500).json({ error: "Internal server error" });
-  }
-});
-
-app.listen(3000);
-```
-
-**Tips:**
-
-- Use `delivery.deliveryId` and `chunkIndex` to **deduplicate** retried chunks (the same chunk may be delivered more than once on retry).
-- Use `chunkIndex` and `totalChunks` to track progress and know when the full import is complete (`chunkIndex === totalChunks - 1` for the last chunk).
-- Store `metadata` alongside imported records if you need to correlate the import with a specific user or action in your app.
-
-### Combined Local Callback and Webhook
-
-You can use both `onData` and `webhook` simultaneously:
+ExpressCSV delivers imported data through `onData`. Your app receives validated chunks in the browser and can forward them to your backend using whatever request shape fits your stack.
 
 ```typescript
 importer.open({
   chunkSize: 500,
   onData: async (chunk, next) => {
-    await saveToLocalDatabase(chunk.records);
+    const response = await fetch("/api/import-users/chunks", {
+      method: "POST",
+      headers: {
+        "Content-Type": "application/json",
+      },
+      body: JSON.stringify({
+        sessionId: chunk.sessionId,
+        chunkIdempotencyKey: chunk.chunkIdempotencyKey,
+        records: chunk.records,
+        currentChunkIndex: chunk.currentChunkIndex,
+        totalChunks: chunk.totalChunks,
+        totalRecords: chunk.totalRecords,
+      }),
+    });
+
+    if (!response.ok) {
+      throw new Error("Backend rejected this import chunk");
+    }
+
     next();
   },
-  webhook: {
-    url: "https://api.example.com/webhooks/csv-import",
-    headers: { Authorization: "Bearer your-api-token" },
-  },
-  onComplete: () => {
-    console.log("Local processing and webhook delivery complete");
+  onComplete: ({ sessionId }) => {
+    console.log("Import complete for", sessionId);
   },
 });
 ```
 
+Each delivered chunk includes:
+
+- `records`
+- `sessionId`
+- `chunkIdempotencyKey`
+- `currentChunkIndex`
+- `totalChunks`
+- `totalRecords`
+
+Use `sessionId` and `chunkIdempotencyKey` to stage writes safely and deduplicate retries from your own app logic.
+
 ## Preloading
 
-By default, the SDK preloads the widget in a hidden iframe for instant display when `open()` is called. This provides the best user experience.
+By default, the SDK preloads the importer in a hidden iframe for instant display when `open()` is called. This provides the best user experience while the initial session bootstrap continues in the background.
 
 ```typescript
 // Preload is enabled by default
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
 });
 
-// Later, the widget appears instantly
+// Later, the importer appears instantly
 importer.open({ onData: (chunk, next) => { /* ... */ next(); } });
 ```
 
@@ -260,7 +130,7 @@ To disable preloading (there will be a brief loading screen instead):
 ```typescript
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
   preload: false,
 });
@@ -284,7 +154,7 @@ const candidateSchema = x.row({
 
 const importer = new CSVImporter({
   schema: candidateSchema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "candidate-import",
   templateDownload: {
     source: "generate",
@@ -302,7 +172,7 @@ const importer = new CSVImporter({
 
 ## Theming and Styling
 
-Customize the widget's appearance with the `theme`, `colorMode`, `customCSS`, and `fonts` options.
+Customize the importer's appearance with the `theme`, `colorMode`, `customCSS`, and `fonts` options.
 
 ### Theme
 
@@ -340,7 +210,7 @@ const dualTheme: ECSVTheme = {
 
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
   theme,
 });
@@ -384,7 +254,7 @@ Control light/dark mode with `colorMode`:
 ```typescript
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
   colorMode: "system", // 'light' | 'dark' | 'system'
 });
@@ -397,7 +267,7 @@ Inject custom CSS for fine-grained styling overrides.
 ```typescript
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
   customCSS: `
     .ecsv [data-step="upload"] {
@@ -417,7 +287,7 @@ Load custom fonts via the `fonts` option:
 ```typescript
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
   fonts: {
     title: { source: "google", name: "Space Grotesk", weights: [400, 600, 700] },
@@ -434,7 +304,7 @@ const importer = new CSVImporter({
 
 The `x` schema builder provides a type-safe, fluent API for defining your CSV structure.
 
-For apps that share schema definitions with backend code or webhook handlers, prefer defining the schema in `@expresscsv/schemas` and importing it into your frontend. That keeps schema authoring free of widget/runtime dependencies.
+For apps that share schema definitions with backend code, prefer defining the schema in `@expresscsv/schemas` and importing it into your frontend. That keeps schema authoring free of importer/runtime dependencies.
 
 ### Field Types
 
@@ -539,7 +409,7 @@ All field types support:
 
 | Modifier | Description |
 |---|---|
-| `.label(text)` | User-facing label shown in the widget |
+| `.label(text)` | User-facing label shown in the importer |
 | `.description(text)` | Help text for the field |
 | `.example(text)` | Example value shown as placeholder |
 | `.optional()` | Makes the field optional (default is required) |
@@ -590,7 +460,7 @@ x.string().refine(
 )
 ```
 
-**Object-returning validator** — inline `valid`, `message`, and an optional `suggestedFix` the widget can offer the user:
+**Object-returning validator** — inline `valid`, `message`, and an optional `suggestedFix` the importer can offer the user:
 
 ```typescript
 x.string().refine((value) => ({
@@ -657,7 +527,7 @@ x.string().refineBatch((values) => {
 
 ### `suggestedFix`
 
-Both `.refine()` and `.refineBatch()` support returning a `suggestedFix` object that the widget offers as a one-click fix for the user:
+Both `.refine()` and `.refineBatch()` support returning a `suggestedFix` object that the importer offers as a one-click fix for the user:
 
 ```typescript
 interface SuggestedFix {
@@ -680,24 +550,27 @@ new CSVImporter(options: SDKOptions)
 | Option | Type | Required | Default | Description |
 |---|---|---|---|---|
 | `schema` | Schema | Yes | - | Schema definition created with `x.row()` |
-| `publishableKey` | `string` | Yes | - | Your publishable key from the [dashboard](https://expresscsv.com) |
+| `getSessionToken` | `() => Promise<string>` | Yes | - | Async callback that asks your backend for a short-lived importer session token |
 | `importIdentifier` | `string` | Yes | - | Unique identifier for this import type |
-| `title` | `string` | No | - | Title shown in the widget header |
-| `preload` | `boolean` | No | `true` | Preload widget for instant display |
+| `title` | `string` | No | - | Title shown in the importer header |
+| `preload` | `boolean` | No | `true` | Preload the importer for instant display |
 | `debug` | `boolean` | No | `false` | Enable debug logging |
 | `theme` | `ECSVTheme` | No | - | Custom theme configuration |
 | `colorMode` | `ColorModePref` | No | - | Light/dark mode (`'light'`, `'dark'`, or `'system'`) |
-| `customCSS` | `string` | No | - | Custom CSS to inject into the widget |
+| `customCSS` | `string` | No | - | Custom CSS to inject into the importer |
 | `fonts` | `Record<string, ECSVFontSource>` | No | - | Custom font sources |
 | `stepDisplay` | `'progressBar' \| 'segmented' \| 'numbered'` | No | `'progressBar'` | Step indicator style |
 | `previewSchemaBeforeUpload` | `boolean` | No | `true` | Show schema preview before upload |
+| `aiColumnMatching` | `boolean` | No | `true` | Enable AI-assisted column matching |
+| `aiTransform` | `boolean` | No | `true` | Enable AI-assisted transform generation |
 | `templateDownload` | `TemplateDownloadOptions<TSchema>` | No | - | Template download configuration with optional schema-typed example rows |
-| `saveSession` | `boolean` | No | - | Persist session state |
+| `storage` | `{ type: "local" } \| { type: "custom", ... }` | No | - | Enable Recovered Sessions with the built-in local backend or a custom adapter implementing `get`, `set`, and `remove` |
 | `locale` | `DeepPartial<ExpressCSVLocaleInput>` | No | - | Localization overrides |
+| `disableStatusStep` | `boolean` | No | - | Skip the success/error status screen |
 
 #### `open(options)`
 
-Opens the widget and begins the import flow. At least one of `onData` or `webhook` must be provided.
+Opens the importer and begins the import flow.
 
 ```typescript
 importer.open(options: OpenOptions): void
@@ -705,21 +578,18 @@ importer.open(options: OpenOptions): void
 
 | Option | Type | Required | Description |
 |---|---|---|---|
-| `onData` | `(chunk: RecordsChunk<T>, next: () => void) => void` | * | Callback for each chunk of records. Call `next()` to continue. |
-| `webhook` | `WebhookConfig` | * | Webhook endpoint for server-side delivery |
+| `onData` | `(chunk: RecordsChunk<T>, next: () => void) => void` | Yes | Callback for each delivered chunk of records. Call `next()` to continue. |
 | `chunkSize` | `number` | No | Records per chunk (default: 1000) |
-| `onComplete` | `() => void` | No | Called when all chunks have been processed |
-| `onCancel` | `() => void` | No | Called when the user cancels the import |
-| `onError` | `(error: Error) => void` | No | Called when an error occurs |
-| `onWidgetOpen` | `() => void` | No | Called when the widget opens |
-| `onWidgetClose` | `(reason: string) => void` | No | Called when the widget closes |
+| `onComplete` | `(context: { sessionId: string }) => void` | No | Called when all chunks have been processed |
+| `onCancel` | `(context: { sessionId: string }) => void` | No | Called when the user cancels the import |
+| `onError` | `(error: Error, context: { sessionId: string }) => void` | No | Called when an error occurs |
+| `onImporterOpen` | `() => void` | No | Called when the importer opens |
+| `onImporterClose` | `(reason: 'user_close' \| 'cancel' \| 'complete' \| 'error') => void` | No | Called when the importer closes |
 | `onStepChange` | `(stepId, previousStepId?) => void` | No | Called when the wizard step changes |
 
-\* At least one of `onData` or `webhook` is required.
-
 #### `close(reason?)`
 
-Closes the widget and cleans up resources.
+Closes the importer and cleans up resources.
 
 ```typescript
 await importer.close(reason?: 'user_close' | 'cancel' | 'complete' | 'error'): Promise<void>
@@ -731,18 +601,17 @@ await importer.close(reason?: 'user_close' | 'cancel' | 'complete' | 'error'): P
 
 | Method | Returns | Description |
 |---|---|---|
-| `getState()` | `WidgetState` | Current widget state |
-| `getIsReady()` | `boolean` | Whether the widget is ready or open |
-| `getIsOpen()` | `boolean` | Whether the widget is currently open |
+| `getState()` | `ImporterState` | Current importer lifecycle state |
+| `getIsReady()` | `boolean` | Whether the importer is ready or open |
+| `getIsOpen()` | `boolean` | Whether the importer is currently open |
 | `getConnectionStatus()` | `boolean` | Whether the iframe connection is active |
-| `getCanRestart()` | `boolean` | Whether the widget can be restarted |
+| `getCanRestart()` | `boolean` | Whether the importer can be restarted |
 | `getLastError()` | `Error \| null` | Last error, if any |
 | `getStatus()` | `object` | Comprehensive status snapshot |
-| `getVersion()` | `string` | SDK version |
 
 #### `restart(newOptions?)`
 
-Restarts the widget, optionally with updated options. Returns `Promise<void>`.
+Restarts the importer, optionally with updated options. Returns `Promise<void>`.
 
 ### `RecordsChunk<T>`
 
@@ -754,19 +623,8 @@ interface RecordsChunk<T> {
   totalChunks: number;
   currentChunkIndex: number;
   totalRecords: number;
-}
-```
-
-### `WebhookConfig`
-
-```typescript
-interface WebhookConfig {
-  url: string;
-  headers?: Record<string, string>;
-  method?: "POST" | "PUT" | "PATCH";
-  timeout?: number;
-  retries?: number;
-  metadata?: Record<string, unknown>;
+  sessionId: string;
+  chunkIdempotencyKey: string;
 }
 ```
 
@@ -788,7 +646,7 @@ type Row = Infer<typeof schema>;
 
 const importer = new CSVImporter({
   schema,
-  publishableKey: "your-publishable-key",
+  getSessionToken: async () => fetchSessionToken(),
   importIdentifier: "user-import",
 });