@updog/data-editor 0.1.3 → 0.1.5

package/README.md

@@ -1,11 +1,13 @@
 # @updog/data-editor
 
-Enterprise-grade spreadsheet editor SDK for React. Designed for editing large datasets (100k+ rows) with multi-source data merging, file imports, validation, and undo/redo.
+Client-side data importer and spreadsheet editor SDK for React. Your users import files, match columns to your schema, fix errors, and submit clean data. Edits happen inline, in the browser, at 1M+ rows.
+
+> **Not using React?** Use the framework-agnostic Web Component build — [`@updog/data-editor-wc`](https://www.npmjs.com/package/@updog/data-editor-wc) — for Angular, Vue, Svelte, or vanilla JS.
 
 ## Requirements
 
 - React 18.x or 19.x
-- An API key — this is a **commercial SDK**. Sign up at [updog.tech](https://updog.tech) to create an account and get a key. The editor will not render without one.
+- An API key — this is a **commercial SDK**. Sign up at [updog.tech](https://updog.tech) to get one.
 
 ## Installation
 
@@ -16,44 +18,71 @@ npm install @updog/data-editor
 ## Quick Start
 
 ```tsx
-import { DataEditor } from "@updog/data-editor";
+import { useState } from "react";
+import { DataEditor, required, email } from "@updog/data-editor";
 import "@updog/data-editor/styles.css";
+import type { DataEditorColumn } from "@updog/data-editor";
 
-function App() {
-  const [open, setOpen] = useState(false);
-
-  const columns = [
-    { id: "name", title: "Name", width: 150 },
-    { id: "email", title: "Email", width: 260 },
-  ];
+type Employee = {
+  id: string;
+  name: string;
+  email: string;
+  role: string;
+};
 
-  const loadData = async (onChunk) => {
-    const res = await fetch("/api/employees");
-    onChunk(await res.json());
-  };
+const columns: DataEditorColumn[] = [
+  { id: "name", title: "Full Name", size: 200, validate: required("Name is required") },
+  { id: "email", title: "Email", size: 250, validate: [required("Email is required"), email("Invalid email")] },
+  { id: "role", title: "Role", editor: { type: "select", options: ["Admin", "Editor", "Viewer"] } },
+];
 
-  const onComplete = (result) => {
-    console.log(result.valid, result.invalid, result.deleted);
-    setOpen(false);
-  };
+export function App() {
+  const [open, setOpen] = useState(false);
 
   return (
     <>
       <button onClick={() => setOpen(true)}>Open Editor</button>
-      <DataEditor
-        apiKey="your-api-key"
+      <DataEditor<Employee>
+        apiKey="your-license-key"
         open={open}
         onClose={() => setOpen(false)}
         columns={columns}
         primaryKey="id"
-        loadData={loadData}
-        onComplete={onComplete}
+        loadData={async (onChunk) => {
+          const res = await fetch("/api/employees");
+          onChunk(await res.json());
+        }}
+        onComplete={async (result, actions) => {
+          for (const source of result.sources) {
+            const inserts = source.rows.filter((r) => r.isNew && !r.isDeleted && r.isValid);
+            const updates = source.rows.filter((r) => !r.isNew && r.isChanged && !r.isDeleted && r.isValid);
+            const deletes = source.rows.filter((r) => r.isDeleted && !r.isNew);
+            await persist(source.sourceId, { inserts, updates, deletes });
+          }
+          actions.reset();
+          setOpen(false);
+        }}
       />
     </>
   );
 }
 ```
 
+## TypeScript
+
+`DataEditor` accepts a generic type parameter for row data. This gives you type-safe access to `primaryKey` and the result rows in `onComplete`.
+
+```tsx
+type Employee = { id: string; name: string; email: string };
+
+<DataEditor<Employee>
+  primaryKey="id"  // autocomplete shows "id" | "name" | "email"
+  // ...
+/>
+```
+
+When you omit the generic, rows are typed as `Record<string, unknown>`.
+
 ## Inline Mode
 
 Render the editor directly in the DOM without a modal overlay:
@@ -61,7 +90,7 @@ Render the editor directly in the DOM without a modal overlay:
 ```tsx
 <DataEditor
   mode="inline"
-  apiKey="your-api-key"
+  apiKey="your-license-key"
   columns={columns}
   primaryKey="id"
   loadData={loadData}
@@ -69,151 +98,296 @@ Render the editor directly in the DOM without a modal overlay:
 />
 ```
 
+In inline mode, `open` and `onClose` don't apply.
+
 ## Props
 
 | Prop | Type | Required | Default | Description |
 |---|---|---|---|---|
-| `mode` | `"modal"` \| `"inline"` | No | `"modal"` | Rendering mode |
-| `apiKey` | `string` | Yes | — | License key |
-| `open` | `boolean` | Modal only | — | Controls modal visibility. Required in modal mode only |
-| `onClose` | `() => void` | Modal only | — | Called when user initiates close. Required in modal mode only |
-| `columns` | `DataEditorColumn[]` | Yes | — | Column definitions |
-| `primaryKey` | `keyof TRow` | Yes | — | Unique row identifier column |
-| `loadData` | `(onChunk) => Promise<void>` | No | — | Async data loader with chunked delivery |
-| `onComplete` | `(result, actions) => void` | No | — | Called on submit with valid/invalid/deleted rows |
-| `variant` | `"editor"` \| `"uploader"` | No | `"editor"` | UI mode |
-| `translations` | `DataEditorTranslations` | No | — | i18n overrides |
-| `locale` | `string` | No | `"en"` | BCP 47 locale tag |
-| `enableDeleteRow` | `"all"` \| `"new"` \| `false` | No | `false` | Row deletion policy |
-| `enableAddRow` | `boolean` | No | `true` | Show "Add row" option |
-| `importFormats` | `DataEditorFormat[]` | No | all | Allowed import formats |
-| `exportFormats` | `DataEditorFormat[]` | No | all | Allowed export formats |
-| `rowHeight` | `number` | No | `34` | Row height in pixels |
-| `headerHeight` | `number` | No | `36` | Header height in pixels |
-| `readonly` | `boolean` | No | `false` | Hide write-oriented UI |
-| `rtl` | `boolean` | No | `false` | Right-to-left layout |
-| `className` | `string` | No | — | CSS class for the modal |
+| `apiKey` | `string` | Yes | — | Your Updog license key. Validated on each open. |
+| `columns` | `DataEditorColumn[]` | Yes | — | Column definitions. |
+| `primaryKey` | `keyof TRow` | Yes | — | Column ID that uniquely identifies each row. |
+| `mode` | `"modal"` \| `"inline"` | No | `"modal"` | Rendering mode. |
+| `open` | `boolean` | Modal only | — | Controlled modal visibility. |
+| `onClose` | `() => void` | Modal only | — | Called when the user closes the modal (X button or Escape). |
+| `loadData` | `(onChunk) => Promise<void>` | No | — | Async data loader. Stream rows in chunks, optionally tagged by source. |
+| `onComplete` | `(result, actions) => void` | No | — | Called on submit. See [Submission Result](#submission-result). |
+| `variant` | `"editor"` \| `"uploader"` | No | `"editor"` | Initial view. `"uploader"` opens the import wizard first. |
+| `translations` | `DataEditorTranslations` | No | — | Partial i18n overrides. |
+| `locale` | `string` | No | `"en"` | BCP 47 locale tag. |
+| `rtl` | `boolean` | No | `false` | Right-to-left layout. |
+| `readonly` | `boolean` | No | `false` | Hide all editing UI. |
+| `enableDeleteRow` | `"all"` \| `"new"` \| `false` | No | `false` | Row deletion policy. |
+| `enableAddRow` | `boolean` | No | `true` | Show the "Add row" button. |
+| `enableCreateColumn` | `boolean` | No | `true` | Allow creating columns for unmatched CSV headers during import. |
+| `importFormats` | `DataEditorFormat[]` | No | all | Allowed import formats. `[]` disables import. |
+| `exportFormats` | `DataEditorFormat[]` | No | all | Allowed export formats. `[]` disables export. |
+| `remoteSources` | `RemoteSource[]` | No | — | Custom import buttons (Google Sheets, S3, etc.) rendered on the upload step. |
+| `rowHeight` | `number` | No | `34` | Row height in pixels. |
+| `headerHeight` | `number` | No | `36` | Header height in pixels. |
+| `server` | `DataEditorServer<TRow>` | No | — | Server-delegated mode: SDK renders, your backend handles queries and mutations. |
+| `chat` | `DataEditorChat<TRow>` | No | — | Bring-your-own AI chat panel. |
+| `onColumnMatch` | `(headers, columns) => ...` | No | — | Override import column matching. |
+| `onValueMatch` | `(valuesToMatch) => ...` | No | — | Override import value matching for `select` columns. |
+| `synonyms` | `Record<string, string[]>` | No | — | Extra synonyms for column auto-matching. |
+| `sampleData` | `Record<string, unknown>[]` | No | — | Rows used in the "Download Example" file. |
+| `localStorage` | `false` \| `{ licenseGrant?: boolean }` | No | `{ licenseGrant: true }` | What the SDK caches in `localStorage`. |
+| `onError` | `(error: UpdogError) => void` | No | — | Called on internal errors. Use for logging or Sentry. |
+| `className` | `string` | No | — | CSS class on the wrapper element. |
 
 ## Column Configuration
 
 ```tsx
+import { required, email } from "@updog/data-editor";
+import type { DataEditorColumn } from "@updog/data-editor";
+
 const columns: DataEditorColumn[] = [
   {
     id: "email",
     title: "Email",
-    width: 260,
+    size: 260,
 
-    // Validation — single or array of validators
-    validate: (value, row) => {
-      if (!value) return { level: "error", message: "Required" };
-      if (!String(value).includes("@")) return { level: "warning", message: "Invalid email" };
-      return null;
-    },
+    // One validator, or an array. Each returns `{ level: "error", message }` or null.
+    validate: [required("Email is required"), email("Enter a valid email")],
 
-    // Uniqueness constraint
+    // Flag duplicates in this column as errors.
     unique: true,
 
-    // Cross-field validation — revalidate these columns when this one changes
+    // Revalidate these columns when this one changes.
     dependentFields: ["confirmEmail"],
 
-    // Cell editor: "text" (default), "date", or "select"
-    editor: { type: "select", options: [{ id: "us", text: "United States" }] },
+    // Cell editor. Default is "text".
+    editor: { type: "select", options: ["US", "UK", "DE"] },
 
-    // Visual formatting (display only, does not change data)
+    // Display-only formatting. Does not mutate stored data.
     formatter: (value) => value.toLowerCase(),
 
-    // Data transformation on input
+    // Runs when rows are uploaded to the editor. Mutates the stored value.
     transformer: (value) => String(value).trim(),
 
-    // Filter panel configuration
+    // Sidebar filter control.
     filter: { type: "select" },
+
+    // Lock cells in this column. `"default"` locks only default-source rows.
+    locked: "default",
   },
 ];
 ```
 
+**Cell editors**: `{ type: "text" }` (default), `{ type: "date", minDate?, maxDate? }`, `{ type: "select", options: string[] }`, `{ type: "number", decimalPlaces?, decimalSeparator?, thousandsSeparator?, allowChars? }`.
+
+**Column filters**: `{ type: "select", label?, placeholder?, options?, multiple? }`, `{ type: "number-range", label? }`, `{ type: "date-range", label? }`.
+
 ## Built-in Validators
 
+Validators are factory functions — call each one with the error message you want displayed.
+
 ```tsx
-import { useDataEditorValidators } from "@updog/data-editor";
-
-function App() {
-  const { required, numeric, email, date, endDateAfterStart } = useDataEditorValidators();
-
-  const columns = [
-    { id: "email", title: "Email", validate: [required, email] },
-    { id: "salary", title: "Salary", validate: numeric },
-    { id: "startDate", title: "Start", validate: date, editor: { type: "date" } },
-    {
-      id: "endDate",
-      title: "End",
-      validate: [date, endDateAfterStart("startDate")],
-      dependentFields: ["startDate"],
-      editor: { type: "date" },
-    },
-  ];
-}
+import {
+  required,
+  numeric,
+  email,
+  date,
+  oneOf,
+  endDateAfterStart,
+} from "@updog/data-editor";
+
+const columns = [
+  { id: "name", title: "Name", validate: required("Required") },
+  { id: "email", title: "Email", validate: [required("Required"), email("Invalid email")] },
+  { id: "salary", title: "Salary", validate: numeric("Must be a number") },
+  { id: "status", title: "Status", validate: oneOf(["active", "inactive"], "Invalid status") },
+  {
+    id: "startDate",
+    title: "Start",
+    validate: date("Use YYYY-MM-DD or DD/MM/YYYY"),
+    editor: { type: "date" },
+    dependentFields: ["endDate"],
+  },
+  {
+    id: "endDate",
+    title: "End",
+    validate: [date("Invalid date"), endDateAfterStart("startDate", "End must be on or after start")],
+    editor: { type: "date" },
+  },
+];
+```
+
+A `ValidationError` with `level: "error"` flags the cell in the grid but does **not** block submission — invalid rows are delivered to `onComplete` tagged via `isValid: false`.
+
+### Custom validators
+
+A validator is `(value, row) => ValidationError | null`. The only `level` is `"error"`.
+
+```tsx
+import type { CellValidator } from "@updog/data-editor";
+
+const mustContainAt: CellValidator = (value) => {
+  if (!value) return null;
+  if (!String(value).includes("@")) return { level: "error", message: "Must contain @" };
+  return null;
+};
 ```
 
 ## Data Loading
 
-Data is loaded in chunks to avoid blocking the UI:
+`loadData` is called once when the editor opens. Call `onChunk` one or more times to stream rows. The grid renders each chunk without blocking.
 
 ```tsx
-const loadData = async (onChunk) => {
-  // Load from API in pages
+// Single source
+loadData={async (onChunk) => {
   for (let page = 0; page < totalPages; page++) {
-    const res = await fetch(`/api/employees?page=${page}`);
-    const rows = await res.json();
-    onChunk(rows); // Each chunk is immediately rendered
+    const rows = await fetch(`/api/employees?page=${page}`).then((r) => r.json());
+    onChunk(rows);
   }
-};
+}}
+
+// Multiple sources — each chunk tagged with a source
+loadData={async (onChunk) => {
+  onChunk(await fetchSalesforce(), { source: "Salesforce", done: true });
+  onChunk(await fetchHubSpot(), { source: "HubSpot", deletable: true, done: true });
+}}
 ```
 
+`ChunkSourceOptions`: `{ source: string; id?: string; deletable?: boolean; done?: boolean }`. Sources are auto-registered on first encounter. Chunks without options go to "Existing Data". When `loadData` resolves, any source still loading is finalized automatically.
+
 ## Submission Result
 
-```tsx
-const onComplete = (result, actions) => {
-  // result.valid    — rows that pass all validation
-  // result.invalid  — rows with validation errors
-  // result.deleted  — rows marked for deletion
-  // result.newCount, result.editedCount, result.deletedCount
+`onComplete(result, actions)` fires when the user submits.
+
+```ts
+type DataEditorResult<TRow> = {
+  sources: {
+    sourceId: string;
+    sourceName: string;
+    rows: {
+      row: TRow;
+      isNew: boolean;      // imported or manually added this session
+      isChanged: boolean;  // cell values differ from origin
+      isDeleted: boolean;  // marked for deletion
+      isValid: boolean;    // passes all validators
+    }[];
+  }[];
+  counts: { new: number; changed: number; deleted: number; invalid: number };
+};
 
-  // actions.reset() — clear all changes and reload
+type DataEditorActions = {
+  reset: () => void; // discard changes and reload via loadData
 };
 ```
 
-## Import/Export Formats
+Pristine backend rows (unchanged, not deleted, not new) are omitted from `sources[].rows` — they're no-ops. The flags are orthogonal: a row can be new, changed, and deleted at once. You own the routing rules:
 
-Supported formats: `"csv"`, `"tsv"`, `"xlsx"`, `"json"`, `"xml"`.
+```tsx
+onComplete={async (result, actions) => {
+  for (const source of result.sources) {
+    const inserts = source.rows.filter((r) => r.isNew && !r.isDeleted && r.isValid);
+    const updates = source.rows.filter((r) => !r.isNew && r.isChanged && !r.isDeleted && r.isValid);
+    const deletes = source.rows.filter((r) => r.isDeleted && !r.isNew);
+    await persist(source.sourceId, { inserts, updates, deletes });
+  }
+  actions.reset();
+}}
+```
+
+If you never tagged sources via `loadData`, you'll get one entry with `sourceId: "backend"`. `result.counts` is an aggregate for summary UI — route with the per-row flags, not counts.
+
+## Import Configuration
+
+The import wizard guides users through uploading a file, mapping columns to your schema, and resolving value mismatches.
+
+### Formats
+
+Supported: `"csv"`, `"tsv"`, `"xlsx"`, `"json"`, `"xml"`.
 
 ```tsx
-// Only allow CSV and XLSX import, disable export
 <DataEditor
-  importFormats={["csv", "xlsx"]}
-  exportFormats={[]}
+  importFormats={["csv", "xlsx"]} // only CSV and XLSX
+  exportFormats={[]}              // disable export entirely
 />
 ```
 
+### Override column matching
+
+```tsx
+onColumnMatch={async (headers, columns) => {
+  const mappings = await myMatchingService.match(headers, columns);
+  return mappings; // { csvHeader: columnId | null }
+}}
+```
+
+Return a map of `{ csvHeader: columnId | null }`. Entries set to `null` or omitted fall back to built-in matching.
+
+### Override value matching
+
+For select columns, override imported-value → option mapping:
+
+```tsx
+onValueMatch={async (valuesToMatch) => {
+  // valuesToMatch = { country: { importedValues: ["espana", "fr"], options: ["Spain", "France"] } }
+  return {
+    country: { espana: "Spain", fr: "France" },
+  };
+}}
+```
+
+Values set to `null` skip auto-matching. Unmapped values fall back to built-in fuzzy matching.
+
+### Synonyms
+
+Extra aliases for the built-in column matcher:
+
+```tsx
+synonyms={{
+  productSku: ["sku", "article_no", "item_code"],
+  firstName: ["first", "given_name", "fname"],
+}}
+```
+
+### Remote sources
+
+Render custom import buttons for third-party sources. You own auth and picker logic; the SDK renders the button and processes the result.
+
+```tsx
+const googleSheets: RemoteSource = {
+  id: "google-sheets",
+  label: "Google Sheets",
+  icon: "<svg>...</svg>",
+  fetch: async () => {
+    const data = await myGoogleSheetsLib.pick();
+    return data.rows; // Record<string, unknown>[] — or return a File
+  },
+};
+
+<DataEditor remoteSources={[googleSheets]} {...rest} />
+```
+
+Return a `File` to parse via the standard CSV/XLSX/JSON/XML pipeline, or return structured records to skip parsing.
+
+## Error Handling
+
+Log SDK-internal errors through `onError`. The SDK recovers gracefully — use this for monitoring, not recovery.
+
+```tsx
+onError={(error) => {
+  Sentry.captureException(error.originalError ?? error, {
+    tags: { code: error.code, source: error.source },
+  });
+}}
+```
+
+Error codes: `PARSE_ERROR`, `RENDER_ERROR`, `TRANSFORM_ERROR`, `VALIDATION_ERROR`, `WORKER_ERROR`, `COMMAND_ERROR`, `OPERATION_ERROR`.
+
 ## Non-React Usage
 
-For Angular, Vue, or vanilla JS, use the Web Component package:
+For Angular, Vue, Svelte, or vanilla JS use the Web Component build:
 
 ```bash
 npm install @updog/data-editor-wc
 ```
 
-See [@updog/data-editor-wc](https://www.npmjs.com/package/@updog/data-editor-wc) for details.
-
-## Key Features
-
-- Virtualized grid — smooth scrolling with 100k+ rows
-- Multi-source data — backend data + multiple CSV/XLSX imports merged into one view
-- Smart CSV column matching — auto-maps imported columns with 90%+ accuracy
-- Undo/redo — Cmd+Z / Cmd+Shift+Z with batch support
-- Find & replace — search across all columns with match highlighting
-- Cell validation — custom validators, uniqueness constraints, cross-field dependencies
-- Dirty tracking — distinguishes new vs edited rows, detects value reverts
+See [@updog/data-editor-wc](https://www.npmjs.com/package/@updog/data-editor-wc).
 
 ## License
 
-Commercial — see [LICENSE](./LICENSE) for the full terms of the Updog SDK Commercial License v1.0. Contact `admin@updog.tech` for enterprise licensing questions. Third-party dependencies bundled with this package are listed in `THIRD_PARTY_NOTICES.txt`.
+Commercial — see [LICENSE](./LICENSE) for the full terms of the Updog SDK Commercial License v1.0. Contact `admin@updog.tech` for enterprise licensing. Third-party dependencies are listed in `THIRD_PARTY_NOTICES.txt`.