@permute/sdk 0.8.0 → 0.9.0

package/README.md

@@ -20,6 +20,10 @@ bunx @permute/sdk auth logout
 
 Choose an organization, optionally limit the connection to one workspace, then select resources and permissions to approve in Permute. The CLI stores renewable credentials in your operating system’s credential store (no plaintext fallback). Active logins have no fixed maximum lifetime and expire after 30 days without renewal; company sign-in rules can require earlier authentication. Logging out revokes access before removing the local credential. Signing in again revokes the previous stored CLI session.
 
+The SDK reuses credentials in memory between requests and reloads the OS store before renewal or when another process changes the saved login. The local auth directory holds only a lock and a non-secret change marker; tokens stay out of ordinary files. Current server permissions and revocation still apply to every request.
+
+The CLI prints the issuer and `permute-cli` entry name to stderr before accessing the OS store. macOS controls its own permission dialog, which may identify the Node or Bun process. SDK applications stay silent by default and can supply `onCredentialStoreAccess: (issuer) => { /* show a notice */ }` to `OAuthLogin`; the callback runs once before the default OS store is accessed and receives no credentials.
+
 ```ts
 import { OAuthLogin } from '@permute/sdk/auth';
 
@@ -321,10 +325,8 @@ const dataset = await client.datasets.create({
   context: [Context.QUICKBOOKS],
 });
 
-await dataset.replace({
-  orders: [{ id: 1 }, { id: 2 }],
-  customers: [{ id: 10 }],
-});
+await dataset.replaceTable('orders', [{ id: 1 }, { id: 2 }]);
+await dataset.replaceTable('customers', [{ id: 10 }]);
 
 console.log(dataset.sourceId, dataset.version, dataset.data.tables);
 ```
@@ -333,15 +335,30 @@ Dataset `context` accepts string identifiers, with `Context` providing named
 constants for known contexts (such as `Context.QUICKBOOKS`). It is not restricted
 to the connector provider list. Use `Providers` when selecting a connector.
 
-`replace()` sends the complete dataset snapshot. Tables omitted from the call
-are removed, while tables with the same name keep their stable identity for
-reports and dashboards. Retrying an identical replacement is a no-op.
+### Add or replace one table
 
-Use `replaceTable()` when the supplied rows are the complete current contents
-of one table. Existing rows not supplied are removed; other tables are unchanged:
+Use `replaceTable()` to add a new named table or replace the complete contents
+of an existing table. Existing rows not supplied are removed; other tables stay
+in the dataset. Await each write before starting the next write to that dataset:
 
 ```ts
 await dataset.replaceTable('orders', allOrders);
+await dataset.replaceTable('customers', allCustomers);
+// Both tables now belong to the same dataset.
+```
+
+### Replace the complete snapshot or append new rows
+
+`replace()` sends the complete dataset snapshot. Tables omitted from the call
+are removed, while tables with the same name keep their stable identity for
+reports and dashboards. Retrying an identical replacement is a no-op:
+
+```ts
+await dataset.replace({
+  orders: [{ id: 1 }, { id: 2 }],
+  customers: [{ id: 10 }],
+});
+// Only orders and customers remain in this dataset.
 ```
 
 Use `appendTable()` only when every supplied row is new and previously written
@@ -359,14 +376,68 @@ and use a new value for a new append. It prevents a retried request from being
 applied twice; reusing it with different rows fails. It does not remove duplicate
 rows sent in different batches.
 
-When source records can change or disappear, collect the source's current rows
-and use `replaceTable()` instead. The SDK serializes custom writes as chunked
-NDJSON, uploads them directly to a signed S3 target, and waits for processing;
-it does not build Parquet locally. The REST upload supports up to 1,000 tables and
-1 GB, but the current SDK object helper encodes the supplied rows in memory, so
-callers should stay comfortably within their process memory. `appendTable()`
-accepts up to 10,000 rows per batch. Durable SDK operations stop polling after
-20 minutes by default; pass `pollTimeoutMs` to set a different bound.
+### Upload a local CSV
+
+With SDK 0.9.0 or later, pass a disk-backed Blob to the same `replaceTable()` method.
+Use Node.js 24 or later with `openAsBlob`; the SDK hashes the file in small blocks,
+uploads it to S3, and waits for processing without loading the whole file into
+memory:
+
+```ts
+import { openAsBlob } from 'node:fs';
+
+await dataset.replaceTable('customers', await openAsBlob('./customers.csv'), {
+  format: 'csv',
+});
+```
+
+Supply comma-separated UTF-8 CSV with unique, nonempty column names in its first row.
+Permute derives the row count and infers column types; an optional `rowCount`
+checks an expected count. Keep the local file unchanged until its upload finishes.
+Header-only CSV can create an empty table.
+
+### Upload a gzipped CSV
+
+Pass an existing CSV.gz file with `compression: 'gzip'`. The SDK sends the file
+as-is (it does not gzip a plain CSV when you set the option). Unpack ZIP archives
+before importing individual table files:
+
+```ts
+await dataset.replaceTable('order_history', await openAsBlob('./order_history.csv.gz'), {
+  format: 'csv',
+  compression: 'gzip',
+});
+```
+
+### Import a large export
+
+The row-array overload still serializes objects as NDJSON and holds both the
+rows and encoded upload in memory. Use the Blob overload for large CSV exports.
+
+Each uploaded file can be at most 5 GiB (5,368,709,120 bytes), with at most 10 GiB
+(10,737,418,240 bytes) after decompression. The upload limit measures compressed
+bytes when using gzip. Raw REST NDJSON writes also accept `compression: 'gzip'`;
+`sizeBytes` and `contentHash` describe the exact uploaded bytes. Each complete
+table replacement can contain at most 10,000,000 rows. A full `replace()` accepts
+up to 1,000 tables within that same upload limit. The byte limit is per write,
+so separate table writes can accumulate several GB in one dataset. Table-by-table
+imports become visible as each write completes; they are not an atomic snapshot
+of the whole export. New empty NDJSON tables require an existing schema.
+
+For CSV exports with hundreds of tables, await each `replaceTable()` call while
+reusing the same dataset key. Use a different key for each separate source export.
+The guide includes [folder imports](https://permute.ai/docs/guides/custom-dataset#folder-of-tables),
+[multiple datasets](https://permute.ai/docs/guides/custom-dataset#multiple-datasets), and
+[signed REST uploads](https://permute.ai/docs/guides/custom-dataset#csv-export-to-s3).
+Validate imported column names, counts, and types before loading the full export.
+These limits apply to custom table writes; `type: 'file'` creates a separate file
+dataset with its own upload limits.
+
+`appendTable()` accepts up to 10,000 rows per batch. Repeated small appends are
+inefficient for a large snapshot; upload its complete CSV with `replaceTable()`.
+Durable SDK operations stop polling after 20 minutes by default; pass
+`pollTimeoutMs` to change the client wait. A polling timeout does not prove the
+server write failed; inspect its status before retrying.
 
 ## Recommended Flow
 

package/dist/auth/oauth.d.ts

@@ -8,6 +8,8 @@ export interface OAuthOptions {
     apiUrl?: string;
     store?: CredentialStore;
     fetch?: typeof fetch;
+    /** Called once before the default OS credential store is accessed; no tokens are passed. */
+    onCredentialStoreAccess?: (issuer: string) => void;
 }
 export declare class OAuthLogin {
     readonly issuer: string;

package/dist/auth/oauth.js

@@ -2,7 +2,7 @@ import { z } from 'zod';
 import { createHash, randomBytes } from 'node:crypto';
 import { setTimeout } from 'node:timers/promises';
 import { PermuteClient } from '../client.js';
-import { SystemCredentialStore } from './store.js';
+import { TOKEN_REFRESH_BUFFER_MS, SystemCredentialStore, } from './store.js';
 export { SystemCredentialStore } from './store.js';
 export const DEFAULT_ISSUER = 'https://auth.permute.ai';
 const CLIENT_ID = 'permute-cli';
@@ -30,7 +30,7 @@ export class OAuthLogin {
     constructor(options = {}) {
         this.issuer = this.endpoint(options.issuer ?? DEFAULT_ISSUER);
         this.apiUrl = this.endpoint(options.apiUrl ?? 'https://api.permute.ai');
-        this.store = options.store ?? new SystemCredentialStore(this.issuer);
+        this.store = options.store ?? new SystemCredentialStore(this.issuer, options.onCredentialStoreAccess);
         this.fetchImpl = options.fetch ?? fetch;
     }
     endpoint(value) {
@@ -65,7 +65,7 @@ export class OAuthLogin {
             throw new Error('Invalid credential response');
         return parsed.data;
     }
-    async persist(tokens) {
+    async persist(tokens, previous) {
         const credentials = {
             issuer: this.issuer,
             apiUrl: this.apiUrl,
@@ -77,9 +77,9 @@ export class OAuthLogin {
             expiresAt: Date.now() + tokens.expires_in * 1000,
         };
         try {
-            const previous = await this.store.read();
-            if (previous && previous.sessionId !== tokens.session_id)
-                await this.revoke(previous.refresh);
+            const existing = previous ?? (await this.store.read());
+            if (existing && existing.sessionId !== tokens.session_id)
+                await this.revoke(existing.refresh);
             await this.store.write(credentials);
         }
         catch (error) {
@@ -172,14 +172,14 @@ export class OAuthLogin {
             const current = await this.store.read();
             if (!current || current.apiUrl !== this.apiUrl)
                 throw new Error('No login for this server. Run permute auth login.');
-            if (current.expiresAt > Date.now() + 60_000)
+            if (current.expiresAt > Date.now() + TOKEN_REFRESH_BUFFER_MS)
                 return current;
             const tokens = this.tokens(await this.post('/token', {
                 client_id: CLIENT_ID,
                 grant_type: 'refresh_token',
                 refresh_token: current.refresh,
             }));
-            return this.persist(tokens);
+            return this.persist(tokens, current);
         });
     }
     async client() {

package/dist/auth/store.d.ts

@@ -1,4 +1,5 @@
 import { z } from 'zod';
+export declare const TOKEN_REFRESH_BUFFER_MS = 60000;
 declare const LoginCredentialsSchema: z.ZodObject<{
     issuer: z.ZodString;
     apiUrl: z.ZodString;
@@ -16,11 +17,16 @@ export interface CredentialStore {
     remove(): Promise<void>;
     exclusive<T>(run: () => Promise<T>): Promise<T>;
 }
-/** Tokens never fall back to a plaintext file. The disk lock contains no credentials. */
+/** Credentials are cached in memory only; disk holds a lock and a non-secret change marker. */
 export declare class SystemCredentialStore implements CredentialStore {
     private readonly issuer;
+    private readonly onAccess?;
     private readonly account;
-    constructor(issuer: string);
+    private readonly directory;
+    private readonly revisionPath;
+    private cached?;
+    private announced;
+    constructor(issuer: string, onAccess?: ((issuer: string) => void) | undefined);
     private entry;
     read(): Promise<{
         issuer: string;
@@ -32,8 +38,10 @@ export declare class SystemCredentialStore implements CredentialStore {
         refresh: string;
         expiresAt: number;
     } | null>;
+    private validate;
     write(value: LoginCredentials): Promise<void>;
     remove(): Promise<void>;
+    private markChanged;
     exclusive<T>(run: () => Promise<T>): Promise<T>;
 }
 export {};

package/dist/auth/store.js

@@ -1,9 +1,10 @@
 import { z } from 'zod';
-import { createHash } from 'node:crypto';
+import { createHash, randomUUID } from 'node:crypto';
 import { homedir } from 'node:os';
 import { join } from 'node:path';
-import { mkdir, rm } from 'node:fs/promises';
+import { mkdir, readFile, rm, writeFile } from 'node:fs/promises';
 import { setTimeout } from 'node:timers/promises';
+export const TOKEN_REFRESH_BUFFER_MS = 60_000;
 const LoginCredentialsSchema = z.object({
     issuer: z.string().url(),
     apiUrl: z.string().url(),
@@ -14,39 +15,74 @@ const LoginCredentialsSchema = z.object({
     refresh: z.string().min(1),
     expiresAt: z.number().finite(),
 });
-/** Tokens never fall back to a plaintext file. The disk lock contains no credentials. */
+/** Credentials are cached in memory only; disk holds a lock and a non-secret change marker. */
 export class SystemCredentialStore {
     issuer;
+    onAccess;
     account;
-    constructor(issuer) {
+    directory = join(homedir(), '.config', 'permute', 'auth');
+    revisionPath;
+    cached;
+    announced = false;
+    constructor(issuer, onAccess) {
         this.issuer = issuer;
+        this.onAccess = onAccess;
         this.account = createHash('sha256').update(issuer).digest('hex');
+        this.revisionPath = join(this.directory, `${this.account}.revision`);
     }
     async entry() {
         const { AsyncEntry } = await import('@napi-rs/keyring');
+        if (!this.announced) {
+            this.onAccess?.(this.issuer);
+            this.announced = true;
+        }
         return new AsyncEntry('permute-cli', this.account, { linux: { store: 'secret-service' } });
     }
     async read() {
+        const revision = await readFile(this.revisionPath, 'utf8').catch((error) => {
+            if (error.code === 'ENOENT')
+                return null;
+            throw error;
+        });
+        if (this.cached?.revision === revision && this.cached.value.expiresAt > Date.now() + TOKEN_REFRESH_BUFFER_MS)
+            return { ...this.cached.value };
+        this.cached = undefined;
         const value = await (await this.entry()).getPassword();
         if (!value)
             return null;
-        const parsed = LoginCredentialsSchema.safeParse(JSON.parse(value));
+        const credentials = this.validate(JSON.parse(value));
+        this.cached = { revision, value: credentials };
+        return { ...credentials };
+    }
+    validate(value) {
+        const parsed = LoginCredentialsSchema.safeParse(value);
         if (!parsed.success || parsed.data.issuer !== this.issuer)
             throw new Error('Stored login is invalid. Run permute auth login again.');
         return parsed.data;
     }
     async write(value) {
-        await (await this.entry()).setPassword(JSON.stringify(value));
+        const credentials = this.validate(value);
+        this.cached = undefined;
+        await (await this.entry()).setPassword(JSON.stringify(credentials));
+        const revision = await this.markChanged();
+        this.cached = { revision, value: credentials };
     }
     async remove() {
+        this.cached = undefined;
         const entry = await this.entry();
         if (await entry.getPassword())
             await entry.deletePassword();
+        await this.markChanged();
+    }
+    async markChanged() {
+        await mkdir(this.directory, { recursive: true, mode: 0o700 });
+        const revision = randomUUID();
+        await writeFile(this.revisionPath, revision, { mode: 0o600 });
+        return revision;
     }
     async exclusive(run) {
-        const directory = join(homedir(), '.config', 'permute', 'auth');
-        await mkdir(directory, { recursive: true, mode: 0o700 });
-        const lock = join(directory, `${this.account}.lock`);
+        await mkdir(this.directory, { recursive: true, mode: 0o700 });
+        const lock = join(this.directory, `${this.account}.lock`);
         const deadline = Date.now() + 30_000;
         while (true) {
             const acquired = await mkdir(lock, { mode: 0o700 }).then(() => true, (error) => {

package/dist/cli.js

@@ -90,7 +90,14 @@ export async function runCli(args = process.argv.slice(2)) {
     }
     if (!!values.issuer !== !!values['api-url'])
         throw new Error('Provide both --issuer and --api-url for another environment.');
-    const login = new OAuthLogin({ issuer: values.issuer, apiUrl: values['api-url'] });
+    const login = new OAuthLogin({
+        issuer: values.issuer,
+        apiUrl: values['api-url'],
+        onCredentialStoreAccess: (issuer) => {
+            const store = process.platform === 'darwin' ? 'macOS Keychain' : 'your OS credential store';
+            console.error(`Permute CLI is accessing ${store} for ${issuer} (entry: permute-cli).`);
+        },
+    });
     if (command === 'logout') {
         await login.logout();
         console.log('CLI login revoked and removed from this device.');

package/dist/client.d.ts

@@ -1,7 +1,7 @@
 import type { EmailSender, EmailSendRequest, EmailReceipt, PermissionGrant, PermissionGrantInput, PermissionGrantUpdate, PermissionGrantQuery, PermissionGrantList, PermissionGrantRevoked, PermissionInspectionInput, PermissionInspection, AccessRequest, AccessRequestInput, AccessRequestQuery, AccessRequestList, AccessRequestDecision } from './types.js';
 export { PermuteApiError } from './errors.js';
 import type { BusinessAction, BusinessActionDetail, BusinessActionList, ActionCreateRequest, ActionDefinition, ActionDefinitionSaveRequest, ActionDefinitionSaved, ActionRequest, ActionReceipt, ActionPublicationRequest, ActionPublicationRequestInput, ActionPublicationRequestList, ActionPublicationDecision } from './types.js';
-import type { ConnectorSession, ConnectorSessionCreated, ConnectorSessionCreateRequest, Database, AppPublicationRequest, AppPublicationRequestInput, AppPublicationRequestList, AppPublicationDecision, AppDraft, AppAccess, AppAddressSettings, AppAddressSetRequest, AppAccessSetRequest, AppDraftList, AppCreateRequest, AppUpdateRequest, AppDeleted, AppUploadsRequest, AppUploads, AppSiteSaveRequest, AppSourceFileQuery, AppSourceFileContent, AppGrants, AppGrantSetRequest, AppUploadFile, DatabaseList, DatabaseCreateRequest, DatabaseDefinition, DatabaseDefinitionSaveRequest, DatabaseDefinitionSaved, DatabaseTables, DatabaseTableCreateRequest, DatabaseTableCreated, DatabaseRecords, DatabaseRecordsQuery, DatabaseTransactionRequest, DatabaseTransactionReceipt, WriteSync, WriteSyncCreateRequest, WriteSyncUpdateRequest, WriteSyncDetail, WriteSyncList, WriteSyncDeleted, WriteSyncDestinations, WriteSyncRun, WriteSyncRunList, WriteSyncRunQuery, WriteSyncExecution, ResourceVersionState, ResourceRevisionList, ResourceRevision, ResourceVersionRequest, ResourcePublishRequest, Fact, FactList, FactWriteRequest, SkillList, SkillContent, SkillWriteRequest, KnowledgeWriteResponse, OperationRunGroup, OperationRunQuery, Operation, OperationList, OperationListQuery, OperationSource, OperationSourceUpdateRequest, OperationUpdateRequest, OperationRunRequest, OperationRunStarted, OperationRunList, OperationRunOutput, DataFlow, DataFlowCreateRequest, DataFlowCommandRequest, DataFlowCommandResponse, Workspace, WorkspaceCreateRequest, WorkspaceDeleteResponse, WorkspaceList, WorkspaceUpdateRequest, AnswerRequest, AnswerResponse, PermuteClientOptions, PermuteRequestOptions, QueryRequest, SearchRequest, SourceDetail, CatalogTable, SourceList, CustomDatasetCreateRequest, DatasetDeleteResponse, DatasetResponse, FileDatasetCreateRequest, FileDatasetMetadataCreateRequest, FileDatasetResponse, DatasetRows, CustomDatasetResponse, DashboardSource, DashboardSourceUpdateRequest } from './types.js';
+import type { ConnectorSession, ConnectorSessionCreated, ConnectorSessionCreateRequest, Database, AppPublicationRequest, AppPublicationRequestInput, AppPublicationRequestList, AppPublicationDecision, AppDraft, AppAccess, AppAddressSettings, AppAddressSetRequest, AppAccessSetRequest, AppDraftList, AppCreateRequest, AppUpdateRequest, AppDeleted, AppUploadsRequest, AppUploads, AppSiteSaveRequest, AppSourceFileQuery, AppSourceFileContent, AppGrants, AppGrantSetRequest, AppUploadFile, DatabaseList, DatabaseCreateRequest, DatabaseDefinition, DatabaseDefinitionSaveRequest, DatabaseDefinitionSaved, DatabaseTables, DatabaseTableCreateRequest, DatabaseTableCreated, DatabaseRecords, DatabaseRecordsQuery, DatabaseTransactionRequest, DatabaseTransactionReceipt, WriteSync, WriteSyncCreateRequest, WriteSyncUpdateRequest, WriteSyncDetail, WriteSyncList, WriteSyncDeleted, WriteSyncDestinations, WriteSyncRun, WriteSyncRunList, WriteSyncRunQuery, WriteSyncExecution, ResourceVersionState, ResourceRevisionList, ResourceRevision, ResourceVersionRequest, ResourcePublishRequest, Fact, FactList, FactWriteRequest, SkillList, SkillContent, SkillWriteRequest, KnowledgeWriteResponse, OperationRunGroup, OperationRunQuery, Operation, OperationList, OperationListQuery, OperationSource, OperationSourceUpdateRequest, OperationUpdateRequest, OperationRunRequest, OperationRunStarted, OperationRunList, OperationRunOutput, DataFlow, DataFlowCreateRequest, DataFlowCommandRequest, DataFlowCommandResponse, Workspace, WorkspaceCreateRequest, WorkspaceDeleteResponse, WorkspaceList, WorkspaceUpdateRequest, AnswerRequest, AnswerResponse, PermuteClientOptions, PermuteRequestOptions, QueryRequest, SearchRequest, SourceDetail, CatalogTable, SourceList, CustomDatasetCreateRequest, DatasetDeleteResponse, DatasetResponse, FileDatasetCreateRequest, FileDatasetMetadataCreateRequest, FileDatasetResponse, DatasetRows, DatasetCsvUploadOptions, CustomDatasetResponse, DashboardSource, DashboardSourceUpdateRequest } from './types.js';
 type DatasetWriteInput = {
     tables: Record<string, DatasetRows>;
     mode: 'replace';
@@ -14,7 +14,11 @@ type DatasetWriteInput = {
     rows: DatasetRows;
     batchId: string;
     mode: 'appendTable';
-};
+} | ({
+    tableName: string;
+    file: Blob;
+    mode: 'replaceTable';
+} & DatasetCsvUploadOptions);
 type DatasetWriter = (id: string, request: DatasetWriteInput, options?: PermuteRequestOptions) => Promise<CustomDatasetResponse>;
 type DatasetReader = (id: string, options?: PermuteRequestOptions) => Promise<DatasetResponse>;
 export declare class PermuteDataset {
@@ -33,6 +37,7 @@ export declare class PermuteDataset {
     get version(): number;
     replace(tables: Record<string, DatasetRows>, options?: PermuteRequestOptions): Promise<this>;
     replaceTable(tableName: string, rows: DatasetRows, options?: PermuteRequestOptions): Promise<this>;
+    replaceTable(tableName: string, file: Blob, options: DatasetCsvUploadOptions & PermuteRequestOptions): Promise<this>;
     appendTable(tableName: string, rows: DatasetRows, options: {
         batchId: string;
     } & PermuteRequestOptions): Promise<this>;

package/dist/client.js

@@ -93,12 +93,45 @@ function inferUploadSize(data) {
     return undefined;
 }
 const DATASET_WRITE_CHUNK_ROWS = 500;
+const DATASET_WRITE_CHUNK_BYTES = 16 * 1024 * 1024;
+async function hashDatasetFile(file, signal) {
+    const hash = createHash('sha256');
+    const reader = file.stream().getReader();
+    try {
+        while (true) {
+            if (signal?.aborted) {
+                await reader.cancel();
+                throw new PermuteApiError({ status: 0, code: 'request_aborted', message: 'Request was aborted' });
+            }
+            const { done, value } = await reader.read();
+            if (done)
+                return hash.digest('hex');
+            hash.update(value);
+        }
+    }
+    finally {
+        reader.releaseLock();
+    }
+}
 function encodeDatasetWrite(request) {
     const entries = request.mode === 'replace' ? Object.entries(request.tables) : [[request.tableName, request.rows]];
     const parts = [];
     const hash = createHash('sha256');
     const addChunk = (tableName, rows) => {
         const line = `${JSON.stringify({ tableName, rows })}\n`;
+        if (new TextEncoder().encode(line).byteLength > DATASET_WRITE_CHUNK_BYTES) {
+            if (rows.length <= 1) {
+                throw new PermuteApiError({
+                    status: 0,
+                    code: 'dataset_row_too_large',
+                    message: 'A dataset row exceeds the 16 MiB upload chunk limit',
+                });
+            }
+            const middle = Math.floor(rows.length / 2);
+            addChunk(tableName, rows.slice(0, middle));
+            addChunk(tableName, rows.slice(middle));
+            return;
+        }
         parts.push(line);
         hash.update(line);
     };
@@ -113,7 +146,6 @@ function encodeDatasetWrite(request) {
     return {
         data,
         contentHash: hash.digest('hex'),
-        tables: entries.map(([name, rows]) => ({ name, rowCount: rows.length })),
     };
 }
 function pollingDeadline(options) {
@@ -208,8 +240,20 @@ export class PermuteDataset {
         this.snapshot = await this.writeTables(this.id, { tables, mode: 'replace' }, options);
         return this;
     }
-    async replaceTable(tableName, rows, options) {
-        this.snapshot = await this.writeTables(this.id, { tableName, rows, mode: 'replaceTable' }, options);
+    async replaceTable(tableName, data, options) {
+        if (data instanceof Blob) {
+            if (!options || !('format' in options)) {
+                throw new PermuteApiError({
+                    status: 0,
+                    code: 'invalid_upload_format',
+                    message: 'CSV file uploads require format: csv',
+                });
+            }
+            const { format, compression, rowCount, ...requestOptions } = options;
+            this.snapshot = await this.writeTables(this.id, { tableName, file: data, mode: 'replaceTable', format, compression, rowCount }, requestOptions);
+            return this;
+        }
+        this.snapshot = await this.writeTables(this.id, { tableName, rows: data, mode: 'replaceTable' }, options);
         return this;
     }
     async appendTable(tableName, rows, options) {
@@ -475,30 +519,39 @@ export class PermuteClient {
         };
         const readDataset = this.bindMethod('datasetsGet', (apiRequest) => (datasetId, requestOptions) => apiRequest({ pathParams: { datasetId } }, requestOptions));
         const writeDatasetTables = async (datasetId, request, requestOptions) => {
-            const encoded = encodeDatasetWrite(request);
-            const body = request.mode === 'replace'
+            const encoded = 'file' in request
+                ? { data: request.file, contentHash: await hashDatasetFile(request.file, requestOptions?.signal) }
+                : encodeDatasetWrite(request);
+            const uploadMetadata = { sizeBytes: encoded.data.size, contentHash: encoded.contentHash };
+            const body = 'file' in request
                 ? {
-                    mode: request.mode,
-                    tables: encoded.tables,
-                    sizeBytes: encoded.data.size,
-                    contentHash: encoded.contentHash,
+                    mode: 'replaceTable',
+                    tableName: request.tableName,
+                    format: request.format,
+                    compression: request.compression,
+                    rowCount: request.rowCount,
+                    ...uploadMetadata,
                 }
-                : request.mode === 'replaceTable'
+                : request.mode === 'replace'
                     ? {
                         mode: request.mode,
-                        tableName: request.tableName,
-                        rowCount: request.rows.length,
-                        sizeBytes: encoded.data.size,
-                        contentHash: encoded.contentHash,
+                        tables: Object.entries(request.tables).map(([name, rows]) => ({ name, rowCount: rows.length })),
+                        ...uploadMetadata,
                     }
-                    : {
-                        mode: request.mode,
-                        tableName: request.tableName,
-                        rowCount: request.rows.length,
-                        sizeBytes: encoded.data.size,
-                        contentHash: encoded.contentHash,
-                        batchId: request.batchId,
-                    };
+                    : request.mode === 'replaceTable'
+                        ? {
+                            mode: request.mode,
+                            tableName: request.tableName,
+                            rowCount: request.rows.length,
+                            ...uploadMetadata,
+                        }
+                        : {
+                            mode: request.mode,
+                            tableName: request.tableName,
+                            rowCount: request.rows.length,
+                            ...uploadMetadata,
+                            batchId: request.batchId,
+                        };
             const initialized = await this.request('datasetsWriteTables', { pathParams: { datasetId }, body }, requestOptions);
             if (initialized.write.status === 'completed')
                 return initialized.dataset;

package/dist/generated/openapi.d.ts

@@ -1404,8 +1404,12 @@ export interface components {
             status: 'pending' | 'processing' | 'completed' | 'failed';
         };
         DatasetWriteTablesRequest: {
+            /** @enum {string} */
+            compression?: 'gzip';
             contentHash: string;
             /** @enum {string} */
+            format?: 'ndjson';
+            /** @enum {string} */
             mode: 'replace';
             sizeBytes: number;
             tables: {
@@ -1413,16 +1417,35 @@ export interface components {
                 rowCount: number;
             }[];
         } | {
+            /** @enum {string} */
+            compression?: 'gzip';
             contentHash: string;
             /** @enum {string} */
+            format?: 'ndjson';
+            /** @enum {string} */
             mode: 'replaceTable';
             rowCount: number;
             sizeBytes: number;
             tableName: string;
+        } | {
+            /** @enum {string} */
+            compression?: 'gzip';
+            contentHash: string;
+            /** @enum {string} */
+            format: 'csv';
+            /** @enum {string} */
+            mode: 'replaceTable';
+            rowCount?: number;
+            sizeBytes: number;
+            tableName: string;
         } | {
             batchId: string;
+            /** @enum {string} */
+            compression?: 'gzip';
             contentHash: string;
             /** @enum {string} */
+            format?: 'ndjson';
+            /** @enum {string} */
             mode: 'appendTable';
             rowCount: number;
             sizeBytes: number;
@@ -1783,6 +1806,12 @@ export interface components {
             dataFlowRevision: number | null;
             dataSourceIds: string[];
             dependencyOperationIds: string[];
+            incremental: {
+                inputColumns: string[];
+                keyColumns: string[];
+                sourceId: string;
+                table: string;
+            } | null;
             instructions: string;
             operationId: string;
             revision: {
@@ -1801,6 +1830,12 @@ export interface components {
             } | null;
             /** Format: date-time */
             expectedUpdatedAt: string;
+            incremental?: {
+                inputColumns: string[];
+                keyColumns: string[];
+                sourceId: string;
+                table: string;
+            } | null;
             source: string;
         };
         OperationUpdateRequest: {

package/dist/generated/runtime-contract.js

@@ -48,7 +48,7 @@ export const publicApiRoutes = {
     "datasetsGet": { "operationId": "datasetsGet", "method": "GET", "path": "/v1/datasets/{datasetId}", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "oneOf": [{ "type": "object", "properties": { "type": { "type": "string", "enum": ["custom"] }, "id": { "type": "string" }, "sourceId": { "type": "string" }, "key": { "type": "string" }, "name": { "type": "string" }, "context": { "type": "array", "items": { "type": "string" } }, "status": { "type": "string", "enum": ["pending", "uploaded", "processing", "completed", "failed"] }, "version": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "tables": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "columnCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 } }, "required": ["name", "rowCount", "columnCount"], "additionalProperties": false } }, "write": { "type": "object", "properties": { "id": { "type": "string" }, "status": { "type": "string", "enum": ["pending", "processing", "completed", "failed"] }, "error": { "type": "string" } }, "required": ["id", "status"], "additionalProperties": false }, "createdAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["type", "id", "sourceId", "key", "name", "context", "status", "version", "tables", "createdAt", "updatedAt"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["file"] }, "id": { "type": "string" }, "sourceId": { "type": "string" }, "name": { "type": "string" }, "status": { "type": "string", "enum": ["pending", "uploaded", "processing", "completed", "failed"] }, "contentType": { "type": "string" }, "sizeBytes": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 }, "createdAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "progress": { "type": "object", "properties": { "percent": { "type": "number", "minimum": 0, "maximum": 100 }, "step": { "type": "string" }, "stepLabel": { "type": "string" }, "details": { "type": "string" } }, "required": ["percent"], "additionalProperties": false }, "error": { "type": "string" }, "upload": { "type": "object", "properties": { "method": { "type": "string", "enum": ["PUT"] }, "url": { "type": "string", "format": "uri" }, "headers": { "type": "object", "additionalProperties": { "type": "string" } }, "expiresAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["method", "url", "headers", "expiresAt"], "additionalProperties": false } }, "required": ["type", "id", "sourceId", "name", "status", "contentType", "sizeBytes", "createdAt", "updatedAt"], "additionalProperties": false }] } },
     "datasetsDelete": { "operationId": "datasetsDelete", "method": "DELETE", "path": "/v1/datasets/{datasetId}", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "id": { "type": "string" }, "deleted": { "type": "boolean", "enum": [true] } }, "required": ["id", "deleted"], "additionalProperties": false } },
     "datasetsDownload": { "operationId": "datasetsDownload", "method": "GET", "path": "/v1/datasets/{datasetId}/download", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "url": { "type": "string", "format": "uri" } }, "required": ["url"], "additionalProperties": false } },
-    "datasetsWriteTables": { "operationId": "datasetsWriteTables", "method": "POST", "path": "/v1/datasets/{datasetId}/tables", "scope": "workspace", "bodySchema": { "oneOf": [{ "type": "object", "properties": { "tables": { "maxItems": 1000, "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string", "minLength": 1, "maxLength": 128 }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 10000000 } }, "required": ["name", "rowCount"], "additionalProperties": false } }, "sizeBytes": { "type": "integer", "minimum": 0, "maximum": 1073741824 }, "contentHash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, "mode": { "type": "string", "enum": ["replace"] } }, "required": ["tables", "sizeBytes", "contentHash", "mode"], "additionalProperties": false }, { "type": "object", "properties": { "tableName": { "type": "string", "minLength": 1, "maxLength": 128 }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 10000000 }, "sizeBytes": { "type": "integer", "minimum": 0, "maximum": 1073741824 }, "contentHash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, "mode": { "type": "string", "enum": ["replaceTable"] } }, "required": ["tableName", "rowCount", "sizeBytes", "contentHash", "mode"], "additionalProperties": false }, { "type": "object", "properties": { "tableName": { "type": "string", "minLength": 1, "maxLength": 128 }, "rowCount": { "type": "integer", "minimum": 1, "maximum": 10000 }, "batchId": { "type": "string", "minLength": 1, "maxLength": 128 }, "sizeBytes": { "type": "integer", "minimum": 0, "maximum": 1073741824 }, "contentHash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, "mode": { "type": "string", "enum": ["appendTable"] } }, "required": ["tableName", "rowCount", "batchId", "sizeBytes", "contentHash", "mode"], "additionalProperties": false }] }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "dataset": { "type": "object", "properties": { "type": { "type": "string", "enum": ["custom"] }, "id": { "type": "string" }, "sourceId": { "type": "string" }, "key": { "type": "string" }, "name": { "type": "string" }, "context": { "type": "array", "items": { "type": "string" } }, "status": { "type": "string", "enum": ["pending", "uploaded", "processing", "completed", "failed"] }, "version": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "tables": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "columnCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 } }, "required": ["name", "rowCount", "columnCount"], "additionalProperties": false } }, "write": { "type": "object", "properties": { "id": { "type": "string" }, "status": { "type": "string", "enum": ["pending", "processing", "completed", "failed"] }, "error": { "type": "string" } }, "required": ["id", "status"], "additionalProperties": false }, "createdAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["type", "id", "sourceId", "key", "name", "context", "status", "version", "tables", "createdAt", "updatedAt"], "additionalProperties": false }, "write": { "type": "object", "properties": { "id": { "type": "string" }, "status": { "type": "string", "enum": ["pending", "processing", "completed", "failed"] }, "error": { "type": "string" } }, "required": ["id", "status"], "additionalProperties": false }, "upload": { "type": "object", "properties": { "method": { "type": "string", "enum": ["PUT"] }, "url": { "type": "string", "format": "uri" }, "headers": { "type": "object", "additionalProperties": { "type": "string" } }, "expiresAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["method", "url", "headers", "expiresAt"], "additionalProperties": false }, "pollAfterMs": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["dataset", "write", "pollAfterMs"], "additionalProperties": false } },
+    "datasetsWriteTables": { "operationId": "datasetsWriteTables", "method": "POST", "path": "/v1/datasets/{datasetId}/tables", "scope": "workspace", "bodySchema": { "anyOf": [{ "type": "object", "properties": { "tables": { "maxItems": 1000, "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string", "minLength": 1, "maxLength": 128 }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 10000000 } }, "required": ["name", "rowCount"], "additionalProperties": false } }, "sizeBytes": { "type": "integer", "minimum": 0, "maximum": 5368709120 }, "contentHash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, "format": { "type": "string", "enum": ["ndjson"] }, "compression": { "type": "string", "enum": ["gzip"] }, "mode": { "type": "string", "enum": ["replace"] } }, "required": ["tables", "sizeBytes", "contentHash", "mode"], "additionalProperties": false }, { "type": "object", "properties": { "tableName": { "type": "string", "minLength": 1, "maxLength": 128 }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 10000000 }, "sizeBytes": { "type": "integer", "minimum": 0, "maximum": 5368709120 }, "contentHash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, "format": { "type": "string", "enum": ["ndjson"] }, "compression": { "type": "string", "enum": ["gzip"] }, "mode": { "type": "string", "enum": ["replaceTable"] } }, "required": ["tableName", "rowCount", "sizeBytes", "contentHash", "mode"], "additionalProperties": false }, { "type": "object", "properties": { "tableName": { "type": "string", "minLength": 1, "maxLength": 128 }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 10000000 }, "sizeBytes": { "type": "integer", "minimum": 0, "maximum": 5368709120 }, "contentHash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, "format": { "type": "string", "enum": ["csv"] }, "compression": { "type": "string", "enum": ["gzip"] }, "mode": { "type": "string", "enum": ["replaceTable"] } }, "required": ["tableName", "sizeBytes", "contentHash", "format", "mode"], "additionalProperties": false }, { "type": "object", "properties": { "tableName": { "type": "string", "minLength": 1, "maxLength": 128 }, "rowCount": { "type": "integer", "minimum": 1, "maximum": 10000 }, "batchId": { "type": "string", "minLength": 1, "maxLength": 128 }, "sizeBytes": { "type": "integer", "minimum": 0, "maximum": 5368709120 }, "contentHash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, "format": { "type": "string", "enum": ["ndjson"] }, "compression": { "type": "string", "enum": ["gzip"] }, "mode": { "type": "string", "enum": ["appendTable"] } }, "required": ["tableName", "rowCount", "batchId", "sizeBytes", "contentHash", "mode"], "additionalProperties": false }] }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "dataset": { "type": "object", "properties": { "type": { "type": "string", "enum": ["custom"] }, "id": { "type": "string" }, "sourceId": { "type": "string" }, "key": { "type": "string" }, "name": { "type": "string" }, "context": { "type": "array", "items": { "type": "string" } }, "status": { "type": "string", "enum": ["pending", "uploaded", "processing", "completed", "failed"] }, "version": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "tables": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "rowCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "columnCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 } }, "required": ["name", "rowCount", "columnCount"], "additionalProperties": false } }, "write": { "type": "object", "properties": { "id": { "type": "string" }, "status": { "type": "string", "enum": ["pending", "processing", "completed", "failed"] }, "error": { "type": "string" } }, "required": ["id", "status"], "additionalProperties": false }, "createdAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["type", "id", "sourceId", "key", "name", "context", "status", "version", "tables", "createdAt", "updatedAt"], "additionalProperties": false }, "write": { "type": "object", "properties": { "id": { "type": "string" }, "status": { "type": "string", "enum": ["pending", "processing", "completed", "failed"] }, "error": { "type": "string" } }, "required": ["id", "status"], "additionalProperties": false }, "upload": { "type": "object", "properties": { "method": { "type": "string", "enum": ["PUT"] }, "url": { "type": "string", "format": "uri" }, "headers": { "type": "object", "additionalProperties": { "type": "string" } }, "expiresAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["method", "url", "headers", "expiresAt"], "additionalProperties": false }, "pollAfterMs": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["dataset", "write", "pollAfterMs"], "additionalProperties": false } },
     "searchCreate": { "operationId": "searchCreate", "method": "POST", "path": "/v1/search", "scope": "workspace", "bodySchema": { "type": "object", "properties": { "query": { "type": "string", "minLength": 1 }, "sourceIds": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } }, "types": { "type": "array", "items": { "type": "string", "enum": ["records", "documents", "tables", "text_blocks"] } }, "limit": { "type": "integer", "minimum": 1, "maximum": 50 }, "rerank": { "type": "boolean" } }, "required": ["query"], "additionalProperties": false }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "items": { "type": "array", "items": { "type": "object", "properties": { "type": { "type": "string", "enum": ["record", "document", "table", "text_block", "unknown"] }, "sourceId": { "type": "string" }, "title": { "type": "string" }, "content": { "type": "string" }, "score": { "type": "number" }, "citation": { "type": "object", "additionalProperties": {} }, "metadata": { "type": "object", "additionalProperties": {} } }, "required": ["type", "content"], "additionalProperties": false } }, "meta": { "type": "object", "properties": { "strategy": { "type": "string", "enum": ["semantic"] }, "reranked": { "type": "boolean" }, "candidateCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "totalMs": { "type": "number", "minimum": 0 } }, "required": ["strategy", "reranked"], "additionalProperties": false } }, "required": ["items", "meta"], "additionalProperties": false } },
     "answerJobsCreate": { "operationId": "answerJobsCreate", "method": "POST", "path": "/v1/answer-jobs", "scope": "workspace", "bodySchema": { "type": "object", "properties": { "question": { "type": "string", "minLength": 1 }, "sourceIds": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } } }, "required": ["question"], "additionalProperties": false }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "answerJobId": { "type": "string" }, "status": { "type": "string", "enum": ["queued", "running", "completed", "failed", "cancelled"] }, "result": { "type": "object", "properties": { "answer": { "type": "string" }, "citations": { "type": "array", "items": { "type": "object", "properties": { "type": { "type": "string", "enum": ["text", "table", "web", "fact"] }, "sourceId": { "type": "string" }, "ref": { "type": "string" }, "title": { "type": "string" }, "url": { "type": "string" }, "metadata": { "type": "object", "additionalProperties": {} } }, "required": ["type"], "additionalProperties": false } }, "assumptions": { "type": "array", "items": { "type": "object", "additionalProperties": {} } }, "charts": { "type": "array", "items": { "type": "object", "additionalProperties": {} } }, "followUps": { "type": "array", "items": { "type": "string" } }, "confidence": { "type": "object", "properties": { "score": { "type": "number", "minimum": 0, "maximum": 100 }, "queryComplexity": { "type": "string", "enum": ["high", "medium", "low"] }, "reason": { "type": "string" } }, "required": ["score", "queryComplexity", "reason"], "additionalProperties": false } }, "required": ["answer", "citations"], "additionalProperties": false }, "error": { "type": "string" }, "pollAfterMs": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 }, "createdAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "startedAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "completedAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["answerJobId", "status"], "additionalProperties": false } },
     "answerJobsGet": { "operationId": "answerJobsGet", "method": "GET", "path": "/v1/answer-jobs/{answerJobId}", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "answerJobId": { "type": "string" }, "status": { "type": "string", "enum": ["queued", "running", "completed", "failed", "cancelled"] }, "result": { "type": "object", "properties": { "answer": { "type": "string" }, "citations": { "type": "array", "items": { "type": "object", "properties": { "type": { "type": "string", "enum": ["text", "table", "web", "fact"] }, "sourceId": { "type": "string" }, "ref": { "type": "string" }, "title": { "type": "string" }, "url": { "type": "string" }, "metadata": { "type": "object", "additionalProperties": {} } }, "required": ["type"], "additionalProperties": false } }, "assumptions": { "type": "array", "items": { "type": "object", "additionalProperties": {} } }, "charts": { "type": "array", "items": { "type": "object", "additionalProperties": {} } }, "followUps": { "type": "array", "items": { "type": "string" } }, "confidence": { "type": "object", "properties": { "score": { "type": "number", "minimum": 0, "maximum": 100 }, "queryComplexity": { "type": "string", "enum": ["high", "medium", "low"] }, "reason": { "type": "string" } }, "required": ["score", "queryComplexity", "reason"], "additionalProperties": false } }, "required": ["answer", "citations"], "additionalProperties": false }, "error": { "type": "string" }, "pollAfterMs": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 }, "createdAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "startedAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "completedAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["answerJobId", "status"], "additionalProperties": false } },
@@ -57,8 +57,8 @@ export const publicApiRoutes = {
     "operationsList": { "operationId": "operationsList", "method": "GET", "path": "/v1/operations", "scope": "workspace", "bodySchema": null, "querySchema": { "type": "object", "properties": { "kind": { "type": "string", "enum": ["dataFlow", "dataTable", "report", "dashboard"] }, "dataFlowId": { "type": "string", "minLength": 1 } }, "required": [], "additionalProperties": false }, "responseSchema": { "type": "object", "properties": { "items": { "type": "array", "items": { "type": "object", "properties": { "id": { "type": "string" }, "name": { "type": "string" }, "kind": { "type": "string" }, "instructions": { "type": "string" }, "dataSourceIds": { "type": "array", "items": { "type": "string" } }, "parentDataFlowId": { "nullable": true, "type": "string" }, "status": { "type": "string" }, "codeStale": { "type": "boolean" }, "schedule": { "anyOf": [{ "oneOf": [{ "type": "object", "properties": { "type": { "type": "string", "enum": ["manual"] } }, "required": ["type"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["hourly"] }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["daily"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "hour", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["weekly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfWeek": { "type": "number", "minimum": 0, "maximum": 6 } }, "required": ["type", "hour", "minute", "dayOfWeek"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["monthly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfMonth": { "type": "number", "minimum": 1, "maximum": 31 } }, "required": ["type", "hour", "minute", "dayOfMonth"], "additionalProperties": false }] }, { "type": "object", "nullable": true, "enum": [null] }] }, "isEnabled": { "type": "boolean" }, "nextRunAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "draft": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "published": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false } }, "required": ["id", "name", "kind", "instructions", "dataSourceIds", "parentDataFlowId", "status", "codeStale", "schedule", "isEnabled", "nextRunAt", "draft", "published"], "additionalProperties": false } } }, "required": ["items"], "additionalProperties": false } },
     "operationsGet": { "operationId": "operationsGet", "method": "GET", "path": "/v1/operations/{operationId}", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "id": { "type": "string" }, "name": { "type": "string" }, "kind": { "type": "string" }, "instructions": { "type": "string" }, "dataSourceIds": { "type": "array", "items": { "type": "string" } }, "parentDataFlowId": { "nullable": true, "type": "string" }, "status": { "type": "string" }, "codeStale": { "type": "boolean" }, "schedule": { "anyOf": [{ "oneOf": [{ "type": "object", "properties": { "type": { "type": "string", "enum": ["manual"] } }, "required": ["type"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["hourly"] }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["daily"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "hour", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["weekly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfWeek": { "type": "number", "minimum": 0, "maximum": 6 } }, "required": ["type", "hour", "minute", "dayOfWeek"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["monthly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfMonth": { "type": "number", "minimum": 1, "maximum": 31 } }, "required": ["type", "hour", "minute", "dayOfMonth"], "additionalProperties": false }] }, { "type": "object", "nullable": true, "enum": [null] }] }, "isEnabled": { "type": "boolean" }, "nextRunAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "draft": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "published": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false } }, "required": ["id", "name", "kind", "instructions", "dataSourceIds", "parentDataFlowId", "status", "codeStale", "schedule", "isEnabled", "nextRunAt", "draft", "published"], "additionalProperties": false } },
     "operationsUpdate": { "operationId": "operationsUpdate", "method": "PATCH", "path": "/v1/operations/{operationId}", "scope": "workspace", "bodySchema": { "type": "object", "properties": { "schedule": { "anyOf": [{ "oneOf": [{ "type": "object", "properties": { "type": { "type": "string", "enum": ["manual"] } }, "required": ["type"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["hourly"] }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["daily"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "hour", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["weekly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfWeek": { "type": "number", "minimum": 0, "maximum": 6 } }, "required": ["type", "hour", "minute", "dayOfWeek"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["monthly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfMonth": { "type": "number", "minimum": 1, "maximum": 31 } }, "required": ["type", "hour", "minute", "dayOfMonth"], "additionalProperties": false }] }, { "type": "object", "nullable": true, "enum": [null] }] }, "isEnabled": { "type": "boolean" } }, "additionalProperties": false }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "id": { "type": "string" }, "name": { "type": "string" }, "kind": { "type": "string" }, "instructions": { "type": "string" }, "dataSourceIds": { "type": "array", "items": { "type": "string" } }, "parentDataFlowId": { "nullable": true, "type": "string" }, "status": { "type": "string" }, "codeStale": { "type": "boolean" }, "schedule": { "anyOf": [{ "oneOf": [{ "type": "object", "properties": { "type": { "type": "string", "enum": ["manual"] } }, "required": ["type"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["hourly"] }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["daily"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 } }, "required": ["type", "hour", "minute"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["weekly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfWeek": { "type": "number", "minimum": 0, "maximum": 6 } }, "required": ["type", "hour", "minute", "dayOfWeek"], "additionalProperties": false }, { "type": "object", "properties": { "type": { "type": "string", "enum": ["monthly"] }, "hour": { "type": "number", "minimum": 0, "maximum": 23 }, "minute": { "type": "number", "minimum": 0, "maximum": 59 }, "dayOfMonth": { "type": "number", "minimum": 1, "maximum": 31 } }, "required": ["type", "hour", "minute", "dayOfMonth"], "additionalProperties": false }] }, { "type": "object", "nullable": true, "enum": [null] }] }, "isEnabled": { "type": "boolean" }, "nextRunAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "draft": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "published": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false } }, "required": ["id", "name", "kind", "instructions", "dataSourceIds", "parentDataFlowId", "status", "codeStale", "schedule", "isEnabled", "nextRunAt", "draft", "published"], "additionalProperties": false } },
-    "operationsSourceGet": { "operationId": "operationsSourceGet", "method": "GET", "path": "/v1/operations/{operationId}/source", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "operationId": { "type": "string" }, "source": { "type": "string" }, "revision": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "instructions": { "type": "string" }, "dataSourceIds": { "type": "array", "items": { "type": "string" } }, "dependencyOperationIds": { "type": "array", "items": { "type": "string" } }, "dataFlowRevision": { "nullable": true, "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["operationId", "source", "revision", "instructions", "dataSourceIds", "dependencyOperationIds", "dataFlowRevision", "updatedAt"], "additionalProperties": false } },
-    "operationsSourceUpdate": { "operationId": "operationsSourceUpdate", "method": "PUT", "path": "/v1/operations/{operationId}/source", "scope": "workspace", "bodySchema": { "type": "object", "properties": { "source": { "type": "string", "minLength": 1, "maxLength": 1000000 }, "expectedRevision": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "expectedDataFlowRevision": { "nullable": true, "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "expectedUpdatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["source", "expectedRevision", "expectedDataFlowRevision", "expectedUpdatedAt"], "additionalProperties": false }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "operationId": { "type": "string" }, "source": { "type": "string" }, "revision": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "instructions": { "type": "string" }, "dataSourceIds": { "type": "array", "items": { "type": "string" } }, "dependencyOperationIds": { "type": "array", "items": { "type": "string" } }, "dataFlowRevision": { "nullable": true, "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["operationId", "source", "revision", "instructions", "dataSourceIds", "dependencyOperationIds", "dataFlowRevision", "updatedAt"], "additionalProperties": false } },
+    "operationsSourceGet": { "operationId": "operationsSourceGet", "method": "GET", "path": "/v1/operations/{operationId}/source", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "operationId": { "type": "string" }, "source": { "type": "string" }, "incremental": { "nullable": true, "type": "object", "properties": { "sourceId": { "type": "string", "minLength": 1 }, "table": { "type": "string", "minLength": 1 }, "keyColumns": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } }, "inputColumns": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } } }, "required": ["sourceId", "table", "keyColumns", "inputColumns"], "additionalProperties": false }, "revision": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "instructions": { "type": "string" }, "dataSourceIds": { "type": "array", "items": { "type": "string" } }, "dependencyOperationIds": { "type": "array", "items": { "type": "string" } }, "dataFlowRevision": { "nullable": true, "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["operationId", "source", "incremental", "revision", "instructions", "dataSourceIds", "dependencyOperationIds", "dataFlowRevision", "updatedAt"], "additionalProperties": false } },
+    "operationsSourceUpdate": { "operationId": "operationsSourceUpdate", "method": "PUT", "path": "/v1/operations/{operationId}/source", "scope": "workspace", "bodySchema": { "type": "object", "properties": { "source": { "type": "string", "minLength": 1, "maxLength": 1000000 }, "incremental": { "nullable": true, "type": "object", "properties": { "sourceId": { "type": "string", "minLength": 1 }, "table": { "type": "string", "minLength": 1 }, "keyColumns": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } }, "inputColumns": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } } }, "required": ["sourceId", "table", "keyColumns", "inputColumns"], "additionalProperties": false }, "expectedRevision": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "expectedDataFlowRevision": { "nullable": true, "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "expectedUpdatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["source", "expectedRevision", "expectedDataFlowRevision", "expectedUpdatedAt"], "additionalProperties": false }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "operationId": { "type": "string" }, "source": { "type": "string" }, "incremental": { "nullable": true, "type": "object", "properties": { "sourceId": { "type": "string", "minLength": 1 }, "table": { "type": "string", "minLength": 1 }, "keyColumns": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } }, "inputColumns": { "minItems": 1, "type": "array", "items": { "type": "string", "minLength": 1 } } }, "required": ["sourceId", "table", "keyColumns", "inputColumns"], "additionalProperties": false }, "revision": { "nullable": true, "type": "object", "properties": { "revisionId": { "type": "string", "minLength": 1 }, "sequence": { "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 } }, "required": ["revisionId", "sequence"], "additionalProperties": false }, "instructions": { "type": "string" }, "dataSourceIds": { "type": "array", "items": { "type": "string" } }, "dependencyOperationIds": { "type": "array", "items": { "type": "string" } }, "dataFlowRevision": { "nullable": true, "type": "integer", "minimum": 0, "maximum": 9007199254740991 }, "updatedAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" } }, "required": ["operationId", "source", "incremental", "revision", "instructions", "dataSourceIds", "dependencyOperationIds", "dataFlowRevision", "updatedAt"], "additionalProperties": false } },
     "operationsRunsList": { "operationId": "operationsRunsList", "method": "GET", "path": "/v1/operations/{operationId}/runs", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "items": { "type": "array", "items": { "type": "object", "properties": { "id": { "type": "string" }, "workflowId": { "type": "string" }, "revisionId": { "nullable": true, "type": "string" }, "status": { "type": "string" }, "createdAt": { "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "completedAt": { "nullable": true, "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$" }, "error": { "nullable": true, "type": "string" }, "outputDocId": { "nullable": true, "type": "string" } }, "required": ["id", "workflowId", "revisionId", "status", "createdAt", "completedAt", "error", "outputDocId"], "additionalProperties": false } } }, "required": ["items"], "additionalProperties": false } },
     "operationsRun": { "operationId": "operationsRun", "method": "POST", "path": "/v1/operations/{operationId}/runs", "scope": "workspace", "bodySchema": { "type": "object", "properties": { "version": { "type": "string", "enum": ["draft", "published"] } }, "required": ["version"], "additionalProperties": false }, "querySchema": null, "responseSchema": { "type": "object", "properties": { "workflowId": { "type": "string" }, "runId": { "type": "string" } }, "required": ["workflowId"], "additionalProperties": false } },
     "operationsRunOutput": { "operationId": "operationsRunOutput", "method": "GET", "path": "/v1/operations/{operationId}/runs/{runId}/output", "scope": "workspace", "bodySchema": null, "querySchema": null, "responseSchema": { "type": "object", "properties": { "runId": { "type": "string" }, "revisionId": { "nullable": true, "type": "string" }, "rowLimitPerTable": { "type": "number", "enum": [1000] }, "tables": { "type": "object", "additionalProperties": { "type": "array", "items": {} } }, "names": { "type": "object", "additionalProperties": { "type": "string" } }, "schemas": { "type": "object", "additionalProperties": {} } }, "required": ["runId", "revisionId", "rowLimitPerTable", "tables", "names", "schemas"], "additionalProperties": false } },

package/dist/types.d.ts

@@ -66,6 +66,9 @@ export type DatasetCreateRequest = CustomDatasetCreateRequest | FileDatasetCreat
 export type DatasetWriteTablesRequest = PublicApiSchema<'DatasetWriteTablesRequest'>;
 export type DatasetWriteTablesResponse = PublicApiSchema<'DatasetWriteTablesResponse'>;
 export type DatasetRows = Record<string, unknown>[];
+export type DatasetCsvUploadOptions = Pick<Extract<DatasetWriteTablesRequest, {
+    format: 'csv';
+}>, 'format' | 'compression' | 'rowCount'>;
 export type SearchRequest = PublicApiSchema<'SearchRequest'>;
 export type SearchResult = PublicApiSchema<'SearchResult'>;
 export type SearchResponse = PublicApiSchema<'SearchResponse'>;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@permute/sdk",
-  "version": "0.8.0",
+  "version": "0.9.0",
   "description": "TypeScript client for the Permute public API",
   "type": "module",
   "license": "Apache-2.0",