@alvera-ai/platform-sdk 0.11.0 → 0.12.0

package/.agent/AGENTS.md

@@ -438,6 +438,10 @@ full wire shape.
   built-in row-mapping Liquid templates.
 - [invite-team](./cookbook/invite-team.md) — invite a teammate into your tenant
   (root / tenantless / tenant-scoped sessions in one flow).
+- [talk-to-data](./cookbook/talk-to-data.md) — turn a datalake conversational:
+  natural language → reviewable SQL (`datalakes.textToSql`, data-free by
+  construction) → read-only execution returning a `{ data, meta }` row page or a
+  CSV export (`datalakes.executeSql`).
 
 ## Utility namespaces
 

package/.agent/ai_sandbox.md

@@ -179,6 +179,31 @@ via consumer-authored fragments is structurally impossible: the
 fragment is the WHERE *expression*, with values supplied via
 positional bindings the platform builds.
 
+### Datalake "talk to data" — `textToSql` + `executeSql`
+
+The datalake BI surface (`api.datalakes.textToSql` /
+`api.datalakes.executeSql`, see `datalakes.md` "Talk to data") is
+the **full-statement** sibling of the WHERE-clause lane above, and
+it sits inside the same Layer-3 boundary:
+
+- **Generation is data-free by construction.** `textToSql` sends
+  only the natural-language `prompt` plus the datalake **schema**
+  to the LLM — never rows. The model returns SQL for review; no
+  datalake data ever crosses the LLM boundary, so generation is
+  safe in both `regulated` and `unregulated` mode. The two-call
+  split (generate, then execute) exists so a human or agent can
+  inspect and edit the SQL before any data is read.
+- **Execution is read-only and mode-routed.** `executeSql` runs the
+  statement against the mode-appropriate schema with a read-only
+  connection: INSERT/UPDATE/DELETE/DDL are rejected (app-layer
+  `@deny` + a read-only DB transaction + an EXPLAIN preflight), and
+  `mode` (`regulated` vs `unregulated`) plus the session's tenant
+  (RLS) decide which schema and which rows are reachable — the same
+  Layer-1 compliance gates that govern every other read.
+- **Pagination caps blast radius.** `page` / `page_size` map to a
+  server-side `LIMIT/OFFSET` window capped at the platform default,
+  so a single call cannot exfiltrate an unbounded result set.
+
 ## How the three layers compose
 
 ```

package/.agent/cookbook/talk-to-data.md

@@ -0,0 +1,139 @@
+---
+title: "Capability: talk to your datalake — natural language → SQL → rows"
+summary: A capability walk for datalake text-to-SQL. Generate a read-only SQL statement from a natural-language prompt (`api.datalakes.textToSql`) — only the prompt + schema cross the LLM boundary, never rows — then run it read-only and read back the page (`api.datalakes.executeSql`), with both the JSON `{ data, meta }` envelope and a CSV export. The BI surface on a datalake.
+industry: foundation
+slug: talk-to-data
+vitest_source:
+  - integration-tests/tests/foundation/text-to-sql.test.ts
+status: green
+---
+
+# Capability
+
+**What you get:** a two-call BI surface on a datalake — describe what you want in
+plain language, get back reviewable SQL, then run it read-only and read the rows.
+
+The split is deliberate: `textToSql` *generates*, `executeSql` *runs*. A UI or
+agent can show the SQL, let a human edit it, and re-run — and the generate step
+is **data-free by construction** (only the prompt + the datalake schema reach the
+LLM, never rows), so it is safe in both `regulated` and `unregulated` mode.
+
+1. `datalakes.textToSql(...)` — natural language → `{ sql, model, provider, explanation }`.
+2. `datalakes.executeSql(...)` — read-only execution → `{ data, meta }` (data is an
+   array-of-arrays row page aligned to `meta.columns`), or a CSV string with
+   `{ format: 'csv' }`.
+
+See `datalakes.md` "Talk to data" for the wire reference and `ai_sandbox.md`
+Layer 3 for why the boundary holds.
+
+# Walkthrough
+
+The `_setup/foundation.md` bootstrap left `api`, `tenantSlug`, and `datalakeSlug`
+populated, pointing at a freshly-migrated foundation datalake.
+
+## 001 — generate SQL from a natural-language prompt
+
+`textToSql` runs ordered multi-provider LLM failover and returns the generated
+`sql` plus the winning `model` / `provider` and a best-effort `explanation`
+(`null` when the explainer is unavailable). It does **not** execute the SQL — it
+hands it back for review. Nothing but the prompt and the datalake schema is sent
+to the model, so no datalake rows leave the trust boundary.
+
+```typescript
+const { data: gen } = await api.datalakes.textToSql(tenantSlug, datalakeSlug, {
+  prompt: 'how many legal entities are there?',
+  mode: 'unregulated',
+})
+if (typeof gen.sql !== 'string' || gen.sql.trim() === '') {
+  throw new Error(`textToSql returned no SQL (got: ${JSON.stringify(gen.sql)})`)
+}
+if (typeof gen.model !== 'string' || typeof gen.provider !== 'string') {
+  throw new Error('textToSql response missing model/provider attribution')
+}
+// gen.explanation is best-effort plain-language prose, or null.
+```
+
+## 002 — run a read-only query and read back the page
+
+`executeSql` runs the SQL **read-only** (INSERT/UPDATE/DELETE/DDL are rejected)
+and returns `{ data, meta }`. `data` is an **array-of-arrays** row page — positional,
+aligned to `meta.columns`, because arbitrary SQL can have duplicate / expression
+column names. `meta` carries the structural + pagination fields. A deterministic
+`SELECT 1 AS n` keeps this step independent of the generated SQL and seeded data.
+
+The curated return is `ExecuteSqlResponse | string` (the CSV branch is a string),
+so narrow with a `typeof` guard before reading the JSON envelope.
+
+```typescript
+const jsonResult = await api.datalakes.executeSql(tenantSlug, datalakeSlug, {
+  sql: 'SELECT 1 AS n',
+  mode: 'unregulated',
+})
+if (typeof jsonResult.data === 'string') {
+  throw new Error('expected the JSON envelope, got a CSV string')
+}
+const page = jsonResult.data
+if (!Array.isArray(page.data) || !Array.isArray(page.data[0])) {
+  throw new Error('executeSql data is not an array-of-arrays')
+}
+if (page.meta.columns[0] !== 'n' || page.data[0][0] !== 1 || page.meta.num_rows !== 1) {
+  throw new Error(`unexpected SELECT 1 result: ${JSON.stringify(page)}`)
+}
+// page.meta also carries total_count, page, page_size, total_pages, duration_ms, command.
+```
+
+## 003 — export the same page as CSV
+
+Pass `{ format: 'csv' }` to get the page as a CSV **string** instead of the JSON
+envelope — the same read, content-negotiated for download.
+
+```typescript
+const csvResult = await api.datalakes.executeSql(
+  tenantSlug,
+  datalakeSlug,
+  { sql: 'SELECT 1 AS n', mode: 'unregulated' },
+  { format: 'csv' },
+)
+if (typeof csvResult.data !== 'string') {
+  throw new Error('expected a CSV string for { format: "csv" }')
+}
+const csv = csvResult.data
+if (!/\bn\b/.test(csv) || !csv.includes('1')) {
+  throw new Error(`unexpected CSV body: ${JSON.stringify(csv)}`)
+}
+```
+
+# Branches
+
+- **Generation failure.** If every configured LLM provider fails, `textToSql`
+  returns `422` (`AlveraApiError`) — `Text-to-SQL generation failed for all
+  providers (...)`. Surface it; do not retry blindly.
+- **Write SQL rejected.** `executeSql` with a non-read-only statement
+  (`INSERT`/`UPDATE`/`DELETE`/DDL) is rejected `422` — the read-only boundary is
+  enforced at the app `@deny` regex, a read-only DB transaction, and an EXPLAIN
+  preflight.
+- **Pagination.** Pass `page` / `page_size` to window large result sets;
+  `meta.total_count` / `meta.total_pages` describe the full set (`page_size` is
+  capped server-side).
+
+# Rollback
+
+Nothing to tear down — both calls are read-only (or generate-only). The
+`_setup/foundation.md` tenant + datalake are reset with `mix ecto.reset` on the
+platform; per-run `runSuffix` names avoid collisions across reruns.
+
+# Outcome
+
+A datalake becomes conversational: a prompt yields reviewable SQL with provider
+attribution and a plain-language explanation, that SQL runs read-only, and the
+page comes back either as a structured `{ data, meta }` envelope or a CSV export
+— all without any datalake row ever reaching the LLM.
+
+# See also
+
+- `.agent/datalakes.md` — "Talk to data" (`textToSql` / `executeSql` wire shape)
+  and §6 gotcha on the array-of-arrays `data`.
+- `.agent/ai_sandbox.md` — Layer 3 SQL boundary: how generation stays data-free
+  and execution stays read-only + mode-routed.
+- `integration-tests/tests/foundation/text-to-sql.test.ts` — the green vitest
+  these snippets are lifted from.

package/.agent/datalakes.md

@@ -586,6 +586,52 @@ for (const name of catalog.datasets) {
 }
 ```
 
+### Talk to data — `textToSql` + `executeSql`
+
+Two query methods turn the datalake into a BI surface: natural language → SQL →
+rows. They are **split on purpose** so an agent or UI can show, edit, and re-run
+the SQL between the two calls.
+
+| Method                                                     | Returns                                                   |
+|------------------------------------------------------------|-----------------------------------------------------------|
+| `.textToSql(tenantSlug, datalakeSlug, body)`               | `{ sql, model, provider, explanation }`                   |
+| `.executeSql(tenantSlug, datalakeSlug, body, options?)`    | `{ data, meta }` — or a CSV **string** with `{ format: 'csv' }` |
+
+- **`textToSql`** body is `{ prompt, mode }` (`mode` ∈ `'regulated' | 'unregulated'`,
+  selecting which schema to target). It runs ordered multi-provider LLM failover and
+  returns the generated `sql`, the winning `model` + `provider`, and a best-effort
+  `explanation` (`null` when the explainer is unavailable). **Only the prompt + the
+  schema cross the LLM boundary — never datalake rows**, so it is safe in both modes.
+  It returns SQL for review; it does **not** execute it.
+- **`executeSql`** body is `{ sql, mode, page?, page_size? }`. It runs the SQL
+  **read-only** (INSERT/UPDATE/DELETE/DDL are rejected) on the mode-appropriate
+  schema, with Flop-inspired pagination (`page_size` capped server-side). The JSON
+  shape is `{ data, meta }`; pass `options = { format: 'csv' }` to get the page as a
+  CSV string attachment instead.
+
+```typescript
+const { data: gen } = await api.datalakes.textToSql(tenantSlug, datalakeSlug, {
+  prompt: 'count contacts created this month',
+  mode: 'unregulated',
+})
+// gen.sql — review / edit before running
+
+const { data: page } = await api.datalakes.executeSql(tenantSlug, datalakeSlug, {
+  sql: gen.sql,
+  mode: 'unregulated',
+  page: 1,
+  page_size: 100,
+})
+// page.data  — array-of-arrays rows, aligned to page.meta.columns
+// page.meta  — { columns, num_rows, total_count, page, page_size, total_pages, duration_ms, command }
+
+const { data: csv } = await api.datalakes.executeSql(
+  tenantSlug, datalakeSlug,
+  { sql: gen.sql, mode: 'unregulated' },
+  { format: 'csv' },
+) // csv is a string, not the JSON envelope
+```
+
 ## 6. Gotchas
 
 1. **`create()` does NOT auto-enqueue migration.** The create response
@@ -712,3 +758,12 @@ for (const name of catalog.datasets) {
     `"password"` — and consequently requires both `*_user` and
     `*_pass` to be supplied. The IAM-role relaxation in §2 applies
     only when `auth_method` is explicitly `"iam_role"`.
+
+13. **`executeSql` rows are array-of-arrays, not objects.** `data` is
+    `Array<Array<unknown>>` — a positional row page with **no column
+    keys**, because arbitrary SQL can yield duplicate or expression
+    column names that have no safe object key. Read column names from
+    `meta.columns` and index rows positionally (`row[i]` ↔
+    `meta.columns[i]`); never assume `row.someColumn`. The `?format=csv`
+    branch returns a CSV **string** with no `meta` envelope — for the
+    structural metadata, use the JSON call.

package/README.md

@@ -132,7 +132,7 @@ Every resource is a typed namespace on the client (`api.<resource>.<verb>`):
 | `tenants`                    | `list`, `create`                                                                                                                             |
 | `invitations`                | `list`, `create`, `accept`                                                                                                                   |
 | `datasets`                   | `search`, `metadata`, `createUserSearch`                                                                                                     |
-| `datalakes`                  | `list`, `get`, `create`, `metadata`, `migrate`, `createUploadLink`, `createDownloadLink`                                                     |
+| `datalakes`                  | `list`, `get`, `create`, `metadata`, `migrate`, `createUploadLink`, `createDownloadLink`, `textToSql`, `executeSql`                          |
 | `dataSources`                | `list`, `create`, `update`                                                                                                                   |
 | `tools`                      | `list`, `get`, `create`, `update`, `delete`, `testInvocation`                                                                                |
 | `genericTables`              | `list`, `create`                                                                                                                             |

package/dist/index.d.mts

@@ -22,6 +22,21 @@ declare const DEFAULT_ENVIRONMENT: "prod";
 type EnvironmentName = keyof typeof ENVIRONMENTS;
 //#endregion
 //#region src/generated/types.gen.d.ts
+/**
+ * TextToSqlRequest
+ *
+ * Natural-language prompt to generate datalake SQL for, plus the data access mode.
+ */
+type TextToSqlRequest = {
+  /**
+   * Which datalake schema to target: `unregulated` (tokenized) or `regulated` (raw)
+   */
+  mode: 'regulated' | 'unregulated';
+  /**
+   * Natural-language description of the desired query
+   */
+  prompt: string;
+};
 /**
  * UserSearchResponse
  *
@@ -1055,6 +1070,49 @@ type ToolAwsLambdaResponse = AwsLambdaResponse & {
 type ToolManualUploadResponse = ManualUploadResponse & {
   tool_body_type: 'manual_upload';
 };
+/**
+ * ExecuteSqlMeta
+ *
+ * Structural and pagination metadata for an `execute-sql` result — enough for an agent to
+ * reason about the shape of the result without scanning the rows. The pagination fields
+ * (`page`, `page_size`, `total_count`, `total_pages`) mirror `PaginationMeta`; their values
+ * come from the Lotus window result, not Flop.
+ *
+ */
+type ExecuteSqlMeta = {
+  /**
+   * Result column names, in order
+   */
+  columns: Array<string>;
+  /**
+   * SQL command tag (e.g. `SELECT`)
+   */
+  command?: string | null;
+  /**
+   * Query execution time in milliseconds
+   */
+  duration_ms?: number | null;
+  /**
+   * Rows returned in this page
+   */
+  num_rows: number;
+  /**
+   * 1-based page number
+   */
+  page: number;
+  /**
+   * Rows per page actually applied (after capping)
+   */
+  page_size: number;
+  /**
+   * Total rows across all pages (null if uncounted)
+   */
+  total_count: number | null;
+  /**
+   * Total pages (null if uncounted)
+   */
+  total_pages: number | null;
+};
 /**
  * UpdatePageRequest
  *
@@ -1925,6 +1983,31 @@ type DataSourceRequest = {
    */
   uri: string;
 };
+/**
+ * ExecuteSqlRequest
+ *
+ * A read-only SQL statement to execute against the datalake, the data access mode, and
+ * optional Flop-inspired pagination. `page_size` is capped server-side.
+ *
+ */
+type ExecuteSqlRequest = {
+  /**
+   * Which datalake schema to query: `unregulated` (tokenized) or `regulated` (raw)
+   */
+  mode: 'regulated' | 'unregulated';
+  /**
+   * 1-based page number
+   */
+  page?: number | null;
+  /**
+   * Rows per page (capped at the server's default page size)
+   */
+  page_size?: number | null;
+  /**
+   * Read-only SQL statement to execute
+   */
+  sql: string;
+};
 /**
  * ToolManualUploadRequest
  */
@@ -3212,6 +3295,21 @@ type IngestFileRequest = {
    */
   key: string;
 };
+/**
+ * ExecuteSqlResponse
+ *
+ * Read-only SQL result. `data` is the page of rows as an array-of-arrays (tabular, since
+ * arbitrary SQL can have duplicate or expression column names that object keys would
+ * collapse); the column names and pagination live in `meta`.
+ *
+ */
+type ExecuteSqlResponse = {
+  /**
+   * Page of result rows; each row is an array of cell values aligned to `meta.columns`
+   */
+  data: Array<Array<unknown>>;
+  meta: ExecuteSqlMeta;
+};
 /**
  * ToolCloudWatchLogGroupResponse
  */
@@ -3438,6 +3536,32 @@ type ConnectedAppResponse = {
     url: string;
   }>;
 };
+/**
+ * TextToSqlResponse
+ *
+ * Generated SQL for a natural-language prompt. `explanation` is a best-effort plain-language
+ * description of the SQL (`null` if the explainer was unavailable). The SQL is returned for
+ * review/editing; run it via `POST .../execute-sql`.
+ *
+ */
+type TextToSqlResponse = {
+  /**
+   * Plain-language explanation of what the SQL does; null when unavailable
+   */
+  explanation: string | null;
+  /**
+   * The model that produced the SQL (e.g. `anthropic:claude-opus-4`)
+   */
+  model: string;
+  /**
+   * The provider that produced the SQL (e.g. `anthropic`, `ollama`)
+   */
+  provider: string;
+  /**
+   * The generated SQL statement
+   */
+  sql: string;
+};
 /**
  * AgenticWorkflowListResponse
  *
@@ -6170,6 +6294,20 @@ declare function _buildApi(myClient: Client): {
       request: Request;
       response: Response;
     }>;
+    textToSql: (tenantSlug: string, datalakeSlug: string, body: TextToSqlRequest) => Promise<{
+      data: TextToSqlResponse;
+      request: Request;
+      response: Response;
+    }>;
+    executeSql: (tenantSlug: string, datalakeSlug: string, body: ExecuteSqlRequest, options?: {
+      format?: "csv";
+    }) => Promise<Omit<Awaited<Promise<{
+      data: ExecuteSqlResponse;
+      request: Request;
+      response: Response;
+    }>>, "data"> & {
+      data: ExecuteSqlResponse | string;
+    }>;
   };
   dataSources: {
     list: (tenantSlug: string, datalakeSlug: string, query?: PlatformApiDataSourceControllerIndexData["query"]) => Promise<{
@@ -6694,5 +6832,5 @@ declare function _buildApi(myClient: Client): {
 //#region src/index.d.ts
 declare function isEnvironmentName(name: string): name is EnvironmentName;
 //#endregion
-export { type ActionStatusUpdaterCloudWatchQueryRequest, type ActionStatusUpdaterResponse, type ActionStatusUpdaterRestCallRequest, ActionType, type AgenticWorkflowListResponse, type AgenticWorkflowRequestWritable, type AgenticWorkflowResponse, type AiAgentResponse, type AlveraApiError, type AlveraClient, type ApiConfig, type ApiDebugConfig, type BatchLogListResponse, type BatchLogResponse, type ConnectedAppListResponse, type ConnectedAppRequestWritable, type ConnectedAppResponse, type CreateActionStatusUpdaterRequest, type CreateAiAgentRequest, type CreateGenericTableRequest, type CreateSessionParams, DEFAULT_ENVIRONMENT, type DataActivationClientListResponse, type DataActivationClientLogListResponse, type DataActivationClientLogResponse, type DataActivationClientRequestWritable, type DataActivationClientResponse, type DataSourceRequest, type DataSourceRequestWritable, type DataSourceResponse, type DatalakeRequestWritable, type DatalakeResponse, type DatasetMetadataOptions, type DatasetSearchOptions, type DatasetSearchResponse, type DownloadUrlResponse, ENVIRONMENTS, type EnvironmentName, type ErrorResponse, type ExecuteActionRequest, type ExecuteActionResponse, type GenericTableColumnRequest, type GenericTableColumnResponse, type GenericTableResponse, type IngestFileRequest, type IngestRequest, type InteroperabilityContractAiAgentRequestWritable, type InteroperabilityContractListResponse, type InteroperabilityContractRequestWritable, type InteroperabilityContractResponse, type InteroperabilityRunRequest, type InteroperabilityRunResponse, type MdmVerifyRequest, type MdmVerifyResponse, type PaginationMeta, type PlatformApi, type ResolvePageRequest, type RunManuallyRequestWritable, type RunManuallyResponse, type RunWorkflowRequest, type RunWorkflowResponse, type SessionResponse, type SessionResult, type SyncRoutesResponse, type TemplateConfig, type TenantListResponse, type TenantResponse, ToolIntent, type ToolRequest, type ToolRequestWritable, type ToolResponse, type UpdatePageRequest, type UploadLinkRequest, type UploadLinkResponse, type WorkflowAiAgentRequestWritable, type WorkflowLogListResponse, type WorkflowLogResponse, createIsolatedPlatformApi, createPlatformApi, createSession, isEnvironmentName, revokeSession };
+export { type ActionStatusUpdaterCloudWatchQueryRequest, type ActionStatusUpdaterResponse, type ActionStatusUpdaterRestCallRequest, ActionType, type AgenticWorkflowListResponse, type AgenticWorkflowRequestWritable, type AgenticWorkflowResponse, type AiAgentResponse, type AlveraApiError, type AlveraClient, type ApiConfig, type ApiDebugConfig, type BatchLogListResponse, type BatchLogResponse, type ConnectedAppListResponse, type ConnectedAppRequestWritable, type ConnectedAppResponse, type CreateActionStatusUpdaterRequest, type CreateAiAgentRequest, type CreateGenericTableRequest, type CreateSessionParams, DEFAULT_ENVIRONMENT, type DataActivationClientListResponse, type DataActivationClientLogListResponse, type DataActivationClientLogResponse, type DataActivationClientRequestWritable, type DataActivationClientResponse, type DataSourceRequest, type DataSourceRequestWritable, type DataSourceResponse, type DatalakeRequestWritable, type DatalakeResponse, type DatasetMetadataOptions, type DatasetSearchOptions, type DatasetSearchResponse, type DownloadUrlResponse, ENVIRONMENTS, type EnvironmentName, type ErrorResponse, type ExecuteActionRequest, type ExecuteActionResponse, type ExecuteSqlMeta, type ExecuteSqlRequest, type ExecuteSqlResponse, type GenericTableColumnRequest, type GenericTableColumnResponse, type GenericTableResponse, type IngestFileRequest, type IngestRequest, type InteroperabilityContractAiAgentRequestWritable, type InteroperabilityContractListResponse, type InteroperabilityContractRequestWritable, type InteroperabilityContractResponse, type InteroperabilityRunRequest, type InteroperabilityRunResponse, type MdmVerifyRequest, type MdmVerifyResponse, type PaginationMeta, type PlatformApi, type ResolvePageRequest, type RunManuallyRequestWritable, type RunManuallyResponse, type RunWorkflowRequest, type RunWorkflowResponse, type SessionResponse, type SessionResult, type SyncRoutesResponse, type TemplateConfig, type TenantListResponse, type TenantResponse, type TextToSqlRequest, type TextToSqlResponse, ToolIntent, type ToolRequest, type ToolRequestWritable, type ToolResponse, type UpdatePageRequest, type UploadLinkRequest, type UploadLinkResponse, type WorkflowAiAgentRequestWritable, type WorkflowLogListResponse, type WorkflowLogResponse, createIsolatedPlatformApi, createPlatformApi, createSession, isEnvironmentName, revokeSession };
 //# sourceMappingURL=index.d.mts.map