@alvera-ai/platform-sdk 0.10.0-rc.2 → 0.10.0-rc.21

package/.agent/debugging.md

@@ -0,0 +1,137 @@
+# Debugging
+
+Tracing HTTP traffic through the SDK is opt-in via the
+`ApiDebugConfig` callback interface. Wire it at client
+construction time and the SDK's HTTP interceptor will log
+every request + response through your callback.
+
+**Safety first**: enabling debug logging without redaction
+will leak credentials in the request bodies the SDK sends to
+the platform. The `ApiDebugConfig.redactStrings` array is the
+load-bearing safety feature — pass every active secret literal
+you've installed BEFORE the SDK fires its first request.
+
+## Wiring the debug interceptor
+
+```typescript
+import {
+  createIsolatedPlatformApi,
+  type ApiDebugConfig,
+} from '@alvera-ai/platform-sdk'
+
+const debugConfig: ApiDebugConfig = {
+  log: (msg: string) => process.stderr.write(msg + '\n'),
+  redactStrings: getActiveSecretLiterals(),
+  // e.g. ['AKIA...real-key', 'twilio-real-token', ...]
+}
+
+const api = createIsolatedPlatformApi({
+  baseUrl,
+  sessionToken,
+  debug: debugConfig,
+})
+```
+
+When `debug` is wired:
+- The SDK's HTTP interceptor logs each request + response
+  through the `log` callback. Consumer chooses the sink:
+  stderr, Pino, OpenTelemetry, etc.
+- BEFORE logging, the SDK substitutes every string in
+  `redactStrings` with a fixed-length placeholder
+  (`********`).
+
+When `debug` is omitted (the default), the SDK doesn't log
+HTTP traffic at all — nothing to redact.
+
+## Safety: the load-bearing piece
+
+The SDK never sees secret REFERENCES (`<%= name %>` Eta
+syntax is a consumer-side convention used by some — CLI
+manifests, for instance). By the time a request body reaches the SDK,
+credentials are already substituted to their literal values.
+If you enable debug logging without populating
+`redactStrings`, those literals appear in plaintext in your
+log output — including production logs if `debug` is wired
+in non-dev environments.
+
+The redaction is fixed-length (`********`), NOT length-
+preserving. A length-preserving mask would still leak
+length-as-signal (an AWS access key is 20 characters;
+a 20-char placeholder says "this WAS a 20-char string").
+
+## The ApiDebugConfig contract
+
+```typescript
+type ApiDebugConfig = {
+  log: (message: string) => void
+  redactStrings?: readonly string[]
+}
+```
+
+- `log` is the sink. The SDK emits formatted log messages
+  (request method + URL + body, response status + body) as
+  a single string per HTTP exchange.
+- `redactStrings` is an array of LITERAL values the SDK
+  should scrub from log output. Empty / absent = no
+  redaction (use only when no secrets are in the request
+  bodies — rare).
+
+The SDK's log output is plain strings, not structured. For
+structured logging (per-field JSON, OTel spans, etc.),
+consumers wrap the `log` callback with their own emitter.
+
+## What the SDK does NOT redact
+
+- **Exception bodies thrown by the SDK** (the 422 envelope,
+  etc.) are NOT filtered through `redactStrings`. If your
+  error-rendering path includes the offending request body,
+  redact it yourself before rendering. See `errors.md` for
+  the throw mechanism.
+- **Response bodies that legitimately contain secrets**.
+  Server-derived fields that are credentials (rare but
+  possible) come back un-redacted in the response; the
+  consumer is responsible for not logging them.
+- **Anything OUTSIDE the SDK's HTTP path**. `redactStrings`
+  only filters what the SDK itself writes to `log`. Your own
+  log statements that touch request/response data must
+  redact themselves.
+
+## Defense in depth
+
+`redactStrings` is one layer. Belt-and-suspenders practice:
+- Avoid log statements that interpolate request bodies
+  OUTSIDE the SDK's debug pipeline.
+- Use fixed-length placeholders in your own surfaces (CLI
+  tables, browser consoles, error pages, exception traces).
+- Treat exception bodies as untrusted for log purposes —
+  parse what you need, don't dump the whole thing.
+- Don't ship `debug` enabled in production unless the
+  redact list is provably complete.
+
+## Pattern: snapshotting the redact list
+
+`redactStrings` is read at client construction time
+(`createIsolatedPlatformApi(...)`). If your consumer's
+secret store mutates during the process's lifetime (e.g. a
+new credential gets installed mid-run), you need to
+construct a NEW client to pick up the new redaction string —
+the SDK doesn't observe the source array after construction.
+
+Consumers that hold secrets in a mutable store typically
+build the client lazily (per-request or per-handler) to
+guarantee a fresh snapshot.
+
+## What's in the log line
+
+A typical log message the SDK emits looks roughly like:
+
+```
+[platform-sdk] POST /api/v1/tenants/acme/datalakes/main/tools
+  request body: {"name":"...","intent":"data_exchange",
+    "body":{"tool_body_type":"s3","access_key_id":"********",
+    "secret_access_key":"********",...}}
+  response 201: {"id":"...","slug":"...","name":"...",...}
+```
+
+(Format is plain-string; exact prefix and field ordering are
+SDK-internal and may evolve.)

package/.agent/errors.md

@@ -0,0 +1,196 @@
+# Errors
+
+When you call an SDK method, two things can happen:
+
+1. **Success** — `{ data, meta? }` returned (typed).
+2. **Throw** — the SDK throws the server's response body verbatim after
+   the HTTP call returned a non-2xx status.
+
+The SDK does **not** validate request bodies on the client. Validation is
+server-authoritative: a malformed body is rejected by the platform and
+surfaced to you as a thrown **422** body. There is no synchronous
+client-side validation error to catch — every validation failure comes back
+from the server in the standardized envelope below.
+
+> Driving the platform through the `alvera` **CLI** instead of the SDK? Then
+> `alvera plan` runs a local pre-flight check against the spec and reports
+> errors *before* any HTTP call. That's a CLI feature (it validates the spec
+> locally); it is not SDK runtime behavior. SDK callers get errors from the
+> server.
+
+## The thrown value (post-HTTP)
+
+When the platform returns a non-2xx status, **the SDK throws the parsed
+response body verbatim** as the thrown value — not an `Error` subclass, just
+the body object (decorated with an `_httpStatus` field so you can branch on
+the status). Narrow it with a type guard:
+
+```typescript
+try {
+  const { data } = await api.tools.create(
+    tenantSlug, datalakeSlug, body,
+  )
+} catch (err) {
+  if (
+    typeof err === 'object' && err !== null &&
+    'errors' in err && Array.isArray((err as any).errors)
+  ) {
+    const { errors } = err as {
+      errors: Array<{
+        source: { pointer: string }
+        title?: string
+        detail?: string
+      }>
+    }
+    for (const e of errors) {
+      // e.source.pointer is "/field_name" or "/embed/field"
+    }
+    return
+  }
+  throw err
+}
+```
+
+The thrown body is typed as `AlveraApiError` (exported from the package root)
+for 422s, and the simpler `ErrorResponse` for some other statuses.
+
+## 422 envelope shape
+
+The platform standardizes ALL 422 response bodies on a single shape —
+regardless of which validation layer fired:
+
+```json
+{
+  "errors": [
+    {
+      "source": { "pointer": "/<field_path>" },
+      "title":  "<human-readable summary>",
+      "detail": "<specific reason this field failed>"
+    },
+    ...
+  ]
+}
+```
+
+- `errors` is always an ARRAY of error objects (never a field-keyed map).
+- `source.pointer` is a JSON Pointer (`/field_name` for top-level fields;
+  `/nested/field` for embeds) naming the offending field on the request body.
+- `detail` is the specific reason; `title` is a brief summary.
+- Multiple errors may share the same `source.pointer` if more than one check
+  fired on the same field.
+
+## Two layers behind the standardized 422
+
+The platform validates a request in two layers; both surface through the
+same envelope above:
+
+**Layer 1 — OpenAPI cast (structural)**
+
+Runs at the plug layer (`OpenApiSpex`) before the controller. Catches:
+missing required fields, wrong types, invalid enums, unknown keys, format
+violations (UUID patterns, etc.). This is the *same* OpenAPI schema the SDK's
+types are generated from, so a body that satisfies the TypeScript types
+almost always clears layer 1.
+
+**Layer 2 — server-side semantic validator (post-cast)**
+
+Runs after the structural cast succeeds. Catches: uniqueness, cross-field
+rules (e.g. `auth_method = "iam_role"` combined with conflicting credential
+fields), foreign-key constraints, custom platform validations. Layer-2
+errors are normalized into the same envelope shape as layer 1 — consumers
+don't distinguish layers from the JSON.
+
+Practical consequence: every validation error reaches you as a thrown 422
+body. Read `source.pointer`, fix that field, resubmit.
+
+## Concrete examples
+
+**Layer 1 — missing required fields** (POST `/agentic-workflows` with empty
+body):
+
+```json
+{
+  "errors": [
+    { "source": { "pointer": "/name" },
+      "detail": "Missing field: name" },
+    { "source": { "pointer": "/dataset_type" },
+      "detail": "Missing field: dataset_type" }
+  ]
+}
+```
+
+**Layer 1 — invalid enum value** (`dataset_type: "nonexistent_type"`):
+
+```json
+{
+  "errors": [
+    { "source": { "pointer": "/dataset_type" },
+      "detail": "Invalid value: not in enum" }
+  ]
+}
+```
+
+**Layer 2 — semantic rule** (DELETE on a default-flagged data activation
+client):
+
+```json
+{
+  "errors": [
+    { "source": { "pointer": "/is_default" },
+      "detail": "Default data activation clients cannot be deleted" }
+  ]
+}
+```
+
+**Layer 2 — uniqueness** (POST `/admin/invites` for an already-invited
+email):
+
+```json
+{
+  "errors": [
+    { "source": { "pointer": "/email" },
+      "detail": "is already invited" }
+  ]
+}
+```
+
+## HTTP status codes (under the hood)
+
+The SDK abstracts HTTP responses behind typed methods. Consumers mostly see
+typed data or thrown bodies; the thrown body carries `_httpStatus`, and raw
+status is also visible via the SDK's `--debug`-style interceptor. The
+platform maps:
+
+| Code | When it fires                                                  |
+|------|----------------------------------------------------------------|
+| 200  | GET / PUT / DELETE successful                                   |
+| 201  | POST successful                                                 |
+| 400  | Request malformed (rare; the SDK serializes before send)        |
+| 401  | Missing or invalid Bearer / `x-api-key`                         |
+| 403  | Authenticated but role lacks permission                         |
+| 404  | Resource doesn't exist OR cross-tenant access (404, not 403)    |
+| 422  | Validation failed; envelope above is the thrown body            |
+| 5xx  | Platform-internal failure; safe to retry with backoff           |
+
+## Retry semantics
+
+Treat 5xx as infra noise — retry with exponential backoff. Treat 422 as a
+real consumer-correctable error — fix the body and resubmit. **Never retry a
+422 with the same body.**
+
+## Mapping `source.pointer` back to your TypeScript type
+
+A `source.pointer` like `/regulated_data_db_reader_user` maps directly to the
+`regulated_data_db_reader_user` field on `DatalakeRequestWritable`. For
+nested embeds, the pointer carries the embed path:
+`/unregulated_cloud_storage/access_key_id`.
+
+For polymorphic embeds, check the discriminator field's pointer first — if
+`/tool_body_type` (or `/cloud_storage_type`, etc.) is in the errors array,
+fix that BEFORE chasing other fields on the embed. The wrong discriminator
+value cascades into spurious required-list failures on the wrong branch's
+fields. See `mutations.md` for the discriminator naming convention.
+
+Each per-resource MD names its `<Resource>RequestWritable` TypeScript type —
+the corpus's resource pages are the source of truth for which fields exist on
+which embed.

package/.agent/generic_tables.md

@@ -0,0 +1,351 @@
+# Generic tables
+
+A **generic table** is an operator-defined table whose schema the
+caller authors at create time, distinct from the platform's industry-
+built **system datasets** (see `datalakes.md` §5 `.systemDatasets`).
+The SDK namespace is `api.genericTables`.
+
+Generic tables are **Datalake-DB-resident**: creation triggers an
+async schema-migration job that builds a physical Postgres table
+in the datalake's schema. The parent datalake MUST be
+`status: 'ready'` before the job can run end-to-end — see
+`datalakes.md` §5 Lifecycle.
+
+## 1. Wire shape
+
+```typescript
+import type {
+  GenericTableColumn,
+  GenericTableResponse,
+} from '@alvera-ai/platform-sdk'
+
+const { data: created } = await api.genericTables.create(
+  tenantSlug,
+  datalakeSlug,
+  {
+    title: 'Contact Us Submissions',
+    description: 'Inbound contact-form rows',
+    columns: [
+      {
+        name: 'submission_id',
+        title: 'Submission ID',
+        type: 'string',
+        description: 'Unique submission identifier',
+        is_unique: true,
+        privacy_requirement: 'none',
+      },
+      // ... more columns
+    ],
+  },
+)
+// created.id          — server-derived UUID
+// created.title       === body.title
+// created.name        — server-derived: `alvera_custom_<slug>`
+// created.status      === 'new' | 'processing' (migration not yet done)
+```
+
+The response returns immediately — the row exists in Platform-DB,
+but the *physical* Postgres table does not yet. Consumers MUST
+poll `.get(name)` until `status === 'deployed'` before treating
+the table as queryable. See §5 Lifecycle.
+
+## 2. Rules the type cannot encode
+
+### `name` is server-derived from `title`
+
+Caller supplies `title` (free-form, human-readable);
+server slugifies it and prefixes `alvera_custom_`:
+
+```
+title: 'Contact Us Submissions'
+  ↓ slugify + prefix
+name:  'alvera_custom_contact_us_submissions'
+```
+
+The `name` is what every downstream resource references —
+`.get(name)`, `generic_table_id` foreign keys via the resolved
+row, `datasets.metadataDetails(tenantSlug, datalakeSlug, 'generic_table', { genericTableId })`,
+etc. Never pre-compute `name` client-side; read it back from
+the create response. A server-side slugifier change would flip
+every downstream binding, so specs assert the derived `name`
+directly.
+
+### `columns` is operator-authored schema
+
+Each column declares storage + semantics in one shot:
+
+```
+name                 required string — snake_case identifier
+                                      (physical Postgres column name)
+title                required string — human label (UI / docs)
+type                 required enum   — see "Column types" below
+is_array             optional bool   — column holds an array of `type` values
+description          required string — narrative; surfaces in catalogs
+is_unique            required bool   — Postgres UNIQUE constraint at migration time
+privacy_requirement  required enum   — 'none' | 'tokenize' | 'redact_only'
+```
+
+#### Column types
+
+```
+   string      free-form text
+   integer     whole numbers
+   float       decimal numbers
+   boolean     true / false
+   date        calendar date (no time component)
+   datetime    timestamp with timezone
+   time        time of day (no date component)
+   jsonb       arbitrary JSON document
+```
+
+Each type can be paired with `is_array: true` to model an array
+column (`string[]`, `integer[]`, etc.). The wire enum lives in
+the generated SDK as `GenericTableColumnTypeEnum` — consult it for
+the live set since new types land independently of corpus
+revisions.
+
+#### Privacy × type interactions
+
+`privacy_requirement` selects how the column flows between the
+regulated and unregulated storage tiers (see `datalakes.md` §1
+for the tier split):
+
+- `none` — keeps the raw value in the regulated tier only;
+  the unregulated tier has no column.
+- `tokenize` — stores the raw value in the regulated tier and an
+  opaque stable token in the unregulated tier. AI tools working
+  off the unregulated tier see the token, not the underlying
+  identity.
+- `redact_only` — keeps the raw value in the regulated tier and
+  strips it from the unregulated tier entirely.
+
+Two type-level restrictions the typed validator does not encode:
+
+- **`type: 'jsonb'` rejects `privacy_requirement: 'tokenize'`.**
+  The platform can't tokenize an arbitrary JSON document — there
+  is no canonical "identity" to tokenize. Valid privacy values
+  for jsonb are `'none'` and `'redact_only'` only. Submitting
+  `{ type: 'jsonb', privacy_requirement: 'tokenize' }` returns
+  422 on `/columns/N/privacy_requirement`.
+- **Tokenize widens primitive types at the schema layer.** A
+  `{ type: 'integer', privacy_requirement: 'tokenize' }` column
+  surfaces in the unregulated tier as a string-typed token, not
+  an integer. This is a quirk of how the platform stores tokens
+  (one opaque string format across all source types) — your
+  unregulated-tier queries must treat the column as a string.
+  This affects `integer`, `float`, `boolean`, `is_array string`,
+  `date`, `datetime`, and `time`.
+
+The privacy choice is **immutable post-create** — there is no
+SDK mutation surface for column-level privacy changes.
+
+**Privacy is operator-decided per column, not implied by name.**
+A field called `message` can be `redact_only` in one tenant and
+`none` in another; the SDK doesn't pattern-match column names
+to default privacy levels. Pick deliberately at design time —
+recreating the table is the only way to flip a column's
+treatment later.
+
+### At least one column MUST be `is_unique: true`
+
+The platform requires at least one unique column to identify
+rows for the data activation upsert path. A body with zero
+unique columns returns 422 on `/columns` with a message naming
+the missing-unique-column rule. The unique column is what bound
+data activation clients use as the upsert key.
+
+### Reserved column names
+
+The platform reserves certain column names you cannot redeclare
+in `columns[N].name`:
+
+- **Infrastructure columns** (always reserved, every industry):
+  `id`, `inserted_at`, `updated_at`, `datalake_id`, plus the
+  unregulated/regulated tier-link fields the platform manages.
+- **Industry-specific MDM FK columns** (reserved only inside
+  their owning industry's datalake):
+
+  ```
+     healthcare           patient_id
+     core_banking         party_id
+     accounts_receivable  customer_id
+     (other domains as they land)
+  ```
+
+A `patient_id` column is REJECTED in a healthcare datalake's
+generic table — the platform's MDM machinery owns that name —
+but is ACCEPTED in a `core_banking` datalake's generic table
+(it's just an opaque user-defined column there). Reserved-name
+rejections surface as 422 on `/columns/N/name`.
+
+### Title cannot start with `alvera_system`
+
+The `alvera_system_` prefix is reserved for system-curated
+generic tables the platform manages internally. A `title` whose
+slugified form would start with `alvera_system` is rejected at
+create with 422 on `/title`. Pick a title that slugifies to
+something else (e.g. avoid leading "Alvera System" in the title
+text).
+
+### No update surface
+
+`PUT /generic-tables/:name` is **not** an SDK affordance.
+Generic tables are schema-immutable post-create — to change
+columns, delete + recreate. This is a deliberate constraint:
+mutating a deployed schema mid-ingestion would race the
+data activation client.
+
+## 3. Field ownership
+
+**Server-derived (Response-only).** Universal set from
+`type_naming.md`, plus:
+
+```
+name           string — `alvera_custom_<slugified-title>`
+status         enum   — 'new' | 'processing' | 'deployed' | 'failed'
+```
+
+**Caller-supplied (round-trip).**
+
+```
+title          required string — human-readable label
+description    optional string — narrative
+columns        required array  — column definitions (see §2)
+```
+
+**Write-only (Request-only).** None.
+
+## 4. Error envelopes
+
+Standard JSON:API envelopes per `errors.md`. Common rejections:
+
+| `source.pointer`                 | Cause                                              |
+|----------------------------------|----------------------------------------------------|
+| `/title`                         | empty; generates a name colliding with existing; or starts with `alvera_system` prefix |
+| `/columns`                       | empty array; duplicate `column.name`; or zero columns marked `is_unique: true` |
+| `/columns/0/name`                | reserved column name (infrastructure column or the industry's MDM FK; see §2) |
+| `/columns/0/type`                | value not in supported type enum (see §2 column types)   |
+| `/columns/0/privacy_requirement` | value not in enum, OR `tokenize` on a `jsonb` column   |
+
+All of these come back as a 422 `AlveraApiError` thrown from the
+SDK on submit — structural cases (Layer 1) and semantic conflicts
+like name collision / deployed-name reuse (Layer 2). The SDK does
+not validate client-side; see `errors.md`.
+
+## 5. Lifecycle
+
+Two-phase create, parallel to datalakes (see `datalakes.md` §5):
+
+```
+   .create  ──→  status: 'new' or 'processing'
+                      │
+                      │  (async migration job — see async.md)
+                      ▼
+                 status: 'deployed'   ◄── safe to bind / query
+```
+
+### Create + poll
+
+```typescript
+const { data: created } = await api.genericTables.create(
+  tenantSlug, datalakeSlug, body,
+)
+// created.status === 'new'  (or 'processing' if worker raced)
+
+while (true) {
+  const { data: row } = await api.genericTables.get(
+    tenantSlug, datalakeSlug, created.name,
+  )
+  if (row.status === 'deployed') break
+  if (row.status === 'failed') throw new Error('migration failed')
+  await new Promise((r) => setTimeout(r, 1_000))
+}
+```
+
+`.get(name)` distinguishes "row never created" (404) from
+"deployment in progress" (200 with status sub-`deployed`) —
+two distinct signals on the same endpoint. Use the 404 to
+short-circuit retry loops; use the 200+status to drive the
+poll above.
+
+### Read shapes
+
+```
+.list(tenantSlug, datalakeSlug)              Paged: { data, meta }
+.get(tenantSlug, datalakeSlug, name)         One row, by name
+                                              (NOT slug — generic
+                                               tables expose `name`)
+```
+
+Plus the polymorphic dataset surface for per-table markdown:
+
+```typescript
+const { data: md } = await api.datasets.metadataDetails(
+  tenantSlug,
+  datalakeSlug,
+  'generic_table',
+  { genericTableId },
+)
+// md: string — agent-facing markdown describing the table's schema
+```
+
+For the whole-domain catalog (every system + generic table type
+registered to this datalake) use the sibling
+`api.datasets.metadata(tenantSlug, datalakeSlug)` — same shape as
+`tools.metadata` / `dataSources.metadata`.
+
+See `datalakes.md` §5 `.systemDatasets` for the system-dataset
+sibling of this verb; the `'generic_table'` kind is the entry
+point for generic-table metadata.
+
+### Delete
+
+`DELETE` removes the row and drops the physical table. Tables
+bound to active data activation clients, workflows, or
+interoperability contracts reject with a 422 naming the
+dependency. Detach dependents first.
+
+## 6. Gotchas
+
+1. **`.get` takes `name`, not `slug`.** Generic tables expose
+   `name` as the canonical reference; the `slug` field — if
+   present on `GenericTableResponse` — is incidental and
+   currently unused by downstream resources. Stash `data.name`
+   from create; pass it to `.get` and to every downstream
+   `generic_table` binding.
+
+2. **`status` reaches `deployed` asynchronously.** A bare
+   `.create` response with `status: 'new'` is NOT yet bindable.
+   The poll loop above is non-optional for any spec that
+   creates a generic table and immediately uses it. A 60-second
+   timeout floor survives a saturated worker queue on a busy box;
+   bump for automated test environments with slower migrations.
+
+3. **Column `privacy_requirement` is immutable post-create.**
+   `tokenize` columns surface as an opaque stable token in the
+   unregulated tier; `redact_only` strips the value from that
+   tier entirely; `none` keeps the raw value only in the
+   regulated tier. There is no update surface to flip this —
+   recreate the table to change.
+
+4. **System-dataset enumeration EXCLUDES generic tables.**
+   `api.datalakes.systemDatasets(...)` returns only the
+   industry-built name list; operator-defined generic
+   tables surface only via `api.genericTables.list(...)`.
+   The literal string `"generic_table"` is also absent from
+   the system-datasets enum — see `datalakes.md` §5.
+
+5. **Title-collision returns at create time.** Two generic tables
+   cannot share a `title` that slugifies to the same `name`
+   within one datalake. The 422 surfaces on POST, well before
+   the migration job runs.
+
+6. **Generic tables can carry agentic write-back columns.**
+   They are not exclusively ingestion targets. Columns the
+   data activation client fills at ingestion (e.g.
+   `submission_id`, `name`, `email`) sit alongside columns
+   reserved for downstream resources to update (e.g. a reply
+   column written by a workflow, or a scoring column written
+   by an agent). Plan the column set for the full lifecycle,
+   not just the inbound shape — adding columns later requires
+   delete + recreate.