@alvera-ai/platform-sdk 0.17.0 → 0.18.0

package/.agent/errors.md

@@ -202,10 +202,10 @@ body** — the failure won't clear on its own.
 
 ## Mapping `source.pointer` back to your TypeScript type
 
-A `source.pointer` like `/regulated_data_db_reader_user` maps directly to the
-`regulated_data_db_reader_user` field on `DatalakeRequestWritable`. For
+A `source.pointer` like `/db_reader_user` maps directly to the
+`db_reader_user` field on `DatalakeRequestWritable`. For
 nested embeds, the pointer carries the embed path:
-`/unregulated_cloud_storage/access_key_id`.
+`/cloud_storage/access_key_id`.
 
 For polymorphic embeds, check the discriminator field's pointer first — if
 `/tool_body_type` (or `/cloud_storage_type`, etc.) is in the errors array,

package/.agent/generic_tables.md

@@ -38,7 +38,7 @@ in the datalake's schema. The parent datalake MUST be
 
 ```typescript
 import type {
-  GenericTableColumn,
+  GenericTableColumnRequest,
   GenericTableResponse,
 } from '@alvera-ai/platform-sdk'
 
@@ -121,7 +121,7 @@ type                 required enum   — see "Column types" below
 is_array             optional bool   — column holds an array of `type` values
 description          required string — narrative; surfaces in catalogs
 is_unique            required bool   — Postgres UNIQUE constraint at migration time
-privacy_requirement  required enum   — 'none' | 'tokenize' | 'redact_only'
+privacy_requirement  required enum   — 'none' | 'tokenize' | 'redact'
 ```
 
 #### Column types
@@ -129,7 +129,8 @@ privacy_requirement  required enum   — 'none' | 'tokenize' | 'redact_only'
 ```
    string      free-form text
    integer     whole numbers
-   float       decimal numbers
+   float       approximate numeric (IEEE 754)
+   decimal     exact numeric, stored as `numeric`
    boolean     true / false
    date        calendar date (no time component)
    datetime    timestamp with timezone
@@ -137,54 +138,118 @@ privacy_requirement  required enum   — 'none' | 'tokenize' | 'redact_only'
    jsonb       arbitrary JSON document
 ```
 
+`decimal` and `float` are distinct on purpose: `decimal` emits a
+Postgres `numeric` and comes back as an exact Decimal, while `float`
+is binary floating point and cannot load one. Money and anything
+else that must not drift belongs in `decimal`.
+
 Each type can be paired with `is_array: true` to model an array
-column (`string[]`, `integer[]`, etc.). The wire enum lives in
-the generated SDK as `GenericTableColumnTypeEnum` — consult it for
-the live set since new types land independently of corpus
-revisions.
+column (`string[]`, `integer[]`, etc.) — with one exception that has
+its own section below, because it is the single most expensive
+mistake available on this page. See **`jsonb` + `is_array`**.
+
+The wire enum lives in the generated SDK as the `type` field of
+`GenericTableColumnRequest` — consult it for the live set, since new
+types land independently of corpus revisions.
 
 #### Privacy × type interactions
 
-`privacy_requirement` selects how the column flows between the
-regulated and unregulated storage tiers (see `datalakes.md` §1
-for the tier split):
-
-- `none` — keeps the raw value in the regulated tier only;
-  the unregulated tier has no column.
-- `tokenize` — stores the raw value in the regulated tier and an
-  opaque stable token in the unregulated tier. AI tools working
-  off the unregulated tier see the token, not the underlying
-  identity.
-- `redact_only` — keeps the raw value in the regulated tier and
-  strips it from the unregulated tier entirely.
-
-Two type-level restrictions the typed validator does not encode:
-
-- **`type: 'jsonb'` REQUIRES `privacy_requirement: 'redact_only'` —
-  `'none'` is rejected too, not just `'tokenize'`.** The platform
-  can't tokenize an arbitrary JSON document (no canonical
-  "identity" to tokenize), and it won't let an unredacted JSON
-  blob flow into the unregulated tier either. `'redact_only'` is
-  the ONLY valid privacy value for a jsonb column — submitting
-  `{ type: 'jsonb', privacy_requirement: 'tokenize' }` OR
-  `{ type: 'jsonb', privacy_requirement: 'none' }` both return
-  422 on `/columns/N/privacy_requirement`.
+`privacy_requirement` decides what each of the three lakes hands back
+for this column. **All three lakes carry every column** — the copies
+are full copies, not projections — so the question is never whether
+the column is there but what value it holds. (See `datalakes.md` §1
+for the lake split, and note that the tokenized and redacted copies
+exist only after an explicit `provisionTokenization` call.)
+
+Measured against a live three-lake tenant, one row read three ways:
+
+| `privacy_requirement` | raw | tokenized | redacted |
+|---|---|---|---|
+| `none`     | `Family Medicine` | `Family Medicine` | `Family Medicine` |
+| `tokenize` | `Rosalind Achterberg` | `tok_7a1de41f5aba…` | `xxxx-berg` |
+| `redact`   | the real value | masked | masked |
+
+Three things that table is trying to make unmissable:
+
+- **`none` is byte-identical everywhere.** It is not "raw only". A
+  masking policy that altered a `none` column would silently break
+  every join in the system, so these are the columns you filter and
+  join on in every lake.
+- **`tokenize` and `redact` differ in the TOKENIZED lake, not the
+  redacted one.** A tokenized column yields a stable `tok_<hash>` —
+  the same input gives the same token, so a machine can join across
+  tables on it without ever resolving it. A redacted column yields an
+  unjoinable placeholder. That is the whole distinction: **`tokenize`
+  preserves referential identity without disclosing it; `redact`
+  destroys both.**
+- **The redacted lake masks both.** `xxxx-<tail>` is for a person
+  glancing at a support screen, and it is deliberately useless as a
+  key.
+
+There is **no restriction tying `jsonb` to `redact`.** A jsonb column
+may be `none`, `tokenize` or `redact` like any other. An older rule
+did refuse anything but `redact`, because tokenizing a JSON document
+once produced a value the column could not hold; that rule was
+**removed rather than relaxed**, because masking a JSON column is now
+well-defined (see the next section). If you are reading an older copy
+of this corpus that says otherwise, it is describing a 422 that no
+longer happens.
+
+One type-level quirk the typed validator does not encode:
+
 - **Tokenize widens primitive types at the schema layer.** A
   `{ type: 'integer', privacy_requirement: 'tokenize' }` column
-  surfaces in the unregulated tier as a string-typed token, not
+  surfaces in the tokenized tier as a string-typed token, not
   an integer. This is a quirk of how the platform stores tokens
   (one opaque string format across all source types) — your
-  unregulated-tier queries must treat the column as a string.
+  queries against the tokenized lake must treat the column as a string.
   This affects `integer`, `float`, `boolean`, `is_array string`,
   `date`, `datetime`, and `time`.
 
+#### `jsonb` + `is_array` — the rule that is not what it looks like
+
+Declaring `{ type: 'jsonb', is_array: true }` is how you model a
+column holding a **JSON array** — a list of objects, say. It is
+required alongside `type: 'jsonb'` when the template renders a list:
+without it the column is a single JSON document, and each row
+carrying an array is refused with
+`upsert_row_failed %{<column>: ["is invalid"]}` **while the batch
+reports itself green**.
+
+**`is_array: true` describes the JSON document. It is NOT an
+instruction to make the Postgres column an array type.** That
+misreading is the most expensive one available here, because of what
+it implies for a masked column.
+
+A masked `jsonb` column is `jsonb` in the raw lake and **`text` in
+both copies**, and those types are *supposed* to differ. Logical
+replication ships the publisher's text rendering and feeds it to the
+subscriber column's input function:
+
+```
+   jsonb  ->  text     always succeeds   (any jsonb renders to a string,
+                                          and any string is valid text)
+   jsonb  ->  text[]   always fails      ([{"code":"A1C"}] is JSON, and
+                                          NOT a Postgres array literal)
+```
+
+So the invariant is **scalar-ness, not type parity**: a jsonb column
+is never a Postgres array type, in any lake. Get this wrong and the
+apply worker stops on the first row carrying an array — and the
+failure has no error surface a client can see. You get derived lakes
+that exist, report `status: 'ready'`, and never fill.
+
+Because the copies hold text, a masked jsonb column comes back as a
+masked *string* rather than as an array, and it will not parse as
+JSON. That is expected: on that side the column really is text.
+
 The privacy choice is **immutable post-create** — `update()` exists
 but its migration is additive-only and never re-privacy's an existing
 column (see §2 update note), so the only way to change a column's
 `privacy_requirement` is `delete()` + `create()`.
 
 **Privacy is operator-decided per column, not implied by name.**
-A field called `message` can be `redact_only` in one tenant and
+A field called `message` can be `redact` in one tenant and
 `none` in another; the SDK doesn't pattern-match column names
 to default privacy levels. Pick deliberately at design time —
 recreating the table is the only way to flip a column's
@@ -203,24 +268,35 @@ data activation clients use as the upsert key.
 The platform reserves certain column names you cannot redeclare
 in `columns[N].name`:
 
-- **Infrastructure columns** (always reserved, every industry):
-  `id`, `inserted_at`, `updated_at`, `datalake_id`, plus the
-  unregulated/regulated tier-link fields the platform manages.
-- **Industry-specific MDM FK columns** (reserved only inside
-  their owning industry's datalake):
-
-  ```
-     healthcare           patient_id
-     core_banking         party_id
-     subscription  customer_id
-     (other domains as they land)
-  ```
-
-A `patient_id` column is REJECTED in a healthcare datalake's
-generic table — the platform's MDM machinery owns that name —
-but is ACCEPTED in a `core_banking` datalake's generic table
-(it's just an opaque user-defined column there). Reserved-name
-rejections surface as 422 on `/columns/N/name`.
+The platform creates every generic table with the same base columns,
+and they are yours to read but never to declare:
+
+```
+   id             uuid primary key
+   inserted_at    timestamp
+   updated_at     timestamp
+   batch_id       the ingest run that wrote the row
+   client_id      the activation client that wrote it
+   legal_entity_id the subject the contract's MDM resolve stamped
+```
+
+**`legal_entity_id` is the one that catches people.** It is not a
+column you declare and fill — a contract's `mdm_input_config`
+resolves the subject and the platform stamps it. Declaring a column
+of that name does not shadow the stamp silently: the create is
+REJECTED with 422, and the message names the offending column
+(`contain reserved system column names: legal_entity_id`). You lose
+the create, not the linkage — so read the 422 as the platform telling
+you the column is already there, and drop it from your definition.
+There is correspondingly **nothing to guard against in a contract
+hook**: a table that declares the column is never created, so an
+`assert` on its absence can never fire.
+
+There is no longer a per-domain reserved list. GH-859 removed the
+industry datalakes and their MDM foreign keys, so names like
+`patient_id`, `party_id` and `customer_id` are now ordinary
+user-defined columns you may declare freely. Reserved-name rejections
+surface as 422 on `/columns/N/name`.
 
 ### Title cannot start with `alvera_system`
 
@@ -240,7 +316,7 @@ reconcile drift. On update the platform re-runs the schema migration
 for the new column set — but the migration is **strictly additive**:
 
 - **New columns in the body are added** to the physical table
-  (test-pinned) and to the regulated mirror.
+  (test-pinned) and to each lake's copy of it.
 - **Columns you omit are NOT dropped.** The migration never
   auto-drops a column ("keep down as noop to avoid destructive
   change"), so a shrunken `columns` array does not remove storage.
@@ -285,11 +361,25 @@ Standard JSON:API envelopes per `errors.md`. Common rejections:
 | `source.pointer`                 | Cause                                              |
 |----------------------------------|----------------------------------------------------|
 | `/title`                         | empty; generates a name colliding with existing; or starts with `alvera_system` prefix |
-| `/columns`                       | empty array; duplicate `column.name`; or zero columns marked `is_unique: true` |
-| `/columns/0/name`                | reserved column name (infrastructure column or the industry's MDM FK; see §2) |
+| `/columns`                       | empty array; duplicate `column.name`; zero columns marked `is_unique: true`; or a RESERVED column name (see §2) |
+| `/columns/0/name`                | too short/long, or not matching `[a-z][a-z0-9_]*`         |
 | `/columns/0/type`                | value not in supported type enum (see §2 column types)   |
 | `/columns/0/privacy_requirement` | value not in enum, OR `tokenize` on a `jsonb` column   |
 
+**Reserved names point at `/columns`, not at the offending index** —
+measured, not inferred. The check runs on the parent changeset
+(`add_error(changeset, :columns, …)`), so it names *every* offending
+column at once rather than the first it meets:
+
+```
+alvera: plan halted at generic_table "lead-submissions": server rejected request:
+  /columns: contain reserved system column names: legal_entity_id
+```
+
+Note where that halted: at **plan**, not apply — the server answering
+during plan's refresh, before any mutation. The per-column rows above
+are indexed because those validations run on the column embed itself.
+
 All of these come back as a 422 `AlveraApiError` thrown from the
 SDK on submit — structural cases (Layer 1) and semantic conflicts
 like name collision / deployed-name reuse (Layer 2). The SDK does
@@ -318,7 +408,7 @@ const { data: created } = await api.genericTables.create(
 const deadline = Date.now() + 60_000
 while (Date.now() < deadline) {
   const { data: row } = await api.genericTables.get(
-    tenantSlug, datalakeSlug, created.id,          // UUID id, NOT created.name
+    tenantSlug, datalakeSlug, created.id!,         // UUID id, NOT created.name
   )
   if (row.status === 'deployed') break
   await new Promise((r) => setTimeout(r, 1_000))
@@ -331,7 +421,7 @@ definition drifted; never a fresh-create state). A migration that
 fails does not flip to a `failed` status; the row simply never reaches
 `deployed`, so bound the poll with a **timeout** (as above) rather than
 watching for a failure value. (This is the same timeout-bounded idiom
-the validated `cookbook/generic-tables.md` runs, passing the UUID
+every validated cookbook runs, passing the UUID
 `id`.) `.get(id)` distinguishes "row never
 created" (404) from "deployment in progress" (200 with a
 sub-`deployed` status) — two distinct signals on the same endpoint.
@@ -409,11 +499,10 @@ dependency. Detach dependents first.
 
 3. **Column `privacy_requirement` is immutable post-create.**
    `tokenize` columns surface as an opaque stable token in the
-   unregulated tier; `redact_only` strips the value from that
-   tier entirely; `none` keeps the raw value only in the
-   regulated tier. `update()`'s additive migration never re-privacy's
-   an existing column, so recreate the table to change a column's
-   privacy.
+   tokenized lake as a stable token; `redact` masks it in both
+   copies; `none` passes the real value through to all three
+   unchanged. `update()`'s additive migration never re-privacy's an
+   existing column, so recreate the table to change one.
 
 4. **System-dataset enumeration EXCLUDES generic tables.**
    `api.datalakes.systemDatasets(...)` returns only the

package/.agent/interoperability_contracts.md

@@ -32,12 +32,16 @@ const { data: created } = await api.interoperabilityContracts.create(
   tenantSlug,
   datalakeSlug,
   {
-    name: 'Inbound CRM → Customer',
-    description: 'Vendor CRM rows → Subscription Customer',
-    resource_type: 'customer',
+    name: 'Inbound CRM → Legal Entity',
+    description: 'Vendor CRM rows → the legal entity they identify',
+    resource_type: 'legal_entity',
     filter_template: undefined,                  // optional
     template_config: { type: 'custom', body: liquidBody },
-    mdm_input_config: { type: 'custom', body: mdmBody },
+    // The template IS the subject here, so no entity resolution
+    // runs. A contract that writes a generic table instead pairs
+    // with this one and carries a real mdm_input_config emitting
+    // the SAME (uri, external_id) — see §2 "The contract pair".
+    mdm_input_config: { type: 'null' },
     generic_table_id: null,                      // see §2
   },
 )
@@ -95,16 +99,46 @@ every other `resource_type`, the field must be `null`. The
 TypeScript type permits either value; the validator and server
 enforce the conditional rule.
 
-### `resource_type` is industry-scoped
+### `resource_type` is a closed platform set
+
+GH-859 removed the per-industry vocabularies. Every datalake now
+accepts the same set:
+
+```
+   action_log   beneficial_owner   document
+   generic_table   legal_entity   message
+```
+
+The subject-specific types an industry used to bring — `patient`,
+`customer`, `payment_account`, `compliance_screening` — are gone.
+Model those as a **generic table** and pair it with a
+`legal_entity` contract (see "The contract pair" below).
+
+On the wire `resource_type` is a plain `string`; the set is
+enforced server-side, so an invalid value is a `/resource_type`
+422 at create rather than a compile error. `datalakes.md` §5
+`.systemDatasets` remains the runtime authority.
+
+### The contract pair
+
+One inbound row usually becomes two things, so it needs two
+contracts:
+
+| | `resource_type` | `mdm_input_config` | writes |
+|---|---|---|---|
+| **A** | `legal_entity` | `{ type: 'null' }` | the subject |
+| **B** | `generic_table` (+ `generic_table_id`) | a real template | the row |
+
+Both emit the **same** `(uri, external_id)` identifier pair, so
+they converge on one legal entity and the platform stamps
+`legal_entity_id` onto the generic-table row.
 
-The valid set of `resource_type` values is determined by the
-parent datalake's `data_domain`. Generic concepts like
-`'generic_table'` are accepted everywhere; domain-anchored
-concepts (`'patient'`, `'customer'`, `'payment_account'`,
-`'legal_entity'`, ...) only resolve within their owning industry.
-Don't hardcode a fixed enum across industries — discover the
-valid set at runtime via the platform's domain catalog (see
-`datalakes.md` §5 `.systemDatasets`).
+**Never declare `legal_entity_id` as a column on that table.**
+The name is reserved, so declaring it does not shadow the stamp —
+the create is REJECTED with 422 (`contain reserved system column
+names: legal_entity_id`). You lose the table, not the linkage.
+Drop it from your definition; the platform adds the column itself
+and you read the stamp back off the row.
 
 ### `filter_template` has INVERTED semantics
 
@@ -158,7 +192,7 @@ type           string — mirrors template_config.type at create time
 ```
 name              required string — agent-facing label
 description       optional string — narrative
-resource_type     required string — target shape (industry-scoped)
+resource_type     required string — target shape (closed set, §2)
 filter_template   optional string — Liquid pre-filter (inverted semantics)
 template_config   required embed  — { type, body?, path? }
 mdm_input_config  optional embed  — { type, body?, path? }; omit equals { type: 'null' }
@@ -407,8 +441,8 @@ reject with 422 on `/base` and a literal message:
 System-created contracts include the auto-generated identity
 contracts the platform provisions when a generic table is
 created (see §5 "Generic-table contracts: auto-created identity
-contracts") and any industry-built templates the platform ships
-with the datalake's data_domain. User-created contracts of any
+contracts") and the templates the platform ships with every
+datalake. User-created contracts of any
 `type` (including caller-created `type: 'identity'` rows bound
 to a generic table) ARE editable and deletable — the gate is
 `system_created`, not `type`.
@@ -452,7 +486,7 @@ const { data: contract } = await api.interoperabilityContracts.create(
           body: JSON.stringify({ name: '{{ msg.company_name }}' }),
           // A contract join AUTHORS output_schema — unlike a workflow
           // join, the server does NOT pin it here.
-          output_schema: '{"type":"object","properties":{"lead":{"type":"string"}}}',
+          output_schema: { type: 'object', properties: { lead: { type: 'string' } } },
         },
       },
     ],
@@ -503,11 +537,12 @@ surface, and `workflows.md` for the workflow side of the same join.
    runs". Declaring `{ type: 'null' }` explicitly is allowed
    for readability but not required.
 
-3. **`resource_type` is industry-scoped.** A datalake with
-   `data_domain: 'foundation'` accepts `'legal_entity'` but
-   rejects `'patient'`. The error surfaces as
-   `/resource_type` 422 at create time. Don't ship a fixed
-   enum across industries.
+3. **`resource_type` is a closed platform set.** Every datalake
+   accepts the same six values; the retired per-industry types
+   (`'patient'`, `'customer'`, `'payment_account'`, ...) now
+   reject with a `/resource_type` 422 at create time. Model
+   those as a generic table paired with a `legal_entity`
+   contract instead.
 
 4. **Sandbox-run is not optional in practice.** A template that
    compiles but renders structurally-invalid JSON, or that