@alvera-ai/platform-sdk 0.14.0 → 0.15.0

package/.agent/action_status_updaters.md

@@ -117,6 +117,10 @@ scope — do not loop.
 {"external_id": "{{ notification.messageId }}", "status": "delivered"}
 ```
 
+> **Shape only — do not copy this mapping.** A hardcoded `"delivered"`
+> records every event as delivered, failures included. Real bodies
+> branch on the provider's status; see the SNS and MMS sections below.
+
 It must render a **flat** JSON object:
 
 ```
@@ -139,15 +143,107 @@ events for messages this updater doesn't own.
 Custom Liquid filters (`json_escape`, `e164`, `to_json`, ...) are
 available here, as everywhere.
 
-### MMS delivery tracking uses a system `message_config` template
+### SMS-over-SNS delivery tracking — two `custom` bodies
+
+The SNS SMS delivery receipt is **binary and single-shot**: one terminal
+outcome, `status` of `SUCCESS` or `FAILURE`. Its event shape is
+**disjoint from EUM's** — `status`, `notification.messageId`,
+`notification.timestamp`, `delivery.phoneCarrier`,
+`delivery.providerResponse` all sit at the root. Do not carry an EUM
+template over to SNS or vice versa.
+
+Two convenient consequences, both unlike MMS:
+
+- `delivered_at` is `notification.timestamp` — a UTC
+  `"YYYY-MM-DD HH:MM:SS.sss"` string Ecto casts directly, so **no date
+  filter is needed** (EUM needs `from_unix_ms` on its Unix-ms field).
+- the DLR **does** surface the US carrier as `delivery.phoneCarrier`.
+
+**`message_config`:**
+
+```liquid
+{"external_id": "{{ notification.messageId }}", "status": "{% if status == 'SUCCESS' %}delivered{% else %}failed{% endif %}", "status_description": "{{ delivery.providerResponse | json_escape }}"{% if status == 'SUCCESS' and notification.timestamp %}, "delivered_at": "{{ notification.timestamp }}"{% endif %}{% unless status == 'SUCCESS' %}, "failure_reason": "{{ status | json_escape }}"{% endunless %}{% if delivery.phoneCarrier and delivery.phoneCarrier != '' %}, "sms_carrier": "{{ delivery.phoneCarrier | json_escape }}"{% endif %}}
+```
+
+**`action_log_config`:**
+
+```liquid
+{"external_id": "{{ notification.messageId }}", "status": "{% if status == 'SUCCESS' %}delivered{% else %}failed{% endif %}", "status_description": "{{ delivery.providerResponse | json_escape }}"}
+```
+
+`status_description` is `delivery.providerResponse` verbatim on both
+sides — the carrier's own text, on success and failure alike. Because
+SNS's status is binary, `failure_reason` can only ever be `"FAILURE"`;
+the granular carrier text stays in `status_description`.
+
+**Note how the two SNS bodies differ from each other**: same status
+mapping, fewer emitted fields on the action log (no `delivered_at`,
+`sms_carrier`, or `failure_reason` — those are message concerns). That
+is a *different* reason from MMS's, where the two also disagree on the
+mapping itself. Both channels need two bodies; they need them for
+different reasons.
+
+### MMS delivery tracking needs TWO different templates — author both as `custom`
+
+MMS delivery tracking is a `cloud_watch` updater. Author **both**
+`message_config` and `action_log_config` as `custom` bodies — the two
+render *different* JSON from the same event, so one body cannot serve
+both. This is the single most-missed thing about updaters.
+
+**Why they differ.** A message's status vocabulary includes
+`customer_rejected`; an action log's does not. A recipient or carrier
+rejection (`SPAM` / `BLOCKED` / `CARRIER_BLOCKED`) is a message-level
+outcome — the action *did* deliver — so the action-log render must
+**NO-OP** on those events while the message render records
+`customer_rejected`. The message side also carries `delivered_at`,
+`failure_reason`, and `sms_carrier`; only `status` +
+`status_description` are ever applied to an action log.
+
+AWS End User Messaging (`sms-voice`) event fields: top-level
+`messageId`, `eventType` (`MEDIA_*`), `messageStatus`,
+`messageStatusDescription`, `eventTimestamp` (Unix ms), `isFinal`.
+
+**`message_config`** — map every status, emit the extra fields:
+
+```liquid
+{% case messageStatus %}
+{% when 'DELIVERED' %}{% assign mapped_status = 'delivered' %}
+{% when 'SUCCESSFUL' %}{% assign mapped_status = 'sent' %}
+{% when 'PENDING' %}{% assign mapped_status = 'sent' %}
+{% when 'QUEUED' %}{% assign mapped_status = 'queued' %}
+{% when 'SPAM' %}{% assign mapped_status = 'customer_rejected' %}
+{% when 'BLOCKED' %}{% assign mapped_status = 'customer_rejected' %}
+{% when 'CARRIER_BLOCKED' %}{% assign mapped_status = 'customer_rejected' %}
+{% else %}{% assign mapped_status = 'failed' %}
+{% endcase %}
+{"external_id": "{{ messageId }}", "status": "{{ mapped_status }}", "status_description": "{{ messageStatusDescription | json_escape }}"{% if messageStatus == 'DELIVERED' and eventTimestamp %}, "delivered_at": "{{ eventTimestamp | from_unix_ms }}"{% endif %}{% if mapped_status == 'failed' or mapped_status == 'customer_rejected' %}, "failure_reason": "{{ messageStatus | json_escape }}"{% endif %}{% if carrierName and carrierName != '' %}, "sms_carrier": "{{ carrierName | json_escape }}"{% endif %}}
+```
+
+**`action_log_config`** — the rejection statuses fall through to a
+bare `external_id`, which applies nothing:
+
+```liquid
+{% assign al_status = '' %}{% case messageStatus %}{% when 'DELIVERED' %}{% assign al_status = 'delivered' %}{% when 'SUCCESSFUL' %}{% assign al_status = 'sent' %}{% when 'PENDING' %}{% assign al_status = 'sent' %}{% when 'QUEUED' %}{% assign al_status = 'queued' %}{% when 'SPAM', 'BLOCKED', 'CARRIER_BLOCKED' %}{% else %}{% assign al_status = 'failed' %}{% endcase %}{"external_id": "{{ messageId }}"{% if al_status != '' %}, "status": "{{ al_status }}", "status_description": "{{ messageStatusDescription | json_escape }}"{% endif %}}
+```
 
-MMS delivery tracking is wired as a `cloud_watch` updater whose
-`message_config` points at a system-defined template rather than a
-hand-authored `custom` body — the system template lives at
-`status_poller/end_user_messaging/mms_delivery_status`. Everything
-else about the updater is ordinary: it still needs an
-`action_log_config` like every other updater (see below) — the
-system template covers `message_config` only.
+Note the `{% when 'SPAM', 'BLOCKED', 'CARRIER_BLOCKED' %}` arm is
+deliberately **empty** — it leaves `al_status` blank, so the render
+emits `external_id` alone and the action log is untouched.
+
+**Capture every event, not just the terminal one.** 2–3 events arrive
+per message — interim `MEDIA_QUEUED` / `MEDIA_PENDING` /
+`MEDIA_SUCCESSFUL` (`isFinal: false`), then one terminal event
+(`isFinal: true`), up to 72h late and out of order. Apply is
+idempotent and a monotonic guard only ever advances `status`, so a
+late or interim lower-rank event can never demote a message that
+already reached a higher state. Poll a window that overlaps
+deliberately; nothing is dropped.
+
+`queued_at` and `sent_at` are **your outbound pipeline's** times, set
+when the platform enqueues and dispatches — a delivery poll never
+overwrites them. Only `delivered_at` comes from the carrier event
+(`DELIVERED`'s own `eventTimestamp` via `from_unix_ms`), so
+re-polling a wide window does not drift it.
 
 ### Carrier information is available for SMS, not MMS
 
@@ -160,12 +256,15 @@ events — not something the platform chooses to drop:
   under `delivery.phoneCarrier` (e.g. `"T-mobile USA Inc."`, with
   numeric `mcc`/`mnc`). The SNS status template maps it to
   `sms_carrier`.
-- **MMS via AWS End User Messaging (`sms-voice`)** — the delivery
-  event has no carrier field. Verified on **both** a config-set
-  CloudWatch destination and the EventBridge path: the terminal
-  `MEDIA_DELIVERED` event exposes only `totalCarrierFee` (a cost),
-  never `carrierName`/`mcc`/`mnc`. So `sms_carrier` stays unset for
-  MMS.
+- **MMS via AWS End User Messaging (`sms-voice`)** — the event *can*
+  carry `carrierName`, but it is DLR-populated and comes back
+  **absent for US destinations** (the US DLR does not surface it),
+  unlike the SNS path's `phoneCarrier`, whose US DLR does. So map it
+  when present — `{% if carrierName and carrierName != '' %}` — and
+  expect `sms_carrier` to stay null for US long-code traffic.
+  Verified against real AWS delivery events: the terminal
+  `MEDIA_DELIVERED` event for a US destination exposes
+  `totalCarrierFee` (a cost) and no `carrierName`.
 
 AWS's Nov-2024 EventBridge-integration announcement states MMS
 delivery events carry carrier information in EventBridge, but the

package/.agent/cookbook/birthday-greeting-sms-trigger.md

@@ -175,7 +175,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
       position: 0,
       trigger_template: TRIGGER_TEMPLATE,
       idempotency_template:
-        '{{ subject_id }}-{{ workflow_id }}-{{ action_id }}-{{ "" | uuid }}',
+        '{{ subject_id }}-{{ workflow_id }}-{{ action_id }}',
       connected_app_id: connectedAppId,
       connected_app_route: '/forms/birthday-greeting',
       connected_app_metadata_template:

package/.agent/cookbook/contact-us-triage-with-llm.md

@@ -326,7 +326,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
     tool_id: toolId,
     position: 0,
     trigger_template: 'now',
-    idempotency_template: `{{ subject_id }}-{{ action_id }}-${bucket}-{{ "" | uuid }}`,
+    idempotency_template: `{{ subject_id }}-{{ action_id }}-${bucket}`,
     tool_call: {
       tool_call_type: 'sms_request',
       to: { type: 'custom', body: '+15551234567' },

package/.agent/cookbook/dunning-sms-for-delinquent.md

@@ -189,7 +189,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
       decision_key: DECISION_KEY,
       position: 0,
       trigger_template: 'now',
-      idempotency_template: '{{ customer_id }}-{{ decision_key }}-{{ "" | uuid }}',
+      idempotency_template: '{{ customer_id }}-{{ decision_key }}',
       connected_app_id: connectedAppId,
       connected_app_route: '/portal/pay',
       connected_app_metadata_template:

package/.agent/cookbook/kyc-notification-on-account-activation.md

@@ -188,7 +188,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
       decision_key: DECISION_KEY,
       position: 0,
       trigger_template: 'now',
-      idempotency_template: '{{ payment_account.id }}-{{ decision_key }}-{{ "" | uuid }}',
+      idempotency_template: '{{ payment_account.id }}-{{ decision_key }}',
       connected_app_id: connectedAppId,
       connected_app_route: '/portal/kyc',
       connected_app_metadata_template: '{"payment_account_id":"{{ payment_account.id }}"}',

package/.agent/cookbook/paginated-restapi-poller.md

@@ -128,7 +128,17 @@ try {
     updater_tool_id: ctx.restToolId,
     sender_tool_ids: [ctx.senderToolId],
     datalake_id: ctx.datalakeId,
-    events_output_schema: { type: 'array' },
+    // FLOOR the platform enforces: an array whose items are objects listing
+    // `external_id` in `required`. A bare `{ type: 'array' }` is a 422 at
+    // create — every event has to name the message it reconciles.
+    events_output_schema: {
+      type: 'array',
+      items: {
+        type: 'object',
+        required: ['external_id'],
+        properties: { external_id: { type: 'string' } },
+      },
+    },
     pagination_context_output_schema: {
       type: 'object',
       required: ['has_next'],
@@ -140,7 +150,18 @@ try {
       // Static on both — the cursor is captured below and never read back.
       path: { type: 'custom', body: '/wiremock.domain/events' },
       params: { type: 'custom', body: '{"event": "delivered"}' },
-      events_template: { type: 'custom', body: '{{ response.items | to_json }}' },
+      // Providers name their own id — `message-id` here, `messageId` / `sid`
+      // elsewhere. The events_template is where that becomes `external_id`:
+      // the apply path POPS that key off every rendered event to find the row
+      // it updates, so the render PROJECTS each event rather than passing the
+      // provider body through untouched.
+      events_template: {
+        type: 'custom',
+        body:
+          '[{% for item in response.items %}' +
+          '{"external_id": "{{ item.message.headers[\'message-id\'] }}", "event": "{{ item.event }}"}' +
+          '{% unless forloop.last %},{% endunless %}{% endfor %}]',
+      },
       pagination_context_template: {
         type: 'custom',
         body:
@@ -150,11 +171,11 @@ try {
     },
     message_config: {
       type: 'custom',
-      body: '{"external_id": "{{ message.headers.message-id }}", "status": "delivered"}',
+      body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
     },
     action_log_config: {
       type: 'custom',
-      body: '{"external_id": "{{ message.headers.message-id }}", "status": "delivered"}',
+      body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
     },
   })
 } catch (err) {
@@ -182,7 +203,14 @@ const { data: asu } = await api.actionStatusUpdaters.create(tenantSlug, datalake
   updater_tool_id: ctx.restToolId,
   sender_tool_ids: [ctx.senderToolId],
   datalake_id: ctx.datalakeId,
-  events_output_schema: { type: 'array' },
+  events_output_schema: {
+    type: 'array',
+    items: {
+      type: 'object',
+      required: ['external_id'],
+      properties: { external_id: { type: 'string' } },
+    },
+  },
   pagination_context_output_schema: {
     type: 'object',
     required: ['has_next'],
@@ -201,7 +229,16 @@ const { data: asu } = await api.actionStatusUpdaters.create(tenantSlug, datalake
       type: 'custom',
       body: '{% unless msg.pagination_context %}{"event": "delivered"}{% endunless %}',
     },
-    events_template: { type: 'custom', body: '{{ response.items | to_json }}' },
+    // The provider's own id (`message-id`) becomes `external_id` HERE — the
+    // apply path pops that key to find the row it reconciles, so project each
+    // event instead of passing the provider body through untouched.
+    events_template: {
+      type: 'custom',
+      body:
+        '[{% for item in response.items %}' +
+        '{"external_id": "{{ item.message.headers[\'message-id\'] }}", "event": "{{ item.event }}"}' +
+        '{% unless forloop.last %},{% endunless %}{% endfor %}]',
+    },
     pagination_context_template: {
       type: 'custom',
       body:
@@ -211,11 +248,11 @@ const { data: asu } = await api.actionStatusUpdaters.create(tenantSlug, datalake
   },
   message_config: {
     type: 'custom',
-    body: '{"external_id": "{{ message.headers.message-id }}", "status": "delivered"}',
+    body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
   },
   action_log_config: {
     type: 'custom',
-    body: '{"external_id": "{{ message.headers.message-id }}", "status": "delivered"}',
+    body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
   },
 })
 actionStatusUpdaterId = asu.id!
@@ -298,6 +335,23 @@ config and wait for another tick.
   `events_output_schema` on every cycle; the pagination render is validated
   against `pagination_context_output_schema` (which must require `has_next`).
   Both schemas are REQUIRED for `restapi` updaters — blank is a 422.
+- **Each schema has a FLOOR the platform enforces, above which the contract is
+  yours.** `events_output_schema` must describe an **array whose `items` are
+  objects listing `external_id` in `required`**; a bare `{ type: 'array' }` is a
+  422 at create. `pagination_context_output_schema` must be an **object listing
+  `has_next` in `required`** — that key is what ends the page loop. Demand more
+  of your provider on top if you like; only the floor is checked.
+- **The floor exists because `external_id` is how reconciliation finds the row.**
+  `apply_status_update` pops that key off every rendered event, so mapping the
+  provider's own id (`message-id` / `messageId` / `sid`) into `external_id` is
+  the **`events_template`'s job** — project each event, never pass the provider
+  body through untouched. A schema that satisfies the floor while the template
+  emits raw provider rows creates cleanly and then reconciles nothing on every
+  cycle. `message_config` / `action_log_config` then read the **mapped**
+  `{{ external_id }}`, not the provider's original key.
+- **`action_log_config` is required alongside `message_config`** for every
+  updater type, and both are cast against the *same* pinned reconciliation
+  schema — so one template body satisfies both.
 - **A newly created poller is `status: 'active'`.** `cycle_detected` is only
   ever set at runtime by the poll driver — never by a caller; you cannot
   create your way into it.

package/.agent/cookbook/sanctions-screening-with-agent-review.md

@@ -286,7 +286,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
     tool_id: toolId,
     position: 0,
     trigger_template: 'now',
-    idempotency_template: `{{ compliance_screening.id }}-${verdict}-{{ "" | uuid }}`,
+    idempotency_template: `{{ compliance_screening.id }}-${verdict}`,
     tool_call: {
       tool_call_type: 'sms_request',
       to: { type: 'custom', body: '+15550000000' },

package/.agent/cookbook/score-leads-with-llm-categorization.md

@@ -337,7 +337,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
     tool_id: toolId,
     position: 0,
     trigger_template: 'now',
-    idempotency_template: `{{ subject_id }}-{{ action_id }}-${band}-{{ "" | uuid }}`,
+    idempotency_template: `{{ subject_id }}-{{ action_id }}-${band}`,
     tool_call: {
       tool_call_type: 'sms_request',
       to: { type: 'custom', body: '+15551234567' },

package/.agent/cookbook/triage-prospects-by-priority.md

@@ -309,7 +309,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
     tool_id: toolId,
     position: 0,
     trigger_template: 'now',
-    idempotency_template: `{{ customer_id }}-${band}-{{ "" | uuid }}`,
+    idempotency_template: `{{ customer_id }}-${band}`,
     tool_call: {
       tool_call_type: 'sms_request',
       to: { type: 'custom', body: '{{ mdm_output.regulated_customer.phone }}' },

package/.agent/cookbook/welcome-sms-for-customers.md

@@ -182,7 +182,7 @@ const workflowResp = await api.workflows.create(tenantSlug, datalakeSlug, {
       decision_key: DECISION_KEY,
       position: 0,
       trigger_template: 'now',
-      idempotency_template: '{{ customer_id }}-{{ decision_key }}-{{ "" | uuid }}',
+      idempotency_template: '{{ customer_id }}-{{ decision_key }}',
       connected_app_id: connectedAppId,
       connected_app_route: '/portal/welcome',
       connected_app_metadata_template: '{"customer_id":"{{ mdm_output.customer.id }}"}',

package/.agent/datalakes.md

@@ -240,7 +240,7 @@ a hostname pattern; values like `"invalid_host!"` (with
 non-hostname punctuation) are rejected.
 
 This is the only field-format constraint surfaced before the
-synchronous reachability probe runs (see §6 Gotcha 9) — a
+synchronous reachability probe runs (see §8 Gotcha 9) — a
 malformed identifier never gets as far as a connection attempt.
 
 ### Schema names must be unique within a database
@@ -576,7 +576,7 @@ agent-facing markdown, or the whole-domain catalog can be fetched
 in one shot via `api.datasets.metadata(tenantSlug, datalakeSlug)`. Pair `.systemDatasets()` with a
 `page_size: 1` probe per name as a post-migration smoke (proves
 each industry-built table is queryable; the post-ready probe
-code is in §6 Gotcha 3).
+code is in §8 Gotcha 3).
 
 ```typescript
 const { data: catalog } = await api.datalakes.systemDatasets(
@@ -641,7 +641,48 @@ const { data: csv } = await api.datalakes.executeSql(
 ) // csv is a string, not the JSON envelope
 ```
 
-## 6. Gotchas
+## 6. Logical name → physical table
+
+The name you author is not the name you query. Datasets are addressed
+**logically** everywhere in a manifest or SDK call, and **physically** in SQL:
+
+| You author | You query in SQL | Schema |
+|---|---|---|
+| `message` | `regulated_messages` | `app_regulated` |
+| `action_log` | `action_logs` | `app_unregulated` |
+| a generic table `clients` | `regulated_alvera_custom_clients` | `app_regulated` |
+
+Custom (generic) tables carry an `alvera_custom_` prefix, and the regulated side
+adds a further `regulated_` prefix on top. The schema follows `execute-sql`'s
+`--mode` (`regulated` → `app_regulated`, `unregulated` → `app_unregulated`).
+
+**Don't derive these by hand — read them.** Guessing column and table names is
+how six turns get spent on `column "error_message" does not exist` (the real
+column is `failure_reason`):
+
+- `alvera get-metadata dataset --only <type>` prints the **physical dataset
+  name** plus every column, from the Ecto struct itself.
+- `alvera get-metadata generic-table --only <slug>` prints the exact **primary
+  table alias** to use in a `WHERE` clause, with a worked `SELECT`.
+
+An `execute-sql` failure on an undefined column, relation, table, or schema now
+names these verbs in its error, so the correction path is one command away.
+
+## 7. The roster owns the dedupe uri — don't inject it per source
+
+MDM matches on the full identification triple, so a `source_uri` injected
+per data source participates in matching. Two sources describing the same
+subject then produce two different triples, and the same subject is written
+**twice** instead of being reconciled onto one entity.
+
+Derive the dedupe uri **once, at the roster**, and let every data source inherit
+it. Never compute a per-source `source_uri` and expect MDM to see through it.
+
+This divergence is **not visible through `get-metadata`**: the roster value and
+the per-source values each look well-formed in isolation, and only the duplicate
+rows downstream reveal the mismatch. Check the derivation, not the metadata.
+
+## 8. Gotchas
 
 1. **`create()` does NOT auto-enqueue migration.** The create response
    returns immediately with `status: 'new'`; Datalake-DB-resident

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@alvera-ai/platform-sdk",
-  "version": "0.14.0",
+  "version": "0.15.0",
   "description": "Typed SDK for the Alvera platform API — manage data sources, tools, generic tables, AI agents, and action status updaters.",
   "type": "module",
   "sideEffects": false,