@alvera-ai/platform-sdk 0.21.1 → 0.21.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agent/action_status_updaters.md +14 -14
- package/.agent/advanced_migrations.md +8 -0
- package/.agent/async.md +4 -3
- package/.agent/cookbook/organic-marketing.md +307 -31
- package/.agent/cookbook/subscription-saas.md +7 -2
- package/.agent/data_activation_clients.md +79 -33
- package/.agent/generic_tables.md +39 -10
- package/.agent/interoperability_contracts.md +4 -4
- package/.agent/mock-services.md +4 -2
- package/.agent/templates.md +6 -1
- package/.agent/tools.md +16 -8
- package/.agent/workflows.md +60 -2
- package/dist/index.d.mts +64 -17
- package/dist/index.d.mts.map +1 -1
- package/dist/index.mjs +8 -5
- package/dist/index.mjs.map +1 -1
- package/package.json +1 -1
|
@@ -555,24 +555,24 @@ flag:
|
|
|
555
555
|
|
|
556
556
|
```
|
|
557
557
|
last_run_at when the poll worker last made progress
|
|
558
|
-
last_run_status 'ok' | '
|
|
559
|
-
last_run_error why the last run
|
|
558
|
+
last_run_status 'ok' | 'error'
|
|
559
|
+
last_run_error why the last run stopped; null when it read its whole window
|
|
560
560
|
last_run_events_found events fetched so far in the current run
|
|
561
561
|
status 'active' | 'cycle_detected'
|
|
562
562
|
```
|
|
563
563
|
|
|
564
|
-
**A poll run has
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
564
|
+
**A poll run has two outcomes:**
|
|
565
|
+
|
|
566
|
+
- `'ok'` — the run read its entire window. The reconciliation is
|
|
567
|
+
complete.
|
|
568
|
+
- `'error'` — the run stopped; `last_run_error` says why.
|
|
569
|
+
|
|
570
|
+
**There is no partial run.** The poller drains the window as a chain of
|
|
571
|
+
jobs, one page per job, carrying the provider's next-page token into the
|
|
572
|
+
next job, and there is no cap on how many pages it reads — so a run
|
|
573
|
+
either reads every page of its window or stops with an error. A CloudWatch
|
|
574
|
+
poll that is handed the same next-page token twice stops with an error
|
|
575
|
+
naming that page rather than reading it again.
|
|
576
576
|
|
|
577
577
|
`last_run_events_found` is the event count for the current run; because
|
|
578
578
|
`refresh` is asynchronous (below), a read taken before the run finishes can
|
|
@@ -104,6 +104,14 @@ END $$;
|
|
|
104
104
|
The same applies to `down_statement`, which is likelier to be run against a
|
|
105
105
|
database that is not in the state you assumed.
|
|
106
106
|
|
|
107
|
+
### One statement per body
|
|
108
|
+
|
|
109
|
+
The platform runs each body as a single prepared statement. Two commands in one
|
|
110
|
+
`up_statement` are accepted at create and fail at migrate with Postgres `42601`
|
|
111
|
+
`cannot insert multiple commands into a prepared statement`. The two examples
|
|
112
|
+
above are therefore two migrations, not one body. A `DO $$ … $$` block counts as
|
|
113
|
+
one statement, so steps that must travel together go inside one.
|
|
114
|
+
|
|
107
115
|
### The table names are the platform's, not yours
|
|
108
116
|
|
|
109
117
|
A generic table you declared as `contact_submissions` is physically
|
package/.agent/async.md
CHANGED
|
@@ -46,9 +46,10 @@ resource-specific. Common examples:
|
|
|
46
46
|
**A data-activation-client run is NOT on this list — it is not a
|
|
47
47
|
`waitUntilReady` status-poll target.** `.ingest` / `.ingestFile` /
|
|
48
48
|
`.runManually` return a `202` + a `batch_id` (not a saved row and not
|
|
49
|
-
a pollable status enum); progress is tracked
|
|
50
|
-
and
|
|
51
|
-
|
|
49
|
+
a pollable status enum); progress is tracked through the run record
|
|
50
|
+
and the DAC's log rows for that `batch_id` — a dataset row carries no
|
|
51
|
+
`batch_id` of its own, it points at its log row — not by polling a
|
|
52
|
+
`success`/`error` status. See `data_activation_clients.md` §6.
|
|
52
53
|
|
|
53
54
|
The Datalake row is the odd one: its top-level `status` enum is
|
|
54
55
|
`new | processing | ready` only — there is no terminal failure
|
|
@@ -38,7 +38,7 @@ is one call away from here.
|
|
|
38
38
|
| Tenant + raw datalake | The marketing team's own world |
|
|
39
39
|
| SMS tool, email tool, LLM tool | Shared by every scenario below |
|
|
40
40
|
| Score Lead AI agent | Bands inbound leads |
|
|
41
|
-
|
|
|
41
|
+
| Four generic tables | Leads, the roster, the audience, the replies |
|
|
42
42
|
| Three workflows | Band fan-out, birthday greeting, campaign |
|
|
43
43
|
| Authenticated links | The greeting's link and the campaign's link |
|
|
44
44
|
| Action status updater | Delivery outcomes, reconciled on a cron |
|
|
@@ -1255,6 +1255,11 @@ await ctx.waitForBatches(ctx.leadFormDacSlug, [ctx.batchBirthday, ctx.batchNoDob
|
|
|
1255
1255
|
`manual_override: false` so the filter genuinely evaluates rather than
|
|
1256
1256
|
being bypassed.
|
|
1257
1257
|
|
|
1258
|
+
The `sql_where_clause` picks the two contacts by the batches that wrote
|
|
1259
|
+
them, and that goes **through the log row**: a legal entity carries no
|
|
1260
|
+
`batch_id` of its own, it points at the `data_activation_logs` row of the
|
|
1261
|
+
ingest that wrote it through `alvera_data_activation_log_id`.
|
|
1262
|
+
|
|
1258
1263
|
**This run never reaches a terminal status, and waiting for one would hang
|
|
1259
1264
|
until the deadline.** The birthday row's action is scheduled for a date in
|
|
1260
1265
|
the future, so its execution log sits at `:executing` for the life of the
|
|
@@ -1263,7 +1268,9 @@ execution log, which is the moment the next step can read them.
|
|
|
1263
1268
|
|
|
1264
1269
|
```typescript
|
|
1265
1270
|
const bdRunResp = await api.workflows.run(tenantSlug, datalakeSlug, ctx.birthdayWorkflowSlug, {
|
|
1266
|
-
sql_where_clause:
|
|
1271
|
+
sql_where_clause:
|
|
1272
|
+
`le.alvera_data_activation_log_id IN (SELECT id FROM data_activation_logs` +
|
|
1273
|
+
` WHERE batch_id IN ('${ctx.batchBirthday}', '${ctx.batchNoDob}'))`,
|
|
1267
1274
|
mode: 'live',
|
|
1268
1275
|
manual_override: false,
|
|
1269
1276
|
})
|
|
@@ -1319,11 +1326,14 @@ if ((bdByStatus.filtered ?? 0) !== 1) {
|
|
|
1319
1326
|
## 027 — find the contact the greeting is for
|
|
1320
1327
|
|
|
1321
1328
|
`workflows.execute` is addressed by dataset id, so the birthday contact's
|
|
1322
|
-
legal-entity id has to be resolved first. Search scoped to its own batch
|
|
1329
|
+
legal-entity id has to be resolved first. Search scoped to its own batch —
|
|
1330
|
+
through the log row again, since the row has no `batch_id` column.
|
|
1323
1331
|
|
|
1324
1332
|
```typescript
|
|
1325
1333
|
const { data: bdSearchDef } = await api.datasets.createUserSearch(tenantSlug, datalakeSlug, 'legal_entity', {
|
|
1326
|
-
search_query:
|
|
1334
|
+
search_query:
|
|
1335
|
+
`le.alvera_data_activation_log_id IN` +
|
|
1336
|
+
` (SELECT id FROM data_activation_logs WHERE batch_id = '${ctx.batchBirthday}')`,
|
|
1327
1337
|
})
|
|
1328
1338
|
if (bdSearchDef.status !== 'completed') {
|
|
1329
1339
|
throw new Error(`legal_entity user-search status=${bdSearchDef.status} error=${bdSearchDef.error_message ?? '(none)'}`)
|
|
@@ -1580,27 +1590,44 @@ await ctx.deployGenericTable(ctx.rosterTableName)
|
|
|
1580
1590
|
await ctx.deployGenericTable(ctx.audienceTableName)
|
|
1581
1591
|
```
|
|
1582
1592
|
|
|
1583
|
-
## 032 — the inbound-messages table, which
|
|
1593
|
+
## 032 — the inbound-messages table, which the app creates
|
|
1584
1594
|
|
|
1585
|
-
Replies land in `inbound_messages
|
|
1586
|
-
|
|
1587
|
-
|
|
1595
|
+
Replies land in `inbound_messages`, and the app defines that table like any
|
|
1596
|
+
other generic table. The platform ships no inbound table: a lake starts with
|
|
1597
|
+
none, and an app that receives replies declares its own — with the columns
|
|
1598
|
+
its replies carry, nothing reserved about the name.
|
|
1588
1599
|
|
|
1589
|
-
|
|
1590
|
-
|
|
1591
|
-
|
|
1600
|
+
`message_id` is the unique column, which makes a re-ingest of the same reply
|
|
1601
|
+
an upsert (§041 relies on it). `handle` and `body` are the sender's contact
|
|
1602
|
+
and words, so both are `tokenize`. `potential_duplicate` is the flag a human
|
|
1603
|
+
reviews when a handle maps to more than one customer.
|
|
1592
1604
|
|
|
1593
1605
|
```typescript
|
|
1594
|
-
const
|
|
1595
|
-
|
|
1596
|
-
|
|
1597
|
-
|
|
1598
|
-
}
|
|
1599
|
-
|
|
1600
|
-
|
|
1601
|
-
}
|
|
1606
|
+
const inbound = await ctx.ensure(
|
|
1607
|
+
'Inbound Messages table',
|
|
1608
|
+
async () => {
|
|
1609
|
+
const { data } = await api.genericTables.list(tenantSlug, datalakeSlug)
|
|
1610
|
+
return (data.data ?? []).find((t: { title?: string }) => t.title === 'Inbound Messages')
|
|
1611
|
+
},
|
|
1612
|
+
async () => {
|
|
1613
|
+
const { data } = await api.genericTables.create(tenantSlug, datalakeSlug, {
|
|
1614
|
+
title: 'Inbound Messages',
|
|
1615
|
+
description: 'Inbound customer messages — campaign replies re-ingested through the reply client.',
|
|
1616
|
+
columns: [
|
|
1617
|
+
{ name: 'message_id', title: 'Message ID', type: 'string', description: 'Vendor-supplied unique inbound-message id', is_unique: true, privacy_requirement: 'none' },
|
|
1618
|
+
{ name: 'handle', title: 'Handle', type: 'string', description: 'Sender phone/email the message arrived from', is_unique: false, privacy_requirement: 'tokenize', semantic_kind: 'string' },
|
|
1619
|
+
{ name: 'channel', title: 'Channel', type: 'string', description: 'Channel the message arrived on: sms | email', is_unique: false, privacy_requirement: 'none' },
|
|
1620
|
+
{ name: 'body', title: 'Body', type: 'string', description: 'Free-text message body', is_unique: false, privacy_requirement: 'tokenize', semantic_kind: 'text' },
|
|
1621
|
+
{ name: 'potential_duplicate', title: 'Potential Duplicate', type: 'boolean', description: 'Set when the sender handle maps to more than one LegalEntity — flagged for human review', is_unique: false, privacy_requirement: 'none' },
|
|
1622
|
+
],
|
|
1623
|
+
})
|
|
1624
|
+
return data
|
|
1625
|
+
},
|
|
1626
|
+
)
|
|
1602
1627
|
ctx.inboundTableId = inbound.id!
|
|
1603
1628
|
ctx.inboundTableName = inbound.name!
|
|
1629
|
+
|
|
1630
|
+
await ctx.deployGenericTable(ctx.inboundTableName)
|
|
1604
1631
|
```
|
|
1605
1632
|
|
|
1606
1633
|
## 033 — the roster contracts, on two clients
|
|
@@ -2393,7 +2420,7 @@ if (!campaignTracked.message.opened_at) {
|
|
|
2393
2420
|
## 041 — the reply comes back, and attaches to whoever sent it
|
|
2394
2421
|
|
|
2395
2422
|
A reply arrives on the same handle the campaign sent to. It ingests into the
|
|
2396
|
-
|
|
2423
|
+
inbound table §032 created through a client whose `mdm_input_config` keys on
|
|
2397
2424
|
the same `(uri, value)` namespace §034 wrote — so the platform resolves the
|
|
2398
2425
|
handle straight back to the customer and stamps their subject on the row.
|
|
2399
2426
|
|
|
@@ -2799,7 +2826,249 @@ if (advancingPoller.status !== 'active') {
|
|
|
2799
2826
|
}
|
|
2800
2827
|
```
|
|
2801
2828
|
|
|
2802
|
-
## 045 —
|
|
2829
|
+
## 045 — an SES sender on its own endpoint, and its delivery reconciled
|
|
2830
|
+
|
|
2831
|
+
The campaign's email went out through the mock provider. A real deployment
|
|
2832
|
+
sends through SES, and three things about SES are worth proving rather than
|
|
2833
|
+
reading about.
|
|
2834
|
+
|
|
2835
|
+
**An SES tool sends to its own `endpoint_url`.** It is the whole request URL;
|
|
2836
|
+
left blank, the tool uses `https://email.<region>.amazonaws.com`. Pointed at
|
|
2837
|
+
a local AWS stand-in, every send goes there instead, which is how this walk
|
|
2838
|
+
sends without an AWS account. A send that reached SES comes back with SES's
|
|
2839
|
+
message id, and the message row keeps it as `external_id`.
|
|
2840
|
+
|
|
2841
|
+
**A message row comes from a workflow, not from a test call.** A tool's
|
|
2842
|
+
`testInvocation` sends, but writes no message row, so there would be nothing
|
|
2843
|
+
to reconcile. The step sends one email to one recipient from a workflow
|
|
2844
|
+
whose only action uses the SES tool. The action fires after the run is
|
|
2845
|
+
recorded, so the step waits for the row. A user search keeps the rows that
|
|
2846
|
+
matched when it was created and never picks up a later one, so each try
|
|
2847
|
+
creates a new search.
|
|
2848
|
+
|
|
2849
|
+
**Name a status template by its provider's path.** Mailgun and SES both ship
|
|
2850
|
+
an `email_message_status` under `status_poller`, so the bare name answers
|
|
2851
|
+
409 with both paths as candidates. The updater names
|
|
2852
|
+
`status_poller/aws_ses/email_message_status`.
|
|
2853
|
+
|
|
2854
|
+
**Delivery comes from SES's events.** SES has no CloudWatch Logs destination
|
|
2855
|
+
of its own: its events travel configuration set → EventBridge → rule → log
|
|
2856
|
+
group, and the platform's SES template reads each one. `detail.mail.messageId`
|
|
2857
|
+
finds the message by its `external_id`; a Delivery moves it to `delivered`;
|
|
2858
|
+
a Bounce, Complaint or Reject moves it to `failed`; any other event leaves it
|
|
2859
|
+
where it is. The local stand-in keeps every email it was sent but publishes
|
|
2860
|
+
none of SES's events, so locally the poll finds nothing and the message stays
|
|
2861
|
+
`sent`. Against real SES, with the configuration set routed to the updater's
|
|
2862
|
+
log group, the same poll moves it to `delivered` at SES's own delivery time.
|
|
2863
|
+
The step accepts either, and nothing else.
|
|
2864
|
+
|
|
2865
|
+
**`refresh` only enqueues a poll.** It answers 202 with the row as it was;
|
|
2866
|
+
read the row again until `last_run_at` moves. `last_run_status` is `ok` when
|
|
2867
|
+
the run read its whole window and `error` when it stopped, never anything in
|
|
2868
|
+
between.
|
|
2869
|
+
|
|
2870
|
+
```typescript
|
|
2871
|
+
const SES_ENDPOINT = 'http://localhost:4566'
|
|
2872
|
+
const SES_LOG_GROUP = '/aws/events/cookbook-ses-delivery'
|
|
2873
|
+
const SES_DECISION_KEY = 'ses_delivery_check'
|
|
2874
|
+
|
|
2875
|
+
const sesTool = await ctx.ensure(
|
|
2876
|
+
'SES email sender',
|
|
2877
|
+
async () => {
|
|
2878
|
+
const { data } = await api.tools.list(tenantSlug, datalakeSlug)
|
|
2879
|
+
return (data.data ?? []).find((t: { name?: string }) => t.name === 'Cookbook SES Email Sender')
|
|
2880
|
+
},
|
|
2881
|
+
async () => {
|
|
2882
|
+
const { data } = await api.tools.create(tenantSlug, datalakeSlug, {
|
|
2883
|
+
name: 'Cookbook SES Email Sender',
|
|
2884
|
+
description: 'SES email dispatcher, pointed at the local AWS stand-in by endpoint_url.',
|
|
2885
|
+
intent: 'email',
|
|
2886
|
+
status: 'active',
|
|
2887
|
+
datalake_id: ctx.datalakeId,
|
|
2888
|
+
body: {
|
|
2889
|
+
tool_body_type: 'email',
|
|
2890
|
+
provider: 'ses',
|
|
2891
|
+
from_email: 'campaigns@bloom-salon.example.com',
|
|
2892
|
+
from_name: 'Bloom Salon',
|
|
2893
|
+
region: 'us-east-1',
|
|
2894
|
+
access_key_id: 'localkey',
|
|
2895
|
+
secret_access_key: 'localsecret',
|
|
2896
|
+
endpoint_url: SES_ENDPOINT,
|
|
2897
|
+
},
|
|
2898
|
+
})
|
|
2899
|
+
return data
|
|
2900
|
+
},
|
|
2901
|
+
)
|
|
2902
|
+
ctx.sesToolId = sesTool.id!
|
|
2903
|
+
|
|
2904
|
+
const { data: storedSes } = await api.tools.get(tenantSlug, datalakeSlug, ctx.sesToolId)
|
|
2905
|
+
const storedSesBody = storedSes.body as { provider?: string; endpoint_url?: string | null }
|
|
2906
|
+
if (storedSesBody.provider !== 'ses' || storedSesBody.endpoint_url !== SES_ENDPOINT) {
|
|
2907
|
+
throw new Error(`the SES tool must keep its endpoint, read back ${JSON.stringify(storedSesBody)}`)
|
|
2908
|
+
}
|
|
2909
|
+
|
|
2910
|
+
// The bare name is ambiguous between Mailgun and SES.
|
|
2911
|
+
let bareName: unknown
|
|
2912
|
+
try {
|
|
2913
|
+
await api.templates.metadataDetails(tenantSlug, datalakeSlug, 'email_message_status', 'status_poller')
|
|
2914
|
+
} catch (err) {
|
|
2915
|
+
bareName = err
|
|
2916
|
+
}
|
|
2917
|
+
if ((bareName as { _httpStatus?: number } | undefined)?._httpStatus !== 409) {
|
|
2918
|
+
throw new Error(`the bare email_message_status must answer 409, got ${JSON.stringify(bareName)}`)
|
|
2919
|
+
}
|
|
2920
|
+
if (!JSON.stringify(bareName).includes('status_poller/aws_ses/email_message_status')) {
|
|
2921
|
+
throw new Error(`the 409 must name the SES path among its candidates: ${JSON.stringify(bareName)}`)
|
|
2922
|
+
}
|
|
2923
|
+
|
|
2924
|
+
// One email, to one recipient, through the SES tool.
|
|
2925
|
+
const sesCheck = await ctx.ensure(
|
|
2926
|
+
'SES delivery check workflow',
|
|
2927
|
+
async () => {
|
|
2928
|
+
const { data } = await api.workflows.list(tenantSlug, datalakeSlug)
|
|
2929
|
+
return (data.data ?? []).find((w: { name?: string }) => w.name === 'Cookbook SES Delivery Check')
|
|
2930
|
+
},
|
|
2931
|
+
async () => {
|
|
2932
|
+
const { data } = await api.workflows.create(tenantSlug, datalakeSlug, {
|
|
2933
|
+
name: 'Cookbook SES Delivery Check',
|
|
2934
|
+
description: 'One email through the SES tool, so its delivery can be reconciled.',
|
|
2935
|
+
dataset_type: 'generic_table',
|
|
2936
|
+
generic_table_id: ctx.audienceTableId,
|
|
2937
|
+
status: 'live',
|
|
2938
|
+
tags: ['marketing', 'ses'],
|
|
2939
|
+
skip_mdm_resolution: false,
|
|
2940
|
+
filter_config: { type: 'custom', body: '{% if event_dataset.email and event_dataset.email != "" %}true{% endif %}' },
|
|
2941
|
+
decision_config: {
|
|
2942
|
+
type: 'custom',
|
|
2943
|
+
body: `["${SES_DECISION_KEY}"]`,
|
|
2944
|
+
output_schema: { type: 'array', items: { type: 'string' } },
|
|
2945
|
+
},
|
|
2946
|
+
actions: [
|
|
2947
|
+
{
|
|
2948
|
+
action_type: 'email',
|
|
2949
|
+
tool_id: ctx.sesToolId,
|
|
2950
|
+
decision_key: SES_DECISION_KEY,
|
|
2951
|
+
position: 0,
|
|
2952
|
+
trigger_template: 'now',
|
|
2953
|
+
idempotency_template: '{{ event_dataset.end_customer_id }}/ses-delivery-check',
|
|
2954
|
+
tool_call: {
|
|
2955
|
+
tool_call_type: 'email_request',
|
|
2956
|
+
to: { type: 'custom', body: '{{ event_dataset.email }}' },
|
|
2957
|
+
subject: { type: 'custom', body: 'Your slot at Bloom Salon' },
|
|
2958
|
+
body: {
|
|
2959
|
+
type: 'custom',
|
|
2960
|
+
body:
|
|
2961
|
+
`<p>Hi {{ event_dataset.name }}, we have kept your slot.</p>` +
|
|
2962
|
+
`<p><a href="${ctx.bookingPageUrl}">See your booking</a></p>`,
|
|
2963
|
+
},
|
|
2964
|
+
link_prefix: ctx.campaignLinkPrefix,
|
|
2965
|
+
},
|
|
2966
|
+
},
|
|
2967
|
+
],
|
|
2968
|
+
})
|
|
2969
|
+
return data
|
|
2970
|
+
},
|
|
2971
|
+
)
|
|
2972
|
+
ctx.sesCheckWorkflowSlug = sesCheck.slug!
|
|
2973
|
+
|
|
2974
|
+
const { data: sesRun } = await api.workflows.run(tenantSlug, datalakeSlug, ctx.sesCheckWorkflowSlug, {
|
|
2975
|
+
sql_where_clause: `end_customer_id = '${ctx.adaId}'`,
|
|
2976
|
+
mode: 'live',
|
|
2977
|
+
manual_override: false,
|
|
2978
|
+
})
|
|
2979
|
+
await ctx.waitForFiredRun(datalakeSlug, sesRun.workflow_run_id)
|
|
2980
|
+
|
|
2981
|
+
// The message row, found by its idempotency key, which names the same send on every rerun.
|
|
2982
|
+
// A search keeps the rows that matched when it was created, so each try creates a new one.
|
|
2983
|
+
const sesSentDeadline = Date.now() + 45_000
|
|
2984
|
+
let sesMessage: { status?: string; external_id?: string | null; sender_tool_id?: string } | undefined
|
|
2985
|
+
while (Date.now() < sesSentDeadline) {
|
|
2986
|
+
const { data: sesSearch } = await api.datasets.createUserSearch(tenantSlug, datalakeSlug, 'message', {
|
|
2987
|
+
search_query: `m.idempotency_key = '${ctx.adaId}/ses-delivery-check'`,
|
|
2988
|
+
})
|
|
2989
|
+
if (sesSearch.status !== 'completed') {
|
|
2990
|
+
throw new Error(`message user-search status=${sesSearch.status} error=${sesSearch.error_message ?? '(none)'}`)
|
|
2991
|
+
}
|
|
2992
|
+
const { data } = await api.datasets.search(tenantSlug, datalakeSlug, 'message', {
|
|
2993
|
+
userSearchId: sesSearch.id!,
|
|
2994
|
+
dataAccessMode: 'raw',
|
|
2995
|
+
})
|
|
2996
|
+
sesMessage = (data.data ?? [])[0] as typeof sesMessage
|
|
2997
|
+
if (sesMessage?.external_id) break
|
|
2998
|
+
await new Promise((r) => setTimeout(r, 1_000))
|
|
2999
|
+
}
|
|
3000
|
+
if (!sesMessage?.external_id) {
|
|
3001
|
+
throw new Error(`the SES send must leave a message carrying SES's id, got ${JSON.stringify(sesMessage)}`)
|
|
3002
|
+
}
|
|
3003
|
+
if (sesMessage.sender_tool_id !== ctx.sesToolId) {
|
|
3004
|
+
throw new Error(`the message must name the SES tool as its sender, got ${sesMessage.sender_tool_id}`)
|
|
3005
|
+
}
|
|
3006
|
+
if (!['sent', 'delivered'].includes(sesMessage.status ?? '')) {
|
|
3007
|
+
throw new Error(`the SES send must leave the message sent, got ${sesMessage.status}`)
|
|
3008
|
+
}
|
|
3009
|
+
|
|
3010
|
+
const sesUpdater = await ctx.ensure(
|
|
3011
|
+
'SES delivery reconciler',
|
|
3012
|
+
async () => {
|
|
3013
|
+
const { data } = await api.actionStatusUpdaters.list(tenantSlug, datalakeSlug)
|
|
3014
|
+
return (data.data ?? []).find((u: { name?: string }) => u.name === 'Cookbook SES Delivery Updater')
|
|
3015
|
+
},
|
|
3016
|
+
async () => {
|
|
3017
|
+
const { data } = await api.actionStatusUpdaters.create(tenantSlug, datalakeSlug, {
|
|
3018
|
+
name: 'Cookbook SES Delivery Updater',
|
|
3019
|
+
cron_expression: '*/30 * * * *',
|
|
3020
|
+
updater_type: 'cloud_watch',
|
|
3021
|
+
updater_tool_id: ctx.cwToolId,
|
|
3022
|
+
sender_tool_ids: [ctx.sesToolId],
|
|
3023
|
+
datalake_id: ctx.datalakeId,
|
|
3024
|
+
updater_body: {
|
|
3025
|
+
updater_body_type: 'cloud_watch_request',
|
|
3026
|
+
log_group_name: SES_LOG_GROUP,
|
|
3027
|
+
start_time: '{{ now_msec | minutes_ago: 45 }}',
|
|
3028
|
+
end_time: '{{ now_msec }}',
|
|
3029
|
+
},
|
|
3030
|
+
message_config: { type: 'system', path: 'status_poller/aws_ses/email_message_status' },
|
|
3031
|
+
action_log_config: { type: 'system', path: 'status_poller/aws_ses/email_action_log_status' },
|
|
3032
|
+
})
|
|
3033
|
+
return data
|
|
3034
|
+
},
|
|
3035
|
+
)
|
|
3036
|
+
ctx.sesUpdaterId = sesUpdater.id!
|
|
3037
|
+
|
|
3038
|
+
// refresh only enqueues; the outcome lands on the row.
|
|
3039
|
+
const { data: beforePoll } = await api.actionStatusUpdaters.get(tenantSlug, datalakeSlug, ctx.sesUpdaterId)
|
|
3040
|
+
await api.actionStatusUpdaters.refresh(tenantSlug, datalakeSlug, ctx.sesUpdaterId)
|
|
3041
|
+
let polled = beforePoll
|
|
3042
|
+
for (let i = 0; i < 60 && polled.last_run_at === beforePoll.last_run_at; i++) {
|
|
3043
|
+
await new Promise((r) => setTimeout(r, 1_000))
|
|
3044
|
+
polled = (await api.actionStatusUpdaters.get(tenantSlug, datalakeSlug, ctx.sesUpdaterId)).data
|
|
3045
|
+
}
|
|
3046
|
+
if (polled.last_run_at === beforePoll.last_run_at) {
|
|
3047
|
+
throw new Error('the SES poll never ran: last_run_at did not move in 60 seconds')
|
|
3048
|
+
}
|
|
3049
|
+
if (polled.last_run_status !== 'ok') {
|
|
3050
|
+
throw new Error(`the SES poll must read its whole window, got ${polled.last_run_status}: ${polled.last_run_error}`)
|
|
3051
|
+
}
|
|
3052
|
+
|
|
3053
|
+
// The message as the poll left it: delivered where SES published a Delivery, still sent where nothing did.
|
|
3054
|
+
const sesExternalId = sesMessage.external_id
|
|
3055
|
+
const { data: sesPage } = await api.datalakes.executeSql(tenantSlug, datalakeSlug, {
|
|
3056
|
+
sql: `SELECT status, external_id FROM messages WHERE sender_tool_id = '${ctx.sesToolId}' AND channel = 'email'`,
|
|
3057
|
+
mode: 'raw',
|
|
3058
|
+
})
|
|
3059
|
+
if (typeof sesPage === 'string') {
|
|
3060
|
+
throw new Error('executeSql answered CSV where it should answer rows')
|
|
3061
|
+
}
|
|
3062
|
+
const sesAfter = (sesPage.data as unknown[][]).find((row) => row[1] === sesExternalId)
|
|
3063
|
+
if (!sesAfter) {
|
|
3064
|
+
throw new Error(`no message with SES's id ${sesExternalId} after the poll: ${JSON.stringify(sesPage.data)}`)
|
|
3065
|
+
}
|
|
3066
|
+
if (!['sent', 'delivered'].includes(String(sesAfter[0]))) {
|
|
3067
|
+
throw new Error(`after the poll the message must be sent or delivered, got ${sesAfter[0]}`)
|
|
3068
|
+
}
|
|
3069
|
+
```
|
|
3070
|
+
|
|
3071
|
+
## 046 — ask the lake a question in plain language
|
|
2803
3072
|
|
|
2804
3073
|
The last thing a marketing team needs is a way to look at any of this
|
|
2805
3074
|
without writing SQL.
|
|
@@ -2827,7 +3096,7 @@ if (typeof generated.model !== 'string' || typeof generated.provider !== 'string
|
|
|
2827
3096
|
console.log(` ↳ ${generated.provider}/${generated.model} proposed: ${generated.sql.trim().slice(0, 120)}`)
|
|
2828
3097
|
```
|
|
2829
3098
|
|
|
2830
|
-
##
|
|
3099
|
+
## 047 — run it read-only, then take the same page as CSV
|
|
2831
3100
|
|
|
2832
3101
|
`executeSql` runs read-only — inserts, updates, deletes and DDL are all
|
|
2833
3102
|
refused — and returns `{ data, meta }`. **`data` is an array of arrays**,
|
|
@@ -2835,7 +3104,7 @@ positional and aligned to `meta.columns`, because arbitrary SQL can produce
|
|
|
2835
3104
|
duplicate or expression column names that object keys would collapse.
|
|
2836
3105
|
|
|
2837
3106
|
The deterministic count below keeps the step independent of whatever the
|
|
2838
|
-
model proposed a moment ago, which is what you want in a walk: §
|
|
3107
|
+
model proposed a moment ago, which is what you want in a walk: §046 proves
|
|
2839
3108
|
the generation, this proves the execution, and neither can mask a failure in
|
|
2840
3109
|
the other.
|
|
2841
3110
|
|
|
@@ -2879,7 +3148,7 @@ if (!csvExport.data.includes('end_customer_id') || !csvExport.data.includes(ctx.
|
|
|
2879
3148
|
}
|
|
2880
3149
|
```
|
|
2881
3150
|
|
|
2882
|
-
##
|
|
3151
|
+
## 048 — one person, two identifiers, and what a second ingest leaves alone
|
|
2883
3152
|
|
|
2884
3153
|
Every recipient so far carries **one** identifier: §034's template picks
|
|
2885
3154
|
phone-or-email and writes a single slot. A real roster does not look like that.
|
|
@@ -3077,7 +3346,7 @@ if (afterSubjects.size !== 1 || !afterSubjects.has(twoIdSubject)) {
|
|
|
3077
3346
|
|
|
3078
3347
|
# Outcome
|
|
3079
3348
|
|
|
3080
|
-
After all forty-
|
|
3349
|
+
After all forty-eight steps run green, on one tenant and **one raw
|
|
3081
3350
|
datalake**:
|
|
3082
3351
|
|
|
3083
3352
|
- **Every inbound lead is banded.** Three leads land in the Lead Submissions
|
|
@@ -3096,18 +3365,21 @@ datalake**:
|
|
|
3096
3365
|
is contacted at all, and its split sends each of them one variant on both
|
|
3097
3366
|
SMS and email, every message carrying its own link to the booking page
|
|
3098
3367
|
(§031–§040).
|
|
3099
|
-
- **A reply attaches to whoever sent it.** It ingests into the
|
|
3100
|
-
table and resolves back to the recipient's subject without creating
|
|
3101
|
-
one (§041).
|
|
3368
|
+
- **A reply attaches to whoever sent it.** It ingests into the app's own
|
|
3369
|
+
inbound table and resolves back to the recipient's subject without creating
|
|
3370
|
+
a new one (§032, §041).
|
|
3102
3371
|
- **Delivery is reconciled, not assumed.** An action status updater turns
|
|
3103
3372
|
provider delivery events into updates on a schedule, its stored body is
|
|
3104
3373
|
read back against its checksum, and a poller whose request could never move
|
|
3105
|
-
to the next page is refused at create (§042–§044).
|
|
3374
|
+
to the next page is refused at create (§042–§044). An SES sender sends to
|
|
3375
|
+
its own endpoint, a workflow's email through it leaves a message carrying
|
|
3376
|
+
SES's id, and an updater reconciles that message through the platform's SES
|
|
3377
|
+
template (§045).
|
|
3106
3378
|
- **The lake answers in plain language.** Text-to-SQL proposes a query
|
|
3107
3379
|
without running it; the read-only executor runs one and returns the same
|
|
3108
|
-
page as CSV (§
|
|
3380
|
+
page as CSV (§046–§047).
|
|
3109
3381
|
- **One person carries two identifiers.** A second ingest that names only the
|
|
3110
|
-
email leaves the phone slot untouched on the same subject (§
|
|
3382
|
+
email leaves the phone slot untouched on the same subject (§048).
|
|
3111
3383
|
|
|
3112
3384
|
Running this walk again reuses all of it.
|
|
3113
3385
|
|
|
@@ -3136,6 +3408,10 @@ Running this walk again reuses all of it.
|
|
|
3136
3408
|
generic-table row is never its own subject. With resolution on, the
|
|
3137
3409
|
message and every link minted with it name the stamped subject as their
|
|
3138
3410
|
`legal_entity_id`; skip it and the subject is nil — §038.
|
|
3411
|
+
6. **Name a status template by its provider's path.** Mailgun and SES both
|
|
3412
|
+
ship an `email_message_status` under `status_poller`, so the bare name
|
|
3413
|
+
answers 409 with both paths as candidates; use
|
|
3414
|
+
`status_poller/aws_ses/email_message_status` — §045.
|
|
3139
3415
|
|
|
3140
3416
|
# See also
|
|
3141
3417
|
|
|
@@ -1931,7 +1931,10 @@ ctx.bulkBatchId = await ctx.ingestFileAndWait(ctx.dataDacSlug, 4)
|
|
|
1931
1931
|
|
|
1932
1932
|
Both halves, scoped to the batch the worker allocated. The table side is read
|
|
1933
1933
|
through read-only SQL; the identity side through the dataset search, where
|
|
1934
|
-
`le` is the legal-entity alias the query compiler expects.
|
|
1934
|
+
`le` is the legal-entity alias the query compiler expects. A legal entity
|
|
1935
|
+
carries no `batch_id` of its own — it points at the `data_activation_logs`
|
|
1936
|
+
row of the ingest that wrote it — so the batch scope goes through that log
|
|
1937
|
+
row.
|
|
1935
1938
|
|
|
1936
1939
|
The stamp check is the one that matters, and it is the same check §014 ran on
|
|
1937
1940
|
the inline rows. Whether a row arrived in a JSON body or in a CSV, it is the
|
|
@@ -1966,7 +1969,9 @@ if (unstamped.length > 0) {
|
|
|
1966
1969
|
}
|
|
1967
1970
|
|
|
1968
1971
|
const { data: search } = await api.datasets.createUserSearch(tenantSlug, datalakeSlug, 'legal_entity', {
|
|
1969
|
-
search_query:
|
|
1972
|
+
search_query:
|
|
1973
|
+
`le.alvera_data_activation_log_id IN` +
|
|
1974
|
+
` (SELECT id FROM data_activation_logs WHERE batch_id = '${ctx.bulkBatchId}')`,
|
|
1970
1975
|
})
|
|
1971
1976
|
if (search.status !== 'completed') {
|
|
1972
1977
|
throw new Error(`legal_entity user-search did not compile: ${search.status}`)
|