@alvera-ai/platform-sdk 0.16.3 → 0.17.0-next.g3186053

package/.agent/AGENTS.md

@@ -576,6 +576,13 @@ of restating:
 | Async + readiness                             | `async.md`       |
 | Debugging (HTTP interceptor + safe redaction) | `debugging.md`   |
 | Type naming + server-derived fields           | `type_naming.md` |
+| Local mock surfaces (WireMock / LocalStack)   | `mock-services.md` |
+
+`mock-services.md` answers the local-development questions the other
+pages assume away: how the platform reaches mocked third-party APIs on
+a laptop, the `endpoint_url` override on the sender tools, and why End
+User Messaging points at WireMock rather than LocalStack. It is fetched
+from the platform-metadata channel, not authored here.
 
 ## Just getting started?
 

package/.agent/cookbook/_setup/access-control.md

@@ -0,0 +1,341 @@
+---
+title: Access control bootstrap setup
+summary: Auth as root, sign up + confirm a fresh admin user, create a tenant + datalake. Shared by every access-control cookbook.
+use_case: access-control
+slug: access-control
+vitest_source:
+  - integration-tests/tests/foundation/bootstrap.test.ts
+status: green
+---
+
+# Problem
+
+Every foundation scenario cookbook needs the same starting point:
+a fresh tenant on the platform with a foundation-domain datalake
+attached, accessed by an industry-admin user. Authoring this
+prelude inline inside every scenario cookbook would duplicate the
+auth + tenant + datalake chain across two cookbooks (and four more
+across the other industries). The validator auto-discovers this
+file from each cookbook's `industry: foundation` front-matter and
+inlines its numbered steps ahead of the scenario's own, so the
+generated bun:test spec is hermetic without the cookbook author
+having to copy-paste this prelude.
+
+# Composition
+
+| Resource provisioned          | Owner       |
+|-------------------------------|-------------|
+| Root session                  | setup       |
+| Industry-admin user (`sarah`) | setup       |
+| Foundation tenant             | setup       |
+| Foundation datalake           | setup       |
+
+# Walkthrough
+
+## 001 — root admin signs in and provisions a fresh industry-admin user
+
+Authenticate as the platform's root admin (`admin@dev.local` /
+`devpassword` in local dev) via the tenantless bootstrap login —
+keyless by structural necessity (no tenant exists yet to scope a
+key to), and a dev/test-only surface. Then sign up a per-run industry-admin
+user (`sarah`) under a unique email derived from `runSuffix`, and
+confirm them so they can sign in. Both the signup and the confirm
+endpoints require root scope.
+
+```typescript
+ctx.rootSession = await createBootstrapSession({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  email: process.env.ALVERA_ROOT_EMAIL!,
+  password: process.env.ALVERA_ROOT_PASSWORD!,
+})
+ctx.rootApi = createIsolatedPlatformApi({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  sessionToken: ctx.rootSession.sessionToken,
+  apiKey: '',
+})
+
+ctx.sarahEmail = `cookbook-sarah-${runSuffix}@dev.local`
+ctx.sarahPassword = 'CookbookPass1!'
+
+const signUpResp = await ctx.rootApi.admin.signUp({
+  email: ctx.sarahEmail,
+  password: ctx.sarahPassword,
+  first_name: 'Cookbook',
+  last_name: 'Sarah',
+})
+ctx.sarahUserId = signUpResp.data.id
+await ctx.rootApi.admin.confirmUser(ctx.sarahUserId)
+```
+
+## 002 — sarah signs in (tenantless) and creates the foundation tenant
+
+Sarah signs in for the first time without a tenant scope (no
+tenant exists yet for her), then immediately creates a fresh
+foundation tenant. The tenant's server-derived `slug` is captured
+into the closure-scoped `tenantSlug` slot for downstream steps.
+
+```typescript
+ctx.sarahTenantlessSession = await createBootstrapSession({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  email: ctx.sarahEmail,
+  password: ctx.sarahPassword,
+})
+ctx.sarahTenantlessApi = createIsolatedPlatformApi({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  sessionToken: ctx.sarahTenantlessSession.sessionToken,
+  apiKey: '',
+})
+
+const tenantResp = await ctx.sarahTenantlessApi.tenants.create({
+  name: `Cookbook Access control ${runSuffix}`,
+})
+tenantSlug = tenantResp.data.slug!
+```
+
+## 003 — sarah signs in tenant-scoped — the canonical client
+
+Re-mint sarah's session with the new tenant slug. This bearer is
+the canonical tenant-scoped client every subsequent step uses; it
+is assigned to the closure-scoped `api` slot so the scenario
+cookbook's steps inherit it via inlining.
+
+```typescript
+// A tenant-scoped login requires X-API-Key — mint a public_api key for
+// the fresh tenant via the platform-admin side door (root Bearer). In the
+// web console this is Settings -> API Keys; any of the tenant's keys
+// satisfies the login gate, and the minted Bearer inherits this key's
+// origin policy.
+const { data: mintedKey } = await ctx.rootApi.admin.createTenantApiKey(tenantSlug, {
+  name: `Cookbook Bootstrap Key ${runSuffix}`,
+  data_access_mode: 'tokenized',
+})
+ctx.tenantApiKey = mintedKey.api_key
+
+const sarahTenantSession = await createSession({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  email: ctx.sarahEmail,
+  password: ctx.sarahPassword,
+  tenantSlug,
+  apiKey: ctx.tenantApiKey,
+})
+api = createIsolatedPlatformApi({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  sessionToken: sarahTenantSession.sessionToken,
+  apiKey: ctx.tenantApiKey,
+})
+```
+
+## 004 — sarah creates the datalake
+
+Provision a datalake on the new tenant. The DB schema is scoped to the cookbook run via `runSuffix` so parallel
+cookbook runs do not collide on schema names. Local-dev defaults
+(`postgres` on `localhost:5432`, `alvera_dev_foundation`) match
+the seeded `dev.exs` setup; LocalStack S3 (`localhost:4566`)
+serves the lake bucket (`alvera-platform-dev`, scoped by `base_path`). The datalake's
+server-derived `slug` is captured into `datalakeSlug`.
+
+```typescript
+const DB_HOST = 'localhost'
+const DB_PORT = 5432
+const DB_USER = 'postgres'
+const DB_PASS = 'postgres'
+const DB_NAME = 'alvera_dev_foundation'
+const DB_SCHEMA = `cookbook_${runSuffix}`
+
+const S3 = {
+  cloud_storage_type: 'aws' as const,
+  region: 'us-east-1',
+  access_key_id: 'test',
+  secret_access_key: 'test',
+  endpoint: 'http://localhost:4566',
+}
+
+const datalakeResp = await api.datalakes.create(tenantSlug, {
+  name: `Cookbook Access control Datalake ${runSuffix}`,
+  description: 'Datalake provisioned by cookbook doctest for access-control.',
+  timezone: 'America/New_York',
+  pool_size: 5,
+
+  db_writer_host: DB_HOST,
+  db_writer_port: DB_PORT,
+  db_writer_name: DB_NAME,
+  db_writer_schema: DB_SCHEMA,
+  db_writer_auth_method: 'password',
+  db_writer_user: DB_USER,
+  db_writer_pass: DB_PASS,
+  db_writer_enable_ssl: false,
+  db_reader_host: DB_HOST,
+  db_reader_port: DB_PORT,
+  db_reader_name: DB_NAME,
+  db_reader_schema: DB_SCHEMA,
+  db_reader_auth_method: 'password',
+  db_reader_user: DB_USER,
+  db_reader_pass: DB_PASS,
+  db_reader_enable_ssl: false,
+
+  // One bucket, scoped per run by `base_path` — the same shape the dev
+  // platform's own datalakes use. The old paired
+  // `<domain>-lake-{un,}regulated` buckets died with the two-tier lake.
+  cloud_storage: { ...S3, bucket: 'alvera-platform-dev', base_path: `cookbook/${runSuffix}` },
+})
+datalakeSlug = datalakeResp.data.slug!
+// `tools.create` and a few other endpoints accept the datalake by
+// UUID in the request body rather than only the slug in the URL,
+// so keep it on `ctx` for downstream steps that need it.
+ctx.datalakeId = datalakeResp.data.id!
+```
+
+## 005 — enqueue the datalake migrations
+
+`datalakes.create` persists the datalake row in `:new` status but
+does not itself run the schema migrations. Migration is a
+separately-triggered async job so the operator controls when the
+(potentially slow) per-industry DDL runs. `datalakes.migrate`
+enqueues the `DatalakeMigrationWorker` Oban job and returns
+immediately with `status: 'enqueued'` plus the Oban `job_id`. The
+poll in §006 then waits for that worker to finish — without this
+call the datalake would sit at `:new` forever.
+
+```typescript
+const migrateResp = await api.datalakes.migrate(tenantSlug, datalakeSlug)
+if (migrateResp.data.status !== 'enqueued') {
+  throw new Error(
+    `datalake migration not enqueued (status: ${migrateResp.data.status})`,
+  )
+}
+```
+
+## 006 — poll the datalake until status is ready
+
+Datalake migrations are async — the §005 migrate call enqueued a
+`DatalakeMigrationWorker` Oban job that runs the per-industry
+schema migrations, deploys PostgREST roles, and provisions the
+service-account bot user. Downstream resource creates (tools,
+workflows) only need `datalakeId` to be persisted (which it is
+the moment §004 returns), but any scenario step that touches
+the regulated/unregulated DBs or the dataset infrastructure
+requires `status: 'ready'`. Polling here makes every foundation
+scenario cookbook deterministic regardless of how long the cold
+migration takes on a given host; the loop exits the moment the
+datalake reaches `:ready`, so the 5-minute cap is paid only in
+failure mode.
+
+```typescript
+const READY_TIMEOUT_MS = 5 * 60_000
+const READY_POLL_MS = 5_000
+const deadline = Date.now() + READY_TIMEOUT_MS
+let datalakeStatus: string | undefined
+while (Date.now() < deadline) {
+  const { data } = await api.datalakes.get(tenantSlug, ctx.datalakeId)
+  datalakeStatus = data.status
+  if (datalakeStatus === 'ready') break
+  await new Promise((r) => setTimeout(r, READY_POLL_MS))
+}
+if (datalakeStatus !== 'ready') {
+  throw new Error(
+    `datalake did not reach :ready within ${READY_TIMEOUT_MS}ms (last status: ${datalakeStatus})`,
+  )
+}
+```
+
+## 007 — a reusable "wait until the run has fired" helper
+
+`workflows.run` only **schedules** a run. It records it and returns
+immediately with `workflow_run_id` / `status` / `scheduled_at`; the
+`workflow_run_log_id` and `batch_id` a scenario needs are written
+later, when the run actually fires, and are read back from
+`workflowRuns.get`.
+
+Two traps live in that gap, so every foundation scenario shares one
+helper rather than re-deriving it:
+
+- **Poll until `workflow_run_log_id` is a string — NOT until `status`
+  leaves `'scheduled'`.** Those are different moments: the run reaches
+  `processing` first and writes the log id a beat later. A predicate on
+  status alone releases you to read a `null`, and because
+  `typeof null === 'object'` the symptom is a baffling *"expected
+  string, got object"* rather than an obvious nil.
+- **Raise on `failed` carrying `failure_reason` rather than polling to
+  the deadline.** A scenario blocked on a run that will never fire
+  should say why on the first read, not thirty seconds later behind a
+  generic timeout.
+
+The helper is stored on the shared `ctx` bag rather than declared as a
+plain function because each numbered step compiles into its own `it()`
+block — a bare `function` here would not be in scope for the steps that
+call it.
+
+```typescript
+const FIRED_TIMEOUT_MS = 60_000
+const FIRED_POLL_MS = 1_000
+
+ctx.waitForFiredRun = async (
+  runDatalakeSlug: string,
+  runId: string,
+  timeoutMs: number = FIRED_TIMEOUT_MS,
+): Promise<{ workflowRunLogId: string; batchId: string | null }> => {
+  const deadline = Date.now() + timeoutMs
+  let lastStatus: string | undefined
+  while (Date.now() < deadline) {
+    const { data } = await api.workflowRuns.get(tenantSlug, runDatalakeSlug, runId)
+    lastStatus = data.status
+    if (data.status === 'failed') {
+      throw new Error(
+        `workflow run ${runId} failed: ${data.failure_reason ?? 'no failure_reason given'}`,
+      )
+    }
+    if (typeof data.workflow_run_log_id === 'string') {
+      return { workflowRunLogId: data.workflow_run_log_id, batchId: data.batch_id ?? null }
+    }
+    await new Promise((r) => setTimeout(r, FIRED_POLL_MS))
+  }
+  throw new Error(
+    `workflow run ${runId} never fired within ${timeoutMs}ms (last status: ${lastStatus})`,
+  )
+}
+```
+
+# Rollback
+
+The cookbook doctest harness does not currently tear down the
+created tenant / datalake / user — each run mints fresh names via
+`runSuffix` so reruns do not collide, and the seeded local DB is
+cheap to reset (`mix ecto.reset` on the platform). When the
+validator graduates to continuous integration at Milestone 18
+(running against the mockoon-backed mock env at `ALVERA_ENV=mock`),
+no real state is created at all.
+
+# Outcome
+
+After this setup runs, the closure-scoped slots are populated as:
+
+- `api` — a tenant-scoped `PlatformApi` client authenticated as
+  the industry-admin user
+- `tenantSlug` — server-derived slug of the fresh foundation
+  tenant
+- `datalakeSlug` — server-derived slug of the fresh foundation
+  datalake
+- `ctx.sarahEmail` / `ctx.sarahPassword` — the per-run admin
+  credentials (kept on `ctx` because no scenario cookbook needs
+  to re-authenticate by default)
+- `ctx.tenantApiKey` — the tenant's publishable API key (minted in
+  setup via `admin.createTenantApiKey`); thread it into any
+  additional tenant-scoped `createSession` a scenario performs
+- `ctx.waitForFiredRun(datalakeSlug, runId)` — polls `workflowRuns.get`
+  until a scheduled run has actually fired, then returns its
+  `{ workflowRunLogId, batchId }`. Use it after every `workflows.run`;
+  reading `workflow_run_log_id` off the run response returns `null`
+  because run-workflow only schedules (see §007)
+
+Scenario cookbooks under `industry: foundation` start at their own
+`§001` with these slots already populated.
+
+# See also
+
+- `.agent/datalakes.md` — datalake create body shape (regulated +
+  unregulated tier configuration)
+- `.agent/AGENTS.md` § SDK auth + client construction — root vs
+  tenantless vs tenant-scoped session scopes
+- `integration-tests/tests/foundation/bootstrap.test.ts` — the
+  green test these snippets are lifted from (the §7b sibling
+  datalake creation is out of scope for this lean cookbook setup)

package/.agent/cookbook/_setup/agent-provider-matrix.md

@@ -0,0 +1,341 @@
+---
+title: Agent provider matrix bootstrap setup
+summary: Auth as root, sign up + confirm a fresh admin user, create a tenant + datalake. Shared by every agent-provider-matrix cookbook.
+use_case: agent-provider-matrix
+slug: agent-provider-matrix
+vitest_source:
+  - integration-tests/tests/foundation/bootstrap.test.ts
+status: green
+---
+
+# Problem
+
+Every foundation scenario cookbook needs the same starting point:
+a fresh tenant on the platform with a foundation-domain datalake
+attached, accessed by an industry-admin user. Authoring this
+prelude inline inside every scenario cookbook would duplicate the
+auth + tenant + datalake chain across two cookbooks (and four more
+across the other industries). The validator auto-discovers this
+file from each cookbook's `industry: foundation` front-matter and
+inlines its numbered steps ahead of the scenario's own, so the
+generated bun:test spec is hermetic without the cookbook author
+having to copy-paste this prelude.
+
+# Composition
+
+| Resource provisioned          | Owner       |
+|-------------------------------|-------------|
+| Root session                  | setup       |
+| Industry-admin user (`sarah`) | setup       |
+| Foundation tenant             | setup       |
+| Foundation datalake           | setup       |
+
+# Walkthrough
+
+## 001 — root admin signs in and provisions a fresh industry-admin user
+
+Authenticate as the platform's root admin (`admin@dev.local` /
+`devpassword` in local dev) via the tenantless bootstrap login —
+keyless by structural necessity (no tenant exists yet to scope a
+key to), and a dev/test-only surface. Then sign up a per-run industry-admin
+user (`sarah`) under a unique email derived from `runSuffix`, and
+confirm them so they can sign in. Both the signup and the confirm
+endpoints require root scope.
+
+```typescript
+ctx.rootSession = await createBootstrapSession({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  email: process.env.ALVERA_ROOT_EMAIL!,
+  password: process.env.ALVERA_ROOT_PASSWORD!,
+})
+ctx.rootApi = createIsolatedPlatformApi({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  sessionToken: ctx.rootSession.sessionToken,
+  apiKey: '',
+})
+
+ctx.sarahEmail = `cookbook-sarah-${runSuffix}@dev.local`
+ctx.sarahPassword = 'CookbookPass1!'
+
+const signUpResp = await ctx.rootApi.admin.signUp({
+  email: ctx.sarahEmail,
+  password: ctx.sarahPassword,
+  first_name: 'Cookbook',
+  last_name: 'Sarah',
+})
+ctx.sarahUserId = signUpResp.data.id
+await ctx.rootApi.admin.confirmUser(ctx.sarahUserId)
+```
+
+## 002 — sarah signs in (tenantless) and creates the foundation tenant
+
+Sarah signs in for the first time without a tenant scope (no
+tenant exists yet for her), then immediately creates a fresh
+foundation tenant. The tenant's server-derived `slug` is captured
+into the closure-scoped `tenantSlug` slot for downstream steps.
+
+```typescript
+ctx.sarahTenantlessSession = await createBootstrapSession({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  email: ctx.sarahEmail,
+  password: ctx.sarahPassword,
+})
+ctx.sarahTenantlessApi = createIsolatedPlatformApi({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  sessionToken: ctx.sarahTenantlessSession.sessionToken,
+  apiKey: '',
+})
+
+const tenantResp = await ctx.sarahTenantlessApi.tenants.create({
+  name: `Cookbook Agent provider matrix ${runSuffix}`,
+})
+tenantSlug = tenantResp.data.slug!
+```
+
+## 003 — sarah signs in tenant-scoped — the canonical client
+
+Re-mint sarah's session with the new tenant slug. This bearer is
+the canonical tenant-scoped client every subsequent step uses; it
+is assigned to the closure-scoped `api` slot so the scenario
+cookbook's steps inherit it via inlining.
+
+```typescript
+// A tenant-scoped login requires X-API-Key — mint a public_api key for
+// the fresh tenant via the platform-admin side door (root Bearer). In the
+// web console this is Settings -> API Keys; any of the tenant's keys
+// satisfies the login gate, and the minted Bearer inherits this key's
+// origin policy.
+const { data: mintedKey } = await ctx.rootApi.admin.createTenantApiKey(tenantSlug, {
+  name: `Cookbook Bootstrap Key ${runSuffix}`,
+  data_access_mode: 'tokenized',
+})
+ctx.tenantApiKey = mintedKey.api_key
+
+const sarahTenantSession = await createSession({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  email: ctx.sarahEmail,
+  password: ctx.sarahPassword,
+  tenantSlug,
+  apiKey: ctx.tenantApiKey,
+})
+api = createIsolatedPlatformApi({
+  baseUrl: process.env.ALVERA_BASE_URL!,
+  sessionToken: sarahTenantSession.sessionToken,
+  apiKey: ctx.tenantApiKey,
+})
+```
+
+## 004 — sarah creates the datalake
+
+Provision a datalake on the new tenant. The DB schema is scoped to the cookbook run via `runSuffix` so parallel
+cookbook runs do not collide on schema names. Local-dev defaults
+(`postgres` on `localhost:5432`, `alvera_dev_foundation`) match
+the seeded `dev.exs` setup; LocalStack S3 (`localhost:4566`)
+serves the lake bucket (`alvera-platform-dev`, scoped by `base_path`). The datalake's
+server-derived `slug` is captured into `datalakeSlug`.
+
+```typescript
+const DB_HOST = 'localhost'
+const DB_PORT = 5432
+const DB_USER = 'postgres'
+const DB_PASS = 'postgres'
+const DB_NAME = 'alvera_dev_foundation'
+const DB_SCHEMA = `cookbook_${runSuffix}`
+
+const S3 = {
+  cloud_storage_type: 'aws' as const,
+  region: 'us-east-1',
+  access_key_id: 'test',
+  secret_access_key: 'test',
+  endpoint: 'http://localhost:4566',
+}
+
+const datalakeResp = await api.datalakes.create(tenantSlug, {
+  name: `Cookbook Agent provider matrix Datalake ${runSuffix}`,
+  description: 'Datalake provisioned by cookbook doctest for agent-provider-matrix.',
+  timezone: 'America/New_York',
+  pool_size: 5,
+
+  db_writer_host: DB_HOST,
+  db_writer_port: DB_PORT,
+  db_writer_name: DB_NAME,
+  db_writer_schema: DB_SCHEMA,
+  db_writer_auth_method: 'password',
+  db_writer_user: DB_USER,
+  db_writer_pass: DB_PASS,
+  db_writer_enable_ssl: false,
+  db_reader_host: DB_HOST,
+  db_reader_port: DB_PORT,
+  db_reader_name: DB_NAME,
+  db_reader_schema: DB_SCHEMA,
+  db_reader_auth_method: 'password',
+  db_reader_user: DB_USER,
+  db_reader_pass: DB_PASS,
+  db_reader_enable_ssl: false,
+
+  // One bucket, scoped per run by `base_path` — the same shape the dev
+  // platform's own datalakes use. The old paired
+  // `<domain>-lake-{un,}regulated` buckets died with the two-tier lake.
+  cloud_storage: { ...S3, bucket: 'alvera-platform-dev', base_path: `cookbook/${runSuffix}` },
+})
+datalakeSlug = datalakeResp.data.slug!
+// `tools.create` and a few other endpoints accept the datalake by
+// UUID in the request body rather than only the slug in the URL,
+// so keep it on `ctx` for downstream steps that need it.
+ctx.datalakeId = datalakeResp.data.id!
+```
+
+## 005 — enqueue the datalake migrations
+
+`datalakes.create` persists the datalake row in `:new` status but
+does not itself run the schema migrations. Migration is a
+separately-triggered async job so the operator controls when the
+(potentially slow) per-industry DDL runs. `datalakes.migrate`
+enqueues the `DatalakeMigrationWorker` Oban job and returns
+immediately with `status: 'enqueued'` plus the Oban `job_id`. The
+poll in §006 then waits for that worker to finish — without this
+call the datalake would sit at `:new` forever.
+
+```typescript
+const migrateResp = await api.datalakes.migrate(tenantSlug, datalakeSlug)
+if (migrateResp.data.status !== 'enqueued') {
+  throw new Error(
+    `datalake migration not enqueued (status: ${migrateResp.data.status})`,
+  )
+}
+```
+
+## 006 — poll the datalake until status is ready
+
+Datalake migrations are async — the §005 migrate call enqueued a
+`DatalakeMigrationWorker` Oban job that runs the per-industry
+schema migrations, deploys PostgREST roles, and provisions the
+service-account bot user. Downstream resource creates (tools,
+workflows) only need `datalakeId` to be persisted (which it is
+the moment §004 returns), but any scenario step that touches
+the regulated/unregulated DBs or the dataset infrastructure
+requires `status: 'ready'`. Polling here makes every foundation
+scenario cookbook deterministic regardless of how long the cold
+migration takes on a given host; the loop exits the moment the
+datalake reaches `:ready`, so the 5-minute cap is paid only in
+failure mode.
+
+```typescript
+const READY_TIMEOUT_MS = 5 * 60_000
+const READY_POLL_MS = 5_000
+const deadline = Date.now() + READY_TIMEOUT_MS
+let datalakeStatus: string | undefined
+while (Date.now() < deadline) {
+  const { data } = await api.datalakes.get(tenantSlug, ctx.datalakeId)
+  datalakeStatus = data.status
+  if (datalakeStatus === 'ready') break
+  await new Promise((r) => setTimeout(r, READY_POLL_MS))
+}
+if (datalakeStatus !== 'ready') {
+  throw new Error(
+    `datalake did not reach :ready within ${READY_TIMEOUT_MS}ms (last status: ${datalakeStatus})`,
+  )
+}
+```
+
+## 007 — a reusable "wait until the run has fired" helper
+
+`workflows.run` only **schedules** a run. It records it and returns
+immediately with `workflow_run_id` / `status` / `scheduled_at`; the
+`workflow_run_log_id` and `batch_id` a scenario needs are written
+later, when the run actually fires, and are read back from
+`workflowRuns.get`.
+
+Two traps live in that gap, so every foundation scenario shares one
+helper rather than re-deriving it:
+
+- **Poll until `workflow_run_log_id` is a string — NOT until `status`
+  leaves `'scheduled'`.** Those are different moments: the run reaches
+  `processing` first and writes the log id a beat later. A predicate on
+  status alone releases you to read a `null`, and because
+  `typeof null === 'object'` the symptom is a baffling *"expected
+  string, got object"* rather than an obvious nil.
+- **Raise on `failed` carrying `failure_reason` rather than polling to
+  the deadline.** A scenario blocked on a run that will never fire
+  should say why on the first read, not thirty seconds later behind a
+  generic timeout.
+
+The helper is stored on the shared `ctx` bag rather than declared as a
+plain function because each numbered step compiles into its own `it()`
+block — a bare `function` here would not be in scope for the steps that
+call it.
+
+```typescript
+const FIRED_TIMEOUT_MS = 60_000
+const FIRED_POLL_MS = 1_000
+
+ctx.waitForFiredRun = async (
+  runDatalakeSlug: string,
+  runId: string,
+  timeoutMs: number = FIRED_TIMEOUT_MS,
+): Promise<{ workflowRunLogId: string; batchId: string | null }> => {
+  const deadline = Date.now() + timeoutMs
+  let lastStatus: string | undefined
+  while (Date.now() < deadline) {
+    const { data } = await api.workflowRuns.get(tenantSlug, runDatalakeSlug, runId)
+    lastStatus = data.status
+    if (data.status === 'failed') {
+      throw new Error(
+        `workflow run ${runId} failed: ${data.failure_reason ?? 'no failure_reason given'}`,
+      )
+    }
+    if (typeof data.workflow_run_log_id === 'string') {
+      return { workflowRunLogId: data.workflow_run_log_id, batchId: data.batch_id ?? null }
+    }
+    await new Promise((r) => setTimeout(r, FIRED_POLL_MS))
+  }
+  throw new Error(
+    `workflow run ${runId} never fired within ${timeoutMs}ms (last status: ${lastStatus})`,
+  )
+}
+```
+
+# Rollback
+
+The cookbook doctest harness does not currently tear down the
+created tenant / datalake / user — each run mints fresh names via
+`runSuffix` so reruns do not collide, and the seeded local DB is
+cheap to reset (`mix ecto.reset` on the platform). When the
+validator graduates to continuous integration at Milestone 18
+(running against the mockoon-backed mock env at `ALVERA_ENV=mock`),
+no real state is created at all.
+
+# Outcome
+
+After this setup runs, the closure-scoped slots are populated as:
+
+- `api` — a tenant-scoped `PlatformApi` client authenticated as
+  the industry-admin user
+- `tenantSlug` — server-derived slug of the fresh foundation
+  tenant
+- `datalakeSlug` — server-derived slug of the fresh foundation
+  datalake
+- `ctx.sarahEmail` / `ctx.sarahPassword` — the per-run admin
+  credentials (kept on `ctx` because no scenario cookbook needs
+  to re-authenticate by default)
+- `ctx.tenantApiKey` — the tenant's publishable API key (minted in
+  setup via `admin.createTenantApiKey`); thread it into any
+  additional tenant-scoped `createSession` a scenario performs
+- `ctx.waitForFiredRun(datalakeSlug, runId)` — polls `workflowRuns.get`
+  until a scheduled run has actually fired, then returns its
+  `{ workflowRunLogId, batchId }`. Use it after every `workflows.run`;
+  reading `workflow_run_log_id` off the run response returns `null`
+  because run-workflow only schedules (see §007)
+
+Scenario cookbooks under `industry: foundation` start at their own
+`§001` with these slots already populated.
+
+# See also
+
+- `.agent/datalakes.md` — datalake create body shape (regulated +
+  unregulated tier configuration)
+- `.agent/AGENTS.md` § SDK auth + client construction — root vs
+  tenantless vs tenant-scoped session scopes
+- `integration-tests/tests/foundation/bootstrap.test.ts` — the
+  green test these snippets are lifted from (the §7b sibling
+  datalake creation is out of scope for this lean cookbook setup)