@alvera-ai/platform-sdk 0.17.0 → 0.18.0

package/.agent/cookbook/_setup/foundation.md

@@ -1,359 +0,0 @@
----
-title: Foundation industry bootstrap setup
-summary: Auth as root, sign up + confirm a fresh industry-admin user, create a foundation tenant + datalake. Shared by every foundation scenario cookbook.
-industry: foundation
-slug: foundation
-vitest_source:
-  - integration-tests/tests/foundation/bootstrap.test.ts
-status: green
----
-
-# Problem
-
-Every foundation scenario cookbook needs the same starting point:
-a fresh tenant on the platform with a foundation-domain datalake
-attached, accessed by an industry-admin user. Authoring this
-prelude inline inside every scenario cookbook would duplicate the
-auth + tenant + datalake chain across two cookbooks (and four more
-across the other industries). The validator auto-discovers this
-file from each cookbook's `industry: foundation` front-matter and
-inlines its numbered steps ahead of the scenario's own, so the
-generated bun:test spec is hermetic without the cookbook author
-having to copy-paste this prelude.
-
-# Composition
-
-| Resource provisioned          | Owner       |
-|-------------------------------|-------------|
-| Root session                  | setup       |
-| Industry-admin user (`sarah`) | setup       |
-| Foundation tenant             | setup       |
-| Foundation datalake           | setup       |
-
-# Walkthrough
-
-## 001 — root admin signs in and provisions a fresh industry-admin user
-
-Authenticate as the platform's root admin (`admin@dev.local` /
-`devpassword` in local dev) via the tenantless bootstrap login —
-keyless by structural necessity (no tenant exists yet to scope a
-key to), and a dev/test-only surface. Then sign up a per-run industry-admin
-user (`sarah`) under a unique email derived from `runSuffix`, and
-confirm them so they can sign in. Both the signup and the confirm
-endpoints require root scope.
-
-```typescript
-ctx.rootSession = await createBootstrapSession({
-  baseUrl: process.env.ALVERA_BASE_URL!,
-  email: process.env.ALVERA_ROOT_EMAIL!,
-  password: process.env.ALVERA_ROOT_PASSWORD!,
-})
-ctx.rootApi = createIsolatedPlatformApi({
-  baseUrl: process.env.ALVERA_BASE_URL!,
-  sessionToken: ctx.rootSession.sessionToken,
-  apiKey: '',
-})
-
-ctx.sarahEmail = `cookbook-sarah-${runSuffix}@dev.local`
-ctx.sarahPassword = 'CookbookPass1!'
-
-const signUpResp = await ctx.rootApi.admin.signUp({
-  email: ctx.sarahEmail,
-  password: ctx.sarahPassword,
-  first_name: 'Cookbook',
-  last_name: 'Sarah',
-})
-ctx.sarahUserId = signUpResp.data.id
-await ctx.rootApi.admin.confirmUser(ctx.sarahUserId)
-```
-
-## 002 — sarah signs in (tenantless) and creates the foundation tenant
-
-Sarah signs in for the first time without a tenant scope (no
-tenant exists yet for her), then immediately creates a fresh
-foundation tenant. The tenant's server-derived `slug` is captured
-into the closure-scoped `tenantSlug` slot for downstream steps.
-
-```typescript
-ctx.sarahTenantlessSession = await createBootstrapSession({
-  baseUrl: process.env.ALVERA_BASE_URL!,
-  email: ctx.sarahEmail,
-  password: ctx.sarahPassword,
-})
-ctx.sarahTenantlessApi = createIsolatedPlatformApi({
-  baseUrl: process.env.ALVERA_BASE_URL!,
-  sessionToken: ctx.sarahTenantlessSession.sessionToken,
-  apiKey: '',
-})
-
-const tenantResp = await ctx.sarahTenantlessApi.tenants.create({
-  name: `Cookbook Foundation ${runSuffix}`,
-})
-tenantSlug = tenantResp.data.slug!
-```
-
-## 003 — sarah signs in tenant-scoped — the canonical client
-
-Re-mint sarah's session with the new tenant slug. This bearer is
-the canonical tenant-scoped client every subsequent step uses; it
-is assigned to the closure-scoped `api` slot so the scenario
-cookbook's steps inherit it via inlining.
-
-```typescript
-// A tenant-scoped login requires X-API-Key — mint a public_api key for
-// the fresh tenant via the platform-admin side door (root Bearer). In the
-// web console this is Settings -> API Keys; any of the tenant's keys
-// satisfies the login gate, and the minted Bearer inherits this key's
-// origin policy.
-const { data: mintedKey } = await ctx.rootApi.admin.createTenantApiKey(tenantSlug, {
-  name: `Cookbook Bootstrap Key ${runSuffix}`,
-  data_access_mode: 'unregulated',
-})
-ctx.tenantApiKey = mintedKey.api_key
-
-const sarahTenantSession = await createSession({
-  baseUrl: process.env.ALVERA_BASE_URL!,
-  email: ctx.sarahEmail,
-  password: ctx.sarahPassword,
-  tenantSlug,
-  apiKey: ctx.tenantApiKey,
-})
-api = createIsolatedPlatformApi({
-  baseUrl: process.env.ALVERA_BASE_URL!,
-  sessionToken: sarahTenantSession.sessionToken,
-  apiKey: ctx.tenantApiKey,
-})
-```
-
-## 004 — sarah creates the foundation datalake
-
-Provision a foundation-domain datalake on the new tenant. The DB
-schemas are scoped to the cookbook run via `runSuffix` so parallel
-cookbook runs do not collide on schema names. Local-dev defaults
-(`postgres` on `localhost:5432`, `alvera_dev_foundation`) match
-the seeded `dev.exs` setup; LocalStack S3 (`localhost:4566`)
-serves the unregulated + regulated buckets. The datalake's
-server-derived `slug` is captured into `datalakeSlug`.
-
-```typescript
-const DB_HOST = 'localhost'
-const DB_PORT = 5432
-const DB_USER = 'postgres'
-const DB_PASS = 'postgres'
-const DB_NAME = 'alvera_dev_foundation'
-const UNREG_SCHEMA = `cookbook_${runSuffix}_unreg`
-const REG_SCHEMA = `cookbook_${runSuffix}_reg`
-
-const S3 = {
-  cloud_storage_type: 'aws' as const,
-  region: 'us-east-1',
-  access_key_id: 'test',
-  secret_access_key: 'test',
-  endpoint: 'http://localhost:4566',
-}
-
-const datalakeResp = await api.datalakes.create(tenantSlug, {
-  name: `Cookbook Foundation Datalake ${runSuffix}`,
-  description: 'Foundation datalake provisioned by cookbook doctest.',
-  data_domain: 'foundation',
-  timezone: 'America/New_York',
-  pool_size: 5,
-
-  unregulated_db_writer_host: DB_HOST,
-  unregulated_db_writer_port: DB_PORT,
-  unregulated_db_writer_name: DB_NAME,
-  unregulated_db_writer_schema: UNREG_SCHEMA,
-  unregulated_db_writer_auth_method: 'password',
-  unregulated_db_writer_user: DB_USER,
-  unregulated_db_writer_pass: DB_PASS,
-  unregulated_db_writer_enable_ssl: false,
-  unregulated_db_reader_host: DB_HOST,
-  unregulated_db_reader_port: DB_PORT,
-  unregulated_db_reader_name: DB_NAME,
-  unregulated_db_reader_schema: UNREG_SCHEMA,
-  unregulated_db_reader_auth_method: 'password',
-  unregulated_db_reader_user: DB_USER,
-  unregulated_db_reader_pass: DB_PASS,
-  unregulated_db_reader_enable_ssl: false,
-
-  regulated_data_db_writer_host: DB_HOST,
-  regulated_data_db_writer_port: DB_PORT,
-  regulated_data_db_writer_name: DB_NAME,
-  regulated_data_db_writer_schema: REG_SCHEMA,
-  regulated_data_db_writer_auth_method: 'password',
-  regulated_data_db_writer_user: DB_USER,
-  regulated_data_db_writer_pass: DB_PASS,
-  regulated_data_db_writer_enable_ssl: false,
-  regulated_data_db_reader_host: DB_HOST,
-  regulated_data_db_reader_port: DB_PORT,
-  regulated_data_db_reader_name: DB_NAME,
-  regulated_data_db_reader_schema: REG_SCHEMA,
-  regulated_data_db_reader_auth_method: 'password',
-  regulated_data_db_reader_user: DB_USER,
-  regulated_data_db_reader_pass: DB_PASS,
-  regulated_data_db_reader_enable_ssl: false,
-
-  unregulated_cloud_storage: { ...S3, bucket: 'foundation-lake-unregulated' },
-  regulated_cloud_storage: { ...S3, bucket: 'foundation-lake-regulated' },
-})
-datalakeSlug = datalakeResp.data.slug!
-// `tools.create` and a few other endpoints accept the datalake by
-// UUID in the request body rather than only the slug in the URL,
-// so keep it on `ctx` for downstream steps that need it.
-ctx.datalakeId = datalakeResp.data.id!
-```
-
-## 005 — enqueue the datalake migrations
-
-`datalakes.create` persists the datalake row in `:new` status but
-does not itself run the schema migrations. Migration is a
-separately-triggered async job so the operator controls when the
-(potentially slow) per-industry DDL runs. `datalakes.migrate`
-enqueues the `DatalakeMigrationWorker` Oban job and returns
-immediately with `status: 'enqueued'` plus the Oban `job_id`. The
-poll in §006 then waits for that worker to finish — without this
-call the datalake would sit at `:new` forever.
-
-```typescript
-const migrateResp = await api.datalakes.migrate(tenantSlug, datalakeSlug)
-if (migrateResp.data.status !== 'enqueued') {
-  throw new Error(
-    `datalake migration not enqueued (status: ${migrateResp.data.status})`,
-  )
-}
-```
-
-## 006 — poll the datalake until status is ready
-
-Datalake migrations are async — the §005 migrate call enqueued a
-`DatalakeMigrationWorker` Oban job that runs the per-industry
-schema migrations, deploys PostgREST roles, and provisions the
-service-account bot user. Downstream resource creates (tools,
-workflows) only need `datalakeId` to be persisted (which it is
-the moment §004 returns), but any scenario step that touches
-the regulated/unregulated DBs or the dataset infrastructure
-requires `status: 'ready'`. Polling here makes every foundation
-scenario cookbook deterministic regardless of how long the cold
-migration takes on a given host; the loop exits the moment the
-datalake reaches `:ready`, so the 5-minute cap is paid only in
-failure mode.
-
-```typescript
-const READY_TIMEOUT_MS = 5 * 60_000
-const READY_POLL_MS = 5_000
-const deadline = Date.now() + READY_TIMEOUT_MS
-let datalakeStatus: string | undefined
-while (Date.now() < deadline) {
-  const { data } = await api.datalakes.get(tenantSlug, ctx.datalakeId)
-  datalakeStatus = data.status
-  if (datalakeStatus === 'ready') break
-  await new Promise((r) => setTimeout(r, READY_POLL_MS))
-}
-if (datalakeStatus !== 'ready') {
-  throw new Error(
-    `datalake did not reach :ready within ${READY_TIMEOUT_MS}ms (last status: ${datalakeStatus})`,
-  )
-}
-```
-
-## 007 — a reusable "wait until the run has fired" helper
-
-`workflows.run` only **schedules** a run. It records it and returns
-immediately with `workflow_run_id` / `status` / `scheduled_at`; the
-`workflow_run_log_id` and `batch_id` a scenario needs are written
-later, when the run actually fires, and are read back from
-`workflowRuns.get`.
-
-Two traps live in that gap, so every foundation scenario shares one
-helper rather than re-deriving it:
-
-- **Poll until `workflow_run_log_id` is a string — NOT until `status`
-  leaves `'scheduled'`.** Those are different moments: the run reaches
-  `processing` first and writes the log id a beat later. A predicate on
-  status alone releases you to read a `null`, and because
-  `typeof null === 'object'` the symptom is a baffling *"expected
-  string, got object"* rather than an obvious nil.
-- **Raise on `failed` carrying `failure_reason` rather than polling to
-  the deadline.** A scenario blocked on a run that will never fire
-  should say why on the first read, not thirty seconds later behind a
-  generic timeout.
-
-The helper is stored on the shared `ctx` bag rather than declared as a
-plain function because each numbered step compiles into its own `it()`
-block — a bare `function` here would not be in scope for the steps that
-call it.
-
-```typescript
-const FIRED_TIMEOUT_MS = 60_000
-const FIRED_POLL_MS = 1_000
-
-ctx.waitForFiredRun = async (
-  runDatalakeSlug: string,
-  runId: string,
-  timeoutMs: number = FIRED_TIMEOUT_MS,
-): Promise<{ workflowRunLogId: string; batchId: string | null }> => {
-  const deadline = Date.now() + timeoutMs
-  let lastStatus: string | undefined
-  while (Date.now() < deadline) {
-    const { data } = await api.workflowRuns.get(tenantSlug, runDatalakeSlug, runId)
-    lastStatus = data.status
-    if (data.status === 'failed') {
-      throw new Error(
-        `workflow run ${runId} failed: ${data.failure_reason ?? 'no failure_reason given'}`,
-      )
-    }
-    if (typeof data.workflow_run_log_id === 'string') {
-      return { workflowRunLogId: data.workflow_run_log_id, batchId: data.batch_id ?? null }
-    }
-    await new Promise((r) => setTimeout(r, FIRED_POLL_MS))
-  }
-  throw new Error(
-    `workflow run ${runId} never fired within ${timeoutMs}ms (last status: ${lastStatus})`,
-  )
-}
-```
-
-# Rollback
-
-The cookbook doctest harness does not currently tear down the
-created tenant / datalake / user — each run mints fresh names via
-`runSuffix` so reruns do not collide, and the seeded local DB is
-cheap to reset (`mix ecto.reset` on the platform). When the
-validator graduates to continuous integration at Milestone 18
-(running against the mockoon-backed mock env at `ALVERA_ENV=mock`),
-no real state is created at all.
-
-# Outcome
-
-After this setup runs, the closure-scoped slots are populated as:
-
-- `api` — a tenant-scoped `PlatformApi` client authenticated as
-  the industry-admin user
-- `tenantSlug` — server-derived slug of the fresh foundation
-  tenant
-- `datalakeSlug` — server-derived slug of the fresh foundation
-  datalake
-- `ctx.sarahEmail` / `ctx.sarahPassword` — the per-run admin
-  credentials (kept on `ctx` because no scenario cookbook needs
-  to re-authenticate by default)
-- `ctx.tenantApiKey` — the tenant's publishable API key (minted in
-  setup via `admin.createTenantApiKey`); thread it into any
-  additional tenant-scoped `createSession` a scenario performs
-- `ctx.waitForFiredRun(datalakeSlug, runId)` — polls `workflowRuns.get`
-  until a scheduled run has actually fired, then returns its
-  `{ workflowRunLogId, batchId }`. Use it after every `workflows.run`;
-  reading `workflow_run_log_id` off the run response returns `null`
-  because run-workflow only schedules (see §007)
-
-Scenario cookbooks under `industry: foundation` start at their own
-`§001` with these slots already populated.
-
-# See also
-
-- `.agent/datalakes.md` — datalake create body shape (regulated +
-  unregulated tier configuration)
-- `.agent/AGENTS.md` § SDK auth + client construction — root vs
-  tenantless vs tenant-scoped session scopes
-- `integration-tests/tests/foundation/bootstrap.test.ts` — the
-  green test these snippets are lifted from (the §7b sibling
-  datalake creation is out of scope for this lean cookbook setup)