@alvera-ai/platform-sdk 0.17.0 → 0.18.0

package/.agent/cookbook/paginated-restapi-poller.md

@@ -1,383 +0,0 @@
----
-title: "Capability: poll a paginated provider API for delivery status (Mailgun-style)"
-summary: A capability walk for restapi action status updaters. Register a REST poller tool against a provider's paginated events API (Mailgun-shaped here) and an action status updater whose request templates FOLLOW the pagination cursor — page 1 renders the collection path, later pages ride `msg.pagination_context.next`. The platform refuses a poller whose request can never advance (`api.actionStatusUpdaters.create` 422), so the guard is walked first, then the accepted shape.
-industry: foundation
-slug: paginated-restapi-poller
-vitest_source:
-  - integration-tests/tests/foundation/action-status-updaters.test.ts
-  - integration-tests/tests/foundation/bootstrap.test.ts
-status: green
----
-
-# Capability
-
-**What you get:** delivery outcomes pulled from a provider's REST events API —
-paginated, cursor-driven — written back onto the messages you sent, on a cron,
-with the platform enforcing that your poll can actually terminate.
-
-A **restapi action status updater** polls an HTTP events endpoint instead of a
-log group:
-
-- a **REST poller tool** (`intent: 'status_poller'`, `tool_body_type:
-  'rest_api'`) supplies the base URL + auth,
-- the ASU's `updater_body` renders the request per page: `path`, `params`,
-  an `events_template` that extracts the page's events as a JSON array, and a
-  `pagination_context_template` that captures `has_next` + the provider's
-  cursor,
-- on every page after the first, the driver binds what your pagination
-  template captured as `msg.pagination_context` (and the page number as
-  `msg.page`) into your `path`/`params` — **your templates must read one of
-  them, or the request is byte-identical for every page and the poll can
-  never advance. The platform rejects that config at create.**
-
-Mailgun's events API is the shape walked here (`GET /v3/<domain>/events` with
-a `paging.next` cursor link), but any cursor- or page-numbered API fits. See
-`action_status_updaters.md` §7 for the wire reference.
-
-# Walkthrough
-
-The `_setup/foundation.md` bootstrap left `api`, `tenantSlug`, `datalakeSlug`,
-and `ctx.datalakeId` populated.
-
-## 001 — register a data source for the tools
-
-Both the sender and the poller attach to a data source — the origin
-registration for the messaging provider.
-
-```typescript
-const { data: ds } = await api.dataSources.create(tenantSlug, datalakeSlug, {
-  name: `Poller Source ${runSuffix}`,
-  uri: 'mailgun.local',
-  description: 'Messaging-provider origin for the paginated delivery poller.',
-  status: 'active',
-  is_default: false,
-})
-dataSourceId = ds.id!
-```
-
-## 002 — create the sender tool whose messages get reconciled
-
-The ASU updates messages a sender produced; `sender_tool_ids` will point here.
-An SNS-backed SMS sender (LocalStack locally) plays that role.
-
-```typescript
-const { data: smsTool } = await api.tools.create(tenantSlug, datalakeSlug, {
-  name: `Poller SMS Sender ${runSuffix}`,
-  description: 'Sender whose delivery status the paginated poller reconciles.',
-  intent: 'sms',
-  status: 'active',
-  datalake_id: ctx.datalakeId,
-  data_source_id: dataSourceId,
-  body: {
-    tool_body_type: 'sns',
-    auth_method: 'access_key',
-    region: 'us-east-1',
-    phone_number: '+15551234567',
-    endpoint_url: 'http://localhost:4566',
-    access_key_id: 'test',
-    secret_access_key: 'test',
-  },
-})
-ctx.senderToolId = smsTool.id!
-```
-
-## 003 — create the REST poller tool
-
-The poller supplies base URL + auth for the provider's events API —
-Mailgun-shaped here (basic auth, `api` / API key). `intent:
-'status_poller'` tags it as a reconciler source, `tool_body_type: 'rest_api'`
-makes the ASU's requests ride this tool's HTTP client.
-
-```typescript
-const { data: restTool } = await api.tools.create(tenantSlug, datalakeSlug, {
-  name: `Mailgun Events Poller ${runSuffix}`,
-  description: 'REST poller — supplies auth for restapi ActionStatusUpdater delivery polling',
-  intent: 'status_poller',
-  status: 'active',
-  datalake_id: ctx.datalakeId,
-  data_source_id: dataSourceId,
-  body: {
-    tool_body_type: 'rest_api',
-    base_url: 'http://localhost:8080/mailgun/v3',
-    auth_method: 'basic',
-    username: 'api',
-    password: 'key-test',
-    request_type: 'json',
-    response_type: 'json',
-    timeout_ms: 30000,
-  },
-})
-ctx.restToolId = restTool.id!
-```
-
-## 004 — the pagination guard: a static request is refused at create
-
-First, the shape that does NOT work — and why the platform refuses it. This
-`path`/`params` pair reads neither `msg.pagination_context` nor `msg.page`,
-so page 500's request would be byte-identical to page 1's: the window never
-moves, `has_next` never goes false, and the run would re-apply the same
-events forever. The create is a 422; the walk catches it to prove the gate.
-
-```typescript
-let rejected = false
-try {
-  await api.actionStatusUpdaters.create(tenantSlug, datalakeSlug, {
-    name: `Mailgun Delivery Poller ${runSuffix} non-advancing`,
-    cron_expression: '*/30 * * * *',
-    updater_type: 'restapi',
-    updater_tool_id: ctx.restToolId,
-    sender_tool_ids: [ctx.senderToolId],
-    datalake_id: ctx.datalakeId,
-    // FLOOR the platform enforces: an array whose items are objects listing
-    // `external_id` in `required`. A bare `{ type: 'array' }` is a 422 at
-    // create — every event has to name the message it reconciles.
-    events_output_schema: {
-      type: 'array',
-      items: {
-        type: 'object',
-        required: ['external_id'],
-        properties: { external_id: { type: 'string' } },
-      },
-    },
-    pagination_context_output_schema: {
-      type: 'object',
-      required: ['has_next'],
-      properties: { has_next: { type: 'boolean' } },
-    },
-    updater_body: {
-      updater_body_type: 'restapi_request',
-      method: 'get',
-      // Static on both — the cursor is captured below and never read back.
-      path: { type: 'custom', body: '/wiremock.domain/events' },
-      params: { type: 'custom', body: '{"event": "delivered"}' },
-      // Providers name their own id — `message-id` here, `messageId` / `sid`
-      // elsewhere. The events_template is where that becomes `external_id`:
-      // the apply path POPS that key off every rendered event to find the row
-      // it updates, so the render PROJECTS each event rather than passing the
-      // provider body through untouched.
-      events_template: {
-        type: 'custom',
-        body:
-          '[{% for item in response.items %}' +
-          '{"external_id": "{{ item.message.headers[\'message-id\'] }}", "event": "{{ item.event }}"}' +
-          '{% unless forloop.last %},{% endunless %}{% endfor %}]',
-      },
-      pagination_context_template: {
-        type: 'custom',
-        body:
-          '{"has_next": {% if response.items.size > 0 %}true{% else %}false{% endif %}, ' +
-          '"next": "{{ response.paging.next }}"}',
-      },
-    },
-    message_config: {
-      type: 'custom',
-      body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
-    },
-    action_log_config: {
-      type: 'custom',
-      body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
-    },
-  })
-} catch (err) {
-  const status = (err as { _httpStatus?: number })._httpStatus
-  if (status !== 422) throw err
-  rejected = true
-}
-if (!rejected) {
-  throw new Error('expected a 422 for a poller whose request never consumes the cursor')
-}
-```
-
-## 005 — the accepted shape: the path follows the cursor
-
-Page 1 (`msg.pagination_context` is falsy) renders the plain collection path
-with bounded query params; every later page rides the cursor link the
-pagination template captured — and drops the params, because Mailgun's
-`paging.next` is a complete URL that already carries them.
-
-```typescript
-const { data: asu } = await api.actionStatusUpdaters.create(tenantSlug, datalakeSlug, {
-  name: `Mailgun Delivery Poller ${runSuffix} advancing`,
-  cron_expression: '*/30 * * * *',
-  updater_type: 'restapi',
-  updater_tool_id: ctx.restToolId,
-  sender_tool_ids: [ctx.senderToolId],
-  datalake_id: ctx.datalakeId,
-  events_output_schema: {
-    type: 'array',
-    items: {
-      type: 'object',
-      required: ['external_id'],
-      properties: { external_id: { type: 'string' } },
-    },
-  },
-  pagination_context_output_schema: {
-    type: 'object',
-    required: ['has_next'],
-    properties: { has_next: { type: 'boolean' } },
-  },
-  updater_body: {
-    updater_body_type: 'restapi_request',
-    method: 'get',
-    path: {
-      type: 'custom',
-      body:
-        '{% if msg.pagination_context %}{{ msg.pagination_context.next }}' +
-        '{% else %}/wiremock.domain/events{% endif %}',
-    },
-    params: {
-      type: 'custom',
-      body: '{% unless msg.pagination_context %}{"event": "delivered"}{% endunless %}',
-    },
-    // The provider's own id (`message-id`) becomes `external_id` HERE — the
-    // apply path pops that key to find the row it reconciles, so project each
-    // event instead of passing the provider body through untouched.
-    events_template: {
-      type: 'custom',
-      body:
-        '[{% for item in response.items %}' +
-        '{"external_id": "{{ item.message.headers[\'message-id\'] }}", "event": "{{ item.event }}"}' +
-        '{% unless forloop.last %},{% endunless %}{% endfor %}]',
-    },
-    pagination_context_template: {
-      type: 'custom',
-      body:
-        '{"has_next": {% if response.items.size > 0 %}true{% else %}false{% endif %}, ' +
-        '"next": "{{ response.paging.next }}"}',
-    },
-  },
-  message_config: {
-    type: 'custom',
-    body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
-  },
-  action_log_config: {
-    type: 'custom',
-    body: '{"external_id": "{{ external_id }}", "status": "delivered"}',
-  },
-})
-actionStatusUpdaterId = asu.id!
-if (asu.status !== 'active') {
-  throw new Error(`a newly created poller must be free to poll — got status ${asu.status}`)
-}
-```
-
-## 006 — discover it (list + metadata)
-
-The poller shows up in the paginated ASU list, and `metadataDetails` renders
-the markdown an agent reads to understand the reconciler.
-
-```typescript
-const { data: list } = await api.actionStatusUpdaters.list(tenantSlug, datalakeSlug)
-if (!(list.data ?? []).some((u) => u.id === actionStatusUpdaterId)) {
-  throw new Error('created restapi ASU not found in the list')
-}
-
-const { data: detail } = await api.actionStatusUpdaters.metadataDetails(tenantSlug, datalakeSlug, actionStatusUpdaterId)
-if (typeof detail !== 'string' || detail.length === 0) {
-  throw new Error('expected non-empty ASU metadata details')
-}
-```
-
-## 007 — write the integration test
-
-End the build with a test you keep: re-read the poller and assert the
-facts the scenario depends on — the create was accepted (so the
-pagination guard passed), the row is free to poll, and the run surface is
-on the wire. This block runs live under `make validate-cookbook`.
-
-```typescript
-// Re-GET — the stored row echoes the authored cron.
-const { data: poller } = await api.actionStatusUpdaters.get(tenantSlug, datalakeSlug, actionStatusUpdaterId)
-if (poller.cron_expression !== '*/30 * * * *') {
-  throw new Error(`cron mismatch on read-back: ${poller.cron_expression}`)
-}
-// cycle_detected is only ever set at runtime by the poll driver —
-// a fresh create MUST read active.
-if (poller.status !== 'active') {
-  throw new Error(`a fresh poller must be free to poll — got status ${poller.status}`)
-}
-// last_run_* is the ONLY run surface (no per-run log); fresh create ⇒
-// legitimately null. Assert the field EXISTS, never a value.
-if (!('last_run_status' in poller)) {
-  throw new Error('poller response carries no last_run_status field')
-}
-// Behavioural probe — the metadata surface an agent reads must render.
-const { data: pollerDetail } = await api.actionStatusUpdaters.metadataDetails(tenantSlug, datalakeSlug, actionStatusUpdaterId)
-if (typeof pollerDetail !== 'string' || pollerDetail.length === 0) {
-  throw new Error('poller metadataDetails came back empty')
-}
-```
-
-If the poller stops reconciling in production, refresh once, re-read
-`last_run_error`, and escalate with that evidence — don't rewrite the
-config and wait for another tick.
-
-# Gotchas
-
-- **The pagination guard is create-time and non-negotiable.** If neither
-  `path` nor `params` reads `msg.pagination_context` / `msg.page`, the create
-  is a 422 — the request could never advance past page 1. The guard reads
-  your template SOURCE, so a dead `{% if false %}{{ msg.page }}{% endif %}`
-  won't fool a reviewer even where it fools a regex; write the real cursor
-  read.
-- **Page 1 is the falsy-context branch.** `msg.pagination_context` is unset
-  on the first page — `{% if msg.pagination_context %}…{% else %}<collection
-  path>{% endif %}` is the canonical shape. Bound page 1's `params`
-  with `{% unless msg.pagination_context %}` when the cursor link already
-  carries the query (Mailgun's `paging.next` does).
-- **`has_next` decides termination — prefer the full-page heuristic in
-  production.** The walked shape (`response.items.size > 0`) terminates on
-  the first empty page, costing one extra request. Where the provider
-  documents a page size, `has_next: {% if response.items.size == 300 %}` (a
-  full page implies more) saves that call; a `paging.next` link that is
-  absent on the last page is an even stronger signal.
-- **`events_template` must render a JSON ARRAY**, validated against
-  `events_output_schema` on every cycle; the pagination render is validated
-  against `pagination_context_output_schema` (which must require `has_next`).
-  Both schemas are REQUIRED for `restapi` updaters — blank is a 422.
-- **Each schema has a FLOOR the platform enforces, above which the contract is
-  yours.** `events_output_schema` must describe an **array whose `items` are
-  objects listing `external_id` in `required`**; a bare `{ type: 'array' }` is a
-  422 at create. `pagination_context_output_schema` must be an **object listing
-  `has_next` in `required`** — that key is what ends the page loop. Demand more
-  of your provider on top if you like; only the floor is checked.
-- **The floor exists because `external_id` is how reconciliation finds the row.**
-  `apply_status_update` pops that key off every rendered event, so mapping the
-  provider's own id (`message-id` / `messageId` / `sid`) into `external_id` is
-  the **`events_template`'s job** — project each event, never pass the provider
-  body through untouched. A schema that satisfies the floor while the template
-  emits raw provider rows creates cleanly and then reconciles nothing on every
-  cycle. `message_config` / `action_log_config` then read the **mapped**
-  `{{ external_id }}`, not the provider's original key.
-- **`action_log_config` is required alongside `message_config`** for every
-  updater type, and both are cast against the *same* pinned reconciliation
-  schema — so one template body satisfies both.
-- **A newly created poller is `status: 'active'`.** `cycle_detected` is only
-  ever set at runtime by the poll driver — never by a caller; you cannot
-  create your way into it.
-- **`action_log_config` is REQUIRED alongside `message_config`** on create
-  AND update (PUT) — same per-event assigns, rendered into the action-log
-  write shape (`external_id` + at least one of `status` / `sent_at` /
-  `metadata`).
-- **One poll cycle can be fired on demand.**
-  `api.actionStatusUpdaters.refresh(tenantSlug, datalakeSlug, id)` runs the
-  cycle the cron would — `202` with the updater row AS-IS (the poll is
-  async; re-read `last_run_status` / `last_run_events_found` /
-  `last_run_error` for the outcome) — a day-two operate surface, not a
-  build step. That outcome has three values: `'ok'` (the whole window was
-  read), `'partial'` (the fetch was truncated — the newest events may be
-  missing, with `last_run_error` explaining), and `'error'` (the run
-  failed). Treat only `last_run_status === 'ok'` as a complete
-  reconciliation; a `!== 'error'` check silently accepts a truncated
-  `partial`.
-
-# See also
-
-- `action_status_updaters.md` §7 — restapi wire shape, the two output
-  schemas, `msg.*` assigns
-- `tools.md` — `rest_api` poller body, `status_poller` intent
-- `cookbook/action-status-updaters.md` — the CloudWatch-flavoured sibling
-  (log-group polling instead of HTTP pagination)
-- `_setup/foundation.md` — the bootstrap this walk starts from
-- `integration-tests/tests/foundation/action-status-updaters.test.ts` —
-  the green test these calls are lifted from (§3d–§3f)

package/.agent/cookbook/rest-fetch.md

@@ -1,273 +0,0 @@
----
-title: "Capability: pull records from a third-party REST API on demand"
-summary: A capability walk for pull-based ingestion. Register a REST-API tool with its auth (Bearer here; an OAuth2 refresh-token variant is shown too), bind it to a contract and a Data Activation Client whose `tool_call` names the path to fetch, trigger a fetch (`api.dataActivationClients.runManually`), and verify the rows landed. The platform calls the API for you — no file to upload.
-industry: subscription
-slug: rest-fetch
-vitest_source:
-  - integration-tests/tests/subscription/run-dac-fetch.test.ts
-  - integration-tests/tests/foundation/oauth2-dac-fetch.test.ts
-  - integration-tests/tests/subscription/bootstrap.test.ts
-status: green
----
-
-# Capability
-
-**What you get:** records pulled straight from a third-party REST API into your
-dataset on demand, with the platform handling auth and the HTTP call — no file
-to export and upload.
-
-A **pull-based** Data Activation Client fetches instead of receiving:
-
-- the **tool** holds the API's base URL + auth (`bearer`, or `oauth2` with a
-  refresh-token grant the platform exchanges server-side before each call),
-- the **DAC's `tool_call`** names the HTTP `method` + `path` to hit and how to
-  paginate; a `response_extractor` unwraps the API's envelope into a row array,
-- `dataActivationClients.runManually(...)` triggers a fetch and returns a
-  `batch_id`.
-
-This is **global** — only the API and its auth differ. See
-`data_activation_clients.md` §6.3 (`.runManually`) and `tools.md` (`rest_api`
-body) for the reference.
-
-# Walkthrough
-
-The `_setup/subscription.md` bootstrap left `api`, `tenantSlug`,
-`datalakeSlug`, and `ctx.datalakeId` populated. The fetch targets a mock Stripe
-API; locally that is the integration stack's WireMock on `localhost:8080`.
-
-## 001 — create the data source
-
-```typescript
-const { data: ds } = await api.dataSources.create(tenantSlug, datalakeSlug, {
-  name: `REST Stripe Source ${runSuffix}`,
-  uri: 'api.stripe.com',
-  description: 'Stripe REST API — origin of the pulled customer rows.',
-  status: 'active',
-  is_default: false,
-})
-dataSourceId = ds.id!
-```
-
-## 002 — create the REST-API tool (Bearer auth)
-
-The tool holds the API's `base_url` and credentials. `auth_method: 'bearer'`
-with a `bearer_token` is the simplest case; `status: 'active'` is required (a
-draft tool is skipped at fetch time — see Gotchas).
-
-```typescript
-const { data: tool } = await api.tools.create(tenantSlug, datalakeSlug, {
-  name: `Stripe REST Tool ${runSuffix}`,
-  description: 'WireMock-mocked Stripe REST API (AR customer fetch).',
-  intent: 'data_exchange',
-  status: 'active',
-  datalake_id: ctx.datalakeId,
-  data_source_id: dataSourceId,
-  body: {
-    tool_body_type: 'rest_api',
-    auth_method: 'bearer',
-    base_url: 'http://localhost:8080/stripe',
-    bearer_token: 'sk_test_vitest_stripe_token',
-    request_type: 'json',
-    response_type: 'json',
-    timeout_ms: 30_000,
-  },
-})
-ctx.restToolId = tool.id!
-```
-
-## 003 — create the customer interoperability contract
-
-The contract shapes each fetched customer into a `Customer` upsert. Reuse the
-vendored production Stripe customer + MDM templates.
-
-```typescript
-const { readFileSync } = await import('node:fs')
-const { join } = await import('node:path')
-const customerTemplate = readFileSync(
-  join(process.env.COOKBOOK_FIXTURES_DIR!, 'subscription/_customers_subscription_customer.liquid'),
-  'utf8',
-)
-const mdmTemplate = readFileSync(
-  join(process.env.COOKBOOK_FIXTURES_DIR!, 'subscription/_customers_subscription_mdm.liquid'),
-  'utf8',
-)
-
-const { data: contract } = await api.interoperabilityContracts.create(tenantSlug, datalakeSlug, {
-  name: `Stripe REST Customer Contract ${runSuffix}`,
-  description: 'Stripe customer fetch → Subscription Customer.',
-  resource_type: 'customer',
-  template_config: { type: 'custom', body: customerTemplate },
-  mdm_input_config: { type: 'custom', body: mdmTemplate },
-  generic_table_id: null,
-})
-interopContractId = contract.id!
-```
-
-## 004 — create the fetch DAC
-
-The Data Activation Client's `tool_call` selects the REST path: `restapi_request`
-with the HTTP `method` and the `path` to hit (`/v1/customers`, appended to the
-tool's `base_url`). `pagination_context_template` declares no further pages here.
-A top-level `response_extractor` unwraps the API's `{ data: [...] }` envelope so
-each element becomes one row.
-
-```typescript
-const { data: dac } = await api.dataActivationClients.create(tenantSlug, datalakeSlug, {
-  name: `Stripe REST Fetch DAC ${runSuffix}`,
-  description: 'Pull-based DAC — fetches Stripe customers over REST.',
-  tool_id: ctx.restToolId,
-  data_source_id: dataSourceId,
-  tool_call: {
-    tool_call_type: 'restapi_request',
-    method: 'get',
-    path: { type: 'custom', body: '/v1/customers' },
-    pagination_context_template: { type: 'custom', body: '{"has_next": false}' },
-  },
-  response_extractor: { type: 'custom', body: '{{ msg.data | to_json }}' },
-  interoperability_contract_ids: [interopContractId],
-})
-dacId = dac.id!
-ctx.dacSlug = dac.slug!
-```
-
-## 005 — trigger a fetch
-
-`runManually` enqueues a fetch and returns the `batch_id` immediately; the actual
-HTTP call + ingestion run in a background worker.
-
-```typescript
-const { data: run } = await api.dataActivationClients.runManually(tenantSlug, datalakeSlug, ctx.dacSlug)
-ctx.batchId = run.batch_id!
-if (!ctx.batchId) {
-  throw new Error('runManually did not return a batch_id')
-}
-```
-
-## 006 — verify the fetched rows landed
-
-Poll the DAC logs until the batch shows a fully-merged ingest (the mock returns 3
-customers), then search the `customer` dataset scoped to the batch.
-
-```typescript
-const deadline = Date.now() + 90_000
-let merged = false
-while (Date.now() < deadline && !merged) {
-  const { data } = await api.dataActivationClients.logs.list(tenantSlug, datalakeSlug, ctx.dacSlug)
-  merged = (data.data ?? []).some((r) => {
-    const row = r as Record<string, unknown>
-    return row.batch_id === ctx.batchId &&
-      typeof row.rows_ingested === 'number' && row.rows_ingested >= 3 &&
-      Array.isArray(row.output_files) && row.output_files.length > 0
-  })
-  if (!merged) await new Promise((r) => setTimeout(r, 1_000))
-}
-if (!merged) {
-  throw new Error('REST fetch did not produce a merged batch within 90s')
-}
-
-const { data: userSearch } = await api.datasets.createUserSearch(tenantSlug, datalakeSlug, 'customer', {
-  search_query: `ra.batch_id = '${ctx.batchId}'`,
-})
-const searchDeadline = Date.now() + 60_000
-let rows: Array<Record<string, unknown>> = []
-while (Date.now() < searchDeadline && rows.length < 3) {
-  const { data: page } = await api.datasets.search(tenantSlug, datalakeSlug, 'customer', {
-    userSearchId: userSearch.id!,
-    dataAccessMode: 'unregulated',
-  })
-  rows = (page.data ?? []) as Array<Record<string, unknown>>
-  if (rows.length < 3) await new Promise((r) => setTimeout(r, 1_000))
-}
-if (rows.length < 3) {
-  throw new Error(`expected 3 fetched customers, found ${rows.length}`)
-}
-```
-
-## 007 — variant: an OAuth2 refresh-token tool
-
-When the API uses OAuth2, the tool carries the full `oauth2_*` grant config
-instead of a static token. The platform exchanges the `oauth2_refresh_token` for
-an access token **server-side before each fetch** — your code never handles the
-token. This step creates such a tool to show the body shape (the fetch itself
-runs the same `runManually` flow once bound to a DAC + contract for that API).
-
-```typescript
-const { data: oauthTool } = await api.tools.create(tenantSlug, datalakeSlug, {
-  name: `OAuth2 REST Tool ${runSuffix}`,
-  description: 'REST tool authenticating via an OAuth2 refresh-token grant.',
-  intent: 'data_exchange',
-  status: 'active',
-  datalake_id: ctx.datalakeId,
-  data_source_id: dataSourceId,
-  body: {
-    tool_body_type: 'rest_api',
-    auth_method: 'oauth2',
-    base_url: 'http://localhost:8080/quickbooks',
-    oauth2_grant_type: 'authorization_code',
-    oauth2_client_id: 'test_client_id',
-    oauth2_client_secret: 'test_client_secret',
-    oauth2_token_url: 'http://localhost:8080/quickbooks/oauth2/v1/tokens/bearer',
-    oauth2_refresh_token: 'test_initial_refresh_token',
-    oauth2_scope: 'com.intuit.quickbooks.accounting',
-    oauth2_token_ttl: 3300,
-    request_type: 'json',
-    response_type: 'json',
-    timeout_ms: 30_000,
-  },
-})
-if (!oauthTool.id) {
-  throw new Error('OAuth2 tool create did not return an id')
-}
-```
-
-## 008 — write the integration test
-
-End the build with a test you keep: trigger one pull and assert the
-enqueue acknowledgement. `runManually` answers with the allocated
-`batch_id` BEFORE the fetch runs — the HTTP call and the ingestion drain
-on a background worker — so the durable test asserts on the call's own
-response, never on polling the merge (that diagnosis walk lives in §006).
-This block runs live under `make validate-cookbook`.
-
-```typescript
-// Re-GET — the client must still be bound to the REST tool.
-const { data: fetchDac } = await api.dataActivationClients.get(tenantSlug, datalakeSlug, dacId)
-if (fetchDac.tool_id !== ctx.restToolId) {
-  throw new Error('DAC is no longer bound to the REST fetch tool')
-}
-// Behavioural probe — one manual pull; the ack carries the batch_id
-// the worker will report under.
-const { data: probePull } = await api.dataActivationClients.runManually(tenantSlug, datalakeSlug, ctx.dacSlug)
-if (typeof probePull.batch_id !== 'string' || probePull.batch_id.length === 0) {
-  throw new Error('runManually probe returned no batch_id')
-}
-```
-
-If the probe fails in production, escalate with the response — and if
-the enqueue succeeds but rows never land, the evidence to escalate with
-is the batch's log row (§006's read), not a re-run storm.
-
-# Gotchas
-
-- **A draft tool is silently skipped at fetch time.** The worker only fetches
-  through `status: 'active'` tools — a `draft` tool produces zero log rows (no
-  error, no data). If a fetch ingests nothing, check the tool status first.
-- **The `response_extractor` unwraps the API envelope.** REST APIs wrap rows in
-  an envelope (`{ data: [...] }`); the extractor (`{{ msg.data | to_json }}`)
-  turns that into the row array the contract maps. Without it, the whole envelope
-  is treated as one row.
-- **OAuth2 token exchange is server-side.** With `auth_method: 'oauth2'`, the
-  platform swaps the refresh token for an access token before each call — your
-  code never sees a token. Bad credentials fail the exchange and produce zero log
-  rows (same silent-skip shape as a draft tool).
-- **`runManually` returns immediately.** The `batch_id` comes back before the
-  fetch runs; poll the logs to know when rows actually landed.
-
-# See also
-
-- `data_activation_clients.md` §6.3 — `.runManually` + the pull-based reference
-- `tools.md` — the `rest_api` tool body, `bearer` vs `oauth2` auth
-- `_setup/subscription.md` — the bootstrap this walk starts from
-- `integration-tests/tests/subscription/run-dac-fetch.test.ts`,
-  `integration-tests/tests/foundation/oauth2-dac-fetch.test.ts` — the green tests
-  these calls are lifted from