@iann29/rastro 0.10.8 → 0.11.0

package/docs/meta-ads.md

@@ -30,6 +30,12 @@ The Meta connection and the Conversions API sender ship in 0.10.0:
 | Spend and catalog sync                                                 | One scheduler chain for every linked ad account           |
 | `RASTRO_SECRETS_KEY`, `RASTRO_META_FAKE`, `RASTRO_META_APP_ID/SECRET`  | Optional env bindings                                     |
 | Conversions sent to the pixel                                          | Sales and new leads from the lead methods; mapped columns |
+| Lead Ads (instant forms) into the host's CRM, `metaLeads` capability   | Unreleased; see [Lead Ads](#lead-ads-instant-forms)       |
+
+Lead Ads (contract §12) is not released yet. Both halves are in this repository:
+the component's Pages, forms, rules and leadgen route, and the control plane's
+webhook and forwarding, which deliver only to hosts that serve the `metaLeads`
+side (§12.4, see [Host side](#host-side)).
 
 ## Turn on the `metaAds` capability
 
@@ -140,18 +146,26 @@ charts.
 
 - Arguments: one to ten unique `siteIds`, an inclusive `from`/`to` covering at
   most 92 UTC days, and at most one filter: `adId` for one ad's leads, or
-  `origin` for one origin's (`"ctwa_ad"` for the leads the ads bought). Passing
-  both fails with `INVALID_ARGUMENT`. Without a filter the series counts every
-  lead.
+  `origin` for one origin's (`"ctwa_ad"` for the click-to-WhatsApp ads' leads),
+  or `origin: "paid"` for every lead an ad bought (`ctwa_ad` and `lead_form`
+  together, the origins `bySite[].paid` counts). Passing both fails with
+  `INVALID_ARGUMENT`. Without a filter the series counts every lead. A host on a
+  release before Lead Ads counts nothing under `"paid"`; it has no `lead_form`
+  leads either, so a dashboard asks such a host for `"ctwa_ad"`. Every host from
+  the Lead Ads release advertises the capability `leadsPaid` with `leads` (not
+  an opt-in: it only says `leadsDaily` understands `"paid"`), so a dashboard
+  asks for `"paid"` whenever the manifest carries it, not only on hosts that
+  advertise `metaLeads`.
 - Answer: every UTC day the range touches, oldest first, empty days included
   with zeros: `{ dayStart, leads, qualified, won, revenueCents, spendCents }`.
 - Counting is by cohort, like `leads.report`: a qualification or sale lands on
   the day its lead first arrived.
 - `spendCents` is that day's spend for the same scope: the site's ads, one ad,
-  or, with `origin: "ctwa_ad"`, the ads again. It is `null` for an organic
-  origin and for sites no linked ad account covers. Spend keeps the ad account's
-  local date while cohorts use the UTC day, so a daily chart may show a day of
-  skew; totals over a week or more agree.
+  or, with `origin: "ctwa_ad"` or `"paid"`, the ads again. It is `null` for an
+  organic origin, for `lead_form` alone (the spend is not split between the two
+  kinds of ad) and for sites no linked ad account covers. Spend keeps the ad
+  account's local date while cohorts use the UTC day, so a daily chart may show
+  a day of skew; totals over a week or more agree.
 
 ### `meta.status` (`metaStatus`)
 
@@ -793,3 +807,695 @@ outros clientes" to the events sheet. Saving with it checked calls
 `onSaveEventMap` for the client, then `onCopyEventMap` with the other clients
 whose board declares the same columns and whose pixel is not another owner's
 (`eventMapCopyTargets` computes them).
+
+## Lead Ads (instant forms)
+
+A Meta instant form's leads go straight into the host's CRM. Rastro talks to
+Meta; the host owns the person. Rastro lists the owner's Facebook Pages and
+their forms, receives Meta's lead notifications, reads each lead with the Page's
+own token, applies the form's rules and hands the result to the host's **form
+lead handler** in the same transaction. The host creates or updates the contact
+and its card, and owns the first contact.
+
+**A lead's personal data is never stored by Rastro.** Name, phone, e-mail and
+every answer typed freely go from Meta to the handler in memory and are not
+written to any table, scheduled job or log. Rastro keeps the origin (Page, form,
+ad, ad set, campaign), the times, the answers to **choice questions** (the ones
+rules read) and what the rules decided.
+
+### Turn it on
+
+1. Pass the federation audience (and the issuer, when it is not the production
+   dashboard) to the component, the same values the host's `auth.config.ts`
+   trusts (the leadgen route checks the control plane's token against them):
+
+   ```ts
+   app.use(rastro, {
+     env: {
+       // ...the Meta bindings above
+       RASTRO_FEDERATION_ISSUER: app.env.RASTRO_FEDERATION_ISSUER,
+       RASTRO_FEDERATION_AUDIENCE: app.env.RASTRO_FEDERATION_AUDIENCE,
+     },
+   });
+   ```
+
+   The component's env is its own: set both on the deployment
+   (`RASTRO_FEDERATION_AUDIENCE` is required) and pass them here. The issuer
+   defaults to `https://site.api.amagerastro.com/federation`, as in the host's
+   federation API. The audience has no default: it is the host's **functions
+   URL** (`https://<deployment>.convex.cloud`, or its custom functions domain),
+   the same `applicationID` the host's `auth.config.ts` names for the issuer
+   ([federation-setup.md](federation-setup.md)), not its site URL. Without it
+   the route answers `503 FEDERATION_NOT_CONFIGURED` to every forward, and
+   `linkPage` refuses with `FEDERATION_NOT_CONFIGURED`, so no Page is linked
+   (and routed) to a host that cannot take its leads. A host paired with another
+   dashboard (a DEV control plane) sets the issuer to that dashboard's:
+   `metaLinkedPages` reports it, and the dashboard routes no lead to a host that
+   trusts another issuer. Mount the component with `httpPrefix: "/rastro/"`: the
+   dashboard posts each forward to `<site URL>/rastro/meta/leadgen`, and a host
+   under another prefix gets its leads only from the reconciliation.
+
+2. Write the handler, an internal **mutation** validated by
+   `formLeadPayloadValidator` and idempotent on `leadgenId` (Meta's webhook and
+   the reconciliation may both bring a lead), and register it once:
+
+   ```ts
+   import { createFunctionHandle } from "convex/server";
+   import { formLeadPayloadValidator } from "@iann29/rastro";
+
+   export const formLeadArrived = internalMutation({
+     args: formLeadPayloadValidator,
+     returns: v.null(),
+     handler: async (ctx, lead) => {
+       // Find the contact by lead.contact.phone, create or update it and its
+       // card in lead.decision's column (or the discard column with
+       // lead.decision.discard), attach lead.answers and lead.origin. Later
+       // stage changes use lead.leadKey with setLeadStage.
+       return null;
+     },
+   });
+
+   export const registerFormLeadHandler = internalMutation({
+     args: {},
+     handler: async (ctx) => {
+       await rastro.setFormLeadHandler(ctx, {
+         handle: await createFunctionHandle(
+           internal.rastroLeads.formLeadArrived,
+         ),
+       });
+     },
+   });
+   ```
+
+3. Opt in on the federated surface with `metaLeads: true` (beside
+   `metaAds: true`, which it needs: the Meta login and the Integrações screen
+   come with `metaAds`, so `metaLeads` alone throws at setup) and re-export the
+   Lead Ads functions from `convex/rastroFederation.ts` (see
+   [Federated Lead Ads functions](#federated-lead-ads-functions)).
+4. Ask the dashboard operator to add the connection's deployment origin
+   (`https://<name>.convex.cloud`, the functions URL) to the control plane's
+   `RASTRO_META_LEADS_DEPLOYMENT_ORIGINS`, and a custom-domain site URL to
+   `RASTRO_FEDERATION_ALLOWED_DEPLOYMENT_ORIGINS` (see
+   [Which hosts may receive leads](#lead-ads-webhook-control-plane)). Until then
+   no webhook is forwarded to the host, and leads only come in through the
+   15-minute reconciliation.
+
+Without a handler nothing is lost: a lead is not marked as ingested, the
+reconciliation reads it again once a handler exists (registering one brings the
+waiting Pages' runs forward), and `metaPages`/`metaForms` read
+`handlerReady: false` with `lastError: "HANDLER_MISSING"`.
+
+### The handler's payload
+
+```ts
+{
+  siteId: string;
+  leadId: string; // Rastro's lead
+  leadKey: string; // `meta_lead:<leadgenId>`: what setLeadStage takes
+  leadgenId: string;
+  receivedAt: number; // when the lead sent the form
+  contact: {
+    name: string | null;
+    phone: string | null;
+    email: string | null;
+  };
+  freeAnswers: Array<{ questionKey: string; label: string; value: string }>;
+  answers: Array<{ questionKey: string; label: string; values: string[] }>;
+  decision: { column: string | null } | { discard: string };
+  rulePath: number[]; // the rules looked at, in order
+  ruleIndex: number | null; // the rule that decided
+  origin: {
+    pageId: string;
+    pageName: string | null;
+    formId: string;
+    formName: string | null;
+    adId: string | null; // with adName, adsetId, adsetName, campaignId,
+    // campaignName (null until the catalog sync names them)
+    platform: string | null; // "fb" or "ig"
+    isOrganic: boolean;
+  };
+  firstContact: {
+    mode: "lead_starts" | "one_click" | "automatic";
+    text?: string;
+    brakes?: {
+      waitMinutes: number;
+      maxPerHour: number;
+      businessHoursOnly: boolean;
+    };
+  };
+}
+```
+
+The component records the lead before it calls the handler: channel `form`,
+entry point `lead_form`, the ad as its origin, the decided column (when the
+rules named one) and the form. Do not `recordLead` it again under another key;
+move it with `setLeadStage` and `leadKey`, so the sale reaches the reports and
+the pixel. A handler that throws rolls the whole ingestion back (the lead, its
+record, the rules' counts), and the next reconciliation brings it again.
+
+### Pages
+
+On a host that registered its form lead handler (`setFormLeadHandler`, the
+opt-in), every connect (`setMetaConnection`, `connectMetaWithCode`) and every
+`listMetaAdAccounts` also reads the owner's Pages
+(`GET /me/accounts?fields=id,name,access_token,tasks`) and seals each Page's own
+token with `RASTRO_SECRETS_KEY`; the first registration reads the Pages of the
+owners already connected. A host without the handler makes no Pages read and
+keeps no Page token. A login without the Page permissions lists no Page and
+still connects. `leadsAllowed` says Meta granted `leads_retrieval` for the Page
+(from `debug_token`'s granular scopes when the app env is set, else from the
+token's permissions) and the login has a task that reaches its leads
+(`ADVERTISE` or `MANAGE`).
+
+- `metaPages({ ownerKey })` (query): `pageId`, `name`, `tasks`, `leadsAllowed`,
+  `linkedSiteId` (whoever linked it: one Page feeds one site), `own`,
+  `subscribedAt`, `lastError` and `handlerReady` (the host registered its form
+  lead handler). Never a token.
+- `lastError` is a code, never a message (the message stays in the component),
+  or `null`. Each says who can fix it:
+
+  | Code                      | Cause                                                    | Fixed by                        |
+  | ------------------------- | -------------------------------------------------------- | ------------------------------- |
+  | `META_TOKEN_INVALID`      | Meta refused the Page's (or the connection's) token      | reconnecting Meta               |
+  | `META_PERMISSION_MISSING` | no `leads_retrieval` for the Page, or the login lost it  | reconnecting Meta               |
+  | `HANDLER_MISSING`         | the host removed its form lead handler                   | the host (`setFormLeadHandler`) |
+  | `HANDLER_FAILED`          | the host's handler threw; the lead is read again         | the host                        |
+  | `HANDLER_GAVE_UP`         | the handler threw 5 times; the lead was skipped for good | the host                        |
+  | `SECRETS_KEY_MISSING`     | the component cannot open the Page token                 | the host's env                  |
+  | `META_TRUNCATED`          | a busy backfill goes on next run                         | itself                          |
+  | `META_RATE_LIMITED`       | Meta's rate limit: the next run waits an hour            | itself                          |
+  | `META_UNAVAILABLE`        | Meta failed otherwise; retried with backoff              | itself                          |
+  | `RUN_TIMEOUT`             | the previous reconciliation did not finish               | itself                          |
+
+  The package exports the list as `META_PAGE_ERROR_CODES`. A Page another owner
+  linked reads `null`: its state is that owner's.
+
+- `linkPage({ siteId, pageId, ownerKey, backfillDays? })` (action) subscribes
+  the app to the Page's `leadgen` webhook
+  (`POST /{page}/subscribed_apps?subscribed_fields=leadgen` with the Page token;
+  Meta must answer `success: true`), reads its forms, records the link and
+  starts the reconciliation. A Page another site has is `CONFLICT`; one without
+  lead access is `META_PERMISSION_MISSING`; on a host without
+  `RASTRO_FEDERATION_AUDIENCE` every link is `FEDERATION_NOT_CONFIGURED`.
+  `backfillDays` (0–30, default 0) also brings the leads of the days before. At
+  most 20 Pages a site.
+- `unlinkPage({ siteId, pageId, ownerKey? })` (action) forgets the link and its
+  reconciliation. It leaves the app's `leadgen` subscription on the Page: the
+  subscription belongs to the Page and the one app every host shares, so taking
+  it off would silence another host that links the Page now (the control plane
+  drops notifications for a Page no host claims). Every 6 hours the
+  reconciliation of a linked Page subscribes it again (idempotent), so a host
+  keeps its notifications even after an older host's unlink took them off. The
+  site's form rules and the leads already in stay.
+- A Page the login no longer lists, or one without lead access, stops with
+  `META_PERMISSION_MISSING`; one whose token Meta refuses with
+  `META_TOKEN_INVALID`; a connect or refresh that lists it again resumes it. A
+  connection whose token Meta refuses stops its Pages too. Removing the
+  connection removes its Pages and links.
+- `metaLinkedPages({ siteIds })` (query):
+  `{ pages: Array<{ pageId, siteId }>, leadgenReady, issuer, audience, handlerReady }`,
+  what the dashboard's control plane reads to route Meta's webhooks to this
+  host. `handlerReady` says the host registered its form lead handler: the
+  handler is the Lead Ads opt-in, and without it no Page is read, so Formulários
+  with no Pages says the host is not ready instead of asking to reconnect Meta.
+  `leadgenReady` says the component has `RASTRO_FEDERATION_AUDIENCE` (its
+  leadgen route can verify a forward); `issuer` is the issuer that route checks
+  forwards against (`RASTRO_FEDERATION_ISSUER`, trailing slash dropped, or the
+  production default); `audience` is the `RASTRO_FEDERATION_AUDIENCE` it expects
+  (trailing slash dropped), or `null`. The control plane routes the Pages only
+  when `leadgenReady` is true, `issuer` is its own and `audience` is the
+  connection's deployment URL.
+
+### Forms and their rules
+
+- `metaForms({ siteId, now?, ownerKey? })` (query): the Pages linked to the
+  site, each with `leadsAllowed`, `subscribedAt`, `lastReconcileAt`,
+  `lastLeadAt`, `lastError` (the same codes as `metaPages`), `handlerReady` and
+  its forms: `formId`, `name`, `status` (`ACTIVE`, `ARCHIVED`…), `questions`
+  (choice questions carry their `options`), `whatsappButton` (`true`/`false`
+  from the end screen's button type, `null` when Meta does not say),
+  `whatsappButtonConfirmed` (the host ticked "Já liguei"), `createdTime`,
+  `leads30d` and `daily` (the 30 UTC days up to `now`, null without it),
+  `lastLeadAt` and a `rules` summary. At most 500 forms a site in all, every
+  Page's `ACTIVE` forms first, then the others while room is left (Meta keeps
+  archived forms forever). New forms show up by themselves on every
+  reconciliation run; `refreshForms({ siteId, pageId? })` (action) reads them at
+  once.
+- `setFormRules({ siteId, formId, entryColumn, rules, firstContact, whatsappButtonConfirmed?, ownerKey?, onlyOwn? })`
+  (mutation) replaces the form's rules. Each rule is
+  `{ questionKey, answers, to }`: `questionKey` is a choice question of the
+  form, `answers` name its options (by key or shown value; stored by key) and
+  `to` is `{ column }` or `{ discard: reason }`. The first matching rule
+  decides; with none, the lead goes to `entryColumn` (`null`: the host's own
+  entry). Columns must be on the site's pipeline (`setPipeline`).
+  `firstContact.mode` is `lead_starts`, `one_click` (needs `text`;
+  `{primeiro_nome}` is the host's to fill) or `automatic` (needs `brakes`: wait
+  15–1440 minutes, 1–120 an hour, business hours or not). Editing decides the
+  next leads only. At most 20 rules.
+- `formRules({ siteId, formId, now?, ownerKey? })` (query): the rules, each with
+  `monthCount` (the leads it decided in the 30 days up to `now`; a rule's count
+  follows its question, answers and target, so reordering keeps it and editing
+  starts a new one), `entryColumn`, `firstContact`, `configured` and, with an
+  owner, `own`.
+- `simulateFormRules({ siteId, formId, entryColumn, rules, now })` (query): the
+  last 30 days' leads under a draft: per rule, per column, per discard reason,
+  and those left to the entry. It reads the choice answers Rastro keeps, never
+  Meta. A rule still being written (no answer yet) counts zero. `entry` counts
+  the leads no rule matched; when the draft names an `entryColumn`, `columns`
+  counts them under that column too, so a screen adds `entry` only when no
+  `columns` row names the entry (unlike `formsReport`, whose `entry` is only the
+  leads sent to the host's own entry).
+- `returnFormLead({ siteId, leadId, ownerKey? })` (mutation) sends a discarded
+  lead back to the funnel: its decision becomes the host's entry and the reason
+  stays as history (`returned`). The host moves its card itself.
+- `formsReport({ siteIds, from, to })` (query): per form, over the cohort days
+  in the range, `leads`, `qualified`, `won`, `revenueCents` (from the lead
+  rollups), the leads sent to each column, to the `entry`, `discarded` per
+  reason, and `returned`. `partial` when a read cap was hit. There is one row
+  per `(siteId, formId)`: a Page moved from one site to another inside the range
+  has a row under each, so key rows by both and read the one of the site the
+  Page feeds now. The range is whole UTC days: `from` floors to its day and `to`
+  takes in the whole UTC day it falls on, so a 30-day window is
+  `to = end of a UTC day`, `from = to + 1 − 30 days`.
+
+### How a lead comes in
+
+1. Meta posts the `leadgen` webhook to the Rastro dashboard's control plane,
+   which knows which host links the Page (`metaLinkedPages`) and forwards
+   `{ pageId, leadgenId }` to `POST <host site>/rastro/meta/leadgen`.
+2. The component's route checks the control plane's token: RS256 under the
+   issuer's published keys (`<issuer>/.well-known/openid-configuration`), issuer
+   and audience as the host trusts them, subject `rastro:meta-webhook`, at most
+   10 minutes of lifetime (the control plane signs 2), purpose `meta_leadgen`,
+   naming its connection, organization and connection version, and bound to
+   exactly that Page and leadgen id. The names come from `META_LEADGEN_TOKEN`
+   (exported by the package; the control plane signs with the same object,
+   contract §12.3). Then it schedules the ingestion and answers `202`
+   `{ accepted: true }`; a Page this host does not link answers `200`
+   `{ accepted: false, reason: "PAGE_NOT_LINKED" }`, a bad token `401`
+   `{ error: { code: "UNAUTHENTICATED" } }`, a host without the audience env
+   `503` `{ error: { code: "FEDERATION_NOT_CONFIGURED" } }`, and an issuer whose
+   keys cannot be read right now (unreachable, `5xx`, `429`, not JSON), or whose
+   keys were read less than a minute ago and lack the token's key id (a key
+   rotated meanwhile), `503` `{ error: { code: "ISSUER_UNREACHABLE" } }`, which
+   is retried. The control plane reads these bodies (see
+   [Forwarding](#lead-ads-webhook-control-plane)).
+3. The ingestion reads the lead from Meta with the Page token
+   (`GET /{leadgen}?fields=created_time,field_data,ad_id,adset_id,campaign_id,form_id,platform,is_organic`),
+   splits its answers (identity and free ones in memory only), applies the
+   rules, records the lead and calls the handler. Idempotent on the leadgen id.
+4. A scheduler chain (no cron) reads every linked Page every 15 minutes: the
+   Page's forms, then each active form's leads since its cursor
+   (`GET /{form}/leads` filtered on `time_created`), oldest first, through the
+   same ingestion. Meta lists a form's leads newest first and one listing stops
+   at 2,000, so a capped listing is never trusted: the run halves the window
+   (`time_created` between the cursor and an end) until Meta lists it whole,
+   ingests it and moves the cursor to its end, then reads the next window (12
+   listings a form a run at most; a busy backfill goes on next run, with
+   `META_TRUNCATED` shown meanwhile). Rate limits push the next run back an
+   hour; other failures retry after 15 minutes, doubling up to 6 hours. It never
+   uses `paginate()`.
+5. A lead that cannot come in holds its form's cursor there (the handler
+   missing, or throwing); the leads after it are still ingested and skipped by
+   their leadgen id when read again. Each failed commit is counted by leadgen id
+   in `formLeadFailures` (ids, attempts and the error's code, never its
+   message). A lead that failed 5 times while later leads of its form came in is
+   given up on: the cursor moves past it, the link shows
+   `HANDLER_GAVE_UP: … skipped after 5 failed attempts (first: <leadgenId>)` for
+   that run and the row keeps `skippedAt`. While every lead fails (the host's
+   CRM is down) none is given up on. The host's error message is never logged
+   (it may carry the contact): only a `ConvexError`'s `data.code` or the error's
+   class name.
+6. A lead the host removes (`leads.remove`) leaves its leadgen id in
+   `formLeadTombstones` (nothing else), so no later read (the webhook retried,
+   the reconciliation, a new link's backfill) brings it back.
+
+### Reports
+
+A form lead's origin is `lead_form` when an ad brought it and
+`lead_form_organic` when the form was filled without one. `lead_form` counts as
+paid: it is in `bySite[].paid` beside `ctwa_ad`, and its ad's row in the
+campaign tree counts it. Spend stays on the `ctwa_ad` origin row, as before;
+cost per lead and return come from `bySite[].paid`. `leads.daily` takes
+`origin: "lead_form"` (no spend) and `origin: "paid"` (`ctwa_ad` and `lead_form`
+together, with the spend that bought them): a sales × spend line asks for
+`"paid"`. A lead's journey (`leads.journey`) carries `form`: its choice answers
+with their labels, the origin, the decision, `rulePath` and `returned`.
+
+### Federated Lead Ads functions
+
+With `metaLeads: true` the federated surface serves the forms screen to the
+Rastro dashboard, acting for the dashboard connection's own Meta owner like the
+rest of the Meta surface.
+
+| Function                | Kind     | Needs     |
+| ----------------------- | -------- | --------- |
+| `metaPages`             | query    | reader    |
+| `metaForms`             | query    | reader    |
+| `metaFormRules`         | query    | reader    |
+| `simulateMetaFormRules` | query    | reader    |
+| `metaFormsReport`       | query    | reader    |
+| `metaLinkedPages`       | query    | reader    |
+| `linkMetaPage`          | action   | configure |
+| `unlinkMetaPage`        | action   | configure |
+| `refreshMetaForms`      | action   | configure |
+| `setMetaFormRules`      | mutation | configure |
+| `returnMetaFormLead`    | mutation | configure |
+
+A Page linked to the site by another owner of the host, and the rules of its
+forms, are locked to the dashboard (`CONFLICT`: "Ligado pelo uze.ai"); a Page
+linked to a site outside the grant reads `linkedOutsideScope: true` without its
+site. The three actions authorize through `authorizeMetaAction`, which the host
+already exports. Without `metaLeads` the reads refuse with
+`FEDERATION_INVALID_SCOPE` ("This host does not bring Meta lead forms").
+`metaLeads` needs `metaAds`: `exposeFederatedAnalyticsApi` throws at setup when
+it gets `metaLeads: true` alone, since the dashboard shows the Formulários
+section inside Integrações and links Pages with the Meta login, both of
+`metaAds`.
+
+```ts
+export const {
+  // ...
+  metaPages,
+  metaForms,
+  metaFormRules,
+  simulateMetaFormRules,
+  metaFormsReport,
+  metaLinkedPages,
+  linkMetaPage,
+  unlinkMetaPage,
+  refreshMetaForms,
+  setMetaFormRules,
+  returnMetaFormLead,
+} = federated;
+```
+
+### Lead Ads in the demo
+
+With `RASTRO_META_FAKE=true` every login lists a Facebook Page per sample client
+(Loja X, Clínica Sorriso, Pet Mania, Casa Aurora) plus "Sorriso Kids", whose
+leads Meta did not grant. Clínica Sorriso has the four forms of the approved
+screens ("Avaliação gratuita", "Clareamento — orçamento", "Lente de contato
+dental", archived "Promo setembro"); the others one each. Every active form
+brings made-up leads at a steady pace (Avaliação gratuita one every 11h15), so a
+linked Page keeps receiving them as time passes. The example host registers its
+handler in the demo's Meta step; `demoAgency:seedAgencyForms` links each
+client's Page with 30 days of leads (never part of `seedDemo`, whose numbers are
+the screens' sample).
+
+## Lead Ads webhook (control plane)
+
+Meta sends the lead notifications of every Page to one callback per app. For the
+"Amage Rastro" app that callback is the Rastro control plane, which only routes:
+it checks Meta's signature, finds which connected host linked the Page, and
+hands that host the Page id and the lead id. It never sees the lead itself; the
+host reads it from Meta with its own Page token. What a host must serve for it
+is under [Host side](#host-side) below.
+
+**The callback.** `GET` and `POST /meta/webhook` on the control plane's
+HTTP-actions origin, in production
+`https://site.api.amagerastro.com/meta/webhook`. Two control-plane env vars:
+
+- `RASTRO_META_APP_SECRET`: the app secret. Every `POST` must carry
+  `X-Hub-Signature-256: sha256=<hex>`, the HMAC-SHA256 of the raw body under
+  this secret, compared in constant time; a missing or wrong signature is `401`
+  and nothing in the body is read. Without the variable, `POST` answers `503`,
+  so Meta keeps the notifications and retries.
+- `RASTRO_META_WEBHOOK_VERIFY_TOKEN`: a random value typed once in the app's
+  Webhooks setup. `GET` echoes `hub.challenge` only for `hub.mode=subscribe` and
+  this token (constant-time compare); anything else is `403`.
+
+In the app dashboard: Webhooks, object **Page**, callback URL above, the verify
+token, field **`leadgen`**. Each host subscribes its Pages to the app when a
+site links them. A signed `POST` answers `200` as soon as its lead ids are
+recorded; forwarding runs scheduled, so a slow host never makes Meta retry or
+disable the subscription. A signed body that is not JSON, or changes other than
+`leadgen`, are acknowledged and ignored.
+
+**Which hosts may receive leads.** A host's Page list is its own word, and Page
+ids are public, so the control plane only takes it from deployments the operator
+lists in `RASTRO_META_LEADS_DEPLOYMENT_ORIGINS` (exact origins of the
+connections' deployment URLs, comma-separated). Unset, no host receives leads
+and an unknown-Page drop logs `META_LEADS_DEPLOYMENT_ORIGINS_UNSET`. A
+connection off the list is never read, and the routes it held are ignored and
+then dropped, so it can neither take a Page nor block another host's.
+
+**Which host owns a Page.** Federation only goes from the dashboard to the host,
+so the control plane reads the hosts: on a listed connection advertising
+`metaLeads`, it calls the federated query `metaLinkedPages({ siteIds })` →
+`{ pages: Array<{ pageId; siteId }>; leadgenReady; issuer; audience }` (ten
+sites per call, at most the `maxSitesPerConnection` first sites of `listSites`,
+within 45 seconds per host) with a two-minute token of its own:
+`analytics:read`, subject `rastro:meta-page-routes`. The answer becomes
+`metaPageRoutes` rows (one per Page and connection), each keeping the host's
+`leadgenReady`, `issuer` and `audience`. A Page is routed only when
+`leadgenReady` is true and `issuer` equals the control plane's own federation
+issuer; otherwise its route row records why (`HOST_NOT_READY`: the host has no
+`RASTRO_FEDERATION_AUDIENCE`; `ISSUER_MISMATCH`: the host trusts another
+dashboard; `AUDIENCE_MISMATCH`: the host's `audience` is not the connection's
+deployment URL; `HOST_SITE_URL_MISSING` / `HOST_SITE_URL_UNTRUSTED`: the
+connection has no site origin a forward may be posted to), the reason is logged
+once per refresh, and no forward is signed for it, so a lead never waits 24
+hours on a host that would refuse it. The rows are refreshed:
+
+1. when the connection is verified ("Verificar novamente"), and when a transfer
+   of it settles;
+2. right after the dashboard links or unlinks a Page (`linkMetaPage`,
+   `unlinkMetaPage`): it calls the control plane's
+   `metaPageRoutes.refreshConnection({ connectionId })` (a member of the
+   connection's organization only), which schedules that connection's refresh;
+3. every 30 minutes for every listed connection advertising `metaLeads`;
+4. when a notification names a Page no route knows: every such host is read by a
+   refresh that started after the notification arrived, so a Page linked seconds
+   before its first lead is found. Notifications arriving while a refresh runs
+   wait for it (as long as it may run: five minutes), refreshes keep 10 seconds
+   between them, and a burst shares one. A notification that still matches
+   nothing is dropped, logged as `META_WEBHOOK_UNKNOWN_PAGE` with the Page id
+   alone; it is routed or dropped even when the refresh fails. A Page that
+   refresh found no host claiming is remembered for ten minutes
+   (`metaUnknownPages`): its next leads are dropped at once instead of reading
+   every host again. A host listing the Page clears it.
+5. when a Page two deployments claim has a claim read more than a minute ago:
+   that connection is read again (it may be stale: the Page unlinked on its host
+   without the dashboard hearing of it), and the lead is routed once more after
+   that read.
+
+A host that cannot be read keeps the routes it had, and so does a connection
+whose transfer is pending: its leads are recorded and retried
+(`CONNECTION_TRANSFER_PENDING`) until the transfer settles. A host re-verified
+without `metaLeads`, taken off the list, revoked or deleted loses them. Several
+organizations may connect the same host deployment; their routes for one Page
+are one host, and the oldest connection carries the lead. A Page that two
+different deployments claim routes to neither and logs `PAGE_ROUTE_CONFLICT` as
+an error (the host itself already refuses one Page on two sites); the leads stay
+with Meta, where each host's reconciliation can still read them.
+
+**What the dashboard shows.**
+`metaPageRoutes.connectionRouting({ connectionId })` (a query, members of the
+connection's organization only) answers
+`{ allowed, pages: Array<{ pageId, issue }> }`: `allowed` is false when the
+control plane forwards nothing of that host (off the operator's list, without
+`metaLeads`, not active), and each routed Page's `issue` is its block reason as
+read now, `CONFLICT` when another deployment claims it too, `HOST_REFUSED` when
+the host answered the last forward to this connection with `401` although it
+reports itself ready (a key or `kid` mismatch, a `jwks_uri` that is not the
+issuer's own; kept in `metaLeadgenRefusals` until a forward is delivered again),
+or `null`. Formulários puts it on each linked Page (`routeIssue`, `NOT_ROUTED`
+for a host not forwarded to): the Page card says why its leads arrive every 15
+minutes instead of at once, and "Leads chegam na hora" is not shown.
+
+**Forwarding.** Each routed notification is one `metaLeadgenForwards` row (Page
+id, lead id, connection, site, status; never lead data) and one scheduled
+attempt:
+
+```http
+POST <host site URL>/rastro/meta/leadgen
+Authorization: Bearer <token>
+Content-Type: application/json
+
+{ "pageId": "<page id>", "leadgenId": "<leadgen id>" }
+```
+
+The host site URL is the connection's HTTP-actions URL, and `/rastro/` is the
+component's `httpPrefix`: the control plane cannot learn a host's mount point,
+so Lead Ads forwarding requires the component mounted under `/rastro/` (a host
+under another prefix answers `404` to every forward, retried for 24 hours, and
+its leads arrive only with the reconciliation). An organization admin types that
+URL, so before each attempt it must be `https` on the default port and on
+`.convex.site` or `.synapsepanel.com`, or an origin listed in
+`RASTRO_FEDERATION_ALLOWED_DEPLOYMENT_ORIGINS` (a custom domain); otherwise the
+forward fails as `HOST_SITE_URL_UNTRUSTED` and nothing is signed. A token the
+control plane cannot sign (keys missing or not matching) is retried as
+`ISSUER_MISCONFIGURED`, not blamed on the host. The control plane reads the
+host's small JSON answer:
+
+- `202 { accepted: true }` (any `2xx` that does not say otherwise) is delivered.
+- `200 { accepted: false, reason: "PAGE_NOT_LINKED" }` is `not_linked`, not
+  delivered: the host no longer links the Page (unlinked, or moved to another
+  host). The control plane refreshes that connection's routes at once and
+  accepts the notification again, once, as for an unknown Page, so a Page moved
+  to another host is forwarded there.
+- `503 { error: { code: "FEDERATION_NOT_CONFIGURED" } }` and `401` are final for
+  that forward (no 24-hour retry) and recorded with their code: the host lacks
+  its audience, or does not trust this issuer, and a retry would not change it.
+  `503 { error: { code: "ISSUER_UNREACHABLE" } }` is retried: the host could not
+  read this issuer's keys for a moment.
+- `400`, `403`, `410`, `413` and `422` are final (the host read the request and
+  said no).
+
+Anything else, no answer included, is retried after 30 s, 1, 2, 5, 10, 20 and 40
+minutes, 1 hour, then every 2 hours, until 24 hours after the notification. A
+lead Meta delivers twice is forwarded once. Finished rows are kept a week, then
+deleted.
+
+The token is signed by the federation issuer with the same keys and audience as
+a connection token, so the host verifies it with the trust it already has
+(`RASTRO_FEDERATION_ISSUER`, `RASTRO_FEDERATION_AUDIENCE`). Its names are
+exported by the package as `META_LEADGEN_TOKEN` (from `@iann29/rastro`); the
+control plane signs with them and the host's route checks them, so the two
+cannot drift apart. Its claims:
+
+| Claim                       | Value                                              |
+| --------------------------- | -------------------------------------------------- |
+| `iss`                       | the control plane's federation issuer              |
+| `aud`                       | the host's deployment URL (the audience)           |
+| `sub`                       | `rastro:meta-webhook`                              |
+| `iat`, `exp`, `jti`         | two minutes of life (a host refuses more than ten) |
+| `rastro_purpose`            | `meta_leadgen`                                     |
+| `rastro_connection_id`      | the dashboard connection the Page is routed to     |
+| `rastro_organization_id`    | that connection's organization                     |
+| `rastro_connection_version` | that connection's version                          |
+| `rastro_page_id`            | the Page, equal to the body's `pageId`             |
+| `rastro_leadgen_id`         | the lead, equal to the body's `leadgenId`          |
+
+It has no `rastro_permissions`, so the federation authorizer refuses it
+(`FEDERATION_INVALID_CLAIMS`) and it can never be replayed as a read token.
+
+### Host side
+
+The component serves this half; a host turns it on as [Turn it on](#turn-it-on)
+says, and [How a lead comes in](#how-a-lead-comes-in) walks the route. What this
+control plane depends on:
+
+- The `metaLeads` capability and `metaLinkedPages` as a manifest `functions`
+  key: a host that records its functions without it is never read.
+- `metaLinkedPages({ siteIds })` answers
+  `{ pages: Array<{ pageId; siteId }>; leadgenReady; issuer; audience; handlerReady }`
+  for the Pages linked to those sites only, one site per Page, to any reader
+  token. The control plane requires `audience` to equal the connection's
+  deployment URL.
+- `POST /rastro/meta/leadgen` (the component mounted with `httpPrefix`
+  `/rastro/`, the only prefix the control plane posts to) verifies the bearer
+  token against the published keys of the issuer `metaLinkedPages` reports and
+  its own audience, with a lifetime of at most
+  `META_LEADGEN_TOKEN.maxLifetimeSeconds`; requires
+  `rastro_purpose === "meta_leadgen"`, because a dashboard member's connection
+  token is a valid JWT of the same issuer and audience and must not be able to
+  ask a host to ingest a lead; requires `rastro_page_id` and `rastro_leadgen_id`
+  to equal the body's `pageId` and `leadgenId`; then schedules the ingestion and
+  answers `202`, or `200 PAGE_NOT_LINKED`, `401`,
+  `503 FEDERATION_NOT_CONFIGURED` (final) or `503 ISSUER_UNREACHABLE` (retried)
+  as [Forwarding](#lead-ads-webhook-control-plane) reads them.
+- The connection's deployment origin is in
+  `RASTRO_META_LEADS_DEPLOYMENT_ORIGINS`, and a custom-domain site URL in
+  `RASTRO_FEDERATION_ALLOWED_DEPLOYMENT_ORIGINS`.
+
+## The Formulários screens (`FormsView`, Lead Ads)
+
+Lead Ads (instant forms, contract §12) get their own screens in
+`@iann29/rastro/ui`, pure like `IntegrationsView` and with the same props
+conventions: `audience` (`"agency"` or `"business"`, the first site only),
+`productName`, `boardLabel`, `partner`, `otherOwnerLabel` and `permitted`.
+`@iann29/rastro/ui/sample` has the approved demo in `sampleForms` (Clínica
+Sorriso, the form "Avaliação gratuita", the leads Mariana Costa and Pedro
+Alves).
+
+- `FormsView` lists the connection's Pages (`pages`, from `metaPages`) and the
+  forms of the linked ones (`forms`: `metaForms` answers a site's Pages with
+  their forms nested; `metaFormsRows` flattens the sites' answers into the rows
+  the screen reads, with each form's `leads30d`, `daily` and `lastLeadAt`).
+  `report` is `formsReport` as the host answers it (`{ forms, partial }`, the
+  columns and discards as arrays, one row per `(siteId, formId)`, read for the
+  site the Page feeds now), for the summary and the drawer. `rules` is
+  `formRules` per form id: `configured: false` is a form that never had rules
+  ("Novo · sem regra"), `undefined` is still loading, and `null` means the host
+  could not answer, which shows the rules as unreadable and blocks saving so an
+  empty draft never overwrites them. A Page without `leadsAllowed` shows
+  "Reconectar a Meta" (`onReconnect`); an unlinked Page offers
+  `onLinkPage(siteId, pageId)`, a linked one `onUnlinkPage` after a
+  confirmation. A linked Page's `lastError` code gets its own short text: only
+  `META_TOKEN_INVALID` and `META_PERMISSION_MISSING` ask to reconnect Meta;
+  `HANDLER_MISSING` (or `handlerReady: false`) says the host still needs its
+  form lead handler; the codes that pass by themselves (`META_TRUNCATED`,
+  `META_RATE_LIMITED`, `META_UNAVAILABLE`, `RUN_TIMEOUT`) never say something is
+  broken. A Page linked to a site outside the reader's grant
+  (`linkedOutsideScope`, the site not named) shows "Ligada a outro cliente/site"
+  and is never offered. A Page or rules another owner set (`own: false`) show
+  "Ligado pelo <otherOwnerLabel>" and stay read only.
+- `FormRulesDrawer` (a row opens it) edits one form: the entry column (null is
+  the board's own door), the ordered rules (first match wins; drag or arrow keys
+  reorder; `monthCount` per stored rule; a discard keeps its reason on the
+  card), the last 30 days simulated under the draft
+  (`onSimulate(siteId, formId, draft)` asks, debounced, with the entry and the
+  complete rules only, so editing the first message never asks again;
+  `simulation` is `simulateFormRules`' answer, whose `columns` already hold the
+  entry column's leads when the draft names one) and how the first contact
+  happens: the lead writes first (recommended, the button's state), one click by
+  a person (always on, the suggested text with `{primeiro_nome}`) or the
+  automatic send, which turns on only behind the risk checkbox and keeps its
+  brakes (wait at least 15 min, a maximum per hour, business hours; the random
+  interval between sends is fixed). Saving calls
+  `onSaveRules(siteId, formId, rules)` with `setFormRules`'s shape; the drawer
+  closes when that resolves to anything but `false`, and a refusal keeps the
+  draft open beside `error`. It changes only the next leads. When Meta does not
+  say whether the button is there (`whatsappButton: null`), the company marks
+  "Já liguei": that is `whatsappButtonConfirmed`, saved on its own and read back
+  from `formRules`; without the automatic send, who writes first follows it
+  (`lead_starts` with the button, `one_click` without).
+- `WhatsAppButtonGuide` (the drawer's "Ver como ligar") walks the four steps in
+  Meta's form editor and lists what can be checked: the published copy and its
+  button from the Page's forms, plus the host's own `guideChecks`.
+  `onRefreshForms(siteId)` is its "Conferir de novo".
+- `FormLeadBadge` ("Formulário · Avaliação gratuita", or "veio do formulário")
+  and `FormLeadAnswers` (answers, origin and the path the lead followed, as a
+  `card`, a `panel` or the phone's `compact`) are for the host's card and
+  contact panel. The host passes what it holds: Rastro never stores a form
+  lead's name, phone, e-mail or free-text answers.
+  `formLeadFromJourney(journey)` turns a lead journey's `form` (`leads.journey`:
+  origin, choice answers, `rulePath`, decision) into their `FormLead`;
+  Campanhas' lead drawer uses it to show the badge and the answers whenever
+  `journey.form` is there.
+
+Each piece reads its own width with container queries: the list turns into one
+button per form under 860px, and the drawer into the phone layout under 640px.
+
+The dashboard shows a "Formulários" tab in Integrações only for a host that
+advertises `metaLeads` and lists `metaPages`. It calls the federated functions
+by the names of `FEDERATED_ANALYTICS_META_LEAD_FUNCTIONS`: `metaPages`,
+`metaForms` (one call per site, with `now`), `metaFormRules`,
+`simulateMetaFormRules` (the entry, the rules and `now`; no first contact),
+`metaFormsReport` (reads) and `linkMetaPage`, `unlinkMetaPage`,
+`setMetaFormRules` (with `whatsappButtonConfirmed`), `refreshMetaForms` (writes,
+behind `analytics:configure`); the report, the simulation and "Conferir de novo"
+each wait for the host to list their function. The host's 30-day reads take the
+end of the current UTC day as `now`, and the report a 30-day window, both
+following the dashboard's clock; the host counts the report in whole UTC days
+(see `formsReport` above), so a window aligned on UTC days matches the rows'
+`leads30d`. Right after a Page is linked or unlinked, the dashboard asks its
+control plane to refresh that connection's Page routes
+(`metaPageRoutes.refreshConnection`), so the first lead of a new link is routed
+without waiting. A form whose rules the host could not read stays unreadable
+rather than empty. Each site's board comes from the columns its host declared
+with `setPipeline`. `returnMetaFormLead` (sending a discarded lead back) has no
+button on the dashboard yet.
+
+In Campanhas, a form an ad ran arrives with origin `lead_form`, which counts as
+paid next to `ctwa_ad` (it is never listed among the organic origins), and its
+line reads "Formulário · <ad>"; a form filled without an ad arrives as
+`lead_form_organic`, "Formulário instantâneo". Both enter through the
+`lead_form` entry point, and the journey starts with "Enviou o formulário do
+anúncio" (or "Enviou um formulário instantâneo"). Meta's short platform keys
+(`ig`, `fb`) read as the app's name. The Vendas line, and the portfolio's
+sparklines and week alerts, read `leadsDaily` with `origin: "paid"` (every lead
+an ad bought, against the same spend) on a host whose manifest carries
+`leadsPaid` (or `metaLeads`), and `"ctwa_ad"` on a host of an older release.