@iann29/rastro 0.1.0-alpha.1 → 0.1.0-alpha.11

package/README.md

@@ -4,35 +4,91 @@
 
 Privacy-first, real-time web analytics built as an isolated Convex component.
 
-Amage Rastro combines a 1,022-byte gzip browser tracker with reactive traffic
+Amage Rastro combines a 1.1 KB gzip browser tracker with reactive traffic
 metrics, a live visitor atlas, journey timelines, trusted payment attribution,
 goals, funnels, affiliates, and multi-site reports. Data and functions stay
 inside your Convex deployment.
 
 ![Amage Rastro Live Atlas](assets/rastro-live-atlas.png)
 
+## Project status
+
+Amage Rastro is an actively developed alpha with a canonical batched event
+store, protected CI, exact run-scoped benchmark reconciliation, and published
+npm prereleases. The untagged `latest` channel intentionally remains on the
+first public alpha. Federation consumers must statically inspect an
+operator-approved exact registry artifact for every required runtime and type
+export before changing a host; follow the gate in the
+[federation setup runbook](docs/federation-setup.md). Never substitute a mutable
+branch or website-supplied source tarball.
+
+Public integration documentation is available at
+[`www.amagerastro.com/docs/`](https://www.amagerastro.com/docs/). Coding agents
+should start with [`llms.txt`](https://www.amagerastro.com/llms.txt) and verify
+the [machine manifest](https://www.amagerastro.com/agent/manifest.json).
+
+Current evidence supports more than 140 million stored events/day for realistic
+traffic and more than 178 million for lean traffic. Near-limit heavy payloads
+are certified at 55.26 million/day on a data-heavy store (76.62 million/day on a
+fresh store, 2026-08-22). The feature-saturated profile failed its last
+certification on a rollup-contention defect
+([#52](https://github.com/amageweb/amage-rastro/issues/52)); the contended write
+path no longer exists (report rollups are folded by one job per site, off the
+ingestion transaction) and the profile awaits recertification. Workload-specific
+results and caveats are reported separately rather than collapsed into one
+capacity claim.
+
 ## What ships
 
-| Capability                  | Implementation                                                                                      |
-| --------------------------- | --------------------------------------------------------------------------------------------------- |
-| Live visitor map            | Reactive live sessions with coarse city-level coordinates and TTL expiry                            |
-| Traffic analytics           | Sharded hour/day rollups, time series, top pages, sources, countries, devices, browsers, and events |
-| Journey replay              | Bounded pageview/click/custom-event timelines; no DOM, keystroke, or form-value recording           |
-| Payment conversion tracking | Trusted server-side ledger with idempotent event IDs and canonical site currency                    |
-| Goals and funnels           | Exact pageview/custom-event goals and ordered conversion funnels                                    |
-| Affiliates                  | Referral attribution, trusted revenue, and commission accounting                                    |
-| Cross-site analytics        | Reports over up to ten authorized site IDs with a preloaded pseudonymous visitor ID                 |
-| Auto-captured clicks        | Anchors, buttons, outbound links, and explicit `data-rastro-event` controls                         |
-| Cookieless mode             | Session-scoped pseudonymous IDs in `sessionStorage`; no cookies                                     |
-| Tiny tracker                | 1,909 bytes raw, 1,022 bytes gzip; the test suite enforces `< 1,024` bytes                          |
-
-## Install
+| Capability                  | Implementation                                                                                            |
+| --------------------------- | --------------------------------------------------------------------------------------------------------- |
+| Live visitor map            | Reactive live sessions with city-level coordinates, realtime departures, and a 90-second safety net       |
+| Traffic analytics           | Sharded hour/day rollups, time series, top pages, sources, countries, devices, browsers, and events       |
+| Field Web Vitals            | Opt-in tracker module reporting LCP, CLS, INP, FCP, and TTFB into bounded per-page daily histograms       |
+| Site map                    | Daily per-route rollups: entries, derived exits and bounces, time on page, and route-to-route transitions |
+| Journey replay              | Bounded pageview/click/custom-event timelines; no DOM, keystroke, or form-value recording                 |
+| Payment conversion tracking | Trusted server-side ledger with idempotent event IDs and canonical site currency                          |
+| Goals and funnels           | Exact pageview/custom-event goals and ordered conversion funnels                                          |
+| Affiliates                  | Referral attribution, trusted revenue, and commission accounting                                          |
+| Cross-site analytics        | Reports over up to ten authorized site IDs with a preloaded pseudonymous visitor ID                       |
+| Automatic GeoIP             | Opt-in native request metadata plus IPinfo/IPWhois fallback; bounded quota and no raw-IP persistence      |
+| Auto-captured clicks        | Anchors, buttons, and `data-rastro-event` controls with a bounded visible label; outbound origin+path     |
+| Bot filtering               | Self-declared crawlers, unfurlers, monitors, HTTP libraries, and headless browsers rejected at ingestion  |
+| Cookieless mode             | No cookies: persistent pseudonymous visitor id in `localStorage`, per-tab session id in `sessionStorage`  |
+| Tiny tracker                | 2,132 bytes raw, 1,127 bytes gzip; the test suite enforces `< 1,280` bytes gzip                           |
+
+## Quickstart
+
+### Prerequisites
+
+- Convex `^1.43.0` or a compatible Synapse deployment.
+- A way to authorize site administration and local report functions: the host's
+  existing authentication provider, or, for a site with no user accounts, a
+  fixed opaque owner ID and internal functions run with deployment credentials
+  (see [Hosts without user authentication](#hosts-without-user-authentication)).
+  The browser ingestion route itself is public and origin-checked.
+- Two deployment origins recorded separately:
+
+| Purpose                        | Convex Cloud example           | Synapse example                        |
+| ------------------------------ | ------------------------------ | -------------------------------------- |
+| Functions and reactive queries | `https://project.convex.cloud` | `https://project.synapse.example`      |
+| HTTP actions and tracker       | `https://project.convex.site`  | `https://project.site.synapse.example` |
+
+Never append `/rastro/tracker.js` to the functions URL.
+
+### 1. Install
 
 ```bash
-npm install @iann29/rastro
+npm install @iann29/rastro@alpha
 ```
 
-Mount the component under an HTTP prefix:
+The untagged npm `latest` channel intentionally remains on the first public
+alpha. Use `@alpha` until a stable release is promoted.
+
+### 2. Mount the component
+
+Merge the component into the project's existing configuration; preserve every
+component and environment binding already registered there.
 
 ```ts
 // convex/convex.config.ts
@@ -45,14 +101,164 @@ app.use(rastro, { httpPrefix: "/rastro/" });
 export default app;
 ```
 
-This creates isolated component tables and three HTTP routes:
+This creates isolated component tables and five HTTP routes:
 
 - `POST /rastro/events` — browser event ingestion
 - `OPTIONS /rastro/events` — CORS preflight
 - `GET /rastro/tracker.js` — the pre-gzipped tracker
+- `GET /rastro/vitals.js` — the pre-gzipped, opt-in Web Vitals module
 - `GET /rastro/health` — integration health
 
-## Create a site
+Push the component and regenerate host bindings before importing
+`components.rastroAnalytics`:
+
+```bash
+# Convex Cloud development deployment
+npx convex dev
+
+# Synapse development deployment
+synapse dev --once
+```
+
+For production, use the product's existing release workflow. On an uncoordinated
+Convex project that is `npx convex deploy`; on Synapse use
+`synapse release --execute` when the project has a release manifest, otherwise
+`synapse deploy`. Never deploy before confirming the target.
+
+Verify the HTTP origin independently from the functions origin:
+
+```bash
+curl -fsS https://YOUR_HTTP_ORIGIN/rastro/health
+```
+
+A successful health response proves that the component HTTP routes are mounted;
+it does not prove that a browser event was accepted.
+
+### Automatic GeoIP
+
+Rastro uses Convex request metadata as the canonical client IP and can resolve
+coarse city-level coordinates on the first request of each session. GeoIP is
+opt-in because the configured provider receives the request IP. Declare the
+settings as optional host environment variables and pass them through, so each
+deployment decides on its own provider, token, and cap without a code change:
+
+```ts
+// convex/convex.config.ts
+import { defineApp } from "convex/server";
+import { v } from "convex/values";
+import rastro from "@iann29/rastro/convex.config.js";
+
+const app = defineApp({
+  env: {
+    RASTRO_GEOIP_PROVIDER: v.optional(
+      v.union(v.literal("ipinfo"), v.literal("ipwhois")),
+    ),
+    RASTRO_GEOIP_TOKEN: v.optional(v.string()),
+    RASTRO_GEOIP_DAILY_LIMIT: v.optional(v.string()),
+    RASTRO_TRUST_PROXY: v.optional(v.literal("true")),
+  },
+});
+app.use(rastro, {
+  httpPrefix: "/rastro/",
+  env: {
+    RASTRO_GEOIP_PROVIDER: app.env.RASTRO_GEOIP_PROVIDER,
+    RASTRO_GEOIP_TOKEN: app.env.RASTRO_GEOIP_TOKEN,
+    RASTRO_GEOIP_DAILY_LIMIT: app.env.RASTRO_GEOIP_DAILY_LIMIT,
+    RASTRO_TRUST_PROXY: app.env.RASTRO_TRUST_PROXY,
+  },
+});
+
+export default app;
+```
+
+This is the shape of the [example host](example/convex/convex.config.ts). With
+nothing set, provider lookups stay off. To enable IPinfo Core on one deployment:
+
+```bash
+# Convex Cloud development deployment
+npx convex env set --deployment dev RASTRO_GEOIP_PROVIDER ipinfo
+npx convex env set --deployment dev RASTRO_GEOIP_TOKEN 'YOUR_IPINFO_TOKEN'
+npx convex env set --deployment dev RASTRO_GEOIP_DAILY_LIMIT 1000
+
+# Synapse development deployment
+synapse convex --dev env set RASTRO_GEOIP_PROVIDER ipinfo
+synapse convex --dev env set RASTRO_GEOIP_TOKEN 'YOUR_IPINFO_TOKEN'
+synapse convex --dev env set RASTRO_GEOIP_DAILY_LIMIT 1000
+```
+
+Binding a literal instead of a passthrough (`RASTRO_GEOIP_PROVIDER: "ipinfo"`)
+is also valid when every deployment of the project shares the provider.
+
+`ipwhois` is also supported. Its token is optional for the documented
+1,000-request/day free endpoint and recommended for production. Bind it as
+`RASTRO_GEOIP_TOKEN` in the same way as the IPinfo example.
+`RASTRO_GEOIP_DAILY_LIMIT` defaults to 1,000 lookups per site per UTC day and
+may be set from `0` through `1000000` to match the provider plan. Admission is
+transactional, so concurrent first-session requests cannot exceed the cap.
+
+When Rastro runs behind a reverse proxy that supplies `X-Forwarded-For` or
+Vercel/Cloudflare geographic headers, set `RASTRO_TRUST_PROXY` to `true` on that
+deployment. Leave it unset for direct Convex deployments. Rastro uses forwarded
+data only when this opt-in is enabled and the runtime IP identifies a private
+immediate peer. Caller-supplied proxy or geographic headers are ignored when
+native metadata is absent or public.
+
+The lookup has a 1.5-second timeout and fails open: telemetry ingestion
+continues without location when the provider is unavailable. A session reserves
+its single lookup attempt atomically only after its initial event is persisted;
+heartbeats and concurrent duplicate requests do not consume provider quota. The
+raw IP is never passed to a mutation or stored in component tables; only
+sanitized country, city, and coordinates rounded to one decimal place are
+persisted.
+
+`GET /rastro/health` reports whether the runtime exposes a usable client IP and
+whether GeoIP is configured and the daily cap, without returning the address or
+token. Deployment administrators can exercise the configured provider through
+the internal `geo:probe` action. If health reports no usable client IP, use
+trusted Vercel/Cloudflare geographic headers with `RASTRO_TRUST_PROXY` or fix
+the deployment's proxy forwarding. Health also names the build the deployment
+serves under `tracker`: the package `version` plus the content `hash` of
+`tracker.js` and the `vitalsHash` of `vitals.js`. Both scripts carry that hash
+as a strong `ETag`, answer `If-None-Match` with `304`, and are cached for five
+minutes with up to one hour of `stale-while-revalidate`, so a returning visitor
+runs a new build after at most one stale page load; an open tab keeps its script
+until it reloads.
+
+### Browser and device classification
+
+The tracker sends no client identification. Ingestion derives `browser`, `os`,
+and `device` from headers the browser attaches itself, and persists only those
+three normalized values; the raw `User-Agent` never reaches a mutation or a
+component table.
+
+Chromium attaches the low-entropy `Sec-CH-UA`, `Sec-CH-UA-Mobile`, and
+`Sec-CH-UA-Platform` client hints to cross-origin requests from secure origins,
+with no `Accept-CH` opt-in required of the tracked site. They are preferred when
+present, because they are the only signal separating Brave, Opera, Vivaldi,
+Samsung Internet, Yandex, and bare Chromium from Chrome: Brave ships Chrome's
+`User-Agent` verbatim by design. Hint brands are allowlisted, so an unrecognized
+or forged brand falls back to the `User-Agent` instead of entering the browser
+dimension verbatim.
+
+The `User-Agent` remains the fallback for Firefox and Safari, which implement no
+client hints, and for insecure origins, which receive none. Known limits of that
+path:
+
+- iPadOS 13+ requests sites with a Macintosh `User-Agent` by default and Safari
+  sends no hints, so those sessions are reported as desktop macOS.
+- Without hints every Android client is reported as mobile;
+  `Sec-CH-UA-Mobile: ?0` on the Android platform is the only available tablet
+  signal, because the explicit `Sec-CH-UA-Form-Factors` hint is high entropy and
+  would require an `Accept-CH` opt-in and a permissions-policy delegation from
+  every tracked site.
+- Known bots are rejected at ingestion by `User-Agent`: self-declared crawlers,
+  link unfurlers, uptime monitors, HTTP libraries, headless browsers, and any
+  agent carrying a URL. The response keeps the usual counters with `rejected`
+  set and adds `X-Rastro-Rejected: bot`; overview metadata reports
+  `botDetection` as `userAgent`. A scripted browser that spoofs a stock
+  `User-Agent` is still counted as an ordinary session.
+
+### 3. Create a site
 
 Components cannot read the host app's authentication state. The host must
 authenticate the caller and pass an opaque owner ID.
@@ -78,7 +284,7 @@ export const createSite = mutation({
     if (!identity) throw new Error("Not authenticated");
 
     return await analytics.createSite(ctx, {
-      ownerId: identity.subject,
+      ownerId: identity.tokenIdentifier,
       name: args.name,
       domains: args.domains,
       currency: args.currency,
@@ -88,12 +294,90 @@ export const createSite = mutation({
 });
 ```
 
+Call this mutation from the host application's authenticated administration UI
+and persist or display the returned `siteId`:
+
+```tsx
+const createSite = useMutation(api.analyticsAdmin.createSite);
+const siteId = await createSite({
+  name: "Amage Product",
+  domains: ["product.example.com", "localhost"],
+  currency: "BRL",
+});
+```
+
+Do not call the authenticated example with a bare CLI command: without an
+explicit test identity, `ctx.auth.getUserIdentity()` correctly returns `null`.
+
 Allowed domains are checked at ingestion. Exact hosts and wildcard subdomains
 such as `*.example.com` are supported. `Origin` is an abuse signal, not
 authentication; Amage Rastro also enforces per-session and fixed-shard per-site
 event/byte budgets.
 
-## Add the tracker
+Domain entries are hostnames, not host-and-port pairs. Use `localhost` or a full
+origin such as `http://localhost:3000`; both normalize to the `localhost` host.
+A site deployed on Vercel needs its preview hosts too: list `*.vercel.app` next
+to the production host, or preview deployments are rejected as
+`ORIGIN_NOT_ALLOWED`.
+
+#### Hosts without user authentication
+
+An institutional site with no user accounts has nobody to authenticate. Use one
+fixed opaque owner ID and internal functions, which only deployment credentials
+can run:
+
+```ts
+// convex/analyticsAdmin.ts
+import { v } from "convex/values";
+import { Rastro } from "@iann29/rastro";
+import { components } from "./_generated/api";
+import { internalMutation } from "./_generated/server";
+
+const analytics = new Rastro(components.rastroAnalytics);
+
+// Opaque and stable. Reuse the same value as `ownerId` when provisioning a
+// federation grant.
+const SITE_OWNER_ID = "acme-site-owner";
+
+export const createSite = internalMutation({
+  args: {
+    name: v.string(),
+    domains: v.array(v.string()),
+    currency: v.string(),
+  },
+  returns: v.string(),
+  handler: async (ctx, args) =>
+    await analytics.createSite(ctx, {
+      ownerId: SITE_OWNER_ID,
+      name: args.name,
+      domains: args.domains,
+      currency: args.currency,
+      cookieless: true,
+    }),
+});
+```
+
+Run it once per site with the deployment's own credentials, never from a
+browser:
+
+```bash
+# Convex Cloud development deployment
+npx convex run --deployment dev analyticsAdmin:createSite \
+  '{"name":"Acme","domains":["acme.example","*.vercel.app"],"currency":"BRL"}'
+
+# Synapse development deployment
+synapse convex --dev run analyticsAdmin:createSite \
+  '{"name":"Acme","domains":["acme.example","*.vercel.app"],"currency":"BRL"}'
+```
+
+Such a host needs no local report functions: the central dashboard reads the
+[federated surface](#federated-dashboard-backend), and its `auth.config.ts`
+lists only the Rastro federation provider, exactly as the
+[example host](example/convex/auth.config.ts) does. Goals, funnels, affiliates,
+and retention policies are configured the same way, through `internalMutation`s
+that call the `Rastro` class with `SITE_OWNER_ID`.
+
+### 4. Add the tracker
 
 Use the HTTP-actions URL for your deployment, not the Convex function URL.
 
@@ -123,8 +407,61 @@ serves the script and ingestion from different origins:
 ></script>
 ```
 
+In a Next.js App Router layout, `next/script` injects the same tag;
+`document.currentScript` is set when it runs, so `data-site` and the inferred
+endpoint work exactly as with a plain tag:
+
+```tsx
+// app/layout.tsx
+import Script from "next/script";
+
+export default function RootLayout({ children }) {
+  return (
+    <html lang="pt-BR">
+      <body>
+        {children}
+        <Script
+          src="https://YOUR_DEPLOYMENT.convex.site/rastro/tracker.js"
+          data-site="YOUR_SITE_ID"
+          strategy="afterInteractive"
+        />
+      </body>
+    </html>
+  );
+}
+```
+
+Remember the site's `domains`: a Vercel preview runs on `*.vercel.app`, not on
+the production host.
+
+### Field Web Vitals (optional)
+
+A second, separate module measures real-visitor LCP, CLS, INP, FCP, and TTFB
+with `PerformanceObserver` and reports them once per page load, when the tab is
+first hidden. It stays out of `tracker.js` so the base tracker keeps its byte
+budget; add it only where you want performance data:
+
+```html
+<script
+  defer
+  data-site="YOUR_SITE_ID"
+  src="https://YOUR_DEPLOYMENT.convex.site/rastro/vitals.js"
+></script>
+```
+
+The module shares the tracker's session identity through `sessionStorage`,
+accepts the same `data-endpoint` and `data-visitor` overrides, and works with or
+without the base tracker on the page. Measurements are attributed to the landing
+path of the page load, never to later SPA routes. Vitals feed only the
+`vitalsReport` histograms — analytics event counts, sessions, and live presence
+do not change. Browsers without an entry type simply skip that metric (Safari
+reports no LCP, CLS, or INP).
+
 ### React
 
+React is an optional peer dependency. It is required only when importing the
+`@iann29/rastro/react` entry point; backend-only consumers do not need it.
+
 ```tsx
 import { useState } from "react";
 import { RastroScript, useRastro } from "@iann29/rastro/react";
@@ -154,6 +491,34 @@ the same value on authorized sites to connect their journeys. Amage Rastro
 intentionally has no mutable post-load `identify` command: changing identity
 after a session starts would make the session ambiguous.
 
+### 5. Verify ingestion
+
+1. Open an allowed site origin in a fresh browser tab.
+2. In browser developer tools, filter Network requests by `rastro/events`.
+3. Wait for the normal tracker flush, which is debounced by approximately 1.2
+   seconds.
+4. Confirm `POST /rastro/events` returns HTTP 200 and a response containing
+   `accepted`, `duplicates`, and `rejected` counters.
+5. Run `analytics:liveVisitors` (with `siteIds` and `now`) or
+   `analytics:sessionJourney` for the returned `siteId`: both reflect the
+   request as soon as it is accepted. `analytics:listSessions` and the
+   aggregate-backed `analytics:overview` can trail the live view by a few
+   seconds, and the overview only reports complete UTC hour/day buckets, so an
+   empty result from those two right after the first request is not a failed
+   ingestion.
+
+The tracker intentionally does not interrupt the product UI when transport
+fails. A loaded `tracker.js` or successful `/health` response alone is not proof
+of ingestion. The usual causes of a rejected or missing request are:
+
+- using the functions URL instead of the HTTP-actions URL;
+- omitting the exact browser host from the site's `domains`, including
+  `*.vercel.app` for preview deployments;
+- including query strings or paths in a configured base URL;
+- querying a partial UTC bucket or a site the authenticated user does not own;
+- reading `listSessions` or `overview` seconds after the first request instead
+  of `liveVisitors`.
+
 ## Track custom events
 
 ```js
@@ -168,6 +533,12 @@ For declarative click capture:
 </button>
 ```
 
+Auto-captured clicks record the control's visible text, bounded to 64 characters
+and whitespace-normalized, so journeys read "Clicked «View pricing»" without
+annotation. `data-rastro-label` replaces that text; use it on controls whose
+copy may contain personal data. Input values are never read. Outbound links keep
+their origin and pathname; internal links keep the pathname.
+
 Properties accept bounded string, number, boolean, or null values. Never send
 emails, names, form values, secrets, or other personal data as analytics
 properties.
@@ -178,6 +549,47 @@ session or with a timestamp in another minute is allowed. The tracker preserves
 timestamps when retrying. Trusted payment event IDs remain separately
 payload-bound by the site's financial ledger.
 
+## Attribute campaigns
+
+The tracker reads `utm_source`, `utm_medium`, `utm_campaign`, `utm_term`, and
+`utm_content` from the landing URL, keeps them in `sessionStorage` for the rest
+of the session, and sends them as properties of every pageview; no other query
+parameter ever leaves the page. A session's `source` is the lowercase
+`utm_source` when present and the referrer host otherwise, so campaign traffic
+no longer reads as `direct`. `utmSource`, `utmMedium`, and `utmCampaign` stay on
+the session for journeys, and `overview.topCampaigns` ranks `utm_campaign` by
+event volume next to `topSources`.
+
+## Identify visitors after signup
+
+Anonymous sessions carry a random `visitorId` until the host supplies a
+pseudonymous one through `data-visitor` or the `RastroScript` `visitorId` prop.
+To keep the anonymous history when a visitor creates an account or signs in,
+link the two ids from the trusted server path, never from the browser:
+
+```js
+const { visitorId: anonymousVisitorId } = window.rastro("context");
+// Send anonymousVisitorId with the signup or sign-in request.
+```
+
+```ts
+await analytics.linkVisitor(ctx, {
+  siteId,
+  visitorId: user.analyticsId, // the value later passed to the tracker
+  previousVisitorId: anonymousVisitorId,
+});
+```
+
+`visitorJourney` for either id then returns one chronological journey; every
+event keeps the id it was recorded with. Links are idempotent, an anonymous id
+belongs to at most one identity, a linked id cannot be linked again, chains and
+identity merges are rejected, and an identity holds at most ten aliases. The
+tracker still sets no cookie: the host's own session identifies the account and
+the anonymous ids live in the browser's own storage (the visitor id in
+`localStorage`, the session id in `sessionStorage`). Linking anonymous history
+to an account turns it into personal data under the host's privacy policy;
+disclose it and offer an opt-out.
+
 ## Attribute trusted payments
 
 Browser-created conversion events are untrusted telemetry and never affect
@@ -239,6 +651,7 @@ export const {
   goalsReport,
   funnelsReport,
   affiliatesReport,
+  siteMap,
 } = exposeAnalyticsApi(components.rastroAnalytics, {
   authorize: async (ctx, request) => {
     const identity = await ctx.auth.getUserIdentity();
@@ -246,12 +659,12 @@ export const {
 
     for (const siteId of request.siteIds) {
       const site = await analytics.getSite(ctx, siteId);
-      if (!site || site.ownerId !== identity.subject) {
+      if (!site || site.ownerId !== identity.tokenIdentifier) {
         throw new Error("Not authorized for analytics site");
       }
     }
 
-    if (request.ownerId && request.ownerId !== identity.subject) {
+    if (request.ownerId && request.ownerId !== identity.tokenIdentifier) {
       throw new Error("Not authorized for analytics owner");
     }
   },
@@ -262,45 +675,171 @@ Do not copy the unauthenticated demo boundary into production. The included demo
 exposes only fixed-scope read reports plus bounded, idempotent seed/live-refresh
 functions.
 
+### Federated dashboard backend
+
+The reference control plane in `control-plane/` can connect a central dashboard
+while telemetry remains in the customer's deployment. It uses
+`exposeFederatedAnalyticsApi` with issuer and connection-resolver options. The
+helper constructs its authorization boundary internally; it cannot accept a
+permissive host authorizer. Its surface contains a public protocol v1 manifest,
+authenticated redacted connection status, redacted site summaries, the reactive
+dashboard report queries, and — behind the optional `configure` capability — the
+functions that manage goals, funnels, affiliates, a site's name, domains and
+timezone, and the retention policy from the dashboard.
+
+The JWT only identifies the requested connection. Site access remains
+authoritative in a local host record resolved on every query, so revocation is
+immediate and a token cannot grant itself another site. The same record decides
+whether the dashboard may configure: a configure function runs only when the
+token carries `analytics:configure` (the control plane claims it for
+organization owners and admins alone) and the host's grant lists it too;
+otherwise the dashboard shows the same forms but hands over the code the host
+runs itself. `listSites` accepts no site arguments and omits owner, domain, and
+network configuration. Stable manifest names, capabilities, limits, and
+authorization error codes are exported for connector clients. The host must
+already have Convex authentication and a local authoritative grant. Follow
+[`docs/federation-setup.md`](docs/federation-setup.md) for the executable setup
+and verification sequence; use [`docs/federation.md`](docs/federation.md) as the
+protocol reference. The production control plane is available at
+`https://www.amagerastro.com`; before connecting a customer deployment, verify
+the selected package exports the federation helper as shown in the setup
+runbook.
+
 ## Backend API
 
 The `Rastro` class wraps the component boundary for host functions:
 
 - Sites: `createSite`, `updateSite`, `getSite`, `listSites`
-- Ingestion: `ingest`, `trackConversion`
+- Ingestion: `ingest`, `trackConversion`, `linkVisitor`
 - Reports: `overview`, `liveVisitors`, `listSessions`, `sessionJourney`,
   `listConversions`, `visitorJourney`, `goalsReport`, `funnelsReport`,
-  `affiliatesReport`
+  `affiliatesReport`, `vitalsReport`, `siteMap`, `dataCoverage`
 - Configuration: `upsertGoal`, `listGoals`, `removeGoal`, `upsertFunnel`,
   `listFunnels`, `removeFunnel`, `upsertAffiliate`, `listAffiliates`,
   `removeAffiliate`
-- Retention: `cleanup`
+- Retention: `cleanup`, `setRetentionPolicy`, `disableRetentionPolicy`,
+  `runRetentionPolicy`, `retentionStatus`
 
 Every public function has argument and return validators. Growing reads are
 indexed and bounded. Component pagination uses scope-bound opaque keyset cursors
 because native `.paginate()` is not supported inside Convex components; page
 cost stays constant instead of increasing with an offset.
 
+`siteMap` reads the daily route rollups newest day first and returns per-route
+totals — pageviews, estimated visitors, entries, derived exits and bounces
+(`exits = entries + transitions in − transitions out`,
+`bounces = entries − sessions that continued`), and time on page — plus
+route-to-route transition counts and a per-route daily series. Numeric, UUID,
+and long hexadecimal path segments collapse to `:id` in the map only; the
+package exports `normalizeRoute` so dashboards bucket live visitors by the same
+rule, and `liveVisitors` reports `previousPath` and `movedAt` when a visitor
+moves between routes. Map coverage begins at the first ingest after the upgrade
+that introduced the rollups.
+
+Overview responses label visitor values as either HLL-estimated unique visitors
+or legacy `bucketOccurrences`; legacy aggregate rows are never presented as
+exact uniques. Anonymous visitors carry a persistent pseudonymous id from the
+tracker's `localStorage`, so the same person counts once across tabs, days, and
+returns. When storage is blocked or an old cached tracker reports no id,
+sessions from the same device and network within one UTC day still fold into one
+visitor through a salted daily key; runtimes without a client IP fall back to
+one visitor per session, which `/rastro/health` reports as
+`uniqueVisitors: "session"` instead of `"dailyHash"`. Breakdown metadata says
+what each ranking counts: pages by pageviews and sources, campaigns, countries,
+devices, and browsers by sessions once every bucket in the range carries those
+counters (recorded since alpha.11), and raw event volume for ranges that reach
+back before them; events always rank by volume. Totals carry the engagement
+counters recorded since the same stamp — summed session duration, sessions that
+reached a second pageview, and first-ever sessions of a visitor — and
+`metadata.engagement` reports `counted` when the whole range has them, so bounce
+rate, average duration, pages per session, and the share of new visitors derive
+from one response; `unavailable` means the range reaches back before the
+counters and those four stay blank. Feature status is explicit: outbound-link
+event volume is supported, bot detection rejects self-declared agents by
+`User-Agent`, and session replay and error insights are currently unsupported.
+
+All report ranges use integer Unix milliseconds with inclusive `from` and `to`
+boundaries, and range metadata reports `boundary: "inclusive"`. Hourly overview
+queries must contain complete UTC hours. A daily overview query must contain
+either complete UTC days or, when every requested site has the same timezone,
+complete calendar days in that zone: a site with a timezone gets one aggregate
+bucket per local calendar day at ingestion, next to the UTC hour and day
+buckets, so "yesterday" in São Paulo ends at midnight in São Paulo. Range
+metadata then reports `timezone` as the zone the buckets follow, `"UTC"`
+otherwise. Local-day coverage starts at the site's first ingest after the
+upgrade (`analyticsCoverage.localDaySince`) and restarts when its timezone
+changes; a range reaching further back is reported as partial coverage, and
+`sites.localDays` (federated `listSites.localDays`) tells a dashboard the zone
+and the stamp up front. A zone that never leaves UTC keeps no extra buckets.
+Funnel and affiliate reports must contain complete UTC days. Those
+aggregate-backed reports fail with `REPORT_INCOMPLETE` instead of silently
+including the rest of a partial bucket. Goal reports retain exact millisecond
+filtering while bounded goal-completion rows are available.
+
+Goal, funnel, and affiliate reports use definition-ID keyed daily rollups when
+the low-churn rollout control proves that generation is available. Each loader
+enforces day, row, document, and byte headroom before returning a complete
+result; it never returns a silently truncated report. Ordinary ingestion never
+patches or schedules work against a shared coverage document. `dataCoverage`
+derives availability through bounded indexed reads of source and rollup records,
+while the control document stores only rollout generation and durable retention
+watermarks. Heartbeats can advance session availability but do not claim event
+or aggregate availability. Hourly and daily overview availability are reported
+separately as `overviewHour` and `overviewDay` because their retention
+boundaries use different bucket alignment.
+
 ## Operational limits
 
 - HTTP body: 64 KiB
 - Events per batch: 50
 - Session-minute groups per batch: 4
 - Active goals per exact matcher: 1
+- Web Vitals pages per site and UTC day: 48 tracked page rows, then `(other)`;
+  device classes are fixed (desktop, mobile, tablet, unknown)
+- Web Vitals report: complete reads bounded to 5,000 rollup rows across at most
+  90 UTC days; wider saturated ranges fail with `REPORT_INCOMPLETE`
+- Site map routes per site-day shard: 200 tracked route rows, then `(other)`; 24
+  outgoing-transition slots per route row
+- Site map report: reads newest day first, bounded to 8,000 rollup rows across
+  at most 90 UTC days; older days beyond the budget are declared as partial
+  coverage instead of failing the report
 - Primary aggregate shards: 256 hourly and 128 daily, derived from stable
   session identity
 - Funnel and affiliate aggregate shards: 16
-- Site ingress budget: 4,096 deterministic per-minute shards, each capped at
-  300 events and 512 KiB (1,228,800 admitted events/minute at uniform load)
+- Report rollups (goal, funnel, affiliate, route, and Web Vitals daily rows) are
+  never written by the ingestion transaction: a batch appends one document to
+  the site's rollup queue and one leased fold job per site merges up to 200
+  queued batches per run into the rows, so every rollup row has a single writer
+  and concurrent ingestion cannot conflict on it. Reports see a batch's rollups
+  a few hundred milliseconds after ingestion accepts it; a fold chain that dies
+  is restarted by the next batch once its 60-second lease lapses
+- Site ingress budget: 4,096 deterministic per-minute shards, each capped at 300
+  events and 512 KiB (1,228,800 admitted events/minute at uniform load)
 - Rate-window storage: one reusable row per active session and site shard
-- Live session timeout: 60 seconds with one bounded five-second expiry sweep per
-  site; legacy per-session jobs hand off to that sweep during the cutover
+- Live session timeout: a `leave` beacon on `pagehide` retires the visitor about
+  ten seconds after they leave and a back/forward-cache return revives them
+  immediately; the safety net for a browser that never says goodbye is 90
+  seconds since the last event or heartbeat (heartbeats fire every 20 seconds,
+  within background-tab throttling), with one bounded five-second expiry sweep
+  per site; legacy per-session jobs hand off to that sweep during the cutover
+- Clock alignment: the tracker and the vitals module stamp each batch with
+  `sentAt`; when that clock is more than 30 seconds from the server's, every
+  event in the batch is re-anchored by the offset before ingestion, so a visitor
+  whose device clock is wrong still goes live and is filed under the right hour
 - Dashboard overview: up to ten sites and bounded report ranges
 - Hourly overview: up to 24 hours; wider ranges use daily buckets
-- Retention cleanup: bounded batches; schedule host maintenance for the data
-  classes your policy retains. The `events` cleanup kind covers both legacy
-  event rows and new event batches and processes at most 32 heavy batches per
-  transaction.
+- Retention cleanup: bounded batches. `setRetentionPolicy` automatically starts
+  leased, persisted cleanup chains for every policy-managed dataset; interrupted
+  attempts resume after lease expiry and stale generations cannot mutate the
+  active chain. A completed chain schedules its next cycle after one day and
+  recomputes its cutoff from the then-current policy. Policy updates replace the
+  generation, and `disableRetentionPolicy` makes every queued attempt stale.
+  `runRetentionPolicy` remains available for an explicit bounded run. The
+  `events` cleanup kind removes canonical event-batch documents and processes at
+  most 32 heavy batches per transaction.
+- Retention cutoffs are bucket-safe: event minutes, hourly aggregates, daily
+  aggregates and feature rollups preserve the bucket containing a cutoff and
+  return the exact effective `before` boundary they applied.
 
 Capacity claims use declared workload profiles. Complete release evidence
 reports ten-minute `lean`, `realistic`, `heavy`, and browser `feature` profiles
@@ -308,12 +847,64 @@ separately. The browser `feature` profile exercises configured goals, funnels,
 and affiliate attribution; trusted conversions require a separate server-side
 driver and certification result.
 
-The historical development evidence and its certification caveats are recorded
-in [`docs/benchmarks/2026-08-20-realistic.md`](docs/benchmarks/2026-08-20-realistic.md).
+### Certified ingest evidence (2026-08-30)
+
+| Profile   | Workload                                          | Certified stored/day | p95 latency | 100M/day target |
+| --------- | ------------------------------------------------- | -------------------: | ----------: | --------------- |
+| Lean      | Minimal events, maximum batches                   |              178.95M |    4,167 ms | Pass            |
+| Realistic | Reused sessions, browser mix, context, heartbeats |              142.04M |    4,953 ms | Pass            |
+| Heavy     | 62 KiB requests with bounded custom properties    |               55.26M |    7,702 ms | Below target    |
+| Feature   | Goals, funnels, and affiliate attribution         |           **failed** |           — | Defect [#52]    |
+
+[#52]: https://github.com/amageweb/amage-rastro/issues/52
+
+Lean, realistic, and heavy each ran at least ten minutes, completed with zero
+request failures, duplicates, or rejected events, and exactly reconciled the
+expected stored events (the realistic run's reconciliation was re-executed
+exactly after a client-side DNS failure — see the report). The realistic row's
+driver session pool is 4,096: at current speed the historical 2,048-session pool
+pushes single sessions past the documented 120 events/minute budget. Results
+came from one client process against the same non-production Synapse DEV
+deployment and site as the 2026-08-21 certification; they are engineering
+evidence, not a production SLA.
+
+Two caveats stand deliberately. The heavy row measured a store already holding
+the battery's own multi-million-event history; the 2026-08-22 fresh-store result
+(76.62M/day) remains valid for that state, and neither number overwrites the
+other. The feature profile is not certified: on alpha.6, affiliate daily-rollup
+contention lost ~4–5% of batches under concurrent load
+([#52](https://github.com/amageweb/amage-rastro/issues/52)). Since alpha.11
+ingestion no longer writes rollup rows at all (see the operational limits), but
+no feature-profile capacity is claimed until a new battery certifies it. GeoIP
+with a provider enabled, trusted server-side conversions, and vitals-bearing
+traffic remain separate pending certifications.
+
+The full evidence is recorded in
+[`docs/benchmarks/2026-08-30-alpha6-recertification.md`](docs/benchmarks/2026-08-30-alpha6-recertification.md).
+Prior results remain in
+[`docs/benchmarks/2026-08-21-formal-certification.md`](docs/benchmarks/2026-08-21-formal-certification.md)
+and
+[`docs/benchmarks/2026-08-20-realistic.md`](docs/benchmarks/2026-08-20-realistic.md).
 
 From a repository checkout, run a duration-based profile against a confirmed
 non-production deployment and a dedicated idle benchmark site:
 
+For a deployment-free CI smoke benchmark of write-heavy ingestion plus reactive
+overview and feature report queries, run:
+
+```bash
+npm run benchmark:local
+```
+
+The local harness uses `convex-test`, enforces a 12-document ingest transaction
+budget, checks reactive totals before and after a write, and queries proven
+daily rollups with the maximum configured 50 goals, 20 funnels, and 100
+affiliates under deterministic 1,000-document and 4 MiB read ceilings. It also
+drives concurrent ingest calls and asserts that their shared rollout/retention
+control document is unchanged. It fails when its generous CI latency budgets are
+exceeded. This is a deterministic local regression guard, not evidence of live
+deployment throughput or capacity.
+
 ```bash
 npm run --silent benchmark:ingest -- \
   --url https://example.site/rastro/events \
@@ -322,7 +913,8 @@ npm run --silent benchmark:ingest -- \
   --origin https://your-allowed-origin.example \
   --profile realistic \
   --duration 600 \
-  --concurrency 100 \
+  --concurrency 75 \
+  --batch-size 50 \
   --certify
 ```
 
@@ -332,49 +924,74 @@ visible in Live Atlas. Only `feature` intentionally matches the demo goal,
 funnel, and affiliate definitions; it does not call the trusted conversion API.
 Use `--dry-run` to inspect generated request size without sending traffic. When
 `--convex-url` is supplied, it must identify the same deployment as `--url` and
-expose this repository's `example:overview` query. The final reconciliation is
-a site-wide before/after delta, so unrelated traffic on that site invalidates
-the result. Reports distinguish accepted ingress, heartbeats, and stored events.
-Shorter completed runs report extrapolated daily estimates only. `--certify`
-requires at least ten minutes and successful reconciliation before emitting
-certified daily capacity fields or exiting successfully as a certification.
+expose this repository's `benchmark:reconcileSessionsPage` query. Reconciliation
+generates exact session IDs from a cryptographic run UUID and reads them through
+bounded index lookups, so unrelated traffic cannot alter the result. The query
+uses the session counters persisted transactionally with event batches; it
+certifies ingest accounting, not raw-event availability after later retention.
+Reports distinguish accepted ingress, heartbeats, and stored events. Shorter
+completed runs report extrapolated daily estimates only. `--certify` requires at
+least ten minutes and successful reconciliation before emitting certified daily
+capacity fields or exiting successfully as a certification.
 
 The ingestion transaction folds metric/dimension deltas, session rate counters,
 site-shard counters, session/live state, visitor attribution, and exact funnel
 progress before writing. Accepted non-heartbeat telemetry from one request is
 grouped into one immutable raw document per session and event-minute, without
 duplicating session geo or user-agent context. A 50-event same-session,
-same-minute baseline stays within 10 document writes. Each touched aggregate
-bucket is written at most once per batch, and dimension overflow folds into an
-explicit `(other)` bucket instead of relabeling old counts.
+same-minute baseline stays within 10 document writes plus the rollup queue entry
+and its fold lease. Each touched aggregate bucket is written at most once per
+batch, and dimension overflow folds into an explicit `(other)` bucket instead of
+relabeling old counts. Goal, funnel, affiliate, route, and Web Vitals rollup
+rows are written only by the site's fold job, never by ingestion.
 Click/outbound/browser-conversion telemetry skips goal and funnel work that
 cannot match those event types.
 
-This is an additive storage migration: existing `events` rows are not deleted or
-backfilled. Journey and trusted-conversion reads merge legacy rows with new
-`eventBatches` rows. Journey `_id` values are stable strings for both formats,
-and session-journey cursors use timestamp, creation time, and ID so pagination
-matches both storage indexes. The compatibility logic is isolated in
-`src/component/eventStore.ts`; ingestion, reports, and retention do not branch
-on physical telemetry storage.
+Raw telemetry is stored only in immutable `eventBatches`. Migrated events retain
+their original per-event browser and geo context inside those batches, while new
+events hydrate context from their session. Journey `_id` values are stable
+strings derived from batch and nested-event identity, and session-journey
+cursors use timestamp, creation time, and ID.
+
+Existing deployments that still contain the retired `events` table must not
+upgrade directly to this cutover. Follow the exact intermediate release and
+verification sequence in [`docs/upgrading.md`](docs/upgrading.md). Fresh
+deployments need no legacy-event migration.
 
 ## Privacy contract
 
 Amage Rastro's default tracker:
 
 - sets no cookies;
-- stores a pseudonymous session ID and referral slug in `sessionStorage`;
-- does not collect or persist raw visitor IP addresses;
-- stores coarse location only when the host edge provides it;
-- does not record the DOM, screenshots, keystrokes, form values, or arbitrary
-  page text;
-- captures only bounded paths, explicit properties, button/anchor targets, and
-  outbound pathnames;
+- stores a pseudonymous session ID and referral slug in `sessionStorage`, and a
+  persistent pseudonymous visitor ID in `localStorage` — scoped to the site's
+  origin, never shared across sites, gone when the visitor clears site data;
+- never persists raw visitor IP addresses in component tables;
+- counts unique visitors by that persistent pseudonymous ID; when storage is
+  blocked or the id is absent, a salted hash of the request IP and `User-Agent`
+  that rotates every UTC day takes over, derived in the HTTP action from a
+  per-deployment secret; a host-supplied `visitorId` always takes precedence,
+  and the raw IP never reaches a mutation or a table;
+- never persists the raw `User-Agent`, only a normalized browser, operating
+  system, and device;
+- processes an IP only when an edge supplies geography or an operator enables a
+  GeoIP provider, which receives that IP under its own privacy terms;
+- stores only country, city, and coordinates rounded to one decimal place;
+- does not record the DOM, screenshots, keystrokes, form values, input contents,
+  or arbitrary page text;
+- captures only bounded paths, explicit properties, the visible label of a
+  clicked link or button (at most 64 characters, replaced by `data-rastro-label`
+  when present), and outbound link origins and pathnames;
 - strips query strings and fragments from stored paths and links.
 
-Cookieless does not automatically mean consent-exempt in every jurisdiction.
-Review your collection purpose, custom properties, identity strategy, retention
-policy, and local law.
+The optional Web Vitals module inherits this contract: it reports at most five
+timing numbers per page load (milliseconds, or CLS scaled by 1000) against the
+landing path, and nothing else.
+
+Cookieless does not automatically mean consent-exempt in every jurisdiction, and
+the persistent visitor ID is an identifier under most privacy laws. Review your
+collection purpose, custom properties, identity strategy, retention policy, and
+local law.
 
 The example Live Atlas uses MapLibre with OpenFreeMap Positron. Map tiles are
 requested by the dashboard viewer; Amage Rastro visitor/session data is rendered
@@ -392,6 +1009,10 @@ synapse dev --once
 npm run dev:frontend
 ```
 
+The root `convex.json` targets the control plane in `control-plane/convex`. The
+reference consumer host in `example/` is a separate Convex app; see
+[`example/README.md`](example/README.md) to run, seed, and pair it.
+
 Quality gates:
 
 ```bash
@@ -399,20 +1020,23 @@ npm test
 npm run typecheck
 npm run lint
 npm run build
-cd example && npx vite build
+npm exec vite -- build example
 ```
 
 The test suite covers origin enforcement, event/privacy sanitization,
 idempotency, reusable session/site rate limits, bounded live expiry chains,
 folded aggregate writes, goals, funnels, trusted payments, currency safety,
 affiliate commission, cross-site journeys, keyset pagination beyond 1,000 rows,
-retention, HTTP routes, React integration, and tracker size/behavior.
+retention, request metadata and GeoIP, exact byte boundaries, HTTP routes, React
+integration, and tracker size/behavior.
 
-## Synapse demo
+## Synapse deployments
 
-The active private development project is `ianProjects / Rastro Analytics`. The
-public-facing site URL is resolved from `.env.local`; credentials and
-`.synapse/` are gitignored.
+The repository root is linked to the private Synapse project whose DEV and PROD
+deployments run the control plane in `control-plane/`. The public-facing site
+URL is resolved from `.env.local`; credentials and `.synapse/` are gitignored.
+The consumer host in `example/` is a separate Convex app with its own deployment
+link; run its CLI commands from that directory.
 
 ## License