@iann29/rastro 0.2.0 → 0.4.0

package/docs/upgrading.md

@@ -23,12 +23,94 @@ hosts still on them.
 | `0.1.0-alpha.11`                                      | Install `0.1.0-alpha.12` directly; same code, the package now ships `src/tracker/generated.ts`                                                  | A `convex/` test importing `@iann29/rastro/test` typechecks on push |
 | `0.1.0-alpha.12`                                      | Install `0.1.0` directly; every schema change is an optional field or a new table; add `analytics:public` to a grant only to allow public links | `/rastro/health` reports `tracker.version: "0.1.0"`                 |
 | `0.1.0`                                               | Install `0.2.0` directly; add the optional host callback and grant version to enable transfers                                                  | Health reports `0.2.0`; opted-in connectors advertise `transfer`    |
+| `0.2.0`                                               | Install `0.3.0` directly; no new table; export `getSession` and `revenueSummary` from the federation module                                     | Health reports `0.3.0`; the manifest lists `sessionDetails`         |
+| `0.3.0`                                               | Install `0.4.0` directly; no re-export; the deploy rebuilds the `reportRollupQueue` index and the tracker changes                               | Health reports `0.4.0`; a hidden tab leaves **Ao vivo** in ~10 s    |
 | Any release whose packed declarations lack federation | Stop and request an eligible exact registry release                                                                                             | Packed declarations contain the federation exports                  |
 
 Do not upgrade a populated legacy deployment directly to a release that removes
 the `events` table. Schema acceptance alone does not prove that telemetry was
 migrated.
 
+## Sessions, presence and rollups (0.4.0)
+
+`0.4.0` needs no host edit: no new function, capability, or re-export. Install
+`@iann29/rastro@0.4.0`, deploy the host through its normal workflow, then open
+**Conexões** and click **Verificar novamente** on the active connection.
+
+What reports show differently:
+
+- **Sessions.** A session is one identity's visit in one tab: a sign-in (a new
+  `data-visitor`) or 30 minutes without an event opens a new one, keeping the
+  referral and campaign. A tab returning after 30 idle minutes counts a pageview
+  of its current page. A tab open during the upgrade starts one new session on
+  its next page load.
+- **Presence.** A hidden tab sends `leave` and drops out of **Ao vivo** about
+  ten seconds later; shown again, it sends a heartbeat at once and returns. A
+  hidden tab sends no requests.
+- **Web Vitals** need the base tracker on the page; `vitals.js` now derives its
+  ids by the tracker's rule, so the default install's vitals are accepted.
+- **Duration** is counted at departure. Heartbeats no longer rewrite aggregate
+  buckets, so a visitor still on a page shows the time up to their last page or
+  event; the rest lands when they leave (about ten seconds after a `leave`, at
+  most 90 seconds after the last heartbeat). Finished visits are unchanged.
+- **Rollups** fold about once a second per site, so bucketed reports trail
+  ingestion by about a second.
+- **Overview.** A busy range fails with `REPORT_INCOMPLETE` instead of a raw
+  read-limit error, and `LIMIT_EXCEEDED` names the real cap: 101 days for one
+  site, 10 for ten; hourly, 24 hours for two sites, 5 for ten.
+- **Funnels.** A completed funnel can be entered again once its conversion
+  window has passed.
+
+Operations:
+
+- The deploy swaps the `reportRollupQueue` index for
+  `by_siteId_and_deadLetteredAt_and_createdAt` and backfills it during the push.
+  The queue holds only batches waiting for their fold, so no staged phase is
+  needed. A batch that fails three fold runs is logged and set aside for 30
+  days.
+- Ingestion admits at most 1,200 events and 2 MiB a minute per site from one
+  client network (an IPv4 address or an IPv6 /64) when `/rastro/health` reports
+  `clientIpAvailable: true`; visitors behind one NAT share it. The per-site
+  budget drops to 384,000 events a minute. Over budget answers 429
+  `RATE_LIMITED`.
+- `RASTRO_CLIENT_RATE_LIMIT=off` lifts the per-client budget for load-test
+  canaries only; leave it unset everywhere else.
+
+Acceptance for this upgrade: `/rastro/health` reports `tracker.version: "0.4.0"`
+and a new `tracker.hash`; `GET /rastro/tracker.js` answers with a new `ETag`,
+which returning browsers pick up within the cache window (see
+[Tracker cache](#tracker-cache)); in a browser context opened after the deploy,
+a hidden tab disappears from **Ao vivo** within about ten seconds and returns
+when shown again.
+
+## Session details and complete revenue lists (0.3.0)
+
+After upgrading the package, export `getSession` and `revenueSummary` from the
+object returned by `exposeFederatedAnalyticsApi` in
+`convex/rastroFederation.ts`, then deploy the host and reverify its dashboard
+connection. The manifest advertises `sessionDetails` and `revenueSummary`;
+advertising them without exporting the functions leaves an incomplete host
+surface.
+
+`getSession({siteId, sessionId})` returns the retained session metadata (or
+null). The dashboard pairs it with the existing paginated `sessionJourney` so a
+copied session link keeps its duration, entry/exit and counters even after a
+reload. Older hosts still provide scoped session events, but cannot restore all
+metadata.
+
+`revenueSummary({siteIds, from, to})` returns exact `commissionCents` for the
+inclusive period. Revenue, conversion count, average order and currency reuse
+`overview` totals. The commission report reads existing daily affiliate stats
+for complete UTC days and trusted conversion records only for partial edge days;
+renamed/deleted affiliate definitions do not remove historical commissions. It
+rejects unavailable retained sources, mixed currencies and periods requiring
+more than 5,000 source rows instead of presenting partial totals. No new table,
+backfill or ingestion change is required. Older hosts show commissions as
+unavailable, never as the sum of a loaded list page.
+
+Session and conversion lists now expose per-site keyset continuation, merge rows
+in chronological order and label search as filtering only loaded records.
+
 ## Organization transfers (0.2.0)
 
 Install `@iann29/rastro@0.2.0` after rehearsing on a non-production snapshot.
@@ -399,6 +481,8 @@ control-plane issuer.
   functions, a read-only grant refuses `upsertGoal` with
   `FEDERATION_CONFIGURE_FORBIDDEN`, and a grant listing `analytics:configure`
   lets an owner or admin create a goal from the dashboard.
+- On `0.4.0` or later: hiding the site's tab removes the visitor from the live
+  view within about ten seconds, and showing it brings them back.
 - Protected overview, journey, goal, funnel, and affiliate reports used by the
   host still run.
 - Trusted conversion idempotency and currency checks still pass.

package/llms.txt

@@ -16,7 +16,7 @@ Do not infer federation availability from this website, a dist-tag, or a source
 ## Architecture
 
 - Tracker and telemetry: browser -> customer HTTP-actions URL -> customer Rastro component.
-- Optional field Web Vitals (alpha.6+): a second browser module at `/rastro/vitals.js` reports LCP, CLS, INP, FCP, and TTFB to the same events endpoint, into bounded daily histograms.
+- Optional field Web Vitals (alpha.6+): a second browser module at `/rastro/vitals.js` reports LCP, CLS, INP, FCP, and TTFB to the same events endpoint, into bounded daily histograms; it needs the base tracker on the page.
 - Reactive dashboard reads: amagerastro.com browser -> customer functions URL -> `rastroFederation:*`.
 - Control plane: account, organization, connection metadata, short-lived token issuance, and revocation.
 - The control plane does not proxy or persist customer analytics events.
@@ -49,13 +49,13 @@ Do not infer federation availability from this website, a dist-tag, or a source
 - Protocol: `amage-rastro-analytics` version 1.
 - Maximum sites per connection and report request: 10.
 - Maximum report range: 366 days.
-- Hourly overview maximum: 24 hours.
+- Overview maximum, bounded by the aggregate rows it reads: 101 days for one site and 10 for ten; hourly, 24 hours for two sites and 5 for ten. A wider range fails with `LIMIT_EXCEEDED`, a busy one with `REPORT_INCOMPLETE`.
 - Funnel report maximum: 90 complete UTC days.
 - Affiliate reports require complete UTC-day boundaries.
 - Vitals report maximum: 90 complete UTC days; `vitals` is an optional capability that hosts before `alpha.6` do not advertise.
 - Site map maximum: 90 complete UTC days, read newest day first within 8,000 route rows; older days it could not read are declared as partial coverage. `siteMap` is an optional capability that hosts before `alpha.8` do not advertise, and its data starts at the first ingest after that upgrade.
 - Maximum live visitors: 500.
-- Live window: a `leave` beacon on pagehide retires a visitor about ten seconds after they leave, and a bfcache return revives them; the safety net without a goodbye is 90 seconds since the last event or heartbeat. `POST /rastro/events` re-anchors a batch whose `sentAt` is more than 30 seconds from server time, so a wrong device clock decides neither liveness nor hour buckets.
+- Live window: the tracker sends a `leave` when the tab hides or the page unloads, retiring the visitor about ten seconds later; a shown tab or a bfcache return sends a heartbeat at once and revives them. The safety net without a goodbye is 90 seconds since the last event or heartbeat. `POST /rastro/events` re-anchors a batch whose `sentAt` is more than 30 seconds from server time, so a wrong device clock decides neither liveness nor hour buckets.
 - Maximum journey events: 500.
 - Token lifetime: 10 minutes, RS256. Every member's token carries `analytics:read`; owners and admins also carry `analytics:configure`, which the host honors only when the local grant lists it (`configure` is an optional capability that hosts before `alpha.11` do not advertise). A public link's token carries `analytics:public` alone, plus `rastro_site_ids` naming the link's sites; hosts before `0.1.0` reject it.
 

package/package.json

@@ -6,7 +6,7 @@
   "bugs": {
     "url": "https://github.com/amageweb/amage-rastro/issues"
   },
-  "version": "0.2.0",
+  "version": "0.4.0",
   "license": "Apache-2.0",
   "publishConfig": {
     "access": "public"

package/src/component/_generated/component.ts

@@ -177,6 +177,7 @@ export type ComponentApi<Name extends string | undefined = string | undefined> =
         "mutation",
         "internal",
         {
+          clientKey?: string;
           context?: {
             browser?: string;
             city?: string;
@@ -362,6 +363,47 @@ export type ComponentApi<Name extends string | undefined = string | undefined> =
         }>,
         Name
       >;
+      getSession: FunctionReference<
+        "query",
+        "internal",
+        { sessionId: string; siteId: string },
+        {
+          _creationTime: number;
+          _id: string;
+          affiliateAttributedAt?: number;
+          affiliateExpiresAt?: number;
+          affiliateId?: string;
+          affiliateSlug?: string;
+          browser: string;
+          city?: string;
+          conversionCount: number;
+          country?: string;
+          currency?: string;
+          device: string;
+          durationMs: number;
+          entryPath: string;
+          eventCount: number;
+          exitPath: string;
+          lastSeenAt: number;
+          latitude?: number;
+          longitude?: number;
+          newVisitor?: boolean;
+          os: string;
+          pageviewCount: number;
+          referrer?: string;
+          revenueCents: number;
+          sessionId: string;
+          siteId: string;
+          source: string;
+          startedAt: number;
+          utmCampaign?: string;
+          utmMedium?: string;
+          utmSource?: string;
+          visitorId: string;
+          visitorKey?: string;
+        } | null,
+        Name
+      >;
       goalsReport: FunctionReference<
         "query",
         "internal",
@@ -676,6 +718,13 @@ export type ComponentApi<Name extends string | undefined = string | undefined> =
         },
         Name
       >;
+      revenueSummary: FunctionReference<
+        "query",
+        "internal",
+        { from: number; siteIds: Array<string>; to: number },
+        { commissionCents: number },
+        Name
+      >;
       sessionJourney: FunctionReference<
         "query",
         "internal",

package/src/component/_generated/server.ts

@@ -39,6 +39,7 @@ import type { DataModel } from "./dataModel.js";
 type Env = {
   readonly CONVEX_CLOUD_URL: string;
   readonly CONVEX_SITE_URL: string;
+  readonly RASTRO_CLIENT_RATE_LIMIT: "off" | undefined;
   readonly RASTRO_GEOIP_DAILY_LIMIT: string | undefined;
   readonly RASTRO_GEOIP_PROVIDER: "ipinfo" | "ipwhois" | undefined;
   readonly RASTRO_GEOIP_TOKEN: string | undefined;

package/src/component/constants.ts

@@ -4,9 +4,20 @@ export const MAX_BATCH_BYTES = 64 * 1024;
 export const MAX_EVENTS_PER_SESSION_WINDOW = 120;
 export const RATE_LIMIT_WINDOW_MS = 60_000;
 export const MAX_BYTES_PER_SESSION_WINDOW = 256 * 1024;
-export const SITE_INGEST_SHARDS = 4_096;
-export const MAX_SITE_EVENTS_PER_SHARD_WINDOW = 300;
-export const MAX_SITE_BYTES_PER_SHARD_WINDOW = 512 * 1024;
+// A site admits at most SITE_INGEST_SHARDS × the per-shard caps a minute:
+// 384,000 events and 512 MiB. Shards keep every POST off one shared row per
+// site; random session IDs can only fill them evenly, while a real peak lands
+// unevenly — the realistic benchmark (~100k events/min) peaks near 1,050
+// events in its hottest shard-minute, so fewer shards would need a looser cap
+// per shard and more would need a looser cap per site.
+export const SITE_INGEST_SHARDS = 256;
+export const MAX_SITE_EVENTS_PER_SHARD_WINDOW = 1_500;
+export const MAX_SITE_BYTES_PER_SHARD_WINDOW = 2 * 1024 * 1024;
+// Per client network (an IPv4 address or an IPv6 /64, keyed by a salted daily
+// hash) per site and minute. Generous on purpose: an office or carrier-grade
+// NAT puts many visitors behind one address, and they all share this budget.
+export const MAX_EVENTS_PER_CLIENT_WINDOW = 1_200;
+export const MAX_BYTES_PER_CLIENT_WINDOW = 2 * 1024 * 1024;
 export const LIVE_SESSION_TTL_MS = 90_000;
 export const LIVE_LEAVE_GRACE_MS = 10_000;
 export const LIVE_SWEEP_INTERVAL_MS = 5_000;
@@ -44,6 +55,16 @@ export const RETENTION_LEASE_MS = 60_000;
 export const ROLLUP_FOLD_BATCH_DOCS = 200;
 export const ROLLUP_FOLD_MAX_ROWS = 1_500;
 export const ROLLUP_FOLD_LEASE_MS = 60_000;
+// A batch that finds no fold pending schedules one this far out, and a run
+// that folded anything schedules the next one as far again, so one run per
+// second serves every batch of that second; only a run that finds the queue
+// empty releases the lease.
+export const ROLLUP_FOLD_DEBOUNCE_MS = 1_000;
+// A run that throws retries its oldest batch alone; a batch that fails this
+// many runs in a row is set aside as a dead letter, kept this long (30 days)
+// for inspection, so one bad batch cannot stall a site's reports.
+export const ROLLUP_FOLD_MAX_FAILURES = 3;
+export const ROLLUP_DEAD_LETTER_RETENTION_MS = 30 * 86_400_000;
 // Shared Web Vitals histogram edges. Every Google rating threshold (LCP
 // 2500/4000, CLS×1000 100/250, INP 200/500, FCP 1800/3000, TTFB 800/1800) is
 // an edge, so rating counts stay exact while one layout serves all metrics.

package/src/component/convex.config.ts

@@ -9,5 +9,8 @@ export default defineComponent("rastroAnalytics", {
     RASTRO_GEOIP_TOKEN: v.optional(v.string()),
     RASTRO_GEOIP_DAILY_LIMIT: v.optional(v.string()),
     RASTRO_TRUST_PROXY: v.optional(v.literal("true")),
+    // "off" drops the per-client ingest budget, for a load-test canary whose
+    // driver sends every request from one address.
+    RASTRO_CLIENT_RATE_LIMIT: v.optional(v.literal("off")),
   },
 });

package/src/component/geo.ts

@@ -510,7 +510,7 @@ function classifyIpv6(ip: string): Exclude<IpKind, "unavailable"> {
   return (groups[0]! & 0xe000) === 0x2000 ? "public" : "private";
 }
 
-function parseIpv6(ip: string): number[] | undefined {
+export function parseIpv6(ip: string): number[] | undefined {
   let normalized = ip;
   const dottedTail = normalized.match(/(\d+\.\d+\.\d+\.\d+)$/)?.[1];
   if (dottedTail) {

package/src/component/http.ts

@@ -20,7 +20,7 @@ import {
 } from "./geo.js";
 import { isPlainRecord } from "./guards.js";
 import { alignClock, sanitizeEvent } from "./sanitize.js";
-import { deriveVisitorKey, randomSecret } from "./identity.js";
+import { deriveClientKey, deriveVisitorKey, randomSecret } from "./identity.js";
 import { classifyClient, isKnownBot } from "./useragent.js";
 import type { TrackerEvent } from "./validators.js";
 import {
@@ -179,6 +179,19 @@ http.route({
       ...classifyClient(userAgent, request.headers),
       ...(visitorKey ? { visitorKey } : {}),
     };
+    // Origin is a browser control: any other sender writes the header it
+    // likes. The per-client budget is keyed like the visitor key above (salted,
+    // daily, never the address); without a usable client IP only the per-site
+    // shards bound the batch.
+    const clientKey =
+      clientIp.ip && env.RASTRO_CLIENT_RATE_LIMIT !== "off"
+        ? await deriveClientKey({
+            secret: await visitorSecret(ctx),
+            now: Date.now(),
+            siteId: payload.siteId,
+            ip: clientIp.ip,
+          })
+        : undefined;
 
     try {
       const result = await ctx.runMutation(api.ingest.ingestBatch, {
@@ -186,6 +199,7 @@ http.route({
         origin,
         events,
         context,
+        ...(clientKey ? { clientKey } : {}),
       });
       if (
         clientIp.ip &&

package/src/component/identity.ts

@@ -1,5 +1,6 @@
 import { v } from "convex/values";
 import { DAY_MS } from "./constants.js";
+import { parseIpv6 } from "./geo.js";
 import { internalMutation, internalQuery } from "./_generated/server.js";
 
 /**
@@ -57,9 +58,47 @@ export async function deriveVisitorKey(input: {
   ip: string;
   userAgent: string;
 }): Promise<string> {
-  const dayStart = Math.floor(input.now / DAY_MS) * DAY_MS;
+  return saltedDailyKey(input.secret, input.now, [
+    input.siteId,
+    input.ip,
+    input.userAgent,
+  ]);
+}
+
+/**
+ * The per-client ingest budget's key: the same salted daily hash, over the
+ * client's network instead of its device — the IPv4 address, or the /64 of an
+ * IPv6 address, the block one subscriber gets and could otherwise walk through
+ * one address per request.
+ */
+export async function deriveClientKey(input: {
+  secret: string;
+  now: number;
+  siteId: string;
+  ip: string;
+}): Promise<string> {
+  const groups = parseIpv6(input.ip.toLowerCase());
+  const network = groups
+    ? `${groups
+        .slice(0, 4)
+        .map((group) => group.toString(16))
+        .join(":")}::/64`
+    : input.ip;
+  return saltedDailyKey(input.secret, input.now, [
+    "client",
+    input.siteId,
+    network,
+  ]);
+}
+
+async function saltedDailyKey(
+  secret: string,
+  now: number,
+  parts: string[],
+): Promise<string> {
+  const dayStart = Math.floor(now / DAY_MS) * DAY_MS;
   const material = new TextEncoder().encode(
-    [input.secret, dayStart, input.siteId, input.ip, input.userAgent].join(" "),
+    [secret, dayStart, ...parts].join(" "),
   );
   const digest = new Uint8Array(
     await crypto.subtle.digest("SHA-256", material),

package/src/component/ingest.ts

@@ -14,6 +14,8 @@ import {
   LIVE_SESSION_TTL_MS,
   LIVE_SWEEP_INTERVAL_MS,
   MAX_BATCH_EVENTS,
+  MAX_BYTES_PER_CLIENT_WINDOW,
+  MAX_EVENTS_PER_CLIENT_WINDOW,
   MAX_SITE_BYTES_PER_SHARD_WINDOW,
   MAX_SITE_EVENTS_PER_SHARD_WINDOW,
   MAX_BYTES_PER_SESSION_WINDOW,
@@ -123,7 +125,7 @@ type AggregateDelta = {
   continuedSessions: number;
   newVisitors: number;
 };
-type AggregateDeltas = Map<string, AggregateDelta>;
+export type AggregateDeltas = Map<string, AggregateDelta>;
 type AttributionSnapshot = {
   affiliateId: Id<"affiliates">;
   affiliateSlug: string;
@@ -214,6 +216,8 @@ export const ingestBatch = mutation({
     origin: v.string(),
     events: v.array(trackerEventValidator),
     context: v.optional(ingestContextValidator),
+    // The HTTP action's salted daily client-network key (see identity.ts).
+    clientKey: v.optional(v.string()),
   },
   returns: ingestResultValidator,
   handler: async (ctx, args) => {
@@ -245,6 +249,15 @@ export const ingestBatch = mutation({
 
     const now = Date.now();
     await ensureAnalyticsControl(ctx, args.siteId, now);
+    if (args.clientKey !== undefined) {
+      await consumeClientRateLimit(
+        ctx,
+        args.siteId,
+        args.clientKey,
+        eventByteCounts,
+        now,
+      );
+    }
     await consumeSiteRateLimit(
       ctx,
       args.siteId,
@@ -389,7 +402,10 @@ export const ingestBatch = mutation({
           rejected += 1;
           continue;
         }
-        const durationDeltaMs = await accountHeartbeat(
+        // A heartbeat is presence, not telemetry: it moves the session's
+        // lastSeenAt and live row only. The time it proves reaches the
+        // duration counters once, when the visitor departs (live.ts).
+        await accountHeartbeat(
           ctx,
           args.siteId,
           existingSession,
@@ -398,25 +414,13 @@ export const ingestBatch = mutation({
           now,
           batchState,
         );
-        // A heartbeat is presence, not telemetry, but it is what keeps a
-        // session's duration honest while the visitor reads: the growth is
-        // folded into the buckets the heartbeat landed in.
-        foldSessionDuration(
-          aggregateDeltas,
-          args.siteId,
-          event,
-          durationDeltaMs,
-          timezone,
-        );
         accepted += 1;
         continue;
       }
 
       if (event.type === "leave") {
-        // A departure only shortens live presence: it stores no telemetry,
-        // touches no aggregates, and never extends anything, so a replayed
-        // or late leave is inert. The read side filters by expiresAt and the
-        // sweep deletes the row, so the dashboard sees the exit in seconds.
+        // A departure closes the final duration interval without telemetry
+        // or new presence. Older departures cannot retire a bfcache return.
         if (!existingSession) {
           rejected += 1;
           continue;
@@ -441,7 +445,34 @@ export const ingestBatch = mutation({
           batchState,
         );
         const liveSession = liveState.current;
-        if (liveSession) {
+        if (liveSession && event.timestamp >= liveSession.lastSeenAt) {
+          // Only the still-covered tail is trusted; a delayed departure
+          // cannot lengthen an expired visit or move its start backwards.
+          if (
+            event.timestamp > existingSession.lastSeenAt &&
+            event.timestamp >= now - LIVE_SESSION_TTL_MS &&
+            event.timestamp <= liveSession.expiresAt
+          ) {
+            const durationMs = Math.max(
+              existingSession.durationMs,
+              event.timestamp - existingSession.startedAt,
+            );
+            sessionRow.current = {
+              ...existingSession,
+              lastSeenAt: event.timestamp,
+              exitPath: event.path,
+              durationMs,
+            };
+            sessionRow.dirty = true;
+            foldSessionDuration(
+              aggregateDeltas,
+              args.siteId,
+              event.sessionId,
+              event.timestamp,
+              durationMs - existingSession.durationMs,
+              timezone,
+            );
+          }
           const expiresAt = Math.min(
             liveSession.expiresAt,
             now + LIVE_LEAVE_GRACE_MS,
@@ -1397,6 +1428,12 @@ async function attributionForEvent(
   };
 }
 
+/**
+ * Moves the session's lastSeenAt and its live row. `durationMs` stays: it is
+ * what the duration counters already hold, and the time a heartbeat proves
+ * is counted when the visitor departs (live.ts), so an idle tab never
+ * rewrites an aggregate bucket.
+ */
 async function accountHeartbeat(
   ctx: MutationCtx,
   siteId: Id<"sites">,
@@ -1405,7 +1442,7 @@ async function accountHeartbeat(
   context: IngestContext,
   now: number,
   batchState?: BatchState,
-): Promise<number> {
+): Promise<void> {
   const startedAt = Math.min(session.startedAt, event.timestamp);
   const lastSeenAt = Math.max(session.lastSeenAt, event.timestamp);
   const updates = {
@@ -1413,7 +1450,6 @@ async function accountHeartbeat(
     lastSeenAt,
     exitPath:
       event.timestamp >= session.lastSeenAt ? event.path : session.exitPath,
-    durationMs: Math.max(0, lastSeenAt - startedAt),
     ...(context.country ? { country: context.country } : {}),
     ...(context.city ? { city: context.city } : {}),
     ...(context.latitude !== undefined ? { latitude: context.latitude } : {}),
@@ -1432,7 +1468,6 @@ async function accountHeartbeat(
   } else {
     await ctx.db.patch("sessions", session._id, updates);
   }
-  const durationDeltaMs = Math.max(0, updates.durationMs - session.durationMs);
   await updateLiveSession(
     ctx,
     siteId,
@@ -1462,7 +1497,54 @@ async function accountHeartbeat(
     now,
     batchState,
   );
-  return durationDeltaMs;
+}
+
+/**
+ * One client network's budget per site and minute, kept in the session
+ * windows' table under `~client:<key>`, a key no sanitized session ID can take.
+ * The key is the salted daily hash the HTTP action derived; the address never
+ * reaches this mutation. It is one row per client, so only that client's own
+ * concurrent requests contend on it.
+ */
+async function consumeClientRateLimit(
+  ctx: MutationCtx,
+  siteId: Id<"sites">,
+  clientKey: string,
+  eventByteCounts: number[],
+  now: number,
+): Promise<void> {
+  const windowStart =
+    Math.floor(now / RATE_LIMIT_WINDOW_MS) * RATE_LIMIT_WINDOW_MS;
+  const sessionId = `~client:${clientKey}`;
+  const rows = await ctx.db
+    .query("ingestWindows")
+    .withIndex("by_siteId_and_sessionId_and_windowStart", (range) =>
+      range.eq("siteId", siteId).eq("sessionId", sessionId),
+    )
+    .order("desc")
+    .take(1);
+  const window = rows[0] ?? null;
+  const current = window?.windowStart === windowStart ? window : null;
+  const eventCount = (current?.eventCount ?? 0) + eventByteCounts.length;
+  const byteCount =
+    (current?.byteCount ?? 0) +
+    eventByteCounts.reduce((total, bytes) => total + bytes, 0);
+  if (
+    eventCount > MAX_EVENTS_PER_CLIENT_WINDOW ||
+    byteCount > MAX_BYTES_PER_CLIENT_WINDOW
+  ) {
+    fail("RATE_LIMITED", "client ingest budget exceeded", {
+      windowMs: RATE_LIMIT_WINDOW_MS,
+      eventLimitPerClient: MAX_EVENTS_PER_CLIENT_WINDOW,
+      byteLimitPerClient: MAX_BYTES_PER_CLIENT_WINDOW,
+    });
+  }
+  const fields = { windowStart, eventCount, byteCount, updatedAt: now };
+  if (window) {
+    await ctx.db.patch("ingestWindows", window._id, fields);
+  } else {
+    await ctx.db.insert("ingestWindows", { siteId, sessionId, ...fields });
+  }
 }
 
 async function consumeSiteRateLimit(
@@ -1848,11 +1930,10 @@ async function advanceFunnel(
     foldFunnelRollup(reportRollupDeltas, siteId, funnel._id, event, 0, false);
     return { funnelId: funnel._id, step: 1, steps: funnel.steps.length };
   }
-  if (
-    progress.completedAt !== undefined ||
-    event.timestamp < progress.lastStepAt
-  )
-    return;
+  if (event.timestamp < progress.lastStepAt) return;
+  // An expired window restarts on a first-step event even after a completion,
+  // so a returning buyer can complete again; inside the window a completed
+  // funnel stays done.
   if (event.timestamp - progress.startedAt > funnel.conversionWindowMs) {
     if (!matchesFirst) return;
     const updated = {
@@ -1886,6 +1967,7 @@ async function advanceFunnel(
     foldFunnelRollup(reportRollupDeltas, siteId, funnel._id, event, 0, false);
     return { funnelId: funnel._id, step: 1, steps: funnel.steps.length };
   }
+  if (progress.completedAt !== undefined) return;
   const nextStep = funnel.steps[progress.currentStep];
   if (!nextStep || !stepMatches(nextStep, event)) return;
   const currentStep = progress.currentStep + 1;
@@ -2404,24 +2486,26 @@ function aggregateBucketStart(
   return Math.floor(timestamp / interval) * interval;
 }
 
-/** Folds only session duration growth, for heartbeats. */
-function foldSessionDuration(
+/**
+ * Folds session duration growth without telemetry: a departure's final
+ * interval, or the heartbeat time of a visitor the live sweep retired. The
+ * shard follows `shardKey`: the session for a departure, the site for a
+ * sweep.
+ */
+export function foldSessionDuration(
   aggregateDeltas: AggregateDeltas,
   siteId: Id<"sites">,
-  event: TrackerEvent,
+  shardKey: string,
+  timestamp: number,
   durationDeltaMs: number,
   timezone?: string,
 ) {
   if (durationDeltaMs <= 0) return;
   for (const granularity of aggregateGranularities(timezone)) {
     const shard =
-      stableHash(event.sessionId) %
+      stableHash(shardKey) %
       (granularity === "hour" ? HOURLY_AGGREGATE_SHARDS : AGGREGATE_SHARDS);
-    const bucketStart = aggregateBucketStart(
-      granularity,
-      event.timestamp,
-      timezone,
-    );
+    const bucketStart = aggregateBucketStart(granularity, timestamp, timezone);
     const key = `${siteId}:${granularity}:${bucketStart}:${shard}`;
     const delta = aggregateDeltas.get(key) ?? {
       siteId,
@@ -2448,7 +2532,7 @@ function foldSessionDuration(
   }
 }
 
-async function flushAggregates(
+export async function flushAggregates(
   ctx: MutationCtx,
   aggregateDeltas: AggregateDeltas,
 ) {