@iann29/rastro 0.1.0-alpha.0 → 0.1.0-alpha.1

package/README.md

@@ -4,7 +4,7 @@
 
 Privacy-first, real-time web analytics built as an isolated Convex component.
 
-Amage Rastro combines a 991-byte gzip browser tracker with reactive traffic
+Amage Rastro combines a 1,022-byte gzip browser tracker with reactive traffic
 metrics, a live visitor atlas, journey timelines, trusted payment attribution,
 goals, funnels, affiliates, and multi-site reports. Data and functions stay
 inside your Convex deployment.
@@ -24,7 +24,7 @@ inside your Convex deployment.
 | Cross-site analytics        | Reports over up to ten authorized site IDs with a preloaded pseudonymous visitor ID                 |
 | Auto-captured clicks        | Anchors, buttons, outbound links, and explicit `data-rastro-event` controls                         |
 | Cookieless mode             | Session-scoped pseudonymous IDs in `sessionStorage`; no cookies                                     |
-| Tiny tracker                | 1,890 bytes raw, 991 bytes gzip; the test suite enforces `< 1,024` bytes                            |
+| Tiny tracker                | 1,909 bytes raw, 1,022 bytes gzip; the test suite enforces `< 1,024` bytes                          |
 
 ## Install
 
@@ -172,6 +172,12 @@ Properties accept bounded string, number, boolean, or null values. Never send
 emails, names, form values, secrets, or other personal data as analytics
 properties.
 
+Browser telemetry exact retries are idempotent within
+`(siteId, sessionId, event-minute, eventId)`. Reusing an event ID in another
+session or with a timestamp in another minute is allowed. The tracker preserves
+timestamps when retrying. Trusted payment event IDs remain separately
+payload-bound by the site's financial ledger.
+
 ## Attribute trusted payments
 
 Browser-created conversion events are untrusted telemetry and never affect
@@ -271,25 +277,86 @@ The `Rastro` class wraps the component boundary for host functions:
 - Retention: `cleanup`
 
 Every public function has argument and return validators. Growing reads are
-indexed and bounded. Component pagination uses validated opaque manual cursors
-because native `.paginate()` is not supported inside Convex components.
+indexed and bounded. Component pagination uses scope-bound opaque keyset cursors
+because native `.paginate()` is not supported inside Convex components; page
+cost stays constant instead of increasing with an offset.
 
 ## Operational limits
 
 - HTTP body: 64 KiB
-- Events per batch: 20
-- Aggregate shards: 16, derived from stable session/visitor identity
-- Site ingress budget: eight deterministic per-minute shards with bounded event
-  and byte caps
-- Live session timeout: 60 seconds with one self-rescheduling expiry chain per
-  session
+- Events per batch: 50
+- Session-minute groups per batch: 4
+- Active goals per exact matcher: 1
+- Primary aggregate shards: 256 hourly and 128 daily, derived from stable
+  session identity
+- Funnel and affiliate aggregate shards: 16
+- Site ingress budget: 4,096 deterministic per-minute shards, each capped at
+  300 events and 512 KiB (1,228,800 admitted events/minute at uniform load)
+- Rate-window storage: one reusable row per active session and site shard
+- Live session timeout: 60 seconds with one bounded five-second expiry sweep per
+  site; legacy per-session jobs hand off to that sweep during the cutover
 - Dashboard overview: up to ten sites and bounded report ranges
+- Hourly overview: up to 24 hours; wider ranges use daily buckets
 - Retention cleanup: bounded batches; schedule host maintenance for the data
-  classes your policy retains
+  classes your policy retains. The `events` cleanup kind covers both legacy
+  event rows and new event batches and processes at most 32 heavy batches per
+  transaction.
+
+Capacity claims use declared workload profiles. Complete release evidence
+reports ten-minute `lean`, `realistic`, `heavy`, and browser `feature` profiles
+separately. The browser `feature` profile exercises configured goals, funnels,
+and affiliate attribution; trusted conversions require a separate server-side
+driver and certification result.
+
+The historical development evidence and its certification caveats are recorded
+in [`docs/benchmarks/2026-08-20-realistic.md`](docs/benchmarks/2026-08-20-realistic.md).
+
+From a repository checkout, run a duration-based profile against a confirmed
+non-production deployment and a dedicated idle benchmark site:
+
+```bash
+npm run --silent benchmark:ingest -- \
+  --url https://example.site/rastro/events \
+  --convex-url https://example.convex-backend \
+  --site-id your_site_id \
+  --origin https://your-allowed-origin.example \
+  --profile realistic \
+  --duration 600 \
+  --concurrency 100 \
+  --certify
+```
 
-The ingestion transaction folds metric/dimension deltas and writes each touched
-aggregate bucket at most once per batch. Click/outbound/browser-conversion
-telemetry skips goal and funnel work that cannot match those event types.
+Profiles are `lean`, `realistic`, `heavy`, and `feature`. The realistic profile
+reuses sessions and sends geographic and user-agent headers so the same load is
+visible in Live Atlas. Only `feature` intentionally matches the demo goal,
+funnel, and affiliate definitions; it does not call the trusted conversion API.
+Use `--dry-run` to inspect generated request size without sending traffic. When
+`--convex-url` is supplied, it must identify the same deployment as `--url` and
+expose this repository's `example:overview` query. The final reconciliation is
+a site-wide before/after delta, so unrelated traffic on that site invalidates
+the result. Reports distinguish accepted ingress, heartbeats, and stored events.
+Shorter completed runs report extrapolated daily estimates only. `--certify`
+requires at least ten minutes and successful reconciliation before emitting
+certified daily capacity fields or exiting successfully as a certification.
+
+The ingestion transaction folds metric/dimension deltas, session rate counters,
+site-shard counters, session/live state, visitor attribution, and exact funnel
+progress before writing. Accepted non-heartbeat telemetry from one request is
+grouped into one immutable raw document per session and event-minute, without
+duplicating session geo or user-agent context. A 50-event same-session,
+same-minute baseline stays within 10 document writes. Each touched aggregate
+bucket is written at most once per batch, and dimension overflow folds into an
+explicit `(other)` bucket instead of relabeling old counts.
+Click/outbound/browser-conversion telemetry skips goal and funnel work that
+cannot match those event types.
+
+This is an additive storage migration: existing `events` rows are not deleted or
+backfilled. Journey and trusted-conversion reads merge legacy rows with new
+`eventBatches` rows. Journey `_id` values are stable strings for both formats,
+and session-journey cursors use timestamp, creation time, and ID so pagination
+matches both storage indexes. The compatibility logic is isolated in
+`src/component/eventStore.ts`; ingestion, reports, and retention do not branch
+on physical telemetry storage.
 
 ## Privacy contract
 
@@ -336,10 +403,10 @@ cd example && npx vite build
 ```
 
 The test suite covers origin enforcement, event/privacy sanitization,
-idempotency, session/site rate limits, live expiry scheduling, folded aggregate
-writes, goals, funnels, trusted payments, currency safety, affiliate commission,
-cross-site journeys, manual pagination, retention, HTTP routes, React
-integration, and tracker size/behavior.
+idempotency, reusable session/site rate limits, bounded live expiry chains,
+folded aggregate writes, goals, funnels, trusted payments, currency safety,
+affiliate commission, cross-site journeys, keyset pagination beyond 1,000 rows,
+retention, HTTP routes, React integration, and tracker size/behavior.
 
 ## Synapse demo
 

package/dist/component/_generated/api.d.ts

@@ -9,6 +9,7 @@
 import type * as affiliates from "../affiliates.js";
 import type * as constants from "../constants.js";
 import type * as errors from "../errors.js";
+import type * as eventStore from "../eventStore.js";
 import type * as funnels from "../funnels.js";
 import type * as goals from "../goals.js";
 import type * as guards from "../guards.js";
@@ -25,6 +26,7 @@ declare const fullApi: ApiFromModules<{
     affiliates: typeof affiliates;
     constants: typeof constants;
     errors: typeof errors;
+    eventStore: typeof eventStore;
     funnels: typeof funnels;
     goals: typeof goals;
     guards: typeof guards;

package/dist/component/_generated/api.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"api.d.ts","sourceRoot":"","sources":["../../../src/component/_generated/api.ts"],"names":[],"mappings":"AACA;;;;;;;GAOG;AAEH,OAAO,KAAK,KAAK,UAAU,MAAM,kBAAkB,CAAC;AACpD,OAAO,KAAK,KAAK,SAAS,MAAM,iBAAiB,CAAC;AAClD,OAAO,KAAK,KAAK,MAAM,MAAM,cAAc,CAAC;AAC5C,OAAO,KAAK,KAAK,OAAO,MAAM,eAAe,CAAC;AAC9C,OAAO,KAAK,KAAK,KAAK,MAAM,aAAa,CAAC;AAC1C,OAAO,KAAK,KAAK,MAAM,MAAM,cAAc,CAAC;AAC5C,OAAO,KAAK,KAAK,IAAI,MAAM,YAAY,CAAC;AACxC,OAAO,KAAK,KAAK,MAAM,MAAM,cAAc,CAAC;AAC5C,OAAO,KAAK,KAAK,IAAI,MAAM,YAAY,CAAC;AACxC,OAAO,KAAK,KAAK,OAAO,MAAM,eAAe,CAAC;AAC9C,OAAO,KAAK,KAAK,SAAS,MAAM,iBAAiB,CAAC;AAClD,OAAO,KAAK,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAChD,OAAO,KAAK,KAAK,KAAK,MAAM,aAAa,CAAC;AAC1C,OAAO,KAAK,KAAK,UAAU,MAAM,kBAAkB,CAAC;AAEpD,OAAO,KAAK,EACV,cAAc,EACd,SAAS,EACT,iBAAiB,EAClB,MAAM,eAAe,CAAC;AAGvB,QAAA,MAAM,OAAO,EAAE,cAAc,CAAC;IAC5B,UAAU,EAAE,OAAO,UAAU,CAAC;IAC9B,SAAS,EAAE,OAAO,SAAS,CAAC;IAC5B,MAAM,EAAE,OAAO,MAAM,CAAC;IACtB,OAAO,EAAE,OAAO,OAAO,CAAC;IACxB,KAAK,EAAE,OAAO,KAAK,CAAC;IACpB,MAAM,EAAE,OAAO,MAAM,CAAC;IACtB,IAAI,EAAE,OAAO,IAAI,CAAC;IAClB,MAAM,EAAE,OAAO,MAAM,CAAC;IACtB,IAAI,EAAE,OAAO,IAAI,CAAC;IAClB,OAAO,EAAE,OAAO,OAAO,CAAC;IACxB,SAAS,EAAE,OAAO,SAAS,CAAC;IAC5B,QAAQ,EAAE,OAAO,QAAQ,CAAC;IAC1B,KAAK,EAAE,OAAO,KAAK,CAAC;IACpB,UAAU,EAAE,OAAO,UAAU,CAAC;CAC/B,CAAiB,CAAC;AAEnB;;;;;;;GAOG;AACH,eAAO,MAAM,GAAG,EAAE,SAAS,CACzB,OAAO,OAAO,EACd,iBAAiB,CAAC,GAAG,EAAE,QAAQ,CAAC,CACjB,CAAC;AAElB;;;;;;;GAOG;AACH,eAAO,MAAM,QAAQ,EAAE,SAAS,CAC9B,OAAO,OAAO,EACd,iBAAiB,CAAC,GAAG,EAAE,UAAU,CAAC,CACnB,CAAC;AAElB,eAAO,MAAM,UAAU,EAAqC,EAAE,CAAC"}
+{"version":3,"file":"api.d.ts","sourceRoot":"","sources":["../../../src/component/_generated/api.ts"],"names":[],"mappings":"AACA;;;;;;;GAOG;AAEH,OAAO,KAAK,KAAK,UAAU,MAAM,kBAAkB,CAAC;AACpD,OAAO,KAAK,KAAK,SAAS,MAAM,iBAAiB,CAAC;AAClD,OAAO,KAAK,KAAK,MAAM,MAAM,cAAc,CAAC;AAC5C,OAAO,KAAK,KAAK,UAAU,MAAM,kBAAkB,CAAC;AACpD,OAAO,KAAK,KAAK,OAAO,MAAM,eAAe,CAAC;AAC9C,OAAO,KAAK,KAAK,KAAK,MAAM,aAAa,CAAC;AAC1C,OAAO,KAAK,KAAK,MAAM,MAAM,cAAc,CAAC;AAC5C,OAAO,KAAK,KAAK,IAAI,MAAM,YAAY,CAAC;AACxC,OAAO,KAAK,KAAK,MAAM,MAAM,cAAc,CAAC;AAC5C,OAAO,KAAK,KAAK,IAAI,MAAM,YAAY,CAAC;AACxC,OAAO,KAAK,KAAK,OAAO,MAAM,eAAe,CAAC;AAC9C,OAAO,KAAK,KAAK,SAAS,MAAM,iBAAiB,CAAC;AAClD,OAAO,KAAK,KAAK,QAAQ,MAAM,gBAAgB,CAAC;AAChD,OAAO,KAAK,KAAK,KAAK,MAAM,aAAa,CAAC;AAC1C,OAAO,KAAK,KAAK,UAAU,MAAM,kBAAkB,CAAC;AAEpD,OAAO,KAAK,EACV,cAAc,EACd,SAAS,EACT,iBAAiB,EAClB,MAAM,eAAe,CAAC;AAGvB,QAAA,MAAM,OAAO,EAAE,cAAc,CAAC;IAC5B,UAAU,EAAE,OAAO,UAAU,CAAC;IAC9B,SAAS,EAAE,OAAO,SAAS,CAAC;IAC5B,MAAM,EAAE,OAAO,MAAM,CAAC;IACtB,UAAU,EAAE,OAAO,UAAU,CAAC;IAC9B,OAAO,EAAE,OAAO,OAAO,CAAC;IACxB,KAAK,EAAE,OAAO,KAAK,CAAC;IACpB,MAAM,EAAE,OAAO,MAAM,CAAC;IACtB,IAAI,EAAE,OAAO,IAAI,CAAC;IAClB,MAAM,EAAE,OAAO,MAAM,CAAC;IACtB,IAAI,EAAE,OAAO,IAAI,CAAC;IAClB,OAAO,EAAE,OAAO,OAAO,CAAC;IACxB,SAAS,EAAE,OAAO,SAAS,CAAC;IAC5B,QAAQ,EAAE,OAAO,QAAQ,CAAC;IAC1B,KAAK,EAAE,OAAO,KAAK,CAAC;IACpB,UAAU,EAAE,OAAO,UAAU,CAAC;CAC/B,CAAiB,CAAC;AAEnB;;;;;;;GAOG;AACH,eAAO,MAAM,GAAG,EAAE,SAAS,CACzB,OAAO,OAAO,EACd,iBAAiB,CAAC,GAAG,EAAE,QAAQ,CAAC,CACjB,CAAC;AAElB;;;;;;;GAOG;AACH,eAAO,MAAM,QAAQ,EAAE,SAAS,CAC9B,OAAO,OAAO,EACd,iBAAiB,CAAC,GAAG,EAAE,UAAU,CAAC,CACnB,CAAC;AAElB,eAAO,MAAM,UAAU,EAAqC,EAAE,CAAC"}

package/dist/component/_generated/api.js.map

@@ -1 +1 @@
-{"version":3,"file":"api.js","sourceRoot":"","sources":["../../../src/component/_generated/api.ts"],"names":[],"mappings":"AAAA,oBAAoB;AACpB;;;;;;;GAOG;AAsBH,OAAO,EAAE,MAAM,EAAE,iBAAiB,EAAE,MAAM,eAAe,CAAC;AAE1D,MAAM,OAAO,GAeR,MAAa,CAAC;AAEnB;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,GAAG,GAGZ,MAAa,CAAC;AAElB;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,QAAQ,GAGjB,MAAa,CAAC;AAElB,MAAM,CAAC,MAAM,UAAU,GAAG,iBAAiB,EAAmB,CAAC"}
+{"version":3,"file":"api.js","sourceRoot":"","sources":["../../../src/component/_generated/api.ts"],"names":[],"mappings":"AAAA,oBAAoB;AACpB;;;;;;;GAOG;AAuBH,OAAO,EAAE,MAAM,EAAE,iBAAiB,EAAE,MAAM,eAAe,CAAC;AAE1D,MAAM,OAAO,GAgBR,MAAa,CAAC;AAEnB;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,GAAG,GAGZ,MAAa,CAAC;AAElB;;;;;;;GAOG;AACH,MAAM,CAAC,MAAM,QAAQ,GAGjB,MAAa,CAAC;AAElB,MAAM,CAAC,MAAM,UAAU,GAAG,iBAAiB,EAAmB,CAAC"}

package/dist/component/constants.d.ts

@@ -1,13 +1,17 @@
-export declare const MAX_BATCH_EVENTS = 20;
+export declare const MAX_BATCH_EVENTS = 50;
+export declare const MAX_BATCH_EVENT_GROUPS = 4;
 export declare const MAX_BATCH_BYTES: number;
 export declare const MAX_EVENTS_PER_SESSION_WINDOW = 120;
 export declare const RATE_LIMIT_WINDOW_MS = 60000;
 export declare const MAX_BYTES_PER_SESSION_WINDOW: number;
-export declare const SITE_INGEST_SHARDS = 8;
-export declare const MAX_SITE_EVENTS_PER_SHARD_WINDOW = 120;
+export declare const SITE_INGEST_SHARDS = 4096;
+export declare const MAX_SITE_EVENTS_PER_SHARD_WINDOW = 300;
 export declare const MAX_SITE_BYTES_PER_SHARD_WINDOW: number;
 export declare const LIVE_SESSION_TTL_MS = 60000;
-export declare const AGGREGATE_SHARDS = 16;
+export declare const LIVE_SWEEP_INTERVAL_MS = 5000;
+export declare const AGGREGATE_SHARDS = 128;
+export declare const HOURLY_AGGREGATE_SHARDS = 256;
+export declare const SECONDARY_AGGREGATE_SHARDS = 16;
 export declare const DIMENSION_SLOTS = 16;
 export declare const MAX_SITE_DOMAINS = 32;
 export declare const MAX_SITES_PER_OWNER = 100;
@@ -21,8 +25,7 @@ export declare const MAX_REPORT_RANGE_DAYS = 366;
 export declare const MAX_LIVE_VISITORS = 500;
 export declare const MAX_JOURNEY_EVENTS = 500;
 export declare const MAX_RETENTION_BATCH = 500;
-export declare const MAX_OVERVIEW_AGGREGATE_ROWS = 12000;
+export declare const MAX_OVERVIEW_AGGREGATE_ROWS = 13000;
 export declare const DAY_MS = 86400000;
 export declare const HOUR_MS = 3600000;
-export declare const MAX_MANUAL_PAGINATION_OFFSET = 1000;
 //# sourceMappingURL=constants.d.ts.map

package/dist/component/constants.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"constants.d.ts","sourceRoot":"","sources":["../../src/component/constants.ts"],"names":[],"mappings":"AAAA,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,eAAe,QAAY,CAAC;AACzC,eAAO,MAAM,6BAA6B,MAAM,CAAC;AACjD,eAAO,MAAM,oBAAoB,QAAS,CAAC;AAC3C,eAAO,MAAM,4BAA4B,QAAa,CAAC;AACvD,eAAO,MAAM,kBAAkB,IAAI,CAAC;AACpC,eAAO,MAAM,gCAAgC,MAAM,CAAC;AACpD,eAAO,MAAM,+BAA+B,QAAa,CAAC;AAC1D,eAAO,MAAM,mBAAmB,QAAS,CAAC;AAC1C,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,eAAe,KAAK,CAAC;AAClC,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,mBAAmB,MAAM,CAAC;AACvC,eAAO,MAAM,kBAAkB,KAAK,CAAC;AACrC,eAAO,MAAM,oBAAoB,KAAK,CAAC;AACvC,eAAO,MAAM,uBAAuB,MAAM,CAAC;AAC3C,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,cAAc,KAAK,CAAC;AACjC,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,qBAAqB,MAAM,CAAC;AACzC,eAAO,MAAM,iBAAiB,MAAM,CAAC;AACrC,eAAO,MAAM,kBAAkB,MAAM,CAAC;AACtC,eAAO,MAAM,mBAAmB,MAAM,CAAC;AACvC,eAAO,MAAM,2BAA2B,QAAS,CAAC;AAElD,eAAO,MAAM,MAAM,WAAa,CAAC;AACjC,eAAO,MAAM,OAAO,UAAY,CAAC;AACjC,eAAO,MAAM,4BAA4B,OAAQ,CAAC"}
+{"version":3,"file":"constants.d.ts","sourceRoot":"","sources":["../../src/component/constants.ts"],"names":[],"mappings":"AAAA,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,sBAAsB,IAAI,CAAC;AACxC,eAAO,MAAM,eAAe,QAAY,CAAC;AACzC,eAAO,MAAM,6BAA6B,MAAM,CAAC;AACjD,eAAO,MAAM,oBAAoB,QAAS,CAAC;AAC3C,eAAO,MAAM,4BAA4B,QAAa,CAAC;AACvD,eAAO,MAAM,kBAAkB,OAAQ,CAAC;AACxC,eAAO,MAAM,gCAAgC,MAAM,CAAC;AACpD,eAAO,MAAM,+BAA+B,QAAa,CAAC;AAC1D,eAAO,MAAM,mBAAmB,QAAS,CAAC;AAC1C,eAAO,MAAM,sBAAsB,OAAQ,CAAC;AAC5C,eAAO,MAAM,gBAAgB,MAAM,CAAC;AACpC,eAAO,MAAM,uBAAuB,MAAM,CAAC;AAC3C,eAAO,MAAM,0BAA0B,KAAK,CAAC;AAC7C,eAAO,MAAM,eAAe,KAAK,CAAC;AAClC,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,mBAAmB,MAAM,CAAC;AACvC,eAAO,MAAM,kBAAkB,KAAK,CAAC;AACrC,eAAO,MAAM,oBAAoB,KAAK,CAAC;AACvC,eAAO,MAAM,uBAAuB,MAAM,CAAC;AAC3C,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,cAAc,KAAK,CAAC;AACjC,eAAO,MAAM,gBAAgB,KAAK,CAAC;AACnC,eAAO,MAAM,qBAAqB,MAAM,CAAC;AACzC,eAAO,MAAM,iBAAiB,MAAM,CAAC;AACrC,eAAO,MAAM,kBAAkB,MAAM,CAAC;AACtC,eAAO,MAAM,mBAAmB,MAAM,CAAC;AACvC,eAAO,MAAM,2BAA2B,QAAS,CAAC;AAElD,eAAO,MAAM,MAAM,WAAa,CAAC;AACjC,eAAO,MAAM,OAAO,UAAY,CAAC"}

package/dist/component/constants.js

@@ -1,13 +1,17 @@
-export const MAX_BATCH_EVENTS = 20;
+export const MAX_BATCH_EVENTS = 50;
+export const MAX_BATCH_EVENT_GROUPS = 4;
 export const MAX_BATCH_BYTES = 64 * 1024;
 export const MAX_EVENTS_PER_SESSION_WINDOW = 120;
 export const RATE_LIMIT_WINDOW_MS = 60_000;
 export const MAX_BYTES_PER_SESSION_WINDOW = 256 * 1024;
-export const SITE_INGEST_SHARDS = 8;
-export const MAX_SITE_EVENTS_PER_SHARD_WINDOW = 120;
+export const SITE_INGEST_SHARDS = 4_096;
+export const MAX_SITE_EVENTS_PER_SHARD_WINDOW = 300;
 export const MAX_SITE_BYTES_PER_SHARD_WINDOW = 512 * 1024;
 export const LIVE_SESSION_TTL_MS = 60_000;
-export const AGGREGATE_SHARDS = 16;
+export const LIVE_SWEEP_INTERVAL_MS = 5_000;
+export const AGGREGATE_SHARDS = 128;
+export const HOURLY_AGGREGATE_SHARDS = 256;
+export const SECONDARY_AGGREGATE_SHARDS = 16;
 export const DIMENSION_SLOTS = 16;
 export const MAX_SITE_DOMAINS = 32;
 export const MAX_SITES_PER_OWNER = 100;
@@ -21,8 +25,7 @@ export const MAX_REPORT_RANGE_DAYS = 366;
 export const MAX_LIVE_VISITORS = 500;
 export const MAX_JOURNEY_EVENTS = 500;
 export const MAX_RETENTION_BATCH = 500;
-export const MAX_OVERVIEW_AGGREGATE_ROWS = 12_000;
+export const MAX_OVERVIEW_AGGREGATE_ROWS = 13_000;
 export const DAY_MS = 86_400_000;
 export const HOUR_MS = 3_600_000;
-export const MAX_MANUAL_PAGINATION_OFFSET = 1_000;
 //# sourceMappingURL=constants.js.map

package/dist/component/constants.js.map

@@ -1 +1 @@
-{"version":3,"file":"constants.js","sourceRoot":"","sources":["../../src/component/constants.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,eAAe,GAAG,EAAE,GAAG,IAAI,CAAC;AACzC,MAAM,CAAC,MAAM,6BAA6B,GAAG,GAAG,CAAC;AACjD,MAAM,CAAC,MAAM,oBAAoB,GAAG,MAAM,CAAC;AAC3C,MAAM,CAAC,MAAM,4BAA4B,GAAG,GAAG,GAAG,IAAI,CAAC;AACvD,MAAM,CAAC,MAAM,kBAAkB,GAAG,CAAC,CAAC;AACpC,MAAM,CAAC,MAAM,gCAAgC,GAAG,GAAG,CAAC;AACpD,MAAM,CAAC,MAAM,+BAA+B,GAAG,GAAG,GAAG,IAAI,CAAC;AAC1D,MAAM,CAAC,MAAM,mBAAmB,GAAG,MAAM,CAAC;AAC1C,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,eAAe,GAAG,EAAE,CAAC;AAClC,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,mBAAmB,GAAG,GAAG,CAAC;AACvC,MAAM,CAAC,MAAM,kBAAkB,GAAG,EAAE,CAAC;AACrC,MAAM,CAAC,MAAM,oBAAoB,GAAG,EAAE,CAAC;AACvC,MAAM,CAAC,MAAM,uBAAuB,GAAG,GAAG,CAAC;AAC3C,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,cAAc,GAAG,EAAE,CAAC;AACjC,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,qBAAqB,GAAG,GAAG,CAAC;AACzC,MAAM,CAAC,MAAM,iBAAiB,GAAG,GAAG,CAAC;AACrC,MAAM,CAAC,MAAM,kBAAkB,GAAG,GAAG,CAAC;AACtC,MAAM,CAAC,MAAM,mBAAmB,GAAG,GAAG,CAAC;AACvC,MAAM,CAAC,MAAM,2BAA2B,GAAG,MAAM,CAAC;AAElD,MAAM,CAAC,MAAM,MAAM,GAAG,UAAU,CAAC;AACjC,MAAM,CAAC,MAAM,OAAO,GAAG,SAAS,CAAC;AACjC,MAAM,CAAC,MAAM,4BAA4B,GAAG,KAAK,CAAC"}
+{"version":3,"file":"constants.js","sourceRoot":"","sources":["../../src/component/constants.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,sBAAsB,GAAG,CAAC,CAAC;AACxC,MAAM,CAAC,MAAM,eAAe,GAAG,EAAE,GAAG,IAAI,CAAC;AACzC,MAAM,CAAC,MAAM,6BAA6B,GAAG,GAAG,CAAC;AACjD,MAAM,CAAC,MAAM,oBAAoB,GAAG,MAAM,CAAC;AAC3C,MAAM,CAAC,MAAM,4BAA4B,GAAG,GAAG,GAAG,IAAI,CAAC;AACvD,MAAM,CAAC,MAAM,kBAAkB,GAAG,KAAK,CAAC;AACxC,MAAM,CAAC,MAAM,gCAAgC,GAAG,GAAG,CAAC;AACpD,MAAM,CAAC,MAAM,+BAA+B,GAAG,GAAG,GAAG,IAAI,CAAC;AAC1D,MAAM,CAAC,MAAM,mBAAmB,GAAG,MAAM,CAAC;AAC1C,MAAM,CAAC,MAAM,sBAAsB,GAAG,KAAK,CAAC;AAC5C,MAAM,CAAC,MAAM,gBAAgB,GAAG,GAAG,CAAC;AACpC,MAAM,CAAC,MAAM,uBAAuB,GAAG,GAAG,CAAC;AAC3C,MAAM,CAAC,MAAM,0BAA0B,GAAG,EAAE,CAAC;AAC7C,MAAM,CAAC,MAAM,eAAe,GAAG,EAAE,CAAC;AAClC,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,mBAAmB,GAAG,GAAG,CAAC;AACvC,MAAM,CAAC,MAAM,kBAAkB,GAAG,EAAE,CAAC;AACrC,MAAM,CAAC,MAAM,oBAAoB,GAAG,EAAE,CAAC;AACvC,MAAM,CAAC,MAAM,uBAAuB,GAAG,GAAG,CAAC;AAC3C,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,cAAc,GAAG,EAAE,CAAC;AACjC,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AACnC,MAAM,CAAC,MAAM,qBAAqB,GAAG,GAAG,CAAC;AACzC,MAAM,CAAC,MAAM,iBAAiB,GAAG,GAAG,CAAC;AACrC,MAAM,CAAC,MAAM,kBAAkB,GAAG,GAAG,CAAC;AACtC,MAAM,CAAC,MAAM,mBAAmB,GAAG,GAAG,CAAC;AACvC,MAAM,CAAC,MAAM,2BAA2B,GAAG,MAAM,CAAC;AAElD,MAAM,CAAC,MAAM,MAAM,GAAG,UAAU,CAAC;AACjC,MAAM,CAAC,MAAM,OAAO,GAAG,SAAS,CAAC"}

package/dist/component/eventStore.d.ts

@@ -0,0 +1,94 @@
+import type { Doc, Id } from "./_generated/dataModel.js";
+import type { MutationCtx, QueryCtx } from "./_generated/server.js";
+import type { TrackerEvent } from "./validators.js";
+export type TelemetryMatch = {
+    sessionId: string;
+    visitorId: string;
+    country?: string;
+    city?: string;
+    latitude?: number;
+    longitude?: number;
+    browser?: string;
+    os?: string;
+    device?: string;
+};
+type BufferedEventBatch = {
+    siteId: Id<"sites">;
+    sessionId: string;
+    visitorId: string;
+    bucketStart: number;
+    events: TrackerEvent[];
+};
+export type EventBatchBuffer = Map<string, BufferedEventBatch>;
+export type JourneyCursor = {
+    timestamp: number;
+    creationTime: number;
+    id: string;
+};
+export type RawJourneyEvent = Omit<Doc<"events">, "_id"> & {
+    _id: string;
+};
+export declare function eventBucketStart(timestamp: number): number;
+export declare function eventBatchKey(sessionId: string, bucketStart: number): string;
+export declare function inspectTelemetryMinute(ctx: MutationCtx, input: {
+    siteId: Id<"sites">;
+    sessionId: string;
+    bucketStart: number;
+    candidateEventIds: Set<string>;
+}): Promise<{
+    eventIds: Set<string>;
+    persistedCount: number;
+}>;
+export declare function findTelemetry(ctx: MutationCtx, siteId: Id<"sites">, event: TrackerEvent): Promise<{
+    match: TelemetryMatch | null;
+    saturated: boolean;
+}>;
+export declare function bufferEventBatch(siteId: Id<"sites">, event: TrackerEvent, buffer: EventBatchBuffer): void;
+export declare function insertEventBatch(ctx: MutationCtx, siteId: Id<"sites">, event: TrackerEvent): Promise<void>;
+export declare function flushEventBatches(ctx: MutationCtx, buffer: EventBatchBuffer): Promise<void>;
+export declare function loadSessionJourneyEvents(ctx: QueryCtx, input: {
+    siteId: Id<"sites">;
+    sessionId: string;
+    from: number;
+    to: number;
+    cursor: JourneyCursor | null;
+    resultLimit: number;
+}): Promise<RawJourneyEvent[]>;
+export declare function compareJourneyEvents(left: Pick<RawJourneyEvent, "timestamp" | "_creationTime" | "_id">, right: Pick<RawJourneyEvent, "timestamp" | "_creationTime" | "_id">): number;
+export declare function hydrateEventContexts(ctx: QueryCtx, rows: RawJourneyEvent[]): Promise<{
+    _creationTime: number;
+    _id: string;
+    country?: string | undefined;
+    city?: string | undefined;
+    latitude?: number | undefined;
+    longitude?: number | undefined;
+    browser: string;
+    os: string;
+    device: string;
+    affiliateSlug?: string | undefined;
+    currency?: string | undefined;
+    eventId: string;
+    href?: string | undefined;
+    name?: string | undefined;
+    path: string;
+    properties?: Record<string, string | number | boolean | null> | undefined;
+    referrer?: string | undefined;
+    revenueCents?: number | undefined;
+    sequence: number;
+    sessionId: string;
+    siteId: import("convex/values").GenericId<"sites">;
+    target?: string | undefined;
+    timestamp: number;
+    type: "click" | "conversion" | "custom" | "heartbeat" | "outbound" | "pageview";
+    visitorId: string;
+}[]>;
+export declare function deleteTelemetryBefore(ctx: MutationCtx, input: {
+    siteId: Id<"sites">;
+    before: number;
+    batchSize: number;
+}): Promise<{
+    deleted: number;
+    hasMore: boolean;
+}>;
+export {};
+//# sourceMappingURL=eventStore.d.ts.map

package/dist/component/eventStore.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"eventStore.d.ts","sourceRoot":"","sources":["../../src/component/eventStore.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,GAAG,EAAE,EAAE,EAAE,MAAM,2BAA2B,CAAC;AACzD,OAAO,KAAK,EAAE,WAAW,EAAE,QAAQ,EAAE,MAAM,wBAAwB,CAAC;AAKpE,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,iBAAiB,CAAC;AAEpD,MAAM,MAAM,cAAc,GAAG;IAC3B,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,EAAE,CAAC,EAAE,MAAM,CAAC;IACZ,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB,CAAC;AAEF,KAAK,kBAAkB,GAAG;IACxB,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE,YAAY,EAAE,CAAC;CACxB,CAAC;AAEF,MAAM,MAAM,gBAAgB,GAAG,GAAG,CAAC,MAAM,EAAE,kBAAkB,CAAC,CAAC;AAE/D,MAAM,MAAM,aAAa,GAAG;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;IACrB,EAAE,EAAE,MAAM,CAAC;CACZ,CAAC;AAEF,MAAM,MAAM,eAAe,GAAG,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,EAAE,KAAK,CAAC,GAAG;IAAE,GAAG,EAAE,MAAM,CAAA;CAAE,CAAC;AAE3E,wBAAgB,gBAAgB,CAAC,SAAS,EAAE,MAAM,GAAG,MAAM,CAE1D;AAED,wBAAgB,aAAa,CAAC,SAAS,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM,GAAG,MAAM,CAE5E;AAED,wBAAsB,sBAAsB,CAC1C,GAAG,EAAE,WAAW,EAChB,KAAK,EAAE;IACL,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,iBAAiB,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC;CAChC;;;GA0BF;AAED,wBAAsB,aAAa,CACjC,GAAG,EAAE,WAAW,EAChB,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,EACnB,KAAK,EAAE,YAAY,GAClB,OAAO,CAAC;IAAE,KAAK,EAAE,cAAc,GAAG,IAAI,CAAC;IAAC,SAAS,EAAE,OAAO,CAAA;CAAE,CAAC,CAiC/D;AAED,wBAAgB,gBAAgB,CAC9B,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,EACnB,KAAK,EAAE,YAAY,EACnB,MAAM,EAAE,gBAAgB,QAmBzB;AAED,wBAAsB,gBAAgB,CACpC,GAAG,EAAE,WAAW,EAChB,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,EACnB,KAAK,EAAE,YAAY,iBASpB;AAED,wBAAsB,iBAAiB,CACrC,GAAG,EAAE,WAAW,EAChB,MAAM,EAAE,gBAAgB,iBAKzB;AAED,wBAAsB,wBAAwB,CAC5C,GAAG,EAAE,QAAQ,EACb,KAAK,EAAE;IACL,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,EAAE,EAAE,MAAM,CAAC;IACX,MAAM,EAAE,aAAa,GAAG,IAAI,CAAC;IAC7B,WAAW,EAAE,MAAM,CAAC;CACrB,GACA,OAAO,CAAC,eAAe,EAAE,CAAC,CAqG5B;AAED,wBAAgB,oBAAoB,CAClC,IAAI,EAAE,IAAI,CAAC,eAAe,EAAE,WAAW,GAAG,eAAe,GAAG,KAAK,CAAC,EAClE,KAAK,EAAE,IAAI,CAAC,eAAe,EAAE,WAAW,GAAG,eAAe,GAAG,KAAK,CAAC,GAClE,MAAM,CAIR;AAED,wBAAsB,oBAAoB,CACxC,GAAG,EAAE,QAAQ,EACb,IAAI,EAAE,eAAe,EAAE;;SA/PyC,MAAM;;;;;;;;;;;;;;;;;;;;;;;;KAwSvE;AAED,wBAAsB,qBAAqB,CACzC,GAAG,EAAE,WAAW,EAChB,KAAK,EAAE;IACL,MAAM,EAAE,EAAE,CAAC,OAAO,CAAC,CAAC;IACpB,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,CAAC;CACnB;;;GA+BF"}

package/dist/component/eventStore.js

@@ -0,0 +1,304 @@
+import { MAX_EVENTS_PER_SESSION_WINDOW, RATE_LIMIT_WINDOW_MS, } from "./constants.js";
+export function eventBucketStart(timestamp) {
+    return Math.floor(timestamp / RATE_LIMIT_WINDOW_MS) * RATE_LIMIT_WINDOW_MS;
+}
+export function eventBatchKey(sessionId, bucketStart) {
+    return JSON.stringify([sessionId, bucketStart]);
+}
+export async function inspectTelemetryMinute(ctx, input) {
+    const [legacyEvents, batches] = await loadTelemetryMinute(ctx, input);
+    const eventIds = new Set([
+        ...legacyEvents.map((event) => event.eventId),
+        ...batches.flatMap((batch) => batch.events.map((event) => event.eventId)),
+    ]);
+    if (legacyEvents.length > MAX_EVENTS_PER_SESSION_WINDOW) {
+        await Promise.all([...input.candidateEventIds]
+            .filter((eventId) => !eventIds.has(eventId))
+            .map(async (eventId) => {
+            const legacy = await findLegacyTelemetry(ctx, {
+                ...input,
+                eventId,
+            });
+            if (legacy)
+                eventIds.add(eventId);
+        }));
+    }
+    return {
+        eventIds,
+        persistedCount: telemetryMinuteCount(legacyEvents, batches),
+    };
+}
+export async function findTelemetry(ctx, siteId, event) {
+    const bucketStart = eventBucketStart(event.timestamp);
+    const [legacyEvents, batches] = await loadTelemetryMinute(ctx, {
+        siteId,
+        sessionId: event.sessionId,
+        bucketStart,
+    });
+    const persistedCount = telemetryMinuteCount(legacyEvents, batches);
+    let legacyEvent = legacyEvents.find((candidate) => candidate.eventId === event.eventId);
+    if (!legacyEvent &&
+        legacyEvents.length > MAX_EVENTS_PER_SESSION_WINDOW) {
+        legacyEvent = await findLegacyTelemetry(ctx, {
+            siteId,
+            sessionId: event.sessionId,
+            bucketStart,
+            eventId: event.eventId,
+        }) ?? undefined;
+    }
+    const saturated = persistedCount >= MAX_EVENTS_PER_SESSION_WINDOW;
+    if (legacyEvent)
+        return { match: legacyEvent, saturated };
+    const batch = batches.find((candidate) => candidate.events.some((nested) => nested.eventId === event.eventId));
+    return {
+        match: batch
+            ? { sessionId: batch.sessionId, visitorId: batch.visitorId }
+            : null,
+        saturated,
+    };
+}
+export function bufferEventBatch(siteId, event, buffer) {
+    const bucketStart = eventBucketStart(event.timestamp);
+    const key = eventBatchKey(event.sessionId, bucketStart);
+    const existing = buffer.get(key);
+    if (existing) {
+        if (existing.visitorId !== event.visitorId) {
+            throw new Error("event batch visitor does not match its session");
+        }
+        existing.events.push(eventForBatch(event));
+        return;
+    }
+    buffer.set(key, {
+        siteId,
+        sessionId: event.sessionId,
+        visitorId: event.visitorId,
+        bucketStart,
+        events: [eventForBatch(event)],
+    });
+}
+export async function insertEventBatch(ctx, siteId, event) {
+    await ctx.db.insert("eventBatches", {
+        siteId,
+        sessionId: event.sessionId,
+        visitorId: event.visitorId,
+        bucketStart: eventBucketStart(event.timestamp),
+        events: [eventForBatch(event)],
+    });
+}
+export async function flushEventBatches(ctx, buffer) {
+    for (const batch of buffer.values()) {
+        await ctx.db.insert("eventBatches", batch);
+    }
+}
+export async function loadSessionJourneyEvents(ctx, input) {
+    const startTimestamp = input.cursor?.timestamp ?? input.from;
+    const firstBucket = eventBucketStart(startTimestamp);
+    const lastBucket = eventBucketStart(input.to);
+    // Nested event tiebreakers are not index fields, so the boundary minute must
+    // be read completely before applying the cursor.
+    const loadBatches = async () => {
+        const batches = [];
+        let eventCount = 0;
+        const requiredEvents = input.resultLimit + MAX_EVENTS_PER_SESSION_WINDOW;
+        let completedBucket;
+        const query = ctx.db
+            .query("eventBatches")
+            .withIndex("by_siteId_and_sessionId_and_bucketStart", (range) => range
+            .eq("siteId", input.siteId)
+            .eq("sessionId", input.sessionId)
+            .gte("bucketStart", firstBucket)
+            .lte("bucketStart", lastBucket))
+            .order("asc");
+        for await (const batch of query) {
+            if (completedBucket !== undefined && batch.bucketStart !== completedBucket) {
+                break;
+            }
+            batches.push(batch);
+            eventCount += batch.events.length;
+            if (eventCount >= requiredEvents)
+                completedBucket = batch.bucketStart;
+        }
+        return batches;
+    };
+    const loadLegacyEvents = async () => {
+        const take = input.resultLimit + 1;
+        if (!input.cursor) {
+            return ctx.db
+                .query("events")
+                .withIndex("by_siteId_and_sessionId_and_timestamp", (range) => range
+                .eq("siteId", input.siteId)
+                .eq("sessionId", input.sessionId)
+                .gte("timestamp", input.from)
+                .lte("timestamp", input.to))
+                .order("asc")
+                .take(take);
+        }
+        const rows = await ctx.db
+            .query("events")
+            .withIndex("by_siteId_and_sessionId_and_timestamp", (range) => range
+            .eq("siteId", input.siteId)
+            .eq("sessionId", input.sessionId)
+            .eq("timestamp", input.cursor.timestamp)
+            .gt("_creationTime", input.cursor.creationTime))
+            .order("asc")
+            .take(take);
+        if (rows.length < take) {
+            rows.push(...await ctx.db
+                .query("events")
+                .withIndex("by_siteId_and_sessionId_and_timestamp", (range) => range
+                .eq("siteId", input.siteId)
+                .eq("sessionId", input.sessionId)
+                .gt("timestamp", input.cursor.timestamp)
+                .lte("timestamp", input.to))
+                .order("asc")
+                .take(take - rows.length));
+        }
+        return rows;
+    };
+    const [legacyEvents, batches] = await Promise.all([
+        loadLegacyEvents(),
+        loadBatches(),
+    ]);
+    const rows = legacyEvents.map((event) => ({
+        ...event,
+        _id: String(event._id),
+    }));
+    for (const batch of batches) {
+        for (let index = 0; index < batch.events.length; index += 1) {
+            const event = batch.events[index];
+            rows.push({
+                ...event,
+                _id: `eventBatch:${batch._id}:${index.toString().padStart(3, "0")}`,
+                _creationTime: batch._creationTime,
+                siteId: batch.siteId,
+                sessionId: batch.sessionId,
+                visitorId: batch.visitorId,
+            });
+        }
+    }
+    return rows
+        .filter((event) => event.timestamp >= input.from &&
+        event.timestamp <= input.to &&
+        (!input.cursor || compareJourneyEventToCursor(event, input.cursor) > 0))
+        .sort(compareJourneyEvents)
+        .slice(0, input.resultLimit);
+}
+export function compareJourneyEvents(left, right) {
+    return left.timestamp - right.timestamp ||
+        left._creationTime - right._creationTime ||
+        left._id.localeCompare(right._id);
+}
+export async function hydrateEventContexts(ctx, rows) {
+    const sessionPromises = new Map();
+    for (const row of rows) {
+        const key = `${row.siteId}:${row.sessionId}`;
+        if (!sessionPromises.has(key)) {
+            sessionPromises.set(key, ctx.db
+                .query("sessions")
+                .withIndex("by_siteId_and_sessionId", (range) => range.eq("siteId", row.siteId).eq("sessionId", row.sessionId))
+                .unique());
+        }
+    }
+    const sessions = new Map(await Promise.all([...sessionPromises].map(async ([key, session]) => [key, await session])));
+    return rows.map((row) => {
+        const session = sessions.get(`${row.siteId}:${row.sessionId}`);
+        const country = row.country ?? session?.country;
+        const city = row.city ?? session?.city;
+        const latitude = row.latitude ?? session?.latitude;
+        const longitude = row.longitude ?? session?.longitude;
+        return {
+            ...row,
+            ...(country ? { country } : {}),
+            ...(city ? { city } : {}),
+            ...(latitude !== undefined ? { latitude } : {}),
+            ...(longitude !== undefined ? { longitude } : {}),
+            browser: row.browser ?? session?.browser ?? "Other",
+            os: row.os ?? session?.os ?? "Other",
+            device: row.device ?? session?.device ?? "unknown",
+        };
+    });
+}
+export async function deleteTelemetryBefore(ctx, input) {
+    const batchCutoff = eventBucketStart(input.before);
+    const limit = input.batchSize + 1;
+    const [legacyEvents, batches] = await Promise.all([
+        ctx.db
+            .query("events")
+            .withIndex("by_siteId_and_timestamp", (range) => range.eq("siteId", input.siteId).lt("timestamp", input.before))
+            .take(limit),
+        ctx.db
+            .query("eventBatches")
+            .withIndex("by_siteId_and_bucketStart", (range) => range.eq("siteId", input.siteId).lt("bucketStart", batchCutoff))
+            .take(limit),
+    ]);
+    const rows = [...legacyEvents, ...batches].slice(0, input.batchSize);
+    for (const row of rows) {
+        if ("bucketStart" in row) {
+            await ctx.db.delete("eventBatches", row._id);
+        }
+        else {
+            await ctx.db.delete("events", row._id);
+        }
+    }
+    return {
+        deleted: rows.length,
+        hasMore: legacyEvents.length + batches.length > input.batchSize,
+    };
+}
+async function loadTelemetryMinute(ctx, input) {
+    return Promise.all([
+        ctx.db
+            .query("events")
+            .withIndex("by_siteId_and_sessionId_and_timestamp", (range) => range
+            .eq("siteId", input.siteId)
+            .eq("sessionId", input.sessionId)
+            .gte("timestamp", input.bucketStart)
+            .lt("timestamp", input.bucketStart + RATE_LIMIT_WINDOW_MS))
+            .take(MAX_EVENTS_PER_SESSION_WINDOW + 1),
+        ctx.db
+            .query("eventBatches")
+            .withIndex("by_siteId_and_sessionId_and_bucketStart", (range) => range
+            .eq("siteId", input.siteId)
+            .eq("sessionId", input.sessionId)
+            .eq("bucketStart", input.bucketStart))
+            .take(MAX_EVENTS_PER_SESSION_WINDOW + 1),
+    ]);
+}
+async function findLegacyTelemetry(ctx, input) {
+    const candidates = await ctx.db
+        .query("events")
+        .withIndex("by_siteId_and_eventId", (range) => range.eq("siteId", input.siteId).eq("eventId", input.eventId))
+        .take(2);
+    return candidates.find((candidate) => candidate.sessionId === input.sessionId &&
+        eventBucketStart(candidate.timestamp) === input.bucketStart) ?? null;
+}
+function telemetryMinuteCount(legacyEvents, batches) {
+    return legacyEvents.length + batches.reduce((total, batch) => total + batch.events.length, 0);
+}
+function eventForBatch(event) {
+    return {
+        eventId: event.eventId,
+        sessionId: event.sessionId,
+        visitorId: event.visitorId,
+        type: event.type,
+        ...(event.name ? { name: event.name } : {}),
+        path: event.path,
+        ...(event.referrer ? { referrer: event.referrer } : {}),
+        timestamp: event.timestamp,
+        sequence: event.sequence,
+        ...(event.target ? { target: event.target } : {}),
+        ...(event.href ? { href: event.href } : {}),
+        ...(event.properties ? { properties: event.properties } : {}),
+        ...(event.revenueCents !== undefined
+            ? { revenueCents: event.revenueCents }
+            : {}),
+        ...(event.currency ? { currency: event.currency } : {}),
+        ...(event.affiliateSlug ? { affiliateSlug: event.affiliateSlug } : {}),
+    };
+}
+function compareJourneyEventToCursor(event, cursor) {
+    return event.timestamp - cursor.timestamp ||
+        event._creationTime - cursor.creationTime ||
+        event._id.localeCompare(cursor.id);
+}
+//# sourceMappingURL=eventStore.js.map