@iann29/rastro 0.1.0-alpha.8 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +274 -75
- package/agent/integration.md +223 -65
- package/agent/manifest.json +34 -13
- package/agent/manifest.schema.json +53 -11
- package/dist/client/federation.d.ts +196 -11
- package/dist/client/federation.d.ts.map +1 -1
- package/dist/client/federation.js +160 -6
- package/dist/client/federation.js.map +1 -1
- package/dist/client/heat.d.ts +37 -0
- package/dist/client/heat.d.ts.map +1 -0
- package/dist/client/heat.js +43 -0
- package/dist/client/heat.js.map +1 -0
- package/dist/client/index.d.ts +448 -105
- package/dist/client/index.d.ts.map +1 -1
- package/dist/client/index.js +207 -17
- package/dist/client/index.js.map +1 -1
- package/dist/component/_generated/api.d.ts +6 -0
- package/dist/component/_generated/api.d.ts.map +1 -1
- package/dist/component/_generated/api.js.map +1 -1
- package/dist/component/_generated/component.d.ts +53 -16
- package/dist/component/_generated/component.d.ts.map +1 -1
- package/dist/component/constants.d.ts +5 -1
- package/dist/component/constants.d.ts.map +1 -1
- package/dist/component/constants.js +8 -1
- package/dist/component/constants.js.map +1 -1
- package/dist/component/coverage.d.ts +9 -0
- package/dist/component/coverage.d.ts.map +1 -1
- package/dist/component/coverage.js +36 -6
- package/dist/component/coverage.js.map +1 -1
- package/dist/component/eventStore.d.ts +1 -1
- package/dist/component/geo.d.ts +2 -0
- package/dist/component/geo.d.ts.map +1 -1
- package/dist/component/geo.js +56 -19
- package/dist/component/geo.js.map +1 -1
- package/dist/component/http.d.ts.map +1 -1
- package/dist/component/http.js +49 -27
- package/dist/component/http.js.map +1 -1
- package/dist/component/ingest.d.ts +1 -1
- package/dist/component/ingest.d.ts.map +1 -1
- package/dist/component/ingest.js +252 -305
- package/dist/component/ingest.js.map +1 -1
- package/dist/component/localTime.d.ts +25 -0
- package/dist/component/localTime.d.ts.map +1 -0
- package/dist/component/localTime.js +126 -0
- package/dist/component/localTime.js.map +1 -0
- package/dist/component/reports.d.ts +46 -15
- package/dist/component/reports.d.ts.map +1 -1
- package/dist/component/reports.js +193 -38
- package/dist/component/reports.js.map +1 -1
- package/dist/component/retention.js +26 -8
- package/dist/component/retention.js.map +1 -1
- package/dist/component/rollupStore.d.ts +320 -0
- package/dist/component/rollupStore.d.ts.map +1 -0
- package/dist/component/rollupStore.js +596 -0
- package/dist/component/rollupStore.js.map +1 -0
- package/dist/component/rollups.d.ts +20 -0
- package/dist/component/rollups.d.ts.map +1 -0
- package/dist/component/rollups.js +73 -0
- package/dist/component/rollups.js.map +1 -0
- package/dist/component/sanitize.d.ts +5 -0
- package/dist/component/sanitize.d.ts.map +1 -1
- package/dist/component/sanitize.js +15 -0
- package/dist/component/sanitize.js.map +1 -1
- package/dist/component/schema.d.ts +349 -15
- package/dist/component/schema.js +25 -0
- package/dist/component/schema.js.map +1 -1
- package/dist/component/sites.d.ts +12 -0
- package/dist/component/sites.d.ts.map +1 -1
- package/dist/component/sites.js +30 -0
- package/dist/component/sites.js.map +1 -1
- package/dist/component/validators.d.ts +41 -20
- package/dist/component/validators.d.ts.map +1 -1
- package/dist/component/validators.js +24 -3
- package/dist/component/validators.js.map +1 -1
- package/dist/tracker/generated.d.ts +7 -4
- package/dist/tracker/generated.d.ts.map +1 -1
- package/dist/tracker/generated.js +7 -4
- package/dist/tracker/generated.js.map +1 -1
- package/dist/tracker/tracker.d.ts +1 -1
- package/dist/tracker/tracker.d.ts.map +1 -1
- package/dist/tracker/tracker.js +24 -2
- package/dist/tracker/tracker.js.map +1 -1
- package/dist/tracker.min.js +1 -1
- package/docs/federation-setup.md +104 -27
- package/docs/federation.md +168 -31
- package/docs/upgrading.md +191 -16
- package/llms.txt +9 -7
- package/package.json +2 -1
- package/src/component/_generated/api.ts +6 -0
- package/src/component/_generated/component.ts +50 -12
- package/src/component/constants.ts +8 -1
- package/src/component/coverage.ts +46 -6
- package/src/component/geo.ts +82 -28
- package/src/component/http.ts +62 -26
- package/src/component/ingest.ts +378 -440
- package/src/component/localTime.ts +167 -0
- package/src/component/reports.ts +261 -39
- package/src/component/retention.ts +25 -7
- package/src/component/rollupStore.ts +799 -0
- package/src/component/rollups.ts +82 -0
- package/src/component/sanitize.ts +14 -0
- package/src/component/schema.ts +35 -0
- package/src/component/sites.ts +32 -0
- package/src/component/validators.ts +28 -1
- package/src/tracker/generated.ts +12 -0
package/README.md
CHANGED
|
@@ -13,12 +13,12 @@ inside your Convex deployment.
|
|
|
13
13
|
|
|
14
14
|
## Project status
|
|
15
15
|
|
|
16
|
-
Amage Rastro is
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
operator-approved exact registry artifact for every required runtime
|
|
21
|
-
export before changing a host; follow the gate in the
|
|
16
|
+
Amage Rastro is actively developed, with a canonical batched event store,
|
|
17
|
+
protected CI, exact run-scoped benchmark reconciliation, and `0.1.0` as the
|
|
18
|
+
first release on the untagged npm `latest` channel; the alphas before it stay
|
|
19
|
+
available under their exact versions. Federation consumers must statically
|
|
20
|
+
inspect an operator-approved exact registry artifact for every required runtime
|
|
21
|
+
and type export before changing a host; follow the gate in the
|
|
22
22
|
[federation setup runbook](docs/federation-setup.md). Never substitute a mutable
|
|
23
23
|
branch or website-supplied source tarball.
|
|
24
24
|
|
|
@@ -30,9 +30,11 @@ the [machine manifest](https://www.amagerastro.com/agent/manifest.json).
|
|
|
30
30
|
Current evidence supports more than 140 million stored events/day for realistic
|
|
31
31
|
traffic and more than 178 million for lean traffic. Near-limit heavy payloads
|
|
32
32
|
are certified at 55.26 million/day on a data-heavy store (76.62 million/day on a
|
|
33
|
-
fresh store, 2026-08-22). The feature-saturated profile
|
|
34
|
-
certification on a
|
|
35
|
-
([#52](https://github.com/amageweb/amage-rastro/issues/52))
|
|
33
|
+
fresh store, 2026-08-22). The feature-saturated profile failed its last
|
|
34
|
+
certification on a rollup-contention defect
|
|
35
|
+
([#52](https://github.com/amageweb/amage-rastro/issues/52)); the contended write
|
|
36
|
+
path no longer exists (report rollups are folded by one job per site, off the
|
|
37
|
+
ingestion transaction) and the profile awaits recertification. Workload-specific
|
|
36
38
|
results and caveats are reported separately rather than collapsed into one
|
|
37
39
|
capacity claim.
|
|
38
40
|
|
|
@@ -40,7 +42,7 @@ capacity claim.
|
|
|
40
42
|
|
|
41
43
|
| Capability | Implementation |
|
|
42
44
|
| --------------------------- | --------------------------------------------------------------------------------------------------------- |
|
|
43
|
-
| Live visitor map | Reactive live sessions with
|
|
45
|
+
| Live visitor map | Reactive live sessions with city-level coordinates, realtime departures, and a 90-second safety net |
|
|
44
46
|
| Traffic analytics | Sharded hour/day rollups, time series, top pages, sources, countries, devices, browsers, and events |
|
|
45
47
|
| Field Web Vitals | Opt-in tracker module reporting LCP, CLS, INP, FCP, and TTFB into bounded per-page daily histograms |
|
|
46
48
|
| Site map | Daily per-route rollups: entries, derived exits and bounces, time on page, and route-to-route transitions |
|
|
@@ -52,7 +54,7 @@ capacity claim.
|
|
|
52
54
|
| Automatic GeoIP | Opt-in native request metadata plus IPinfo/IPWhois fallback; bounded quota and no raw-IP persistence |
|
|
53
55
|
| Auto-captured clicks | Anchors, buttons, and `data-rastro-event` controls with a bounded visible label; outbound origin+path |
|
|
54
56
|
| Bot filtering | Self-declared crawlers, unfurlers, monitors, HTTP libraries, and headless browsers rejected at ingestion |
|
|
55
|
-
| Cookieless mode |
|
|
57
|
+
| Cookieless mode | No cookies: persistent pseudonymous visitor id in `localStorage`, per-tab session id in `sessionStorage` |
|
|
56
58
|
| Tiny tracker | 2,132 bytes raw, 1,127 bytes gzip; the test suite enforces `< 1,280` bytes gzip |
|
|
57
59
|
|
|
58
60
|
## Quickstart
|
|
@@ -60,8 +62,11 @@ capacity claim.
|
|
|
60
62
|
### Prerequisites
|
|
61
63
|
|
|
62
64
|
- Convex `^1.43.0` or a compatible Synapse deployment.
|
|
63
|
-
-
|
|
64
|
-
|
|
65
|
+
- A way to authorize site administration and local report functions: the host's
|
|
66
|
+
existing authentication provider, or, for a site with no user accounts, a
|
|
67
|
+
fixed opaque owner ID and internal functions run with deployment credentials
|
|
68
|
+
(see [Hosts without user authentication](#hosts-without-user-authentication)).
|
|
69
|
+
The browser ingestion route itself is public and origin-checked.
|
|
65
70
|
- Two deployment origins recorded separately:
|
|
66
71
|
|
|
67
72
|
| Purpose | Convex Cloud example | Synapse example |
|
|
@@ -74,11 +79,11 @@ Never append `/rastro/tracker.js` to the functions URL.
|
|
|
74
79
|
### 1. Install
|
|
75
80
|
|
|
76
81
|
```bash
|
|
77
|
-
npm install @iann29/rastro
|
|
82
|
+
npm install @iann29/rastro
|
|
78
83
|
```
|
|
79
84
|
|
|
80
|
-
The untagged npm `latest` channel
|
|
81
|
-
|
|
85
|
+
The untagged npm `latest` channel is `0.1.0`. Pin the exact version in the host;
|
|
86
|
+
the changelog and the upgrading guide say what each release changes.
|
|
82
87
|
|
|
83
88
|
### 2. Mount the component
|
|
84
89
|
|
|
@@ -133,7 +138,9 @@ it does not prove that a browser event was accepted.
|
|
|
133
138
|
|
|
134
139
|
Rastro uses Convex request metadata as the canonical client IP and can resolve
|
|
135
140
|
coarse city-level coordinates on the first request of each session. GeoIP is
|
|
136
|
-
opt-in because the configured provider receives the request IP.
|
|
141
|
+
opt-in because the configured provider receives the request IP. Declare the
|
|
142
|
+
settings as optional host environment variables and pass them through, so each
|
|
143
|
+
deployment decides on its own provider, token, and cap without a code change:
|
|
137
144
|
|
|
138
145
|
```ts
|
|
139
146
|
// convex/convex.config.ts
|
|
@@ -141,19 +148,47 @@ import { defineApp } from "convex/server";
|
|
|
141
148
|
import { v } from "convex/values";
|
|
142
149
|
import rastro from "@iann29/rastro/convex.config.js";
|
|
143
150
|
|
|
144
|
-
const app = defineApp({
|
|
151
|
+
const app = defineApp({
|
|
152
|
+
env: {
|
|
153
|
+
RASTRO_GEOIP_PROVIDER: v.optional(
|
|
154
|
+
v.union(v.literal("ipinfo"), v.literal("ipwhois")),
|
|
155
|
+
),
|
|
156
|
+
RASTRO_GEOIP_TOKEN: v.optional(v.string()),
|
|
157
|
+
RASTRO_GEOIP_DAILY_LIMIT: v.optional(v.string()),
|
|
158
|
+
RASTRO_TRUST_PROXY: v.optional(v.literal("true")),
|
|
159
|
+
},
|
|
160
|
+
});
|
|
145
161
|
app.use(rastro, {
|
|
146
162
|
httpPrefix: "/rastro/",
|
|
147
163
|
env: {
|
|
148
|
-
RASTRO_GEOIP_PROVIDER:
|
|
149
|
-
RASTRO_GEOIP_TOKEN: app.env.
|
|
150
|
-
RASTRO_GEOIP_DAILY_LIMIT:
|
|
164
|
+
RASTRO_GEOIP_PROVIDER: app.env.RASTRO_GEOIP_PROVIDER,
|
|
165
|
+
RASTRO_GEOIP_TOKEN: app.env.RASTRO_GEOIP_TOKEN,
|
|
166
|
+
RASTRO_GEOIP_DAILY_LIMIT: app.env.RASTRO_GEOIP_DAILY_LIMIT,
|
|
167
|
+
RASTRO_TRUST_PROXY: app.env.RASTRO_TRUST_PROXY,
|
|
151
168
|
},
|
|
152
169
|
});
|
|
153
170
|
|
|
154
171
|
export default app;
|
|
155
172
|
```
|
|
156
173
|
|
|
174
|
+
This is the shape of the [example host](example/convex/convex.config.ts). With
|
|
175
|
+
nothing set, provider lookups stay off. To enable IPinfo Core on one deployment:
|
|
176
|
+
|
|
177
|
+
```bash
|
|
178
|
+
# Convex Cloud development deployment
|
|
179
|
+
npx convex env set --deployment dev RASTRO_GEOIP_PROVIDER ipinfo
|
|
180
|
+
npx convex env set --deployment dev RASTRO_GEOIP_TOKEN 'YOUR_IPINFO_TOKEN'
|
|
181
|
+
npx convex env set --deployment dev RASTRO_GEOIP_DAILY_LIMIT 1000
|
|
182
|
+
|
|
183
|
+
# Synapse development deployment
|
|
184
|
+
synapse convex --dev env set RASTRO_GEOIP_PROVIDER ipinfo
|
|
185
|
+
synapse convex --dev env set RASTRO_GEOIP_TOKEN 'YOUR_IPINFO_TOKEN'
|
|
186
|
+
synapse convex --dev env set RASTRO_GEOIP_DAILY_LIMIT 1000
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Binding a literal instead of a passthrough (`RASTRO_GEOIP_PROVIDER: "ipinfo"`)
|
|
190
|
+
is also valid when every deployment of the project shares the provider.
|
|
191
|
+
|
|
157
192
|
`ipwhois` is also supported. Its token is optional for the documented
|
|
158
193
|
1,000-request/day free endpoint and recommended for production. Bind it as
|
|
159
194
|
`RASTRO_GEOIP_TOKEN` in the same way as the IPinfo example.
|
|
@@ -162,11 +197,11 @@ may be set from `0` through `1000000` to match the provider plan. Admission is
|
|
|
162
197
|
transactional, so concurrent first-session requests cannot exceed the cap.
|
|
163
198
|
|
|
164
199
|
When Rastro runs behind a reverse proxy that supplies `X-Forwarded-For` or
|
|
165
|
-
Vercel/Cloudflare geographic headers,
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
200
|
+
Vercel/Cloudflare geographic headers, set `RASTRO_TRUST_PROXY` to `true` on that
|
|
201
|
+
deployment. Leave it unset for direct Convex deployments. Rastro uses forwarded
|
|
202
|
+
data only when this opt-in is enabled and the runtime IP identifies a private
|
|
203
|
+
immediate peer. Caller-supplied proxy or geographic headers are ignored when
|
|
204
|
+
native metadata is absent or public.
|
|
170
205
|
|
|
171
206
|
The lookup has a 1.5-second timeout and fails open: telemetry ingestion
|
|
172
207
|
continues without location when the provider is unavailable. A session reserves
|
|
@@ -181,7 +216,13 @@ whether GeoIP is configured and the daily cap, without returning the address or
|
|
|
181
216
|
token. Deployment administrators can exercise the configured provider through
|
|
182
217
|
the internal `geo:probe` action. If health reports no usable client IP, use
|
|
183
218
|
trusted Vercel/Cloudflare geographic headers with `RASTRO_TRUST_PROXY` or fix
|
|
184
|
-
the deployment's proxy forwarding.
|
|
219
|
+
the deployment's proxy forwarding. Health also names the build the deployment
|
|
220
|
+
serves under `tracker`: the package `version` plus the content `hash` of
|
|
221
|
+
`tracker.js` and the `vitalsHash` of `vitals.js`. Both scripts carry that hash
|
|
222
|
+
as a strong `ETag`, answer `If-None-Match` with `304`, and are cached for five
|
|
223
|
+
minutes with up to one hour of `stale-while-revalidate`, so a returning visitor
|
|
224
|
+
runs a new build after at most one stale page load; an open tab keeps its script
|
|
225
|
+
until it reloads.
|
|
185
226
|
|
|
186
227
|
### Browser and device classification
|
|
187
228
|
|
|
@@ -275,6 +316,66 @@ event/byte budgets.
|
|
|
275
316
|
|
|
276
317
|
Domain entries are hostnames, not host-and-port pairs. Use `localhost` or a full
|
|
277
318
|
origin such as `http://localhost:3000`; both normalize to the `localhost` host.
|
|
319
|
+
A site deployed on Vercel needs its preview hosts too: list `*.vercel.app` next
|
|
320
|
+
to the production host, or preview deployments are rejected as
|
|
321
|
+
`ORIGIN_NOT_ALLOWED`.
|
|
322
|
+
|
|
323
|
+
#### Hosts without user authentication
|
|
324
|
+
|
|
325
|
+
An institutional site with no user accounts has nobody to authenticate. Use one
|
|
326
|
+
fixed opaque owner ID and internal functions, which only deployment credentials
|
|
327
|
+
can run:
|
|
328
|
+
|
|
329
|
+
```ts
|
|
330
|
+
// convex/analyticsAdmin.ts
|
|
331
|
+
import { v } from "convex/values";
|
|
332
|
+
import { Rastro } from "@iann29/rastro";
|
|
333
|
+
import { components } from "./_generated/api";
|
|
334
|
+
import { internalMutation } from "./_generated/server";
|
|
335
|
+
|
|
336
|
+
const analytics = new Rastro(components.rastroAnalytics);
|
|
337
|
+
|
|
338
|
+
// Opaque and stable. Reuse the same value as `ownerId` when provisioning a
|
|
339
|
+
// federation grant.
|
|
340
|
+
const SITE_OWNER_ID = "acme-site-owner";
|
|
341
|
+
|
|
342
|
+
export const createSite = internalMutation({
|
|
343
|
+
args: {
|
|
344
|
+
name: v.string(),
|
|
345
|
+
domains: v.array(v.string()),
|
|
346
|
+
currency: v.string(),
|
|
347
|
+
},
|
|
348
|
+
returns: v.string(),
|
|
349
|
+
handler: async (ctx, args) =>
|
|
350
|
+
await analytics.createSite(ctx, {
|
|
351
|
+
ownerId: SITE_OWNER_ID,
|
|
352
|
+
name: args.name,
|
|
353
|
+
domains: args.domains,
|
|
354
|
+
currency: args.currency,
|
|
355
|
+
cookieless: true,
|
|
356
|
+
}),
|
|
357
|
+
});
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
Run it once per site with the deployment's own credentials, never from a
|
|
361
|
+
browser:
|
|
362
|
+
|
|
363
|
+
```bash
|
|
364
|
+
# Convex Cloud development deployment
|
|
365
|
+
npx convex run --deployment dev analyticsAdmin:createSite \
|
|
366
|
+
'{"name":"Acme","domains":["acme.example","*.vercel.app"],"currency":"BRL"}'
|
|
367
|
+
|
|
368
|
+
# Synapse development deployment
|
|
369
|
+
synapse convex --dev run analyticsAdmin:createSite \
|
|
370
|
+
'{"name":"Acme","domains":["acme.example","*.vercel.app"],"currency":"BRL"}'
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
Such a host needs no local report functions: the central dashboard reads the
|
|
374
|
+
[federated surface](#federated-dashboard-backend), and its `auth.config.ts`
|
|
375
|
+
lists only the Rastro federation provider, exactly as the
|
|
376
|
+
[example host](example/convex/auth.config.ts) does. Goals, funnels, affiliates,
|
|
377
|
+
and retention policies are configured the same way, through `internalMutation`s
|
|
378
|
+
that call the `Rastro` class with `SITE_OWNER_ID`.
|
|
278
379
|
|
|
279
380
|
### 4. Add the tracker
|
|
280
381
|
|
|
@@ -306,6 +407,33 @@ serves the script and ingestion from different origins:
|
|
|
306
407
|
></script>
|
|
307
408
|
```
|
|
308
409
|
|
|
410
|
+
In a Next.js App Router layout, `next/script` injects the same tag;
|
|
411
|
+
`document.currentScript` is set when it runs, so `data-site` and the inferred
|
|
412
|
+
endpoint work exactly as with a plain tag:
|
|
413
|
+
|
|
414
|
+
```tsx
|
|
415
|
+
// app/layout.tsx
|
|
416
|
+
import Script from "next/script";
|
|
417
|
+
|
|
418
|
+
export default function RootLayout({ children }) {
|
|
419
|
+
return (
|
|
420
|
+
<html lang="pt-BR">
|
|
421
|
+
<body>
|
|
422
|
+
{children}
|
|
423
|
+
<Script
|
|
424
|
+
src="https://YOUR_DEPLOYMENT.convex.site/rastro/tracker.js"
|
|
425
|
+
data-site="YOUR_SITE_ID"
|
|
426
|
+
strategy="afterInteractive"
|
|
427
|
+
/>
|
|
428
|
+
</body>
|
|
429
|
+
</html>
|
|
430
|
+
);
|
|
431
|
+
}
|
|
432
|
+
```
|
|
433
|
+
|
|
434
|
+
Remember the site's `domains`: a Vercel preview runs on `*.vercel.app`, not on
|
|
435
|
+
the production host.
|
|
436
|
+
|
|
309
437
|
### Field Web Vitals (optional)
|
|
310
438
|
|
|
311
439
|
A second, separate module measures real-visitor LCP, CLS, INP, FCP, and TTFB
|
|
@@ -371,17 +499,35 @@ after a session starts would make the session ambiguous.
|
|
|
371
499
|
seconds.
|
|
372
500
|
4. Confirm `POST /rastro/events` returns HTTP 200 and a response containing
|
|
373
501
|
`accepted`, `duplicates`, and `rejected` counters.
|
|
374
|
-
5. Run
|
|
375
|
-
|
|
502
|
+
5. Run `analytics:liveVisitors` (with `siteIds` and `now`) or
|
|
503
|
+
`analytics:sessionJourney` for the returned `siteId`: both reflect the
|
|
504
|
+
request as soon as it is accepted. `analytics:listSessions` and the
|
|
505
|
+
aggregate-backed `analytics:overview` can trail the live view by a few
|
|
506
|
+
seconds, and the overview only reports complete UTC hour/day buckets, so an
|
|
507
|
+
empty result from those two right after the first request is not a failed
|
|
508
|
+
ingestion.
|
|
376
509
|
|
|
377
510
|
The tracker intentionally does not interrupt the product UI when transport
|
|
378
511
|
fails. A loaded `tracker.js` or successful `/health` response alone is not proof
|
|
379
512
|
of ingestion. The usual causes of a rejected or missing request are:
|
|
380
513
|
|
|
381
514
|
- using the functions URL instead of the HTTP-actions URL;
|
|
382
|
-
- omitting the exact browser host from the site's `domains
|
|
515
|
+
- omitting the exact browser host from the site's `domains`, including
|
|
516
|
+
`*.vercel.app` for preview deployments;
|
|
383
517
|
- including query strings or paths in a configured base URL;
|
|
384
|
-
- querying a partial UTC bucket or a site the authenticated user does not own
|
|
518
|
+
- querying a partial UTC bucket or a site the authenticated user does not own;
|
|
519
|
+
- reading `listSessions` or `overview` seconds after the first request instead
|
|
520
|
+
of `liveVisitors`.
|
|
521
|
+
|
|
522
|
+
Since 0.1.0 a live row also carries the intent facts of its session:
|
|
523
|
+
`entryPath`, `returning` (the visitor had a history before this session; a host
|
|
524
|
+
id linked to anonymous history at sign-up is a return), `utmCampaign`,
|
|
525
|
+
`affiliateSlug`, `conversionCount`, and the furthest `funnel` step the visitor
|
|
526
|
+
reached (`{ funnelId, step, steps }`). `visitorHeat` from the package turns
|
|
527
|
+
those, plus the pages that complete a goal or a funnel step, into the
|
|
528
|
+
hot/warm/cold score the dashboard's live view shows; a host may apply the same
|
|
529
|
+
rules to its own signals. Rows written before the upgrade lack the facts and
|
|
530
|
+
score from what they have.
|
|
385
531
|
|
|
386
532
|
## Track custom events
|
|
387
533
|
|
|
@@ -421,8 +567,13 @@ of the session, and sends them as properties of every pageview; no other query
|
|
|
421
567
|
parameter ever leaves the page. A session's `source` is the lowercase
|
|
422
568
|
`utm_source` when present and the referrer host otherwise, so campaign traffic
|
|
423
569
|
no longer reads as `direct`. `utmSource`, `utmMedium`, and `utmCampaign` stay on
|
|
424
|
-
the session for journeys
|
|
425
|
-
|
|
570
|
+
the session for journeys; `overview.topCampaigns` ranks `utm_campaign` and
|
|
571
|
+
`overview.topMediums` the lowercase `utm_medium` next to `topSources`, by
|
|
572
|
+
sessions once the range is covered by the counters. `overview.topOutbound` ranks
|
|
573
|
+
the hosts outbound clicks reached — the host alone, never the path or query — by
|
|
574
|
+
clicks. Both lists are recorded from `metadata.dimensionsSince` (each site's
|
|
575
|
+
first ingest on 0.1.0 plus the rollup delay); a range that starts earlier ranks
|
|
576
|
+
them from partial data, and the dashboard says so.
|
|
426
577
|
|
|
427
578
|
## Identify visitors after signup
|
|
428
579
|
|
|
@@ -449,9 +600,10 @@ event keeps the id it was recorded with. Links are idempotent, an anonymous id
|
|
|
449
600
|
belongs to at most one identity, a linked id cannot be linked again, chains and
|
|
450
601
|
identity merges are rejected, and an identity holds at most ten aliases. The
|
|
451
602
|
tracker still sets no cookie: the host's own session identifies the account and
|
|
452
|
-
the anonymous
|
|
453
|
-
|
|
454
|
-
|
|
603
|
+
the anonymous ids live in the browser's own storage (the visitor id in
|
|
604
|
+
`localStorage`, the session id in `sessionStorage`). Linking anonymous history
|
|
605
|
+
to an account turns it into personal data under the host's privacy policy;
|
|
606
|
+
disclose it and offer an opt-out.
|
|
455
607
|
|
|
456
608
|
## Attribute trusted payments
|
|
457
609
|
|
|
@@ -545,16 +697,26 @@ while telemetry remains in the customer's deployment. It uses
|
|
|
545
697
|
`exposeFederatedAnalyticsApi` with issuer and connection-resolver options. The
|
|
546
698
|
helper constructs its authorization boundary internally; it cannot accept a
|
|
547
699
|
permissive host authorizer. Its surface contains a public protocol v1 manifest,
|
|
548
|
-
authenticated redacted connection status, redacted site summaries,
|
|
549
|
-
|
|
700
|
+
authenticated redacted connection status, redacted site summaries, the reactive
|
|
701
|
+
dashboard report queries, and — behind the optional `configure` capability — the
|
|
702
|
+
functions that manage goals, funnels, affiliates, a site's name, domains and
|
|
703
|
+
timezone, and the retention policy from the dashboard.
|
|
550
704
|
|
|
551
705
|
The JWT only identifies the requested connection. Site access remains
|
|
552
706
|
authoritative in a local host record resolved on every query, so revocation is
|
|
553
|
-
immediate and a token cannot grant itself another site.
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
707
|
+
immediate and a token cannot grant itself another site. The same record decides
|
|
708
|
+
whether the dashboard may configure: a configure function runs only when the
|
|
709
|
+
token carries `analytics:configure` (the control plane claims it for
|
|
710
|
+
organization owners and admins alone) and the host's grant lists it too;
|
|
711
|
+
otherwise the dashboard shows the same forms but hands over the code the host
|
|
712
|
+
runs itself. A third permission, `analytics:public`, lets the organization
|
|
713
|
+
publish a read-only public link of the granted sites — the overview without
|
|
714
|
+
revenue, conversions, goals, campaigns, or custom events, and the live visitors
|
|
715
|
+
without identities — and, like the configure scope, it works only when the
|
|
716
|
+
host's grant lists it. `listSites` accepts no site arguments and omits owner,
|
|
717
|
+
domain, and network configuration. Stable manifest names, capabilities, limits,
|
|
718
|
+
and authorization error codes are exported for connector clients. The host must
|
|
719
|
+
already have Convex authentication and a local authoritative grant. Follow
|
|
558
720
|
[`docs/federation-setup.md`](docs/federation-setup.md) for the executable setup
|
|
559
721
|
and verification sequence; use [`docs/federation.md`](docs/federation.md) as the
|
|
560
722
|
protocol reference. The production control plane is available at
|
|
@@ -595,22 +757,42 @@ that introduced the rollups.
|
|
|
595
757
|
|
|
596
758
|
Overview responses label visitor values as either HLL-estimated unique visitors
|
|
597
759
|
or legacy `bucketOccurrences`; legacy aggregate rows are never presented as
|
|
598
|
-
exact uniques. Anonymous
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
760
|
+
exact uniques. Anonymous visitors carry a persistent pseudonymous id from the
|
|
761
|
+
tracker's `localStorage`, so the same person counts once across tabs, days, and
|
|
762
|
+
returns. When storage is blocked or an old cached tracker reports no id,
|
|
763
|
+
sessions from the same device and network within one UTC day still fold into one
|
|
764
|
+
visitor through a salted daily key; runtimes without a client IP fall back to
|
|
765
|
+
one visitor per session, which `/rastro/health` reports as
|
|
766
|
+
`uniqueVisitors: "session"` instead of `"dailyHash"`. Breakdown metadata says
|
|
767
|
+
what each ranking counts: pages by pageviews and sources, campaigns, countries,
|
|
768
|
+
devices, and browsers by sessions once every bucket in the range carries those
|
|
769
|
+
counters (recorded since alpha.11), and raw event volume for ranges that reach
|
|
770
|
+
back before them; events always rank by volume. Totals carry the engagement
|
|
771
|
+
counters recorded since the same stamp — summed session duration, sessions that
|
|
772
|
+
reached a second pageview, and first-ever sessions of a visitor — and
|
|
773
|
+
`metadata.engagement` reports `counted` when the whole range has them, so bounce
|
|
774
|
+
rate, average duration, pages per session, and the share of new visitors derive
|
|
775
|
+
from one response; `unavailable` means the range reaches back before the
|
|
776
|
+
counters and those four stay blank. Feature status is explicit: outbound-link
|
|
777
|
+
event volume is supported (with `topOutbound` naming the destinations), bot
|
|
778
|
+
detection rejects self-declared agents by `User-Agent`, and session replay and
|
|
779
|
+
error insights are currently unsupported.
|
|
607
780
|
|
|
608
781
|
All report ranges use integer Unix milliseconds with inclusive `from` and `to`
|
|
609
|
-
boundaries
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
782
|
+
boundaries, and range metadata reports `boundary: "inclusive"`. Hourly overview
|
|
783
|
+
queries must contain complete UTC hours. A daily overview query must contain
|
|
784
|
+
either complete UTC days or, when every requested site has the same timezone,
|
|
785
|
+
complete calendar days in that zone: a site with a timezone gets one aggregate
|
|
786
|
+
bucket per local calendar day at ingestion, next to the UTC hour and day
|
|
787
|
+
buckets, so "yesterday" in São Paulo ends at midnight in São Paulo. Range
|
|
788
|
+
metadata then reports `timezone` as the zone the buckets follow, `"UTC"`
|
|
789
|
+
otherwise. Local-day coverage starts at the site's first ingest after the
|
|
790
|
+
upgrade (`analyticsCoverage.localDaySince`) and restarts when its timezone
|
|
791
|
+
changes; a range reaching further back is reported as partial coverage, and
|
|
792
|
+
`sites.localDays` (federated `listSites.localDays`) tells a dashboard the zone
|
|
793
|
+
and the stamp up front. A zone that never leaves UTC keeps no extra buckets.
|
|
794
|
+
Funnel and affiliate reports must contain complete UTC days. Those
|
|
795
|
+
aggregate-backed reports fail with `REPORT_INCOMPLETE` instead of silently
|
|
614
796
|
including the rest of a partial bucket. Goal reports retain exact millisecond
|
|
615
797
|
filtering while bounded goal-completion rows are available.
|
|
616
798
|
|
|
@@ -644,12 +826,22 @@ boundaries use different bucket alignment.
|
|
|
644
826
|
- Primary aggregate shards: 256 hourly and 128 daily, derived from stable
|
|
645
827
|
session identity
|
|
646
828
|
- Funnel and affiliate aggregate shards: 16
|
|
829
|
+
- Report rollups (goal, funnel, affiliate, route, and Web Vitals daily rows) are
|
|
830
|
+
never written by the ingestion transaction: a batch appends one document to
|
|
831
|
+
the site's rollup queue and one leased fold job per site merges up to 200
|
|
832
|
+
queued batches per run into the rows, so every rollup row has a single writer
|
|
833
|
+
and concurrent ingestion cannot conflict on it. Reports see a batch's rollups
|
|
834
|
+
a few hundred milliseconds after ingestion accepts it; a fold chain that dies
|
|
835
|
+
is restarted by the next batch once its 60-second lease lapses
|
|
647
836
|
- Site ingress budget: 4,096 deterministic per-minute shards, each capped at 300
|
|
648
837
|
events and 512 KiB (1,228,800 admitted events/minute at uniform load)
|
|
649
838
|
- Rate-window storage: one reusable row per active session and site shard
|
|
650
|
-
- Live session timeout:
|
|
651
|
-
|
|
652
|
-
that
|
|
839
|
+
- Live session timeout: a `leave` beacon on `pagehide` retires the visitor about
|
|
840
|
+
ten seconds after they leave and a back/forward-cache return revives them
|
|
841
|
+
immediately; the safety net for a browser that never says goodbye is 90
|
|
842
|
+
seconds since the last event or heartbeat (heartbeats fire every 20 seconds,
|
|
843
|
+
within background-tab throttling), with one bounded five-second expiry sweep
|
|
844
|
+
per site; legacy per-session jobs hand off to that sweep during the cutover
|
|
653
845
|
- Clock alignment: the tracker and the vitals module stamp each batch with
|
|
654
846
|
`sentAt`; when that clock is more than 30 seconds from the server's, every
|
|
655
847
|
event in the batch is re-anchored by the offset before ingestion, so a visitor
|
|
@@ -699,12 +891,13 @@ evidence, not a production SLA.
|
|
|
699
891
|
Two caveats stand deliberately. The heavy row measured a store already holding
|
|
700
892
|
the battery's own multi-million-event history; the 2026-08-22 fresh-store result
|
|
701
893
|
(76.62M/day) remains valid for that state, and neither number overwrites the
|
|
702
|
-
other. The feature profile
|
|
703
|
-
|
|
704
|
-
([#52](https://github.com/amageweb/amage-rastro/issues/52))
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
894
|
+
other. The feature profile is not certified: on alpha.6, affiliate daily-rollup
|
|
895
|
+
contention lost ~4–5% of batches under concurrent load
|
|
896
|
+
([#52](https://github.com/amageweb/amage-rastro/issues/52)). Since alpha.11
|
|
897
|
+
ingestion no longer writes rollup rows at all (see the operational limits), but
|
|
898
|
+
no feature-profile capacity is claimed until a new battery certifies it. GeoIP
|
|
899
|
+
with a provider enabled, trusted server-side conversions, and vitals-bearing
|
|
900
|
+
traffic remain separate pending certifications.
|
|
708
901
|
|
|
709
902
|
The full evidence is recorded in
|
|
710
903
|
[`docs/benchmarks/2026-08-30-alpha6-recertification.md`](docs/benchmarks/2026-08-30-alpha6-recertification.md).
|
|
@@ -766,9 +959,11 @@ site-shard counters, session/live state, visitor attribution, and exact funnel
|
|
|
766
959
|
progress before writing. Accepted non-heartbeat telemetry from one request is
|
|
767
960
|
grouped into one immutable raw document per session and event-minute, without
|
|
768
961
|
duplicating session geo or user-agent context. A 50-event same-session,
|
|
769
|
-
same-minute baseline stays within 10 document writes
|
|
770
|
-
bucket is written at most once per
|
|
771
|
-
explicit `(other)` bucket instead of
|
|
962
|
+
same-minute baseline stays within 10 document writes plus the rollup queue entry
|
|
963
|
+
and its fold lease. Each touched aggregate bucket is written at most once per
|
|
964
|
+
batch, and dimension overflow folds into an explicit `(other)` bucket instead of
|
|
965
|
+
relabeling old counts. Goal, funnel, affiliate, route, and Web Vitals rollup
|
|
966
|
+
rows are written only by the site's fold job, never by ingestion.
|
|
772
967
|
Click/outbound/browser-conversion telemetry skips goal and funnel work that
|
|
773
968
|
cannot match those event types.
|
|
774
969
|
|
|
@@ -788,10 +983,13 @@ deployments need no legacy-event migration.
|
|
|
788
983
|
Amage Rastro's default tracker:
|
|
789
984
|
|
|
790
985
|
- sets no cookies;
|
|
791
|
-
- stores a pseudonymous session ID and referral slug in `sessionStorage
|
|
986
|
+
- stores a pseudonymous session ID and referral slug in `sessionStorage`, and a
|
|
987
|
+
persistent pseudonymous visitor ID in `localStorage` — scoped to the site's
|
|
988
|
+
origin, never shared across sites, gone when the visitor clears site data;
|
|
792
989
|
- never persists raw visitor IP addresses in component tables;
|
|
793
|
-
- counts
|
|
794
|
-
|
|
990
|
+
- counts unique visitors by that persistent pseudonymous ID; when storage is
|
|
991
|
+
blocked or the id is absent, a salted hash of the request IP and `User-Agent`
|
|
992
|
+
that rotates every UTC day takes over, derived in the HTTP action from a
|
|
795
993
|
per-deployment secret; a host-supplied `visitorId` always takes precedence,
|
|
796
994
|
and the raw IP never reaches a mutation or a table;
|
|
797
995
|
- never persists the raw `User-Agent`, only a normalized browser, operating
|
|
@@ -810,9 +1008,10 @@ The optional Web Vitals module inherits this contract: it reports at most five
|
|
|
810
1008
|
timing numbers per page load (milliseconds, or CLS scaled by 1000) against the
|
|
811
1009
|
landing path, and nothing else.
|
|
812
1010
|
|
|
813
|
-
Cookieless does not automatically mean consent-exempt in every jurisdiction
|
|
814
|
-
|
|
815
|
-
policy, and
|
|
1011
|
+
Cookieless does not automatically mean consent-exempt in every jurisdiction, and
|
|
1012
|
+
the persistent visitor ID is an identifier under most privacy laws. Review your
|
|
1013
|
+
collection purpose, custom properties, identity strategy, retention policy, and
|
|
1014
|
+
local law.
|
|
816
1015
|
|
|
817
1016
|
The example Live Atlas uses MapLibre with OpenFreeMap Positron. Map tiles are
|
|
818
1017
|
requested by the dashboard viewer; Amage Rastro visitor/session data is rendered
|