stitchkit 0.93.0 → 0.94.1

package/llms/tracking-server.txt

@@ -0,0 +1,32 @@
+# stitchkit/tracking/server — stitchkit docs slice
+
+> Generated from the `docs/` tree by `bun run gen:llms`. The index of all slices is
+> `llms.txt` in this package.
+
+Contains: API reference: `stitchkit/tracking/server`  (docs/api/reference.md)
+
+Also read:
+- see `llms/tracking.txt` for the Visitor tracking guide
+
+==============================================================================
+# API reference: `stitchkit/tracking/server`  (docs/api/reference.md)
+==============================================================================
+
+## `stitchkit/tracking/server`
+
+The decisions a tracking backend makes, without the storage it makes them
+over. The application reads visits and stored hashes from its database, hands
+them here and writes what comes back; `issueVisitLease` runs the visit
+algorithm over a store the application implements. → ADR 0166
+
+| Export | Kind | Summary |
+|--------|------|---------|
+| `dispositionTrackingBatch` / `DispositionInput` / `DispositionResult` / `DispositionEvent` / `KnownVisit` | function / _type_ | decide a batch: bot → all `excluded-bot`; wrong lineage or owner → `identity-invalid`; stored id → `duplicate` (+ `conflicts` when the hash differs); rest `accepted`; anonymous visits an identified caller may `adopt` |
+| `issueVisitLease` / `IssueVisitLeaseOptions` / `IssuedVisitLease` | function / _type_ | under the lineage lock: continue the named or latest active visit (adopting an anonymous one when `ownership: 'adopting'`), else end what is open and start a new one; a bot gets a lease no store holds |
+| `TrackingVisitStore` / `FindActiveVisitQuery` / `ActiveVisit` / `VisitActor` / `VisitOutboxHealth` / `VisitOwnership` | _type_ | the six-method store the application implements over its database — `withLineageLock`, `findActive`, `touch`, optional `adopt`, `endOpen`, `create` — and the shapes it exchanges |
+| `activeIntervalOf` / `ActiveTimeInterval` / `ActiveIntervalOptions` | function / _type_ | the visible-time interval an event carries, dated by the server clock; `null` for an unshaped or implausible one |
+| `hashTrackingEvent` | function | `sha256(JSON.stringify(parsedEvent))` — the stored payload hash, unsorted on purpose so it matches what applications already hold |
+| `isBotUserAgent` / `DEFAULT_BOT_USER_AGENT_PATTERN` | function / const | crawlers, monitors and `headlesschrome` are not visitors |
+| `createPresenceRegistry` / `PresenceRegistry` / `PresenceEntry` | function / _type_ | who is here now, in this process: `touch` / `snapshot(filter)` / `presentVisitOf` with a 45 s TTL |
+
+---

package/llms/tracking.txt

@@ -0,0 +1,263 @@
+# stitchkit/tracking — stitchkit docs slice
+
+> Generated from the `docs/` tree by `bun run gen:llms`. The index of all slices is
+> `llms.txt` in this package.
+
+Contains: Guide: Visitor tracking  (docs/guide/tracking.md) · API reference: `stitchkit/tracking`  (docs/api/reference.md)
+
+==============================================================================
+# Guide: Visitor tracking  (docs/guide/tracking.md)
+==============================================================================
+
+# Visitor tracking
+
+`stitchkit/tracking` is the browser half of visitor tracking — the outbox, the
+visit lease, the page-leave beacon, visible time, scroll milestones,
+declarative clicks, attribution — and `stitchkit/tracking/server` is the set of
+decisions a tracking backend makes about what arrives. Neither has an event
+vocabulary of its own, a database, or a React component: those are the
+application's, and the boundary is deliberate. → ADR 0166
+
+Both entrypoints are **evolving**.
+
+## What the framework owns, and what you do
+
+| Framework | Application |
+|-----------|-------------|
+| the event envelope, the batch, the dispositions, the visit entry and lease schemas | the event **types**, their metadata, their labels |
+| the tab-shared outbox with reserved sequences and a short flush lease | which storage adapter (IndexedDB in a tab) |
+| the page-leave beacon that arrives, and its queued insurance copy | the session, the router, the React provider |
+| visible time, scroll milestones, `data-track` clicks, UTM first/current touch | the referrer → source map, the attribute names |
+| `dispositionTrackingBatch`, `issueVisitLease`, `activeIntervalOf`, presence | the tables, the transaction, adoption, geo, reports |
+
+## Contract
+
+```ts
+import { createTrackingContract } from 'stitchkit/tracking'
+
+export const tracking = createTrackingContract({
+  scope: 'public',
+  eventTypes: ['PAGE_VIEW', 'PAGE_LEAVE', 'SCROLL_DEPTH', 'SESSION_HEARTBEAT', 'CLICK', 'OUTBOUND_CLICK', 'INTERACTION', 'ITEM_VIEW'],
+  // eventExtras: z.object({ locale: z.string() }),   // fields you carry beside the envelope
+})
+```
+
+Two operations: `bootstrap` (`POST /visit`) issues or renews a visit lease,
+`track` (`POST /events`) receives a batch. `track` declares
+`safelistedBody: true`, because the page-leave event is a string beacon — the
+only body a document that is being unloaded can deliver to another origin —
+and that in turn requires an explicit `cors.origin` allow-list on the server.
+Read [safelisted request bodies](./server.md#safelisted-request-bodies-beacons)
+before deploying: it is one rule, and it is the one that keeps the beacon path
+from being a CSRF hole.
+
+`scope: 'public'` is the usual choice: the landing page is read by anonymous
+visitors, and their path is part of the funnel. Identity comes from the cookie
+on the server side; the client claims nothing about itself.
+
+## Browser
+
+```ts
+import {
+  browserTrackingHost, CONVENTIONAL_TRACKING_EVENT_TYPES, createTrackingClient,
+  createTrackingOutbox, indexedDbOutboxStorage,
+} from 'stitchkit/tracking'
+
+// Only the types the browser writes — a server-side event must not be trackable here.
+type BrowserEvents = Pick<EventMetadataMap, ClientTrackingEventType>
+
+const client = createTrackingClient<BrowserEvents>({
+  host: browserTrackingHost(),
+  buildId: env.NEXT_PUBLIC_BUILD_ID, // a typed env accessor; the client refuses `undefined`
+  builtin: CONVENTIONAL_TRACKING_EVENT_TYPES,       // your names for the events the client emits
+  bootstrap: (entry) => api.tracking.bootstrap(entry),
+  deliver: (batch) => api.tracking.track(batch),   // or a socket first, HTTP as the fallback
+  unloadUrl: urls.tracking.track(),
+  outbox: createTrackingOutbox(indexedDbOutboxStorage('my-app-tracking')),
+  referrerMap: [{ pattern: /t\.me|telegram\.org/, source: 'telegram', medium: 'social' }],
+  isAction: isInteractionAction,
+  onVisit: (visitId) => rum.setVisit(visitId),
+})
+```
+
+`EventMetadataMap` is your `{ [type]: metadata }` map; `track` is typed by it:
+
+```ts
+client.track('ITEM_VIEW', { itemId, itemTitle })
+client.track('SIGN_OUT')
+```
+
+Two knobs match the client to your schema: `batchSize` is the contract's
+`maxEventsPerBatch` (a larger batch is a `400` the outbox can never retire),
+and `decorate` adds your `eventExtras` — a locale, an area, or the identity a
+client that cannot send headers on unload has to carry in the body — to every
+event as it is minted, so the outbox, the beacon and `deliver` all see it:
+
+```ts
+createTrackingClient<BrowserEvents, TrackingEventEnvelope<keyof BrowserEvents> & { locale: string }>({
+  decorate: (event) => ({ ...event, locale: currentLocale() }),
+  // …
+})
+```
+
+A `bootstrap` operation that needs extra fields of its own (a locale, a RUM
+flag) is not the factory's: build the `entry` schema with
+`createTrackingSchemas(…).entry.extend({ … })` and declare that endpoint yourself.
+
+The client owns the mechanics that were the same in every application that
+wrote them: the visit lease and its renewal after a long sleep, a pending queue
+until the visit exists, synchronous event identity from a block of reserved
+sequence numbers, the outbox and its flush lease, one bounded delivery retry,
+the page-leave beacon *and* its queued copy, additive visible time, scroll
+milestones, `data-track` / `data-track-action` clicks, heartbeats. It does not
+own the router or React — tell it about navigation, and wrap it yourself:
+
+```tsx
+const TrackerContext = createContext<TrackFn<BrowserEvents> | null>(null)
+
+export function TrackingProvider({ children }: { children: React.ReactNode }) {
+  const [client] = useState(() => createTrackingClient<BrowserEvents>({ /* as above */ }))
+  const pathname = usePathname()
+  const search = useSearchParams().toString()
+  useEffect(() => client.start(), [client])
+  useEffect(() => client.onNavigate(pathname, search ? `?${search}` : ''), [client, pathname, search])
+  return <TrackerContext.Provider value={client.track}>{children}</TrackerContext.Provider>
+}
+
+export const useTracker = () => useContext(TrackerContext) ?? (() => { throw new Error('outside TrackingProvider') })()
+```
+
+A component that records "opened X" can mount twice on one page; keep that
+memory at document scope with `createOncePerPage`, not in a ref.
+
+### Why these details are not optional
+
+Four of them were found the expensive way, by consuming applications, and each
+is a test in this repository that reddens when the mechanism is put back:
+
+- the beacon body is a **string** — a `Blob` typed `application/json` reports
+  `true` and dies on the preflight it cannot have;
+- the event gets its sequence number **before** it is written — awaiting the
+  outbox inside `pagehide` loses the event with the document;
+- the flush lease is **short and released on leave** — a ten-second lease a
+  dying document kept delayed the next document's first flush by ten seconds;
+- `oncePerPage` lives at document scope — a remounted component has fresh refs.
+
+## Server
+
+The server half is pure. Read from your database, decide, write:
+
+```ts
+import { activeIntervalOf, dispositionTrackingBatch, hashTrackingEvent, issueVisitLease } from 'stitchkit/tracking/server'
+
+track: async ({ input, ipAddress, userAgent }) => {
+  const actorOwnerId = session?.userId ?? null
+  const visits = await db.visit.findMany({ where: { id: { in: input.events.map((e) => e.visitId) } } })
+  const stored = await db.trackingEvent.findMany({ where: { clientEventId: { in: input.events.map((e) => e.eventId) } } })
+  const decided = dispositionTrackingBatch({
+    events: input.events,
+    visits: visits.map((v) => ({ id: v.id, browserStreamId: v.browserStreamId, ownerId: v.userId })),
+    existing: new Map(stored.map((e) => [e.clientEventId, e.clientPayloadHash])),
+    actorOwnerId,
+    userAgent,
+  })
+  const now = new Date()
+  await db.$transaction(async (tx) => {
+    for (const visitId of decided.adoptable) await adoptVisit(tx, visitId, actorOwnerId, now)
+    await tx.trackingEvent.createMany({
+      data: decided.accepted.map((event, ordinal) => ({
+        ...yourColumns(event), clientPayloadHash: hashTrackingEvent(event), serverItemOrdinal: ordinal, createdAt: now,
+      })),
+    })
+    for (const event of decided.accepted) {
+      const interval = activeIntervalOf(event, now)
+      if (interval) await tx.activeTimeInterval.upsert(/* by interval.intervalId */)
+    }
+  })
+  return { accepted: decided.accepted.length, dispositions: decided.dispositions }
+}
+```
+
+`hashTrackingEvent` is `sha256(JSON.stringify(parsedEvent))` with no key
+sorting — exactly what applications already store, so a migration does not
+spend a week reporting conflicts. `decided.conflicts` names duplicates whose
+payload changed; log them.
+
+`issueVisitLease` runs the visit algorithm over a store you implement — six
+methods over your tables, inside a lineage lock you hold (`pg_advisory_xact_lock`
+over the browser stream id is the usual one):
+
+```ts
+const store: TrackingVisitStore<Prisma.TransactionClient> = {
+  withLineageLock: (lineage, fn) => db.$transaction(async (tx) => {
+    await tx.$executeRaw`SELECT pg_advisory_xact_lock(hashtext(${lineage}))`
+    return fn(tx)
+  }),
+  findActive: (tx, q) => tx.visit.findFirst({ where: { /* q.browserStreamId, q.cutoff, ownership */ } }),
+  touch: (tx, id, now, health) => tx.visit.update({ where: { id }, data: { lastActivityAt: now, ...health } }),
+  adopt: (tx, visit, ownerId, now) => /* assign the owner, back-fill events, record the merge */,
+  endOpen: (tx, lineage, now) => tx.visit.updateMany({ where: { browserStreamId: lineage, endedAt: null }, data: { endedAt: now } }),
+  create: (tx, visit, health) => tx.visit.create({ data: { id: visit.id, ...deviceGeoAndSource(visit) , ...health } }),
+}
+
+bootstrap: ({ input, userAgent }) =>
+  issueVisitLease(store, { ownerId: session?.userId ?? null, userAgent }, input, { ownership: 'adopting' })
+```
+
+`ownership` is the one policy two applications disagreed on: `'adopting'`
+continues an anonymous visit and gives it to the caller who signs in — the
+path from the landing page to the first signed-in action becomes one path; `'owned'` never
+continues an anonymous visit, for an application where every visitor already
+has an identity. A bot gets a lease that no store holds, so its events are
+`identity-invalid` and nothing of it reaches a report.
+
+`createPresenceRegistry` is who is here now, in this process: touch it from
+`track`, read it for a live feed. It is honestly empty after a restart.
+
+## Not here
+
+GeoIP, event labels, funnels, reports, the analytics page, Socket.IO push into
+an admin UI, RUM — application code, and the reason two applications' tracking
+looks different even though their mechanics are now one.
+
+==============================================================================
+# API reference: `stitchkit/tracking`  (docs/api/reference.md)
+==============================================================================
+
+## `stitchkit/tracking`
+
+Browser-safe visitor-tracking mechanics, with no event vocabulary of their own
+and no React. The React wrapper (a provider and a hook, ~40 lines) is the
+application's — it owns the router and the session. The server half is
+`stitchkit/tracking/server`. → ADR 0166 — [guide](../guide/tracking.md)
+
+| Export | Kind | Summary |
+|--------|------|---------|
+| `createTrackingClient` | function | the client: visit lease and renewal, pending queue, synchronous identity from reserved sequences, outbox and flush lease, delivery, the string-body page-leave beacon plus a queued copy, visible time, scroll milestones, declarative clicks, heartbeats |
+| `TrackingClient` / `TrackingClientConfig` | _type_ | `track` / `onNavigate` / `start` / `visitId` / `browserStreamId` / `attribution`, and the host, `buildId`, `builtin` names, `bootstrap`, `deliver`, optional `decorate` (your `eventExtras` on every minted event), `unload` / `unloadUrl`, `outbox`, `onUnauthorized`, `onVisit`, `onFailure`, `isAction`, `referrerMap`, `clickAttributes`, `scrollMilestones`, `batchSize`, periods and limits |
+| `TrackFn` / `EventsWithMetadata` / `EventsWithoutMetadata` | _type_ | `track(type)` / `track(type, metadata)` typed by the application's metadata map, and the two halves of that map |
+| `BuiltinTrackingEventTypes` / `CONVENTIONAL_TRACKING_EVENT_TYPES` | _type_ / const | the application's names for the events the client emits itself; the conventional `PAGE_VIEW` … `INTERACTION` set |
+| `browserTrackingHost` / `TrackingHost` / `TrackingPageContext` | function / _type_ | everything the client takes from a browser tab behind one interface — page context, visibility, scroll depth, events, intervals, clocks, storage — so a test or a non-tab host can supply its own |
+| `createTrackingContract` / `TrackingContractConfig` / `TrackingContractEndpoints` | function / _type_ | `bootstrap` (`POST /visit`) and `track` (`POST /events`, `safelistedBody`) from the application's event types, scope and optional extras |
+| `createTrackingSchemas` / `TrackingSchemas` / `TrackingSchemasConfig` | function / _type_ | every schema of one surface — `event`, `request`, `response`, `disposition`, `entry`, `bootstrap` — from `eventTypes`, optional `eventExtras`, `buildIdPattern`, `maxEventsPerBatch` |
+| `TrackingEventEnvelope` / `TrackingEventShape` | _type_ | `eventId`, `visitId`, `browserStreamId`, `browserSequence`, `type`, `page`, `metadata`, `clientTimestamp` |
+| `TrackEventsRequest` / `TrackEventsResponse` / `TrackEventsResponseSchema` | _type_ / schema | a batch (`buildId`, `events`, `utm`) and its answer (`accepted`, `dispositions`) |
+| `TrackingDisposition` / `TrackingDispositionSchema` / `TrackingDispositionStatus` / `TrackingDispositionStatusSchema` | _type_ / schema | `accepted` / `duplicate` / `identity-invalid` / `excluded-bot` — every one terminal |
+| `VisitEntryContext` / `VisitEntryContextShape` / `VisitBootstrapResponse` / `VisitBootstrapResponseSchema` | _type_ / schema | what a browser reports to open a visit (lineage, landing, referrer, UTM, display, screen, `buildId`, outbox health) and the lease it gets back |
+| `TrackingOutboxState` / `TrackingOutboxStateSchema` | _type_ / schema | `available` / `unavailable` |
+| `UtmData` / `UtmDataSchema` / `AttributionData` / `AttributionDataSchema` | _type_ / schema | campaign tags and a touch (UTM, referrer, landing page) |
+| `DEFAULT_BUILD_ID_PATTERN` | const | `dev` or a git SHA |
+| `createTrackingOutbox` / `TrackingOutbox` / `TrackingOutboxOptions` / `TrackingOutboxHealth` | function / _type_ | the tab-shared outbox: `streamId`, `reserveSequences`, `enqueue`, `acquireLease` / `releaseLease` (short, released on leave), `readBatch`, `acknowledge`, `health`; bounded by count and age |
+| `TrackingOutboxStorage` / `TrackingOutboxMeta` / `TrackingOutboxEvents` / `TrackingOutboxRecord` / `TrackingQueuedEvent` | _type_ | one atomic, all-or-nothing `transact` over a key/value store and an event store (`readonly` for pure reads) — the contract an adapter implements |
+| `indexedDbOutboxStorage` / `memoryOutboxStorage` | function | the IndexedDB adapter (one `readwrite` transaction per `transact`) and the in-memory one for tests and degraded mode |
+| `createSequenceReserve` / `SequenceReserve` / `SequenceReserveOptions` | function / _type_ | a block of reserved numbers spent synchronously — `take` / `refill` / `shared` — with a per-tab fallback when the source fails |
+| `deliverTrackingBatch` / `TrackingDeliveryOptions` / `TrackingDeliveryOutcome` | function / _type_ | one batch, one bounded retry on `0` / `5xx`; `delivered` / `failed` / `auth-invalidated` (on `401` / `403`) with an optional `onUnauthorized` recovery |
+| `sendUnloadBeacon` | function | `sendBeacon` with a **string** body — the `text/plain` request a dying document can still complete; `false` without the API |
+| `createVisibleTimeMeter` / `VisibleTimeMeter` / `VisibleInterval` / `VisibleHeartbeat` / `VisibleTimeMeterOptions` | function / _type_ | additive visible time — `checkpoint` / `cut` / `heartbeat`, each interval with its own id; a hidden heartbeat carries no interval |
+| `scrollDepthPercent` / `createScrollMilestones` / `ScrollMilestones` | function / _type_ | depth as a percentage of the scrollable range; milestones fired once each per page |
+| `createOncePerPage` / `OncePerPage` | function / _type_ | one fact per key per page within a window — the guard a twice-mounted component cannot keep itself |
+| `resolveAttribution` / `ResolveAttributionInput` / `ResolvedAttribution` / `AttributionStorage` / `ReferrerRule` | function / _type_ | first-touch once, current-touch on every new UTM; the referrer → source/medium map is the application's, there is no built-in list |
+| `parseUtmFromSearch` / `parseReferrer` | function | the two parsers `resolveAttribution` is built from |
+| `resolveTrackedClick` / `ResolveTrackedClickOptions` / `TrackedClick` / `TrackedClickAttributes` / `ClickTarget` | function / _type_ | `data-track` → click, `data-track-action` → interaction, a link to another origin → outbound; attribute names configurable; `leavesPage` says which path to send on |
+
+---