@sagepilot-ai/web-events-addon 0.0.0-stage → 0.2.0

package/README.md

@@ -1,3 +1,110 @@
-# Temporary Holding Version
+# Sagepilot web events add-on
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Local implementation preview. One installation captures supported browser observations for a workspace SDK installation. It does not initialize chat or require a channel. No package publication, hosted deployment or production collection is included.
+
+The add-on bundles the shared installation client. Install `@sagepilot-ai/web-events-addon` for npm usage, or load the standalone browser bundle.
+
+## Usage
+
+```ts
+import { createSagepilotWebEvents } from '@sagepilot-ai/web-events-addon';
+
+const events = createSagepilotWebEvents({
+  key: 'workspace_uuid:installation_uuid',
+  collectionEnabled: false
+});
+await events.start();
+
+// Call only when your website's collection permission allows it.
+await events.setCollectionEnabled(true);
+```
+
+The shared installation client obtains an anonymous scoped session from the gateway's `/platform-api/v1/sdk/sessions`. SDK observations go directly to the server-configured Event Handler `/events/v1/sdk/observations` endpoint. `apiHost` optionally accepts an HTTPS gateway origin, or loopback HTTP for local tests. Do not pass a path or untrusted host. Installation keys identify a binding; they do not authenticate customer identity.
+
+ESM, CommonJS and type builds are produced alongside the standalone `dist/sagepilot-web-events.global.js`. A host or GTM tag can load that file and call `SagepilotWebEvents.createSagepilotWebEvents(...)`.
+
+`createSagepilotWebEvents` reuses the existing instance for the same installation; the first options win until `destroy()`. `start()` is idempotent. `flush()` attempts a bounded drain, `diagnostics()` exposes counters, and `destroy()` detaches hooks without purging an unacknowledged durable backlog. `setCollectionEnabled(false)` immediately fences capture, aborts SDK requests and purges unsent records. Explicitly disable before destroying if revocation/purge is intended.
+
+Both local permission and the server's `collection_enabled` must allow capture. Server settings refresh at least once per minute while started, including when collection is off so popup publication still propagates. `refreshConfig()` applies changes immediately. Refresh failures suspend capture until a successful refresh. Ingress tokens are short lived; Event Handler performs no database lookup and FJE checks current installation policy before processing. A queue admission receipt is not event normalization or journey execution.
+
+## Captured evidence
+
+| Source | Supported evidence |
+|---|---|
+| Navigation | Initial page and History push/replace/pop/hash transitions, with stripped/fingerprinted routes |
+| Interaction | Bounded structural descriptors, allowlisted action tokens and typed facts from the clicked element's ordinary or semantic product card |
+| Client-only cart | Settled same-product quantity increase in a positively identified cart region, with explicit DOM evidence provenance |
+| Fetch | Method, operation-route fingerprints, timing, HTTP transport outcome; supported app-provided JSON string request bodies |
+| App-consumed fetch bodies | JSON projection when the application calls native `Response.json()` or JSON `Response.text()`; no extra clone or body read |
+| XHR | Standard requests, existing text/JSON responses, transport error/abort/timeout outcomes |
+
+Network evidence uses stable request IDs and separate header/body phases. FJE groups discovery samples, validates an AI proposal and processes published deterministic rules. A click can prove `add_to_cart_clicked`; successful cart/purchase events require explicit positive business response evidence. Ambiguous observations stay unresolved.
+
+Arbitrary element text/labels, IDs/classes, arbitrary merchant form values, headers, cookies, query strings, fragments, screenshots and raw body strings never enter the journal. Button labels contribute only a fixed vocabulary such as `add`, `cart`, `checkout` or `wishlist`. Existing standard `data-product-*`/schema.org markup remains supported, but it is not required: an ordinary card with one image label, a matching visible title and unambiguous displayed price can provide an origin-scoped opaque `product_ref` and price. DOM labels are screened before hashing and never exported. That reference is not a merchant catalog ID or an anonymization guarantee. Currency is omitted unless an ISO currency code is actually present; `$` alone does not mean USD. No framework state is read. Ordinary networking/navigation routes remain fingerprinted; the product snapshot below separately permits validated public product/image/source URLs.
+
+### Product common fields and source metadata
+
+`data.product.snapshot` separates common fields (`product_id`, `variant_id`, `sku`, `name`, `url`, `image_url`, `price`, `currency`) from `metadata`. Metadata contains resolution, original identifier names/string values, selected variant identifiers, safe product scalar/scalar-array attributes, provenance and a `limited` flag. For example, source `id:0` becomes `product_id:'0'` plus `metadata.identifiers.id:'0'`; source `catalog_id:'CAT-1'` remains under its original metadata key and is not guessed to be a product ID. Custom product attributes such as `fabric_gsm`, sizes, style and shipping flags remain JSON metadata. Missing common fields stay absent.
+
+Confirmed product/catalog containers, Product/ProductGroup JSON-LD, bounded inert product JSON and literal public Next hydration records are supported automatically. A narrow Nuxt indexed-record projection reads only independently corroborated product fields; it does not restore framework state or run revivers. No `eval`, application store access or extra merchant request is used.
+
+Existing native commerce identifiers and an explicit public `Product ID:` label can corroborate source identity; arbitrary merchant form values are never exported. Actual checked properties select variants. An initial HTML selection, URL query or SKU alone cannot invent a product/variant ID. Public collapsed product specifications remain usable only with independent source/URL agreement, while private forms and excluded regions remain unreadable. Recommendations and malformed form owners cannot borrow the current page's product.
+
+Customer/order/account sources, private-key subtrees, credentials, contact-like values, markup, personalization, reviews and arbitrary nested objects are excluded before upload. Names and descriptions are permitted only as screened product-source text. Catalog selection requires unique exact SKU/variant/product identifiers, a declared canonical URL, or a unique image with independent product corroboration; title/price alone cannot select a record. Competing aliases, shared assets, truncated source coverage and price/currency conflicts remain unresolved or ambiguous. Cached conflict rejection persists until the source changes. A declared source SKU may match a complete token in an existing image filename; the filename cannot invent a missing SKU. Product/source URLs permit one validated variant selector, without credentials or fragments; ordinary route evidence still strips query values. Unresolved DOM provenance contains only the site origin.
+
+Explicit decimal amount/currency objects and unambiguous schema Offers can supply common money fields. Native minor-unit prices are not guessed into major currency units. Missing or contradictory price/currency stays absent. Source identifier namespaces, including source-declared group/catalog IDs and differing schema SKU roles, remain metadata instead of silently becoming canonical product IDs.
+
+The index retains at most 128 sanitized product records for 15 minutes, at most 32 identifier/attribute fields per map and 16 scalar array values; each snapshot is at most 8 KiB within the existing 32 KiB observation cap. Existing app-consumed catalog text is capped at 256 KiB. Numeric identifier lexemes are preserved from original text when the browser supports JSON parse source context. Already-parsed unsafe numbers are omitted rather than rounded or stringified as if exact. Arbitrary catalog schemas and values beyond these limits are not promised complete coverage.
+
+When local collection permission is already granted at `start()`, startup hooks retain only up to 32 weak XHR references, safe public catalog URLs and request generations for at most 10 seconds. They read no response state, headers or body before the server grants collection. Activation then inspects still-live original completed/in-flight text responses; reused requests, collected references, revocation and failed bootstrap fail safely. The SDK issues no extra catalog requests and never clones or proactively reads fetch bodies. Requests completed before installation, collected startup XHRs and unsafe already-parsed identifiers can remain unavailable. Only sanitized public catalog facts survive identity-only transitions; consent/policy withdrawal and destroy clear them.
+
+For a client-only cart with no network response, the SDK snapshots a positively identified Cart/Checkout region before the existing click handler runs. A bounded two-second observer can emit `cart_change` only for the same product and region with a settled positive integer quantity increase. FJE preserves `dom_cart_quantity_increase` provenance on the normalized event; it proves observed cart UI state, not backend commitment. An unmounted/hidden or unreadable cart baseline is unknown, so a first click may produce intent only. An explicitly rendered empty cart can establish zero. Overlapping same-product actions, network-backed mutations, busy/loading regions, ambiguous variants/prices, late timers and consent/identity changes suppress this DOM proof. Purchase validation is unchanged.
+
+The JSON projector rejects sensitive-key subtrees and accessors and keeps bounded typed scalars. Strings are restricted to short ASCII codes/references; numeric product/variant/SKU references preserve their exact digits. Ordinary free text, common contact formats and long opaque strings are omitted. Original field paths and array indices are retained for permitted fields. Heuristic filtering cannot guarantee that every permitted compact code or number is non-sensitive. Do not enable collection for confidential endpoints or without appropriate permission.
+
+Projection limits: 64 Ki-code-unit input text, depth 8, 256 visited nodes, 48 scalar fields, 32 keys/object and 10 array items. Arbitrary binary/streaming/form bodies, requests before startup, pre-bound/custom networking, app-created Response clones, unconsumed response bodies, closed shadow roots/cross-origin frames and native app networking are unsupported. The SDK never inspects application stores or private framework state. Hooks installed by other libraries after ours are preserved at teardown.
+
+## Ownership and local delivery
+
+The existing `sagepilot_anonymous_id` browser ID is reused so the chat SDK and events recognize the same browser; workspace/installation namespaces remain server-controlled. Browser storage denial falls back to an explicitly reported runtime ID. Sessions are persisted separately in a scoped storage key; tokens never enter observation records.
+
+For cookie mode, connect a storefront route for `POST /sagepilot/visitor/{workspace_id}/{installation_id}`. Proxy it to `POST /platform-api/v1/sdk/visitors/{workspace_id}/{installation_id}` on the SDK gateway. Pass the browser's `Origin`, `Sec-Fetch-Site`, JSON body and only the `sagepilot_visitor` cookie. Relay the upstream response including `Set-Cookie` and `Cache-Control: no-store`. Keep merchant cookies and credentials on the storefront.
+
+In Settings → SDK, select a web installation, turn on **First-party visitor cookie**, and save. The installed SDK reads this setting on startup and refresh. With analytics permission, it can recover the same browser ID and completed popup from the signed storefront cookie. It falls back to the existing browser ID if the route is unavailable. Switching the feature off or calling `setCollectionEnabled(false)` stops using the cookie and clears it when the SDK knows it was set. The cookie is scoped to this browser and installation. A popup completed with email verification can link separate browsers to the same customer.
+
+Records retain event ID, occurrence time, workspace/registered installation, original signed session owner and local identity epoch. A separate credential cache retains at most four owner generations across verification/reload so anonymous backlog stays anonymous. Records whose authority is unavailable or explicitly revoked are dropped with diagnostics; they never block newer owners or get relabeled.
+
+## Popup and verified identity
+
+Settings → SDK publishes a text-only popup template independently from mapping versions. The installed runtime renders the published revision inside a shadow DOM, with exact page paths, delay/scroll conditions, frequency limits, required fields and keyboard dismissal/focus management. Submitting the form sets the existing contact endpoint's `marketing_consent` field to `true`. The renderer and dashboard preview have no consent checkbox or label. Drafts never render. Disabled publication removes an open popup after refresh. Native app registrations do not render website popups.
+
+Email verification happens through the explicit contact endpoint. Contact-only forms save self-reported details and keep anonymous authority. Only a completed server challenge creates an identified session; phone numbers remain unverified. Form values and verification codes never enter ordinary observations. Shown/submitted/dismissed lifecycle observations obey collection permission.
+
+The popup handles submission/verification itself. Host-owned forms can use `submitContact({submission_id,popup_revision,name?,email?,phone?,marketing_consent})` and `verifyContact(challengeId,code)`. `logout()` immediately pauses capture and cancels pending uploads before revoking the API grant and establishing a fresh anonymous owner. Queued events retain their original ownership. The collector discards any records whose credentials the logout revoked. Existing signed ingress credentials expire within 15 minutes.
+
+- At most 128 pending route/click/network emissions per collector, and 128 capture writes / 512 KiB of serialized handoff evidence.
+- The journal holds up to 8,192 records or 8 MiB, whichever comes first. It evicts oldest records at capacity. The 24-hour local expiry fits the server's existing offline renewal window.
+- At most 32 KiB per occurrence and 50 records / 96 KiB per browser batch.
+- One active drain, up to four batches per drain, two-second timer, exponential retry/backoff with jitter capped at roughly one minute.
+- Upload outages leave capture active while local consent and the last confirmed server session are valid. An expired session or rejected authorization pauses capture until policy is confirmed again.
+- IndexedDB is preferred. When it is unavailable, the SDK saves up to 256 sanitized records or 512 KiB in `localStorage` and migrates them when IndexedDB becomes available on a subsequent page load. Records beyond the fallback budget remain in memory.
+- A fresh page waits for the site's consent decision before capture or delivery. Explicit consent withdrawal purges the backlog, including records in an IndexedDB journal that recovers later when the browser can save the purge marker.
+- Explicit terminal per-record receipts remove records; missing/retryable/invalidly formed receipts never acknowledge a whole batch.
+
+`quota_evicted`, `expired_records`, `memory_only_records`, `rejected_payloads`, `collector_dropped` and `storage_failures` report delivery gaps or exposure. A hard page/process termination can lose records before their storage writes complete. Browsers can also evict origin storage.
+
+## Validation
+
+```sh
+npm run typecheck -w @sagepilot-ai/web-events-addon
+npm run build -w @sagepilot-ai/web-events-addon
+node scripts/check-web-events-addon.cjs
+SAGE_EVENTS_BROWSER=safari node scripts/check-web-events-addon.cjs
+SAGE_EVENTS_BROWSER=firefox node scripts/check-web-events-addon.cjs
+node scripts/check-product-index.cjs
+node scripts/check-dom-cart.cjs
+```
+
+The browser test exercises the same built IIFE and localhost contract in Chrome, macOS Safari and Firefox. It checks capture during outages, original event IDs across page navigation, persistent fallback and migration, consent and expiry, and popup behavior. Safari uses Apple's `safaridriver` with **Allow remote automation** enabled in Developer Settings. Firefox uses an installed browser and `geckodriver` on PATH. Set `SAGE_EVENTS_FIREFOX` or `SAGE_EVENTS_GECKODRIVER` to override those paths. Chrome uses an isolated profile and accepts a binary override through `SAGE_EVENTS_CHROME`.
+
+Run `npm run test:sdk-events` in `sage-web-push-test-app` for the cross-service test. It starts the existing website at `/sdk-events`, actual API/Go ingress and FJE local file processing, then validates a Claude Sonnet 5 proposal. The journey and outbox use local dry runs. iOS Safari and production database/provider integration require separate validation.