@proveanything/smartlinks 2.1.13 → 2.1.14

package/docs/connectors.md

@@ -1,7 +1,7 @@
 # Integration connectors
 
 A **connector** is an app that knows one external provider (a PIM, a content network, a shop
-platform…). It is installed on a collection like any app, versioned and released like any app, and
+platform, a loyalty or CRM system…). It is installed on a collection like any app, versioned and released like any app, and
 it adds a manifest `connector` block plus a handful of [server functions](server-functions.md) the
 integration engine and the integrations console call by convention.
 
@@ -83,7 +83,7 @@ Types: `AppConnectorDeclaration` and friends in `@proveanything/smartlinks` (`sr
 | `provider` | Stable provider key (`syndigo`, `oneworldsync`). |
 | `title` | Provider display name; defaults to the app name. |
 | `directions` | `inbound` (provider → SmartLinks), `outbound` (SmartLinks → provider), or both. |
-| `entities` | SmartLinks entities it maps, e.g. `["product"]`. |
+| `entities` | SmartLinks entities it maps: `product`, and for people-shaped providers `interaction`, `contact`, `event` (see [Loyalty and CRM connectors](#loyalty-and-crm-connectors)). |
 | `connection.fields[]` | The console's **Connect** form. A field with `secret` is stored as this app's secret of that name on the collection (write-only; never returned to the browser). Other fields are saved in the installation's connection settings. |
 | `settings` | JSON Schema for installation settings; the console renders it as a form. |
 | `presets[]` | Ready-made flows ("Products → provider"). A preset's `flow.config.fieldMappings` is a **suggested** mapping: the connector's knowledge of the provider, which the customer accepts or edits in core. |
@@ -91,7 +91,7 @@ Types: `AppConnectorDeclaration` and friends in `@proveanything/smartlinks` (`sr
 | `panels[]` | Provider-specific screens the console mounts (an API explorer, a status page). |
 | `aiGuide` | A guide the mapping assistant reads: the provider's quirks, required fields, good defaults. |
 | `recordId` | Path of the provider's own id in a fetched record (e.g. `gtin`). Inbound records are landed and matched by it, so a re-import updates the same product instead of creating another. Default `id`; a flow can override it (`config.upsert.externalIdPath`). |
-| `limits` | `{ requestsPerSecond }` toward the provider, enforced across every SmartLinks worker. |
+| `limits` | `{ requestsPerSecond }` toward the provider, enforced across every SmartLinks worker; `{ maxEvents, maxAttributes }` when the provider caps registered events — a flow's `eventMapping` is checked against them when saved. |
 
 ---
 
@@ -111,7 +111,9 @@ declare the provider's hosts (`network:<host>`) and the secrets it reads (`secre
 | `fetch` | Every inbound run | `ConnectorFetchInput` `{ installation, cursor, limit }` | `ConnectorRecords` `{ records, nextCursor }`: one page per call |
 | `shape` | Every inbound run (and sample dry runs) | `ConnectorShapeInput` `{ installation, records }` | `ConnectorShapeResult` `{ documents, skipped }`: one provider-neutral document per record, same order; `null` skips one |
 | `serialize` | Every outbound run | `ConnectorSerializeInput` `{ installation, documents, entities, targetOptions, eventType? }`: documents core mapped to `describeTarget` paths; `entities` = the SmartLinks record each came from (id, name); `targetOptions` = per-field options the mapping carries (e.g. locale, unit); a deletion arrives as `{ id, _deleted: true }` | `ConnectorSerializeResult` `{ payloads, skipped }`: one provider payload per document, same order; `null` skips one |
-| `send` | Every outbound run | `ConnectorSendInput` `{ installation, payloads }` | `ConnectorSendResult` `{ results: [{ index, ok, status, error, externalId }] }` |
+| `send` | Every outbound run | `ConnectorSendInput` `{ installation, payloads, eventType? }` | `ConnectorSendResult` `{ results: [{ index, ok, status, error, externalId, retryable?, retryAfterSeconds? }] }` — `retryable: false` for a provider validation error (dead at once), `true` / a 5xx or 429 status for anything worth retrying |
+| `describeEvents` | Mapper opens; a flow with an `eventMapping` is saved | `{ installation }` | `ConnectorEventsSchema` `{ events: [{ name, title, attributes: [{ name, type, required }] }], limits: { maxEvents, maxAttributes } }`: the event names the provider has registered for this customer |
+| `lookup` | An entitlement check (`docs/entitlements.md`) | `ConnectorLookupInput` `{ installation, identity: { externalId, email, phone } \| { externalId, emailHash, phoneHash }, want }` | `ConnectorLookupResult` `{ found, memberId, tier, points, status, attributes, asOf }`. Throw for an outage — never `found: false` |
 
 **How a run flows:**
 - **Inbound:** `fetch` (paged; every record is landed as received) → `shape` → core maps the documents
@@ -182,6 +184,41 @@ export async function shape(_ctx: any, event: any): Promise<ConnectorShapeResult
 
 ---
 
+## Loyalty and CRM connectors
+
+A provider that is about **people** rather than products uses the same contract with three more
+pieces; nothing provider-specific lives in core. Full contract: `loyalty-connector-authoring.md` in the
+platform's design docs; the shapes are in [integrations.md](integrations.md#event-sourced-flows-and-the-envelope).
+
+| Flow | What core does | What the connector does |
+|---|---|---|
+| **Earn** (us → them) | an outbound flow with `sourceEntity: 'interaction' \| 'contact' \| 'event'` sends one **envelope** per event, through the flow's identity policy and `eventMapping`, signed, with retries and a run log | `describeEvents` (what the provider has registered), `serialize` (envelope → the provider's event payload), `send` if the generic HTTP destination can't make the call |
+| **Profile** (them → us) | `ctx.sl.members.apply(document)` matches the contact and writes member id, tier, points, tags, external ids; or an inbound flow with `targetEntity: 'contact'` | an **HTTP-trigger function** receiving the provider's member webhook (verify their signature, then apply); or `fetch` + `shape` for a poll |
+| **Gate / display** (on demand) | `entitlements.check` — claims → cached `lookup` → the contact's own loyalty fields → fallback; Valkey cache keyed on an identity hash, brand-set TTL | `lookup`: one identity → tier / points / status |
+
+```ts
+// A member webhook: the provider tells us about a person; we land it on the contact.
+export default async function memberWebhook(event: any, ctx: any) {
+  verifyProviderSignature(event, await ctx.secrets.get('webhookSecret'))
+  const m = event.body
+  return ctx.sl.members.apply({
+    identity: { externalId: m.memberId, email: m.email },
+    contact:  { firstName: m.firstName, lastName: m.lastName },        // fills empty fields only
+    loyalty:  { memberId: m.memberId, tier: m.tier, points: m.points, status: m.status, asOf: m.updatedAt },
+    tags:     { add: [`tier:${m.tier}`] },
+  }, { provider: 'acme-loyalty' })                                     // → { contactId, op: 'created' | 'updated' | 'unchanged' }
+}
+```
+
+Declare it `{ "trigger": { "type": "http" }, "capabilities": ["sl:members:write", "secrets:webhookSecret"] }`;
+`sl:members:write` is approved by the admin on install. The envelope's `identity.externalIds[<provider>]`
+is the member id this wrote back, so **Earn** can address the member by the provider's own id.
+
+A connector should ship an `eventMapping` **preset** with the provider's standard event names and an
+`aiGuide` paragraph, so the console's assistant proposes a customer's mapping in the provider's terms.
+
+---
+
 ## Where configuration lives
 
 | What | Where |
@@ -220,6 +257,7 @@ await secrets.put(collectionId, 'provider-key', value, { appId })              /
 await integrations.saveConnectorInstallation(collectionId, appId, { connection: { environment: 'production' } })
 const test = await integrations.runConnectorStep(collectionId, appId, 'testConnection')
 const target = await integrations.runConnectorStep(collectionId, appId, 'describeTarget')
+const events = await integrations.runConnectorStep(collectionId, appId, 'describeEvents')   // loyalty / engagement providers
 const { records } = await integrations.runConnectorStep(collectionId, appId, 'sample', { limit: 5 })
 const shaped = await integrations.runConnectorStep(collectionId, appId, 'shape', { records })      // dry run
 const wire = await integrations.runConnectorStep(collectionId, appId, 'serialize', { documents })  // dry run

package/docs/entitlements.md

@@ -0,0 +1,90 @@
+# Entitlements: "can this person do this?" without a mirror of their member base
+
+A brand with twenty million members wants a game only Gold members can play. SmartLinks never copies
+the member base to find out. A **gate** says what it needs — a predicate such as `tier>=gold` — and the
+platform asks the collection's configured **sources**, in order, stopping at the first that answers.
+The brand's system stays the record of truth; every answer says where it came from (`basis`) and how
+old it is (`asOf`).
+
+| Source | Data kept here | When it fits |
+|---|---|---|
+| **claims** | none | the brand's SSO or app issues a short-lived signed token carrying the tier; the caller presents it |
+| **lookup** | a hashed key → a small answer, for minutes | the loyalty connector's `lookup` step answers for one person; cached in Valkey under an HMAC of the identity for the TTL the brand sets (default 15 min; "not a member" 5 min); one in-flight lookup per key; the connector's own rate limit applies |
+| **contact** | what the contact already holds | `customFields.loyalty` / `loyaltyTier` written by the connector's member webhook (`ctx.sl.members.apply`) |
+| **fallback** | — | when nothing answered: `deny` (default), `allow`, or `degrade` (allow, flagged `degraded: true`) |
+
+Every decision is one structured log line — gate, predicate, allowed, basis, age, an identity **hash**
+— and never an identifier. A caller sees `attributes` (tier, points, status, member) and never a member
+id.
+
+## Predicates
+
+```
+'tier>=gold'   'tier=silver'   'tier in gold,platinum'   'points>=500'   'member'
+{ tier: { gte: 'gold' } }   { tier: { in: ['gold', 'platinum'] } }   { member: true }
+{ attribute: { path: 'status', eq: 'active' } }
+{ all: [p1, p2] }   { any: [p1, p2] }   { not: p }
+```
+
+Tier comparisons use the collection's ordered `tiers` list (lowest first); a tier the list doesn't know
+never satisfies an ordered comparison. A malformed predicate is a configuration error (`BAD_PREDICATE`),
+never a silent denial.
+
+## Where gates live
+
+**An interaction type** — the public submit route checks it; the caller may present a claim:
+
+```ts
+await SL.interactions.updateInteraction(collectionId, appId, typeId, {
+  permissions: { allowPublicSubmit: true, requires: { entitlement: 'tier>=gold', fallback: 'deny' } },
+})
+// A refused submit: 403 { error: 'FORBIDDEN', reason: 'entitlement_denied' }
+// With a claim: body.entitlementToken = '<jwt>'  or header X-SmartLinks-Entitlement: <jwt>
+```
+
+**A server function** — for the signed-in caller by default:
+
+```ts
+const d = await ctx.sl.entitlements.check('tier>=gold')          // needs sl:entitlements:read
+// → { allowed: true, basis: 'lookup-cache', asOf: '2026-11-08T09:00:00.000Z', attributes: { tier: 'gold', points: 1500, member: true } }
+if (!d.allowed) return new Response(JSON.stringify({ error: 'gold_only' }), { status: 403 })
+// Someone else, or a presented claim:
+await ctx.sl.entitlements.check({ points: { gte: 500 } }, { identity: { email }, token, fallback: 'degrade' })
+```
+
+## Configuring a collection
+
+The `entitlements` setting group (`settings.set(collectionId, 'entitlements', {...})`):
+
+```jsonc
+{
+  "sources": ["claims", "lookup", "contact"],        // tried in this order
+  "fallback": "deny",
+  "tiers": ["bronze", "silver", "gold", "platinum"], // lowest first
+  "claims": {
+    "secretRef": "sec_brand_sso",                     // HS256 shared secret in the secret store …
+    "publicKeyPem": "-----BEGIN PUBLIC KEY-----…",     // … or an RS256 / ES256 public key
+    "issuer": "https://login.brand.example", "audience": "smartlinks",
+    "tierClaim": "tier", "memberIdClaim": "sub", "attributesClaim": "entitlements", "maxAgeSec": 900
+  },
+  "lookup": {
+    "connectorAppId": "acme-loyalty", "provider": "acme-loyalty",
+    "ttlSec": 900, "negativeTtlSec": 300,
+    "identity": "raw",                                // what the connector's lookup receives: 'raw' (email/phone) or 'hashed'
+    "want": ["tier", "points", "status"]
+  }
+}
+```
+
+The connector side (`lookup`) is one step: one identity in, `{ found, memberId, tier, points, status,
+asOf }` out; throw on an outage so the next source is tried — see
+[connectors.md](connectors.md#loyalty-and-crm-connectors).
+
+## What a brand can be told
+
+- We never need a copy of your member base: claims and lookups hold nothing, or a minutes-old answer per
+  active person.
+- Where you want a local copy, it holds no identifiers (the HMAC mirror — on request).
+- Your system remains the record of truth: every answer carries `asOf` and `basis`; TTLs are yours to set.
+- You see every call: the connector's calls are rate-limited, logged, and leave from a fixed egress IP.
+- Nothing leaves without a policy: the envelope's identity policy per flow.

package/docs/integrations.md

@@ -3,16 +3,18 @@
 An **integration flow** is one input/output pipeline between SmartLinks and an external
 system. There are two directions:
 
-- **outbound** — read a SmartLinks entity (v1: a product), transform it with field
-  mappings, and send it to an external endpoint.
-- **inbound** — fetch from an external system and write a SmartLinks entity. *(Executor is
-  outbound-first; inbound lands in a later increment.)*
+- **outbound** — read a SmartLinks **product** that changed, or **one event** (an interaction, a
+  contact change, a tag scan — any spine event) as the [envelope](#event-sourced-flows-and-the-envelope),
+  transform it with field mappings, and send it to an external endpoint (signed, if you want).
+- **inbound** — fetch from an external system and write a SmartLinks **product** or **contact**
+  (a provider's view of a person: member id, tier, points — see [Contact as a target](#contact-as-a-target)).
 
 Flows are triggered three ways, all converging on the same executor:
 
 - **manual** — `integrations.runFlow(...)`, inline (returns a run summary) or enqueued.
-- **event** — an outbound flow subscribed to an event type (e.g. `product.updated`) fires
-  automatically when that entity changes.
+- **event** — an outbound flow subscribed to an event type (e.g. `product.updated`,
+  `interaction.submitted`, `contact.*`, `tag.scanned`, `*`) fires automatically when it happens. See
+  [Event names and patterns](#event-names-and-patterns).
 - **schedule** — a flow carrying a cron/interval `schedule` is run by the scan job. *(next)*
 
 Provider-specific logic (a provider's payload format, its API, its connect form) lives in a
@@ -52,17 +54,21 @@ interface IntegrationFlow {
   direction: 'inbound' | 'outbound'
   name: string
   status: 'draft' | 'active' | 'paused' | 'error'  // only 'active' flows fire on events/schedule
-  eventTypes: string[]                              // e.g. ['product.updated']
+  eventTypes: string[]                              // e.g. ['product.updated'], ['interaction.*'], ['*']
   schedule: string | null                           // cron/interval for scheduled flows
-  sourceEntity: string | null                       // outbound source, v1: 'product'
-  targetEntity: string | null                       // inbound target
+  sourceEntity: 'product' | 'interaction' | 'contact' | 'event' | null   // outbound: what it reads
+  targetEntity: 'product' | 'contact' | null        // inbound: what it writes
   config: {
     connection?: {
       baseUrl?: string
       sendEndpoint?: string                          // outbound: appended to baseUrl
       defaultHeaders?: Record<string, string>
       auth?: { method: 'api_key' | 'bearer' | 'basic' | ..., headerName?: string, credentialRef?: string }
+      timeoutMs?: number
     }
+    signing?: { credentialRef: string, previousCredentialRef?: string }        // X-SmartLinks-Signature (below)
+    identity?: { policy?: 'externalIds' | 'hashed' | 'raw', hashKeyCredentialRef?: string }  // event-sourced flows
+    eventMapping?: EventMapping                       // event-sourced flows: many events → the provider's few
     fieldMappings?: FieldMapping[]
   }
   // ...run watermark/telemetry: lastRunAt, lastRunStatus, lastRunCount, totalSynced
@@ -85,6 +91,150 @@ aborts the whole record.
 
 ---
 
+## Event names and patterns
+
+A flow's `eventTypes` (and a server function's `trigger.eventTypes`) name events exactly or by prefix:
+
+| Subscription | Matches |
+|---|---|
+| `product.updated` | exactly that |
+| `interaction.submitted:<typeId>` | one interaction type's submissions |
+| `interaction.*` | every interaction event |
+| `record.updated:*` | record updates of any type |
+| `*` | everything in the collection — a raw event stream |
+
+Events on the spine today: `product.created / updated / deleted`, `record.<verb>[:<recordType>]`,
+`interaction.submitted[:<typeId>]`, `contact.created / updated / deleted / erased`,
+`tag.scanned[:<eventType>]` (a scan resolves in the redirect, before any page could run a tag),
+`proof.created / transferred`, `attestation.created[:type]`, `booking.*`, `slot.*`.
+
+---
+
+## Event-sourced flows and the envelope
+
+A flow with `sourceEntity: 'interaction' | 'contact' | 'event'` sends **one event per run** — not a
+product. Every such event is wrapped in the same **envelope**, whatever the destination, and only then
+mapped with the flow's field mappings (or handed to a connector's `serialize`):
+
+```jsonc
+{
+  "id": "0b2b…",                               // the event's id — stable across retries; use it as the destination's idempotency / transaction id
+  "type": "interaction.submitted:quiz-1",
+  "entityType": "interaction", "entityId": "0b2b…",
+  "occurredAt": "2026-11-08T10:00:00.000Z",
+  "collectionId": "…", "orgId": "…", "appId": "quiz",
+  "identity": {
+    "contactId": "…", "userId": "…",
+    "externalIds": { "acme-loyalty": "M-9", "crm": "C-1" },     // ids other systems know this person by
+    "emailHash": "…", "phoneHash": "…", "hashAlgorithm": "hmac-sha256"   // policy 'hashed' (default)
+    // "email", "phone": only under policy 'raw'
+  },
+  "identityPolicy": "hashed",
+  "product": { "id": "P1", "name": "Widget", "sku": "W-1", "gtin": "0500…", "variantId": null, "batchId": null },
+  "proof": { "id": "…" },
+  "subject": { "interactionId": "quiz-1", "outcome": "completed", "metadata": { "score": 9 }, "source": "hub" },
+  "event": { "name": "engagement_completed", "attributes": { "kind": "quiz", "score": 9 }, "rule": "quiz" },   // from eventMapping
+  "attributes": {}
+}
+```
+
+Field mappings read envelope paths (`identity.externalIds.acme-loyalty`, `subject.outcome`,
+`event.name`, `product.sku`). With no mappings the envelope itself is sent. A failed event delivery
+keeps its envelope and is retried from it; a product is re-read instead.
+
+```ts
+const flow = await integrations.createFlow(collectionId, {
+  appId: 'core-integrations', direction: 'outbound', name: 'Engagement → loyalty', status: 'active',
+  eventTypes: ['interaction.*', 'tag.scanned:*'],
+  sourceEntity: 'event',
+  config: {
+    connection: { baseUrl: 'https://api.acme.example', sendEndpoint: '/events', auth: { method: 'bearer', credentialRef: ref } },
+    signing: { credentialRef: signingRef },
+    identity: { policy: 'hashed' },
+    eventMapping: {
+      rules: [
+        { id: 'quiz', match: { interactionIds: ['quiz-1'], outcomes: ['completed'] }, event: 'engagement_completed',
+          attributes: { kind: 'quiz', score: '{{ subject.metadata.score }}', sku: { path: 'product.sku' } } },
+        { id: 'scan', match: { types: ['tag.scanned:*'] }, event: 'pack_scanned', attributes: { sku: { path: 'product.sku' } } },
+      ],
+      unmatched: 'skip',
+    },
+    fieldMappings: [
+      { targetPath: 'event_name',     sourcePath: 'event.name', transformType: 'direct' },
+      { targetPath: 'external_id',    sourcePath: 'identity.externalIds.acme-loyalty', transformType: 'direct' },
+      { targetPath: 'transaction_id', sourcePath: 'id', transformType: 'direct' },
+    ],
+  },
+})
+```
+
+### Identity policy
+
+`config.identity.policy` decides what about the person leaves the platform:
+
+| Policy | The envelope's `identity` carries |
+|---|---|
+| `externalIds` | `contactId`, `userId`, `externalIds` — nothing else |
+| `hashed` (default) | the above plus `emailHash` / `phoneHash`: HMAC-SHA256 with a **per-collection** key, or the brand's own key when `identity.hashKeyCredentialRef` names one (so hashes match a mirror the brand built) |
+| `raw` | the above plus `email` and `phone` |
+
+No hash key configured on the server → no hashes, never an unkeyed hash.
+
+### Event mapping
+
+Loyalty and engagement platforms accept a small, pre-registered set of event names. `config.eventMapping`
+folds our many interaction types into those names: rules in order, first match wins; `match` may name
+`types` (exact or patterns), `interactionIds`, `outcomes`, `appIds`; attributes are literals, Liquid
+templates over the envelope, or `{ path, default }`. Unmatched events are skipped (and shown as skipped
+in the run) unless `unmatched: 'passthrough'`. When the flow has a connector with a `describeEvents`
+step, the mapping is **validated when saved** against the provider's registered events and its
+`limits` (`maxEvents`, `maxAttributes`): `createFlow` / `updateFlow` answer `400 EVENT_MAPPING_INVALID`
+with `problems[]`.
+
+### Signed deliveries
+
+With `config.signing.credentialRef`, every HTTP delivery carries:
+
+```
+X-SmartLinks-Signature: t=1730900000,v1=<hex HMAC-SHA256(secret, `${t}.${rawBody}`)>
+X-SmartLinks-Delivery:  <uuid, new per attempt>
+X-SmartLinks-Event:     interaction.submitted:quiz-1
+X-SmartLinks-Run:       <run id>
+Idempotency-Key:        <flowId>:<eventId>:<hash of the body>
+```
+
+The signature covers the exact bytes of the body. A receiver recomputes it, compares in constant time,
+and rejects `t` older than about five minutes. During a rotation `previousCredentialRef` adds a second
+`v1=` so a receiver that knows either secret verifies.
+
+Destinations must be **public hosts**: loopback, private ranges and cloud metadata addresses are
+refused as a permanent failure. (Local development: `INTEGRATION_ALLOW_PRIVATE_DESTINATIONS=1` on the
+server.)
+
+### Contact as a target
+
+An inbound flow with `targetEntity: 'contact'` writes a provider's view of a person. After the
+connector's `shape` (or your field mappings) each record is a **member document**:
+
+```jsonc
+{
+  "identity":    { "externalId": "M-9", "email": "ada@example.com" },      // contactId | externalId | email | phone — at least one
+  "contact":     { "firstName": "Ada", "lastName": "Lovelace" },           // fills EMPTY profile fields only
+  "externalIds": { "acme-crm": "C-1" },                                    // merged
+  "customFields": { "region": "UK" },                                      // merged (never `comms`)
+  "tags":        { "add": ["gold"], "remove": ["silver"] },
+  "loyalty":     { "memberId": "M-9", "tier": "gold", "points": 1500, "status": "active", "asOf": "…", "attributes": { "since": 2024 } }
+}
+```
+
+Matching: `contactId` → `externalIds[<provider>]` → email → phone; a new contact is created once however
+many workers see the same member. Loyalty lands as `customFields.loyalty` plus flat
+`customFields.loyaltyTier / loyaltyPoints / loyaltyMemberId / loyaltyProvider` for segments and comms
+targeting. The same document is what a server function applies with `ctx.sl.members.apply(...)` (a
+provider's webhook, see [connectors.md](connectors.md#loyalty-and-crm-connectors)).
+
+---
+
 ## Secrets (write-only)
 
 The secret store is **write-only from the client**: you can set, rotate, list (refs +