@delmaredigital/payload-experiments 0.4.0-rc.2 → 0.4.0-rc.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,54 +1,56 @@
1
1
  # @delmaredigital/payload-experiments
2
2
 
3
- > ⚠️ Upgrading to 0.4? Run `payload migrate:create` for four nullable plugin columns (`xp_experiments.page_collection` and its `_xp_experiments_v` counterpart, `xp_phases.placement_docs`, `xp_state.retired_slugs`), plus the Experiment block and `xpGoal` columns if you adopt them, then `payload generate:importmap`. Saves on pages, blocks and Puck collections can now be refused while a test runs on them (`page-busy`, `interacting-tests`); Puck tests whose slot is only on a draft can't start until the page is published; host `metricSources` ids may not start with `click:`; `admin.editorUrl` now receives a required `editor` argument. See the CHANGELOG's BREAKING section.
3
+ A/B testing for Payload CMS and Next.js, from the content you already manage. Test a block on a page, a whole page against another page, or a setting your developer makes testable. Pick a goal, including click goals you name and attach to buttons and links, then compare the Original with your new versions. Results appear in the admin: they count only people who actually saw the content, warn when traffic looks wrong, and check guardrail goals (goals you don't want to hurt). You pick a winner and can make it permanent. Your data stays in your own Postgres database. There is no vendor.
4
4
 
5
- > ⚠️ Upgrading to 0.3? For 0.3.0-rc.1, every `linkIdentity` call must supply `verified`; it now returns an outcome. Run `payload migrate:create` for the nullable `xp_identities.verified_at` column and `user_id` index, plus the release-pages columns listed in the CHANGELOG (`decision_pages_released_at` / `decision_pages_released_by` on `xp_experiments` and their `version_` counterparts on `_xp_experiments_v`). Existing identity links become unverified; see the CHANGELOG for optional backfill.
5
+ > ⚠️ Upgrading to 0.4? Run `payload migrate:create` for four nullable plugin columns (`xp_experiments.page_collection` and its `_xp_experiments_v` counterpart, `xp_phases.placement_docs`, `xp_state.retired_slugs`), plus the Experiment block and `xpGoal` columns if you adopt them, then `payload generate:importmap`. Saves on pages, blocks and Puck collections can now be refused while a test runs on them (`page-busy`, `interacting-tests`); Puck tests whose slot is only on a draft can't start until the page is published; host `metricSources` ids may not start with `click:`; `admin.editorUrl` now receives a required `editor` argument. See the [CHANGELOG's BREAKING section](https://github.com/delmaredigital/payload-experiments/blob/main/CHANGELOG.md).
6
6
 
7
- > ⚠️ Upgrading to 0.2? After upgrading, run `payload generate:importmap`: the Tests surface is registered through admin.dependencies, and without it the Tests routes show a 'not installed' message instead of the Tests screen. In 0.2.0-rc.2, the Tests server views moved from `/rsc` to `/admin/rsc`, so the import map must be regenerated when upgrading from rc.1 too. The old experiments edit-form components are gone from `/admin/client`, and the `xp-experiments` admin list and edit views now redirect to the Tests surface.
7
+ ## Install
8
8
 
9
- A/B testing for Payload CMS and Next.js, with results inside the admin and data
10
- in your own Postgres database. Editors can test blocks, whole pages and
11
- registered flag points, choose a winner, and make it the permanent content.
12
- Results count people who actually saw the content, with checks for traffic
13
- imbalances and metrics that must not get worse. Puck support is optional.
14
-
15
- ```sh
9
+ ```bash
16
10
  pnpm add @delmaredigital/payload-experiments
17
11
  ```
18
12
 
19
- ## Setup
13
+ ### Requirements
20
14
 
21
- Requires Node.js `>=22.23.4`, Next.js App Router, and Postgres through
22
- `@payloadcms/db-postgres` with transactions enabled. Other database adapters
23
- are not supported. Install the peers used by your host:
15
+ Use Node.js `>=22.23.4`, the Next.js App Router, and Postgres through `@payloadcms/db-postgres` with transactions enabled. Puck is optional. Install the peers your application uses:
24
16
 
25
- | Peer | Range |
17
+ | Peer | Version |
26
18
  | --- | --- |
27
19
  | `payload`, `@payloadcms/db-postgres`, `@payloadcms/next`, `@payloadcms/ui` | `>=3.90.0 <4` |
28
20
  | `next` | `>=16.4.1 <17` |
29
21
  | `react`, `react-dom` | `>=19.2.1 <20` |
30
22
  | `pg` | `>=8.13.0 <9` |
31
- | `@puckeditor/core` (optional, for Puck integration) | `>=0.24.0` |
23
+ | `@puckeditor/core` (optional) | `>=0.24.0` |
24
+
25
+ ## Quick Start
26
+
27
+ These examples use your existing Payload app and Pages collection. Imports starting with `@/` are files you supply; the rendering example uses Puck.
28
+
29
+ ### 1. Set a secret and create the runtimes
32
30
 
33
- Create one proxy runtime, then a server runtime shared by your Payload config
34
- and server readers. Keep Payload imports lazy so the proxy stays independent
35
- of Payload, React and Puck:
31
+ Set a private `XP_SECRET` in your server environment. Create a shared `pg` pool for the same database as Payload, with a finite connection timeout:
36
32
 
37
33
  ```ts
38
- // experiments/proxyRuntime.ts
34
+ // src/experiments/proxyRuntime.ts
35
+ import { Pool } from 'pg'
39
36
  import {
40
37
  defineExperimentsProxyRuntime,
41
38
  createPostgresManifestLoader,
42
39
  } from '@delmaredigital/payload-experiments/runtime/proxy'
43
- import { manifestPool } from '@/db/manifestPool' // host-owned pg Pool
44
40
 
41
+ const pool = new Pool({
42
+ connectionString: process.env.DATABASE_URI,
43
+ connectionTimeoutMillis: 5_000,
44
+ })
45
45
  export const proxyRuntime = defineExperimentsProxyRuntime({
46
46
  secret: process.env.XP_SECRET!,
47
- apiRoute: '/api', // match Payload's config.routes.api
48
- loadManifest: createPostgresManifestLoader({ pool: manifestPool }),
47
+ apiRoute: '/api', // match Payload's routes.api
48
+ loadManifest: createPostgresManifestLoader({ pool }),
49
49
  })
50
+ ```
50
51
 
51
- // experiments/serverRuntime.ts
52
+ ```ts
53
+ // src/experiments/serverRuntime.ts
52
54
  import { defineExperimentsServerRuntime } from
53
55
  '@delmaredigital/payload-experiments/runtime/server'
54
56
  import { proxyRuntime } from './proxyRuntime'
@@ -60,853 +62,359 @@ export const serverRuntime = defineExperimentsServerRuntime(proxyRuntime, {
60
62
  return getPayload({ config })
61
63
  },
62
64
  })
65
+ ```
66
+
67
+ Keep the secret private and stable across deployments and instances. To rotate it, pass `secret: [newSecret, previousSecret]`, newest first. For a custom Postgres schema, pass the loader's `schemaName`.
68
+
69
+ ### 2. Add the plugin
63
70
 
64
- // In payload.config.ts
71
+ Add this to your existing `payload.config.ts` plugins array. Your permission check must return a boolean; grant it to the staff who will manage tests.
72
+
73
+ ```ts
65
74
  import { createExperimentsPlugin } from '@delmaredigital/payload-experiments'
66
75
  import { serverRuntime } from '@/experiments/serverRuntime'
67
- import { canManageExperiments } from '@/access/experiments' // host permission check
76
+ import { canManageTests } from '@/access/tests'
77
+ import { sharedRateLimit } from '@/experiments/rateLimit'
68
78
 
69
- // Add to buildConfig({ plugins: [...] })
70
79
  createExperimentsPlugin({
71
80
  runtime: serverRuntime,
72
- access: { manage: canManageExperiments },
73
- puck: {
74
- enabled: true,
75
- collections: ['pages', { slug: 'templates', field: 'content' }],
76
- },
81
+ access: { manage: canManageTests },
82
+ events: { rateLimit: sharedRateLimit },
77
83
  })
78
84
  ```
79
85
 
80
- `secret` is required on `defineExperimentsProxyRuntime`, not a top-level plugin
81
- option. Use a stable, private secret shared by all instances. For rotation, pass
82
- an array with the newest secret first; older entries verify existing signatures.
83
- Set a finite `connectionTimeoutMillis` on the host's manifest pool to bound
84
- connection acquisition. `createPostgresManifestLoader` also accepts `schemaName`
85
- when Payload uses a custom Postgres schema. It reads the private compiled
86
- manifest directly, avoiding HTTP calls back into your deployment.
87
-
88
- `createExperimentsPlugin` requires `runtime` and `access.manage`. The permission
89
- callback receives `{ req }` and must return a boolean (or a promise of one);
90
- permission is granted only for `true`. Optional `access.admin` and
91
- `access.readResults` default to `manage`; `access.readIdentities` defaults to
92
- `admin`. Use `admin` for code experiments and changing the analysis method.
93
- Other plugin options are `flagPoints`, `metricSources`, `eventSources`,
94
- `attributeKeys`, `pages`, `puck` and `events`:
95
-
96
- - `attributeKeys` declares targeting attributes as `{ type, values? }`, with
97
- `type` one of `string`, `number`, `boolean`; optional string `values` supplies
98
- an admin select. Pass only non-identifying attributes.
99
- - `puck.collections` registers existing host collections for placement discovery,
100
- content-change markers and making winners permanent. A slug string uses the
101
- `puckData` JSON field; `{ slug, field }` names another field. `puck.enabled`
102
- defaults to `true` when this list is non-empty; set it explicitly in TypeScript
103
- config (the current public type requires it). `false` disables the installed
104
- content-change hook.
105
- - `events.retentionDays` defaults to `180`; `events.maxBodyBytes` to `8192`.
106
- Both must be positive integers. `events.rateLimit` is an optional host adapter,
107
- called with `{ req, unitId }`; return `true` to permit or `false` to drop.
108
- - `events.dimensions` (optional, default none) names up to 8 request attributes
109
- to record on each exposure for later analysis, for example `['loggedIn']`.
110
- Values come only from the attributes the server computes for that request
111
- (`path`, `loggedIn`, `staff` and your `getUnit` attributes), never from what a
112
- browser sends; only strings (first 64 characters), numbers and booleans are
113
- kept. Names that identify a person (such as `userId`, `email` or `ip`) are
114
- refused at startup, but whether an attribute you add yourself identifies
115
- someone is your responsibility. For client-reported exposures `path` is the
116
- page path from the request's `Referer`. Reporting by dimension is not yet
117
- available.
118
- Supply a shared limiter suitable for your deployment; there is no built-in
119
- in-memory limiter.
120
-
121
- The plugin adds `xp-experiments`, `xp-phases`, `xp-layers`, `xp-metrics`,
122
- `xp-events`, `xp-identities`, `xp-results-snapshots` and the `xp-state` global.
123
- The host owns their migrations. After registering the plugin, generate and
124
- review a migration, then apply it in the host:
125
-
126
- ```sh
86
+ Supply a shared rate limiter for your deployment: `events.rateLimit` receives
87
+ `{ req, unitId }` and returns `true` to permit or `false` to drop a request.
88
+ Keep identifying attributes out of `events.dimensions`. Use the proxy runtime's `cookie.consent` to enrol only after consent (`ExperimentsProxyRuntimeOptions`); `eraseUser(payload, { userId })` from `/rsc` handles privacy erasure (typed by that exported function).
89
+
90
+ ### 3. Run migrations and generate the import map
91
+
92
+ Generate and review the migration in your host app, then apply it. Regenerate the import map so the Tests screens and goal pickers are available.
93
+
94
+ ```bash
127
95
  pnpm exec payload migrate:create
128
96
  pnpm exec payload migrate
97
+ pnpm exec payload generate:importmap
129
98
  ```
130
99
 
131
- ## Next.js proxy
100
+ ### 4. Wire the Next.js proxy
132
101
 
133
- For a standalone proxy, export the wrapper from your host's `proxy.ts`:
102
+ Create your app's `proxy.ts`. This assigns the version to show and handles
103
+ whole-page rewrites.
134
104
 
135
105
  ```ts
106
+ // src/proxy.ts
136
107
  import { createExperimentProxy } from '@delmaredigital/payload-experiments/proxy'
137
108
  import { proxyRuntime } from '@/experiments/proxyRuntime'
138
109
 
139
110
  export const proxy = createExperimentProxy(proxyRuntime)
140
111
  ```
141
112
 
142
- To compose with other proxy logic, call `resolveExperiments` and forward its
143
- request headers on **both** the normal response and any page rewrite. Then call
144
- `applyExperiments` to attach response cookies and headers:
113
+ ### 5. Add a results cron route
114
+
115
+ Your host schedules and authenticates this route. Choose a budget below your platform's route timeout, and schedule regular calls so results refresh.
145
116
 
146
117
  ```ts
147
- import { NextResponse, type NextRequest } from 'next/server'
148
- import { resolveExperiments, applyExperiments } from
149
- '@delmaredigital/payload-experiments/proxy'
150
- import { proxyRuntime } from '@/experiments/proxyRuntime'
118
+ // src/app/api/cron/xp-results/route.ts
119
+ import { runResults } from '@delmaredigital/payload-experiments/rsc'
120
+ import { serverRuntime } from '@/experiments/serverRuntime'
121
+ import { isAuthorizedCronRequest } from '@/auth/cron'
151
122
 
152
- export async function proxy(request: NextRequest) {
153
- const result = await resolveExperiments(request, proxyRuntime)
154
- const options = { request: { headers: result.requestHeaders } }
155
- const response = result.rewrite
156
- ? NextResponse.rewrite(result.rewrite, options)
157
- : NextResponse.next(options)
158
- return applyExperiments(response, result)
123
+ export async function GET(request: Request) {
124
+ if (!(await isAuthorizedCronRequest(request))) {
125
+ return new Response('Unauthorized', { status: 401 })
126
+ }
127
+ const payload = await serverRuntime.getPayload()
128
+ const result = await runResults(payload, { budgetMs: 20_000 })
129
+ return Response.json(result, { headers: { 'Cache-Control': 'no-store' } })
159
130
  }
160
131
  ```
161
132
 
162
- The default `xp_id` cookie is a signed opaque visitor id: HttpOnly,
163
- SameSite=Lax, Secure on HTTPS, path `/`, lifetime 365 days. Runtime `cookie`
164
- options are `name`, `maxAgeDays`, `domain` and `consent`. A newly minted cookie
165
- is forwarded into the same request, so server readers see it immediately.
166
- Inbound `x-xp-*` headers are removed before any skip check. The proxy writes
167
- signed `x-xp-assignments` (at most 1 KB) and `x-xp-request` (method, path/search
168
- and consent, at most 2048 bytes). Server readers verify these; unsigned or
169
- changed values are ignored, with a safe fallback when context is missing.
170
-
171
- `cookie.consent(request)` must read **cookies and headers only**: server
172
- conversion and identity helpers call it with a placeholder URL. When it returns
173
- `false`, nobody is enrolled, including signed-in users; no visitor id is minted
174
- or honoured, and an existing unit cookie is removed from the forwarded request.
175
- Running tests show the Original. Decided winners without an active holdout
176
- round still serve the decided content subject to targeting. Archived page
177
- rewrites serve shipped content by path alone. Neither records exposure.
178
- `linkIdentity` and `trackConversion`
179
- also become no-ops when consent is declined. Omitting `consent` permits cookies
180
- and enrolment.
181
-
182
- Optional runtime `getUnit(request)` returns `{ userId?, attributes,
183
- sessionUnknown? }`, synchronously or asynchronously. Use opaque user ids. With
184
- no session token, return known logged-out attributes such as
185
- `{ loggedIn: false, staff: false }`. With a verified session cache, return the
186
- known user and attributes; with an unresolved session, set `sessionUnknown:
187
- true`. User-unit experiments and session-dependent targeting can then be
188
- resolved on the server through `getUser`. Server runtime options `userIdOf`
189
- (default `String(user.id)`) and `attributesOf` map the host user to the unit id
190
- and targeting attributes; server attributes take precedence. `attributesOf`
191
- defaults to `{ loggedIn: Boolean(user) }`. If you declare `staff` in
192
- `attributeKeys`, pass your own `attributesOf` that identifies staff to
193
- `defineExperimentsServerRuntime`; the plugin refuses to start otherwise. The plugin owns the
194
- normalized `path` attribute (no search, no trailing slash except `/`).
195
-
196
- Staff, bots detected by the built-in user-agent list, preview renders and `HEAD`
197
- requests produce no exposures. A signed `xp_preview` token can select a group
198
- for review without counting it; preview responses are not cached. The plugin
199
- skips admin, API, Next internals and asset paths. Host editor renders must also
200
- remain outside exposure recording.
201
-
202
- Whole-page experiments use visitor units only. Requests to any target path are
203
- rewritten to the assigned group's page; a holdout sees the Original page. After
204
- a winner is made permanent, losing paths keep rewriting to the winner while
205
- the archived experiment record exists. A live experiment targeting a path takes
206
- precedence. Decided winners remain subject to targeting.
207
-
208
- ## Server components (`/rsc`)
209
-
210
- Import server helpers from `@delmaredigital/payload-experiments/rsc`.
211
- `getAssignments(serverRuntime, { user? })` reads and verifies request context,
212
- resolves deferred users, and recomputes stale or missing assignments using the
213
- signed cookie and manifest. It is memoised per request. It reads dynamic Next
214
- headers: call it inside `<Suspense>` under Cache Components, outside
215
- `'use cache'`.
216
-
217
- For Puck content, resolve before rendering and record only the returned
218
- `rendered` list after the response. Capture all request values before `after()`:
133
+ ### 6. Render the assigned version
134
+
135
+ For Puck pages, resolve the saved content before rendering it. Your page loader and render config stay in your app; add the [Puck integration](#puck-optional). For a Payload blocks field, see [Payload blocks (without Puck)](#payload-blocks-without-puck).
219
136
 
220
137
  ```tsx
138
+ // src/components/TestedPage.tsx
221
139
  import { after } from 'next/server'
222
- import { getAssignments, resolveExperimentData, recordExposures } from
223
- '@delmaredigital/payload-experiments/rsc'
140
+ import { Render } from '@puckeditor/core'
141
+ import {
142
+ getAssignments, resolveExperimentData, recordExposures,
143
+ } from '@delmaredigital/payload-experiments/rsc'
224
144
  import { serverRuntime } from '@/experiments/serverRuntime'
225
-
226
- // Inside your async Server Component, after loading the host page:
227
- const assignments = await getAssignments(serverRuntime)
228
- const { data, rendered } = resolveExperimentData(page.puckData, assignments)
229
- after(() => recordExposures(serverRuntime, rendered, assignments))
230
- // Render `data` with your host's Puck render config.
145
+ import { loadPage } from '@/pages/loadPage'
146
+ import { renderConfig } from '@/puck/renderConfig'
147
+
148
+ export async function TestedPage({ path }: { path: string }) {
149
+ const page = await loadPage(path)
150
+ const assignments = await getAssignments(serverRuntime)
151
+ const { data, rendered } = resolveExperimentData(page.puckData, assignments)
152
+ after(() => recordExposures(serverRuntime, rendered, assignments))
153
+ return <Render config={renderConfig} data={data} />
154
+ }
231
155
  ```
232
156
 
233
- **Exposure is the denominator.** Reading an assignment is not an exposure.
234
- `resolveExperimentData` removes unselected content and returns only block
235
- experiments actually present with the `render` policy. Pass that list to
236
- `recordExposures(serverRuntime, rendered, assignments)`, never the full
237
- assignment map. Only schedule it when you render that resolved content.
238
- Viewport and pageview exposures come from the client; manual flag/code
239
- exposures require an explicit host call. Non-enrolled units see the Original
240
- without being counted. Same-unit exposures to multiple groups in a round are
241
- excluded and reported.
242
-
243
- Next's default prefetch does not render the dynamic subtree in `<Suspense>`.
244
- Full prefetch (`<Link prefetch={true}>` or full `router.prefetch`) of pages with
245
- `render`-policy slots is unsupported: it would record content before a visit.
246
-
247
- `toClientAssignments(assignments, { pathname, apiRoute?, flagPoints?,
248
- experiments? })` creates the provider value without visitor identifiers. Both
249
- allow-lists default to empty. Use the served page's pathname for pageview
250
- logging; see the client integration below.
251
-
252
- `getFlagPoint(serverRuntime, id, { user? })` returns `arm`, `params`, `source`
253
- (`experiment` or `default`), optional `experiment`/`experimentId`, and
254
- `expose(renderedArm)`. Bind host code to the registered point id. It uses the
255
- live experiment, then saved winner defaults, then the point's `defaultParams`.
256
- Render the returned settings and call `await point.expose(point.arm)` only
257
- when that arm's content is actually delivered; this records only the manual
258
- exposure policy. Nothing is exposed by reading the point.
259
-
260
- Other server integrations:
261
-
262
- - `trackConversion(serverRuntime, { eventName, value?, idempotencyKey, userId?,
263
- headers? })` records a server event (default value `1`). Use a stable, globally
264
- unique idempotency key for each business event. Pass request headers when
265
- outside a Next request; a user-only server conversion can supply `userId`.
266
- Record revenue through trusted server code or a metric source, never a client
267
- beacon.
268
- - `linkIdentity(serverRuntime, { headers, userId, verified })` belongs on every
269
- user-creation and proving sign-in path; see Identity linking below.
270
- - `eraseUser(payload, { userId })` deletes the user's identities and associated
271
- events transactionally and returns `{ identities, events }` counts. Call it
272
- from the host's authorized user-erasure flow.
273
- - `recordContentChange({ doc, previousDoc, req, collection })` can be called
274
- from a host collection's `afterChange` for content tagged with the arm-select
275
- field; for a collection with drafts, also add `captureContentBaseline` to its
276
- `beforeChange` hooks so the previously published content can be compared.
277
- Collections without drafts can also add `captureContentBaseline` for an exact
278
- raw comparison; without it, the helper normalises relationships to ids. No host
279
- change is required for collections without drafts. Supply that hook's transaction request. Puck collections get the hook
280
- automatically when enabled. Markers compare published content; drafts and
281
- autosaves do not add markers (collections without drafts record on save).
282
- Neither the helper nor the installed hooks ever throw into your save: a
283
- failure is logged and the content save proceeds.
284
- - `experimentMetadata(serverRuntime, path)` returns metadata to merge into the
285
- page's Next metadata: live variant targets get noindex and an Original
286
- canonical; archived losing paths point canonically to the winner. It reads
287
- the signed request context via `headers()`, so it decides from the URL the
288
- visitor requested, not the rewritten route, and the Original's URL is never
289
- noindex. While a test is live the Original's URL also gets a canonical to
290
- itself, even when it is served with a variant's route: merge this result after
291
- your own metadata so its `alternates.canonical` wins over one derived from the
292
- rendered route. Call it only from a dynamic `generateMetadata`, never inside
293
- `'use cache'`; `path` is the fallback when the proxy did not run.
294
- `experimentSitemapFilter(path, manifest)` is a synchronous build-time filter:
295
- pass a privately server-loaded manifest and keep paths for which it returns
296
- `true`, excluding live variants and archived losing paths.
297
-
298
- ## Identity linking
299
-
300
- Call `linkIdentity(serverRuntime, { headers, userId, verified })` on every
301
- user-creation and proving sign-in path. It links the signed anonymous cookie
302
- to an opaque user id on its own connection and never throws into sign-in.
303
- `verified` is required. Pass `true` only when **this request proves, in this
304
- browser, that the person owns the account**: a magic-link or one-time-code
305
- callback, password sign-in, or OAuth callback. It does not mean "a session
306
- exists" or "the account's email is verified". Registration alone uses `false`.
157
+ Render this async component inside `<Suspense>` and outside `'use cache'`.
158
+ Only record the returned `rendered` list when you deliver that content. Keep
159
+ editor and preview renders out of this path. Use Next's default prefetch;
160
+ full prefetch of pages that count content on server render is unsupported.
307
161
 
308
- ```ts
309
- // Registration, before proof of account ownership:
310
- await linkIdentity(serverRuntime, { headers, userId, verified: false })
311
- // After successful magic-link/code verification in this browser:
312
- await linkIdentity(serverRuntime, { headers, userId, verified: true })
313
- // After a successful password sign-in (or OAuth callback):
314
- await linkIdentity(serverRuntime, { headers, userId, verified: true })
315
- ```
162
+ ## Running a test
316
163
 
317
- An auth "after" hook that fires whenever a new session cookie is set also
318
- fires on routine session refreshes; pass `verified: true` only from proving
319
- sign-in paths, never from a refresh.
320
-
321
- The result is `'linked'` for an inserted link, `'verified'` for an upgraded
322
- unverified link, `'skipped'` for no consent, no valid unit cookie or an existing
323
- link with nothing to change, and `'failed'` for an error (safely logged).
324
- First link per browser wins; a different account never replaces it.
325
-
326
- For visitor experiments with user-keyed events, verified links are always
327
- eligible. Unverified links are eligible only before the first verification
328
- across all of that user's browsers, or indefinitely if nobody ever verifies.
329
- Each event goes to the earliest exposed eligible visitor. Eligible links
330
- spanning groups are reported as a diagnostic. User-keyed CUPED covariates use
331
- the same eligibility rule at each visitor's first exposure and credit one
332
- aggregate value to the earliest eligible visitor per user.
333
-
334
- Accepted limits:
335
-
336
- - Cross-device: register in A, verify on B, keep using A. A's events after B's
337
- proof are not credited until a proving sign-in on A upgrades A. Events before
338
- B's proof, including the registration funnel, remain credited. Once A is
339
- upgraded, its link is verified and eligible for all its events, so the next
340
- results run also credits A's events from between B's proof and the upgrade.
341
- - Password set before verification: a later password sign-in is proof the
342
- plugin accepts. Whether unverified registration may set a password is a host
343
- authentication concern.
344
- - Shared device: a different user's verified sign-in never relinks an already
345
- linked browser and returns `'skipped'`; that user's events get no credit
346
- through that browser.
347
- - If someone registers another person's email from their own browser and that
348
- person never signs in, a later user-keyed event the host records against that
349
- account (for example an offline gift) is credited to the registering browser.
350
- It needs a deliberate registration of someone else's email plus an event
351
- matched to that account, and it moves one unit's value, never assignment.
352
- There is no time window, so recurring gifts from never-signed-in donors keep
353
- counting.
354
-
355
- ## Results
356
-
357
- The host owns scheduling and authentication. Call
358
- `runResults(payload, { budgetMs })` in-process from your cron route; no Payload
359
- jobs queue or HTTP call back into the application is needed:
164
+ 1. Start from the content: add an **Experiment block** (in Puck's **A/B tests** category),
165
+ or open **Tests** in the admin to test whole pages or a registered setting.
166
+ 2. Keep the **Original**, use **Copy Original here** in Puck to make a version,
167
+ and make your changes. For whole pages, use Payload's **Duplicate** action, give the
168
+ copy its own address, publish it, then choose it as a version in the test.
169
+ 3. Pick a **goal**: a signup, purchase, or a named click goal on a button or
170
+ link. Add guardrail goals for outcomes you don't want to hurt.
171
+ 4. Save, follow the setup checks, preview each version ([wire preview links](#preview-and-editor-links)), and **Start**. The
172
+ results explain whether to keep running or choose a winner.
173
+ 5. Use **Decide** to pick the winner, then **Make it the content** to make it permanent.
174
+ You can also **Stop** to show everyone the Original; starting again begins
175
+ a new round with fresh results.
360
176
 
361
- ```ts
362
- // app/api/cron/xp-results/route.ts
363
- import { runResults } from '@delmaredigital/payload-experiments/rsc'
364
- import { serverRuntime } from '@/experiments/serverRuntime'
365
- import { isAuthorizedCronRequest } from '@/auth/cron' // host-owned auth
177
+ A click tells you someone took an action; it doesn't prove they completed a purchase or signup. Use the outcome itself as a goal when you can measure it.
366
178
 
367
- export async function GET(request: Request) {
368
- if (!(await isAuthorizedCronRequest(request))) {
369
- return new Response('Unauthorized', { status: 401 })
370
- }
371
- const payload = await serverRuntime.getPayload()
372
- const result = await runResults(payload, { budgetMs: 20_000 })
373
- return Response.json(result, { headers: { 'Cache-Control': 'no-store' } })
374
- }
375
- ```
179
+ ## What your app wires
376
180
 
377
- The default budget is 20 seconds. Choose a budget below the host route's timeout.
378
- The runner prioritizes the oldest snapshots, checks the remaining budget before
379
- metrics, bounds waiting for host `query`/`covariate` promises and database
380
- connections, and applies a remaining-budget SQL statement timeout. Hung work
381
- cannot keep the runner waiting beyond that budget. Source failures mark that
382
- metric unavailable. A timed-out source is reported in `unavailableMetrics`;
383
- that experiment's new snapshots are not written, its previous snapshots remain,
384
- and `budgetExhausted` signals the cutoff. Racing a host promise does not cancel
385
- its underlying work: give your own source clients appropriate timeouts too.
386
- The returned summary also includes `ran`, `skipped`, `swept`, `sweepAttempted` and `processedIds`.
387
- Optional `experimentId` limits analysis to one experiment; `sweep: false`
388
- disables retention for that call.
389
-
390
- Analysis uses each round's frozen config and actual exposures. Its window ends
391
- at `endedAt`; later exposures and conversions are excluded. Finished rounds
392
- have a **24-hour late-beacon grace** before their final snapshot freezes: once
393
- a snapshot has been computed after that grace, later runs leave it unchanged.
394
- This does not extend the attribution window beyond the round's end.
395
-
396
- Event retention defaults to **180 days after the experiment's final round
397
- ends**, configurable with `events.retentionDays`. No experiment events are
398
- swept while any of its rounds is open, including a long-lived holdout round.
399
- The runner reserves part of its budget for the batched retention sweep.
400
-
401
- Routes below are relative to Payload's `config.routes.api` (normally `/api`):
402
-
403
- | Endpoint | Permission and response |
404
- | --- | --- |
405
- | `GET /xp/experiments/:id/results?phase=1&series=1` | `access.readResults`; round summaries, newest selected-round `snapshot` (`null` before a run), optional `decisionSnapshot` and requested time series. Without `phase`, a decided experiment opens on its decision round. |
406
- | `GET /xp/experiments/summary?ids=...` | `access.readResults`; map of experiment ids to exposures, days and verdict summary. Encode each id individually, comma-join, and request at most 100 ids. |
407
- | `POST /xp/experiments/:id/recompute` | `access.manage`; runs results for one experiment without sweeping and returns `{ snapshotId }`. Failed or budget-exhausted work returns an error. Frozen rounds stay frozen. |
181
+ ### Proxy
408
182
 
409
- ## Flag points and metric sources
183
+ Keep the [proxy from step 4](#4-wire-the-nextjs-proxy) on your public routes.
184
+ If you already have a proxy, compose with `resolveExperiments` and
185
+ `applyExperiments` from `/proxy`: forward the returned `requestHeaders` on
186
+ both normal responses and rewrites, then apply the returned cookies and headers.
187
+
188
+ ### Whole-page tests
410
189
 
411
- Register stable host integration points in `flagPoints`. `defineFlagPoint` and
412
- `defineMetricSource` are optional typed identity helpers from the root export;
413
- the root also exports the `FlagPoint`, `ParamField` and `MetricSource` types.
414
-
415
- A `FlagPoint` has `id`, `label`, `params` and `defaultParams`. `params` maps
416
- setting names to `ParamField` definitions: `{ type, label, required?, options?,
417
- min?, max? }`. Supported types are `text`, `textarea`, `number`, `checkbox`,
418
- `select`; select options are `{ label, value }`. Keep defaults valid for that
419
- schema. Optional callbacks integrate host-owned content:
420
-
421
- - `placements({ experimentId, arms, req })` returns
422
- `[{ arm, ok, detail? }]` for the required content groups at Start. Holdouts
423
- use Original content and are not passed as required placements.
424
- - `makeContent({ experimentId, winner: { arm, params }, req })` runs when the
425
- chosen settings are made permanent. Match host tags on both experiment id
426
- and arm. Use the manager's `req` and `overrideAccess: false` for host-document
427
- writes. The plugin saves the winning settings as defaults for the point.
428
-
429
- A `MetricSource`, registered in `metricSources`, has `id`, `label`,
430
- `kind: 'binomial' | 'count' | 'revenue'`, and `keyedBy: 'visitor' | 'user'`.
431
- Source ids starting with `click:` are reserved for click goals and refused at boot.
432
- `clientWritable` optionally permits client conversions (revenue is always
433
- server-only). `query({ from, to, unitIds? })` returns a promise of
434
- `[{ unitId, value, ts }]`, **one row per event**, with `ts` a `Date`; use the
435
- same opaque ids as assignment or identity linking. A user experiment requires
436
- a user-keyed source. Visitor experiments can use a user-keyed source through
437
- linked identities.
438
-
439
- Optional `covariate({ units: [{ unitId, firstExposureTs }], lookbackDays })`
440
- returns `[{ unitId, value }]`, one aggregate value per unit for pre-exposure
441
- CUPED adjustment (the plugin asks for a 14-day lookback). Percentile
442
- revenue capping reuses `query` over the 90 days before Start; the resolved cap
443
- is frozen for the round. Capping affects decision statistics, while results
444
- also show uncapped totals. Count and binomial metrics are never capped.
445
-
446
- For simple yes/no outcomes, list names in `eventSources`, for example
447
- `eventSources: ['signup']`. A bare string allows client writes; the object form
448
- sets that permission explicitly:
190
+ Add this to the plugin from step 2 for a drafts-enabled Pages collection:
449
191
 
450
192
  ```ts
451
- eventSources: ['signup', { name: 'donation_completed', clientWritable: false }]
193
+ pages: { collections: [{
194
+ slug: 'pages',
195
+ pathOf: doc => typeof doc.slug === 'string' ? `/${doc.slug}` : null,
196
+ findByPath: async (path, req) => {
197
+ const { docs } = await req.payload.find({
198
+ collection: 'pages', req, overrideAccess: false, draft: false, depth: 0, limit: 1,
199
+ where: { and: [
200
+ { slug: { equals: path.slice(1) } },
201
+ { _status: { equals: 'published' } },
202
+ ] },
203
+ })
204
+ return docs[0] ? { id: docs[0].id, published: true } : null
205
+ },
206
+ }] }
452
207
  ```
453
208
 
454
- Names must match `/^[a-z0-9_]+$/` and be unique across both forms. Editors can
455
- choose `events:signup` or `events:donation_completed` as binomial sources backed
456
- by stored conversions with the corresponding `eventName`. Writable sources
457
- accept both `trackConversion` and `trackConversionBeacon`; server-only sources
458
- are recorded with `trackConversion`. Beacon conversions for a non-writable name
459
- are dropped silently. Only listed names become `events:<name>` sources. They
460
- support visitor and user units and do not represent counts or revenue in v1.
209
+ Adapt both callbacks for your homepage. Add `contentOf: doc => doc.layout` to select content for comparison. Keep shipped winner pages published at the same address.
210
+ In dynamic `generateMetadata`, merge `await experimentMetadata(serverRuntime, path)` from `/rsc` **after** your own metadata so its canonical and robots settings win; keep it outside `'use cache'`. Use `experimentSitemapFilter` from `/rsc` in your sitemap.
461
211
 
462
- ### Testing page SEO in the host
212
+ ### Preview and editor links
463
213
 
464
- `pageKindSeoContract({ secret, original, versions, state, winner? })` from `/rsc`
465
- returns `{ manifest, rows }`. States are `running`, `decided` and `archived`;
466
- provide a winner path for the latter two. Build the host's test runtime with
467
- `loadManifest: async () => manifest` and the same secret. For each row, set
468
- `next/headers` to its signed `headers`, render metadata for `servedPath`, and
469
- assert `expect.robots`, `expect.canonical` and `expect.inSitemap` against the
470
- host's metadata and sitemap functions. This includes the Original served as
471
- each version, including the homepage.
214
+ The setup checks in [Running a test, step 4](#running-a-test) get their preview links from `admin.previewUrl`. Add these options to the plugin, adapting the routes to your app:
472
215
 
473
- A host-level test mocks four boundaries: `next/headers`, the page loader,
474
- the runtime and the site URL. Set `metadataBase` so relative canonical paths
475
- resolve to your site URL.
216
+ ```ts
217
+ admin: {
218
+ previewUrl: ({ collection, slug }) => collection === 'pages' && slug !== undefined
219
+ ? (slug === 'home' ? '/' : `/${slug}`) : null,
220
+ editorUrl: ({ collection, id, editor }) => editor === 'puck'
221
+ ? `/admin/puck-editor/${encodeURIComponent(collection)}/${encodeURIComponent(String(id))}`
222
+ : `/admin/collections/${encodeURIComponent(collection)}/${encodeURIComponent(String(id))}`,
223
+ }
224
+ ```
476
225
 
477
- ### Whole-page options
226
+ `previewUrl` receives `PreviewPlacement`: `collection`, `id`, `title`, `published`, and optional `slug` from the published page (absent for unpublished pages). Return `null` to hide preview. `editorUrl` receives `collection`, `id`, optional `draft` and `doc`, and `editor: 'puck' | 'blocks' | 'page'` to choose the editor route.
478
227
 
479
- | Option | Purpose |
480
- | --- | --- |
481
- | `pages.collections` | Optional array of `{ slug, pathOf, findByPath }`. `pathOf(doc)` derives a public path from the slug (used to protect live test paths); `findByPath(path, req)` resolves a published page to `{ id, published: true }`, or returns `null`. Start and readiness check every non-holdout version, including the Original. Registered collections must exist when this plugin runs. Localized publication status and field-level `beforeChange` hooks that rewrite paths after the collection guard are unsupported; derive paths from fields finalized before collection hooks. Without this option, readiness warns that pages are unchecked and Start keeps its existing behaviour. |
228
+ ### Results schedule
229
+
230
+ The [route in step 5](#5-add-a-results-cron-route) runs results in your app.
231
+ On Vercel, add it to your cron configuration and check its authentication.
232
+ The host owns the schedule; the plugin does not use Payload jobs.
482
233
 
483
- Do not swallow errors from nested updates of guarded page collections inside host hooks: a swallowed error leaves the guard's operation frame in place for the rest of the request because Payload's Local API does not call `afterError`.
234
+ ### Migrations
484
235
 
485
- For a collection with drafts enabled, use a published-only slug lookup with the
486
- manager's request and `overrideAccess: false`:
236
+ The schema belongs to your app. Repeat [step 3](#3-run-migrations-and-generate-the-import-map) after every upgrade that changes it, including fields you add to host blocks.
237
+
238
+ ### Signing users in
239
+
240
+ If your site has logins, extend the [server runtime in step 1](#1-set-a-secret-and-create-the-runtimes)
241
+ with `getUser` for your session lookup and `attributesOf` for non-identifying
242
+ attributes such as `loggedIn` and `staff`. Identify staff so their visits don't
243
+ count. In the proxy runtime, supply `getUnit`: return `{ userId, attributes }`
244
+ from a verified session, with `loggedIn` and `staff` in `attributes`; return
245
+ known logged-out attributes when there is no session token, or set
246
+ `sessionUnknown: true` when a token still needs checking on the server.
247
+ Declare `staff: { type: 'boolean' }` in the plugin's `attributeKeys`.
248
+ Call `linkIdentity` at registration and after a successful sign-in:
487
249
 
488
250
  ```ts
489
- pages: {
490
- collections: [{
491
- slug: 'pages',
492
- pathOf: doc => typeof doc.slug === 'string' ? `/${doc.slug}` : null,
493
- findByPath: async (path, req) => {
494
- const { docs } = await req.payload.find({
495
- collection: 'pages', req, overrideAccess: false, draft: false, depth: 0, limit: 1,
496
- where: { and: [
497
- { slug: { equals: path.slice(1) } },
498
- { _status: { equals: 'published' } },
499
- ] },
500
- })
501
- return docs[0] ? { id: docs[0].id, published: true } : null
502
- },
503
- }],
504
- },
251
+ import { linkIdentity } from '@delmaredigital/payload-experiments/rsc'
252
+ import { serverRuntime } from '@/experiments/serverRuntime'
253
+
254
+ // In your successful password, magic-link/code, or OAuth callback:
255
+ await linkIdentity(serverRuntime, { headers: request.headers, userId, verified: true })
505
256
  ```
506
257
 
507
- Adapt both callbacks for the site's homepage path. These checks run before Start;
508
- they do not protect pages from later deletion, unpublishing or renaming. Keep an
509
- archived winner published at the same address: archived test URLs still point to it.
510
- Whole-page registrations also install the sidebar test status and content-change hooks.
511
- For 0.4.0, run `payload migrate:create` for `xp_state.retired_slugs`,
512
- `xp_experiments.page_collection` and its version column, then
513
- `payload generate:importmap`.
258
+ `verified` is required: use `true` only when this request proves ownership of
259
+ the account in this browser. Registration without that proof uses `false`.
260
+ A session refresh or an account's verified-email flag is not proof here.
514
261
 
515
- `contentOf(doc)` optionally selects the content your site renders (for example,
516
- `doc.layout`). Published copies are compared with nested `id` and `blockName`
517
- keys removed. Without it, content checks and page content-change comparisons
518
- are skipped. `copyHint` (up to 200 characters), `copyUrl` (a root-relative path)
519
- and `copyLabel` (up to 40 characters, requires `copyUrl`) customize the guidance
520
- for making version pages. The default copy link opens the registered collection's
521
- list under the configured admin route.
262
+ ### Erasing a user
522
263
 
523
- ### `admin` options
264
+ When you erase a user, call `eraseUser(payload, { userId })` from `/rsc` so their test data goes too; see the [full reference](https://github.com/delmaredigital/payload-experiments/blob/main/packages/plugin/REFERENCE.md#server-components-rsc).
524
265
 
525
- Host links and kind descriptions are optional. For a host using payload-puck's
526
- editor route, public pages and a `puck-templates` collection:
266
+ ### Puck (optional)
527
267
 
528
- | Option | Signature and behavior |
529
- | --- | --- |
530
- | `admin.previewUrl` | `(placement: PreviewPlacement) => string \| null \| undefined \| Promise<string \| null \| undefined>`. Receives `collection`, `id`, `title`, `published`, and optional `slug`. May be async; return `null` to hide preview for that placement. Each call has a 2 s limit, with at most 8 lookups awaited concurrently. |
268
+ For [steps 2](#2-add-the-plugin) and [6](#6-render-the-assigned-version), add
269
+ `puck: { enabled: true, collections: ['pages'] }` to the plugin. A collection
270
+ slug uses its `puckData` field; use `{ slug: 'pages', field: 'content' }` for another.
271
+ Works with `@delmaredigital/payload-puck >=0.10.3` and experiments `>=0.4.0`.
531
272
 
532
- For example, pass:
273
+ In your render config, import `experimentsPuck` from `/puck` and pass
274
+ `integrations: [experimentsPuck()]` to payload-puck's `extendConfig`. In a
275
+ `'use client'` editor module, import `experimentsPuckEditor` from `/puck/editor`,
276
+ create `const integrations = [experimentsPuckEditor()]` once at module scope,
277
+ and pass it to `PuckConfigProvider`. Pass your custom `routes` to both helpers.
278
+ Load this in your iframe stylesheet, after Tailwind if you use it, and list that stylesheet's URL in payload-puck's `editorStylesheets`:
533
279
 
534
- ```ts
535
- admin: {
536
- editorUrl: ({ collection, id, draft, editor, doc }) =>
537
- `/admin/puck-editor/${encodeURIComponent(collection)}/${encodeURIComponent(String(id))}`,
538
- previewUrl: async ({ collection, slug }) => {
539
- if (collection !== 'pages' || slug === undefined) return null
540
- // Match the site's public page routing, including its homepage convention.
541
- return slug === 'home' ? '/' : `/${slug}`
542
- },
543
- descriptions: {
544
- block: 'Try different content on a landing page.',
545
- document: 'Try different versions of a donation ask.',
546
- setting: 'Try a different signup or donation prompt.',
547
- },
548
- }
280
+ ```css
281
+ @import "@delmaredigital/payload-experiments/puck/editor.css";
549
282
  ```
550
283
 
551
- `PreviewPlacement` is exported from the package root and `/rsc`. Its `slug` is
552
- the published main row's string slug, including for a placement found in a newer
553
- draft with a different slug. It is absent for unpublished documents. `published`
554
- describes the placement's version; the callback receives no other document fields.
555
- An async callback can also look up the public path by `collection` and `id` using
556
- the host's own data access. The old `({ collection, id }) => string` form still works.
557
-
558
- Preflight resolves the URL callbacks on the server for block placements and sends
559
- only URL strings to the client. A null/undefined result, a non-string, an exception,
560
- a rejected promise or a timeout leaves preview hidden for that placement. When an
561
- editor URL exists, the admin offers "Edit on page"; when neither URL exists, it
562
- omits the row. Timed-out host work cannot be cancelled and keeps its concurrency
563
- slot until it settles; if all 8 slots remain occupied, further preview lookups
564
- are skipped for that preflight. Kind descriptions are available to the server views through
565
- `config.custom.xp`.
566
-
567
- A flag point can also declare `description?: string` to explain the setting below its label.
568
-
569
- ## Security
570
-
571
- The unit cookie, assignments header, request-context header and preview tokens
572
- are HMAC-SHA256 signed with purpose-separated inputs and constant-time
573
- verification. Signing prevents a browser from choosing a trusted identity,
574
- assigned arm, request context or preview group by editing a value. The manifest
575
- is private. Store no PII beyond opaque unit ids: keep names, emails and other
576
- identifying values out of attributes, params and event data.
577
-
578
- Every collection/global access operation is explicit: create, read, update,
579
- delete, readVersions and unlock. Managers operate experiments, layers and
580
- metrics; results and identity reads have separate permissions. Internal event,
581
- identity, round, snapshot and state writes are denied through ordinary
582
- collection/global access. Every Payload Local API call declares
583
- `overrideAccess`; request handlers and views pass `req` and
584
- `overrideAccess: false`. The enumerated privileged operations are the events
585
- insert and assignment lookup, `recordExposures`, server `trackConversion`,
586
- `linkIdentity`, manifest reads, transition writes to plugin collections/state,
587
- content-change markers, results writes and retention sweep, and `eraseUser`.
588
- Host content writes when making a winner permanent still use the manager's
589
- access. Privileged side effects follow their authorized operation in the same
590
- transaction; identity linking deliberately uses its own insert-ignore connection
591
- so duplicate links cannot abort sign-in.
592
-
593
- `POST /xp/events` requires a valid signed unit cookie, reads a streamed body
594
- with an **8192-byte default limit**, and accepts at most **20 events** per body.
595
- It parses JSON regardless of content type for beacon compatibility. Exposure
596
- beacons must name the current round, policy and the arm actually rendered;
597
- the server recomputes assignment and drops mismatches while counting them
598
- separately. Only manual, viewport and pageview exposure policies are accepted.
599
- Conversions require a client-writable source, never revenue; client values are
600
- stored as `1`, attributes are allow-listed, and the host's `events.rateLimit`
601
- adapter runs when supplied. Accepted and dropped events both return `202 {}`;
602
- malformed requests can return errors. `click:<key>` conversions are also accepted
603
- for defined click-goal keys matching `[a-z0-9][a-z0-9_-]{0,63}`; undefined or
604
- invalid keys are dropped with `202`. Clicks carry no attributes, and defined
605
- keys may be cached for up to 30 seconds.
606
-
607
- **The real bound on slow or method-override bodies is the host firewall/platform
608
- timeout.** A stream under the byte limit can remain open, and Payload can read
609
- an `X-HTTP-Method-Override` body before the plugin's streamed cap runs. Apply
610
- host firewall rules and platform request timeouts at that boundary.
611
-
612
- Static checks enforce explicit access and keep raw SQL in the database modules,
613
- but have accepted gaps: extracting a Payload method with a computed property
614
- in destructuring; reflection through `Object.getOwnPropertyNames`,
615
- `Object.getOwnPropertyDescriptor` or `Reflect.ownKeys`; capturing a computed
616
- method from an unknown receiver and calling it later; `payload.jobs.*` calls
617
- (the jobs queue is not used or supported here); lowercase SQL through `pg`;
618
- and aliases changed through `let` reassignment. **Code review is the backstop**
619
- for these patterns.
620
-
621
- ## Admin and client
622
-
623
- **Tests** is a regular navigation group for managers only, with **All tests**
624
- and **Goals**. It is the first named group, after Payload's default
625
- Collections/Globals groups when the host has ungrouped entities. Payload's own
626
- dashboard shows the Tests cards. A host collection with `group: 'Tests'` joins
627
- the plugin's group.
628
-
629
- After upgrading, run `payload generate:importmap` to register the two list views
630
- and the Goals edit redirect. Without regeneration, both list URLs and the Goal
631
- create/edit URLs show Payload's raw views.
632
-
633
- ### Client assignments and exposure
634
-
635
- Read assignments on the server and pass them to `ExperimentsProvider` from
636
- `@delmaredigital/payload-experiments/client`. Build its `value` with
637
- `toClientAssignments` from `@delmaredigital/payload-experiments/rsc`:
284
+ ### Payload blocks (without Puck)
638
285
 
639
- ```tsx
640
- <ExperimentsProvider value={toClientAssignments(assignments, {
641
- pathname,
642
- flagPoints: ['signup-form'],
643
- experiments: ['welcome-copy'],
644
- })}>
645
- {children}
646
- </ExperimentsProvider>
286
+ Add `blocks: { collections: [{ slug: 'pages', field: 'layout' }] }` to the plugin from step 2. Add the wrapper to your host's blocks field, then repeat step 3:
287
+
288
+ ```ts
289
+ import { createExperimentBlock } from '@delmaredigital/payload-experiments'
290
+ import { contentBlocks } from '@/blocks'
291
+
292
+ // In the Pages collection's fields array:
293
+ { name: 'layout', type: 'blocks', blocks: [
294
+ ...contentBlocks, createExperimentBlock({ blocks: contentBlocks }),
295
+ ] }
647
296
  ```
648
297
 
649
- `flagPoints` and `experiments` are allow-lists: both default to none. Include
650
- only the flag points and experiments that the client needs. The provider value
651
- contains no visitor identifier. Use **one `apiRoute` for every provider on a
652
- page**, including nested providers; it is the API base, such as `/api`. When the
653
- API isn't at `/api`, set that shared route through
654
- `toClientAssignments(assignments, { pathname, apiRoute })`.
655
-
656
- `useFlagPoint(id)` reads the current group and settings at a registered flag
657
- point. `useExperiment(slug)` reads an allow-listed experiment. An unlisted flag
658
- point or experiment slug throws in development and returns the Original with
659
- one console error in production. Neither hook records exposure automatically.
660
- For manual exposure, call
661
- `expose(renderedArm)` only when that group's content actually showed, passing
662
- the group key you rendered. Reading an assignment alone does not count as
663
- seeing content. `expose()` sends only for experiments using the manual exposure
664
- policy.
665
-
666
- On each whole-page experiment target, render a nested `ExperimentsProvider`
667
- whose value is built for **that page's pathname**, and put
668
- `<ExperimentPageview />` inside it. This component reads from the nearest
669
- provider and records the page that appeared, including after navigation.
670
-
671
- Every page that renders Puck viewport slots also needs a provider. A
672
- layout-level provider with empty allow-lists is enough:
673
- `toClientAssignments(assignments, { pathname })`. Without a provider, viewport
674
- exposures cannot be sent.
675
-
676
- For a client-writable conversion, call
677
- `trackConversionBeacon(eventName, { idempotencyKey })`. Pass a stable key for
678
- conversions that matter, so retries count as the same conversion. It sends
679
- nothing without a mounted provider. The server accepts only conversion sources
680
- configured to allow client writes; record revenue on the server.
681
-
682
- ### Puck render and editor configs
683
-
684
- Works with: payload-puck ≥0.10.3 (experiments ≥0.4.0).
685
-
686
- For payload-puck hosts, add one integration object on each side. In the render
687
- config, pass `integrations` to `extendConfig`:
298
+ Resolve before rendering, as in step 6:
688
299
 
689
300
  ```ts
690
- import { extendConfig } from '@delmaredigital/payload-puck/config'
691
- import { experimentsPuck } from '@delmaredigital/payload-experiments/puck'
692
- import { baseConfig } from './baseConfig'
301
+ import { after } from 'next/server'
302
+ import { getAssignments, resolveExperimentBlocks, recordExposures } from
303
+ '@delmaredigital/payload-experiments/rsc'
693
304
 
694
- export const renderConfig = extendConfig({
695
- base: baseConfig,
696
- integrations: [experimentsPuck()],
697
- })
305
+ const assignments = await getAssignments(serverRuntime)
306
+ const { blocks, rendered } = resolveExperimentBlocks(page.layout, assignments)
307
+ after(() => recordExposures(serverRuntime, rendered, assignments))
308
+ // Render `blocks` with your host's block renderer.
698
309
  ```
699
310
 
700
- Call `experimentsPuckEditor()` once at module scope in a `'use client'` module,
701
- then pass that array to `PuckConfigProvider`. A new object per render rebuilds
702
- Puck's config and plugin list.
311
+ Use the same Suspense, preview and prefetch rules as step 6; record only `rendered` when delivering the content.
703
312
 
704
- ```tsx
705
- 'use client'
313
+ ### Click goals on your blocks
706
314
 
707
- import { PuckConfigProvider } from '@delmaredigital/payload-puck/client'
708
- import { experimentsPuckEditor } from '@delmaredigital/payload-experiments/puck/editor'
709
- import { editorConfig } from './editorConfig'
315
+ After [step 2](#2-add-the-plugin), add a goal picker to your button or link block,
316
+ then repeat [step 3](#3-run-migrations-and-generate-the-import-map):
710
317
 
711
- const integrations = [experimentsPuckEditor()]
318
+ ```tsx
319
+ import { xpGoalField } from '@delmaredigital/payload-experiments'
712
320
 
713
- export function PuckProvider({ children }: { children: React.ReactNode }) {
714
- return <PuckConfigProvider config={editorConfig} integrations={integrations}>
715
- {children}
716
- </PuckConfigProvider>
717
- }
321
+ // Add xpGoalField() to the host block's fields array.
322
+ // In that block's renderer, put the saved value on the actual button or link:
323
+ <a href={href} data-xp-goal={xpGoal}>{label}</a>
718
324
  ```
719
325
 
720
- Each object includes the test block and its **A/B tests** sidebar category. The
721
- editor object also includes the outline plugin: it hides empty version slots
722
- that the block's test doesn't use, while content in unused slots stays visible
723
- with a warning. Both helpers accept `{ routes }`; the editor also accepts
724
- `accent`. Use the host's Payload routes and the plugin's `theme.accent`.
326
+ For Puck components, use `xpGoalPuckField()` from `/puck` in the render fields
327
+ and `xpGoalPuckEditorField()` from `/puck/editor` in the editor fields; pass custom `routes` as
328
+ `xpGoalPuckEditorField({ routes })`. Editors name click goals in **Goals** or the picker. Mount the provider described below
329
+ to count clicks; put the main goal and guardrail goals in every version or
330
+ outside all tested content.
725
331
 
726
- #### Wiring Puck by hand
332
+ ## How it fits a typical site
727
333
 
728
- The separate pieces stay exported:
334
+ For a site with Puck Pages, signed-in users, a Vercel cron and staff who manage tests, the pieces usually live here:
729
335
 
730
- - `/puck`: `ExperimentSlot`, `createExperimentSlot({ routes })`,
731
- `EXPERIMENT_SLOT_TYPE` and `createExperimentSlotField({ routes })`.
732
- - `/puck/editor`: `ExperimentSlotEditor`,
733
- `createExperimentSlotEditor({ routes, accent })` and `experimentOutlinePlugin()`.
336
+ | File | What belongs there |
337
+ | --- | --- |
338
+ | `src/experiments/proxyRuntime.ts` | Private secret, manifest pool and API route |
339
+ | `src/experiments/serverRuntime.ts` | Lazy Payload loader, session lookup and staff attributes |
340
+ | `src/payload.config.ts` | Plugin, staff permissions, shared limiter and Pages registration |
341
+ | `src/proxy.ts` | Version selection and whole-page rewrites |
342
+ | `src/app/api/cron/xp-results/route.ts`, `vercel.json` | Authenticated results route and schedule |
343
+ | `src/components/TestedPage.tsx` | Resolve Puck content and render the assigned version |
344
+ | `src/puck/renderConfig.ts`, `src/puck/EditorProvider.tsx` | Render/editor integrations and iframe CSS |
345
+ | `src/auth/callbacks.ts` | Identity linking after registration and proving sign-in |
346
+ | `src/blocks/Button.ts` and its renderer | Click-goal picker and button/link attribute |
734
347
 
735
- Register the render and editor blocks under the same `EXPERIMENT_SLOT_TYPE` key (import it from `/puck`),
736
- add a sidebar category listing it, and add `experimentOutlinePlugin()` to Puck's
737
- `plugins`. It composes with the host's outline override.
348
+ ## Flag points and metric sources
738
349
 
739
- Routes default to Payload's `/admin` and `/api` when omitted. The existing
740
- `ExperimentSlot` and `ExperimentSlotEditor` exports remain available with these
741
- defaults. Hosts on non-default routes switch to the factories. The standalone
742
- `createExperimentSlotField({ routes })` also accepts the host's routes.
350
+ Register a stable point once; editors can run new tests on it without changing
351
+ your code. Add this option to the plugin from step 2:
743
352
 
744
- Pass the same accent as the plugin's `theme.accent`: the editor cannot read
745
- Payload config. Both use the same validator and report the same error for an
746
- invalid colour. Without an accent the editor uses the neutral palette.
353
+ ```ts
354
+ flagPoints: [{
355
+ id: 'signup-prompt', label: 'Signup prompt',
356
+ params: { heading: { type: 'text', label: 'Heading', required: true } },
357
+ defaultParams: { heading: 'Start learning' },
358
+ }]
359
+ ```
747
360
 
748
- Load the editor stylesheet inside the Puck iframe. With Tailwind v4, add the
749
- package import **after** Tailwind in the CSS that builds your editor iframe
750
- stylesheet (the stylesheet listed in payload-puck's `editorStylesheets`):
361
+ Read it with `getFlagPoint(serverRuntime, 'signup-prompt')` from `/rsc`, render
362
+ its `params`, and call `await point.expose(point.arm)` only when you deliver
363
+ that version. Bind code to the point id. For simple outcome goals, register
364
+ `eventSources: [{ name: 'signup', clientWritable: false }]` and call
365
+ `trackConversion` from `/rsc` with `eventName`, a stable `idempotencyKey` and,
366
+ when needed, `userId` or request `headers`. Editors see **signup** in the goal picker.
367
+ For existing business data, supply `metricSources` adapters with `id`, `label`,
368
+ `kind`, `keyedBy` and `query`, returning one `{ unitId, value, ts: Date }` row
369
+ per event. Revenue belongs in trusted server code, never a browser beacon.
751
370
 
752
- ```css
753
- @import "tailwindcss";
754
- @import "@delmaredigital/payload-experiments/puck/editor.css";
755
- ```
371
+ ## Client counting for viewport and pageview tests
756
372
 
757
- Alternatively, serve the exported `puck/editor.css` file from a public URL and
758
- append that URL to payload-puck's `editorStylesheets`, after the host stylesheet.
759
- The editor CSS includes its own light and dark palette, selected by
760
- `data-theme="dark"` or the `dark` class on the iframe's `<html>`. The client component's CSS import alone
761
- does not reach an iframe configured with `syncHostStyles: false`.
373
+ Pass server assignments through a provider around the rendered content:
762
374
 
763
- If Puck runs outside Payload's admin, the page needs Payload's admin styles (the plugin's tokens map onto them) and the plugin's admin styles, plus `data-theme="light"` or `data-theme="dark"` on `<html>`. Inside the admin all of this is already in place:
375
+ ```tsx
376
+ import { ExperimentsProvider, ExperimentPageview } from
377
+ '@delmaredigital/payload-experiments/client'
378
+ import { toClientAssignments } from '@delmaredigital/payload-experiments/rsc'
764
379
 
765
- ```css
766
- @import "@payloadcms/next/css";
767
- @import "@payloadcms/ui/styles.css";
768
- @import "@delmaredigital/payload-experiments/admin/theme.css";
769
- @import "@delmaredigital/payload-experiments/admin/ui.css";
380
+ <ExperimentsProvider value={toClientAssignments(assignments, { pathname })}>
381
+ <ExperimentPageview />
382
+ {children}
383
+ </ExperimentsProvider>
770
384
  ```
771
385
 
772
- Resolve the Puck data on the server with `resolveExperimentData` before
773
- rendering it. The render component shows only the selected group's slot. The
774
- editor lets you select a block experiment, edit each group's content, copy the
775
- Original into a variant, or create an experiment. It warns when edits would
776
- change content during a running test. `createExperimentSlotField` is available
777
- from `/puck` if you need the experiment picker in another custom component.
778
-
779
- The picker lists the 100 most recently updated block experiments.
780
- Viewport exposure watches a fixed, invisible **100 px sentinel**
781
- at the top of the slot and counts it when at least half is visible. The
782
- sentinel stays the same size for short and tall content.
783
-
784
- ### Tag host content with a group
785
-
786
- Add `createArmSelectField({ flagPoint: 'signup-form' })` from
787
- `@delmaredigital/payload-experiments` to a host collection's fields. By default
788
- the field is named `experimentArm`; it stores
789
- `{ experiment: String(experimentId), arm }`.
790
-
791
- Match host content on **`String(experimentId)` plus the arm key**, never on
792
- the experiment slug. The field shows group labels, leaves out the holdout,
793
- warns about tags from an earlier experiment, and lets you clear them. A running
794
- experiment shows a warning before you change its content.
795
-
796
- ### Inside the admin
797
-
798
- The experiment's edit tabs guide setup:
799
-
800
- - **What:** what you are testing and why, including the flag point when needed.
801
- - **Who:** who takes part, targeting, allocation, and space in a layer.
802
- - **Groups:** the Original, variants and optional holdout, their percentages,
803
- settings, or page paths.
804
- - **Metrics:** the main outcome, supporting metrics, guardrails, and planned
805
- duration.
806
- - **Review:** saved setup checks, placements and preview links, recent traffic,
807
- and a duration estimate when enough information is available. Apply copies
808
- the suggested duration and sensitivity into the form; save before you start.
809
-
810
- The **Results** view lets you select a round, read the verdict, compare groups,
811
- check data quality, and see results over time with content-change markers.
812
- Holdouts are measured without being compared during the test. Revenue results
813
- show uncapped totals alongside any capped decision statistic. Recompute
814
- requests fresh results. The experiment list shows status and a results summary;
815
- round history and layer usage provide context in the admin.
816
-
817
- Actions depend on the experiment's status and your permissions. **Unsaved
818
- changes block actions**; save first so they use the saved experiment.
819
-
820
- - **Start** runs a draft. **Start a new round** after stopping warns:
821
- “Starting again begins a new round; results restart.”
822
- - **Stop** says: “Everyone will see the Original. Starting again begins a new
823
- round; results restart.”
824
- - **Raise allocation** adds people without changing the group of anyone
825
- already taking part. Lowering allocation needs a new round.
826
- - **Add a metric** adds a supporting metric or guardrail to a running test.
827
- - **Decide** starts with the recommended group. Choosing the Original says
828
- “Everyone will see the Original.” Choosing a variant says everyone will see
829
- it, except an existing holdout, which keeps seeing the Original so you can
830
- measure the shipped change. When results need caution or you choose against
831
- the recommendation, enter a note and type the chosen group's label.
832
- - **Release holdout** says the holdout will see the winner too and measuring
833
- since shipping ends.
834
- - **Make it the content** copies the winner into page content or saves its
835
- settings as defaults, then archives the experiment. Failed page updates are
836
- listed with Retry. **Finish archiving** completes archiving when the content
837
- has already been updated.
838
- - **Archive** closes a stopped experiment. **Discard winner** says: “Everyone
839
- goes back to the Original. The decision note is kept.” It requires typed
840
- confirmation. For code experiments, the action confirming that the developer
841
- has hard-coded the winner also requires typed confirmation before archiving.
842
-
843
- ### Known limitations (v1)
844
-
845
- - Preview links open after a token request, so Safari's popup blocker may block
846
- them. Allow popups for the admin.
847
- - “Raise allocation” is offered even at 100%.
848
- - The metric picker in “Add a metric” lists 100 metrics.
849
- - The Puck picker lists 100 experiments.
850
- - The Puck experiment picker always calls `/api`; hosts with a custom API route
851
- see an empty picker.
852
- - Content-change markers are recorded on publish only (on save for collections
853
- without drafts).
854
- - Results cover the whole round; breakdowns by segment or dimension are not
855
- available in v1.
856
-
857
- ### Limits
858
-
859
- | Capability | Limit |
860
- |---|---|
861
- | Make it the content: localized collections | not supported yet |
862
- | Make it the content: field defaults | An update-denied field empty in both versions may publish its default because Payload applies defaults when reading. |
386
+ Use the served page's own pathname, including after a rewrite. Puck viewport
387
+ blocks count when visible; whole-page tests need `ExperimentPageview` on each
388
+ target page. The provider also enables click goals. Pass `apiRoute` if yours
389
+ isn't `/api`. Client readers need explicit `flagPoints` or `experiments`
390
+ allow-lists in `toClientAssignments`; reading alone never counts a visit.
863
391
 
864
392
  ## Theme
865
393
 
866
- The admin inherits Payload's neutral palette by default. Set one host accent:
394
+ The admin follows Payload's light and dark themes. To use your own accent, add:
867
395
 
868
396
  ```ts
869
397
  theme: { accent: '#1f3a68' }
870
398
  ```
871
399
 
872
- `theme.accent` accepts `#rgb` or `#rrggbb`; empty or undefined uses the default.
873
- The plugin derives darker hover/active fills (clamped to keep text contrast), a
874
- contrasting focus ring, and an opaque 8% subtle tint.
875
- In dark mode it raises HSL lightness when needed to reach 3:1 against the
876
- reference dark surface, then selects white or near-black foreground text.
877
- The WCAG 2.x gate requires at least 4.5:1 **foreground text on the accent fill**
878
- in each theme (the accent is used as a fill, not as page text). A failing
879
- colour is refused with its ratio. Status chips retain Payload semantic colours.
880
- `deriveAccent` is also exported from `/puck/editor` for editor integrations.
881
-
882
- ## Click goals
883
-
884
- Click goals use a goal's short name as their key (`click:<short-name>`). Add
885
- `xpGoalField()` from the root entry to a host block or array row. Its manager
886
- picker includes inline creation with "+ New click goal"; run
887
- `payload generate:importmap` to register the picker. For Puck's `xpGoal` prop,
888
- use `xpGoalPuckEditorField()` from `/puck/editor` in your editor config and
889
- `xpGoalPuckField()` from `/puck` in your render config. Run the
890
- host's `payload migrate:create`: each table using the Payload field (and its
891
- version table) gains a nullable text column. Render the value on the actual
892
- interactive element as `data-xp-goal={xpGoal}`; code can also set
893
- `data-xp-goal="join"` directly. Define a click goal with that short name first.
894
- For managers, an undefined key blocks the save through the Payload field's
895
- server validation, refreshing a cached miss immediately.
896
- Draft saves are also refused when the collection validates drafts
897
- (`versions.drafts.validate`). Editors without goal-management access see a
898
- plain text input. Goals in
899
- related documents and keys placed only in code are invisible to the content
900
- fairness check.
901
-
902
- `ExperimentsProvider` records native clicks when its server assignments arm
903
- click goals. Nested providers share one document listener; the deepest armed
904
- provider supplies the route and pageview id. Enter on links/buttons and Space
905
- on buttons count through the browser's native click. A focusable `div` needs
906
- `role="button"` and the host's own keyboard handler. Beacons use `sendBeacon`
907
- (with a keepalive fallback) so navigation/unload does not discard the click.
908
- Click and exposure beacons share `events.rateLimit`; size the adapter for both.
909
- New click goals are accepted within 30 seconds. A click goal may be primary;
910
- primary and guardrail goals must appear in every tested version or outside all
911
- of them. Secondary asymmetry warns, and results health checks published rows
912
- again on every run.
400
+ Use a hex colour with three or six digits; colours without enough contrast are refused. Pass the same `accent` to `experimentsPuckEditor` for Puck.
401
+
402
+ ## Access options
403
+
404
+ Callbacks receive `{ req }` and return a boolean or a promise of one.
405
+
406
+ | Option | Who it permits | Default |
407
+ | --- | --- | --- |
408
+ | `manage` | Create and run tests, manage goals and choose winners | Required |
409
+ | `admin` | Code tests and analysis-method changes | `manage` |
410
+ | `readResults` | Read results | `manage` |
411
+ | `readIdentities` | Read identity links | `admin` |
412
+
413
+ ## Full reference (until the docs site)
414
+
415
+ The longer option-by-option reference from before this rewrite is in [REFERENCE.md](https://github.com/delmaredigital/payload-experiments/blob/main/packages/plugin/REFERENCE.md) until the documentation site exists.
416
+
417
+ ## License
418
+
419
+ [MIT](https://github.com/delmaredigital/payload-experiments/blob/main/LICENSE).
420
+ See the [statistics engine README](https://github.com/delmaredigital/payload-experiments/blob/main/packages/stats/README.md) for the methods behind the results.