@delmaredigital/payload-experiments 0.2.0-rc.7 → 0.3.0-rc.4

package/README.md

@@ -1,5 +1,7 @@
 # @delmaredigital/payload-experiments
 
+> ⚠️ Upgrading to 0.3? For 0.3.0-rc.1, every `linkIdentity` call must supply `verified`; it now returns an outcome. Run `payload migrate:create` for the nullable `xp_identities.verified_at` column and `user_id` index, plus the release-pages columns listed in the CHANGELOG (`decision_pages_released_at` / `decision_pages_released_by` on `xp_experiments` and their `version_` counterparts on `_xp_experiments_v`). Existing identity links become unverified; see the CHANGELOG for optional backfill.
+
 > ⚠️ Upgrading to 0.2? After upgrading, run `payload generate:importmap`: the Tests surface is registered through admin.dependencies, and without it the Tests routes show a 'not installed' message instead of the Tests screen. In 0.2.0-rc.2, the Tests server views moved from `/rsc` to `/admin/rsc`, so the import map must be regenerated when upgrading from rc.1 too. The old experiments edit-form components are gone from `/admin/client`, and the `xp-experiments` admin list and edit views now redirect to the Tests surface.
 
 A/B testing for Payload CMS and Next.js, with results inside the admin and data
@@ -87,7 +89,7 @@ permission is granted only for `true`. Optional `access.admin` and
 `access.readResults` default to `manage`; `access.readIdentities` defaults to
 `admin`. Use `admin` for code experiments and changing the analysis method.
 Other plugin options are `flagPoints`, `metricSources`, `eventSources`,
-`attributeKeys`, `puck` and `events`:
+`attributeKeys`, `pages`, `puck` and `events`:
 
 - `attributeKeys` declares targeting attributes as `{ type, values? }`, with
   `type` one of `string`, `number`, `boolean`; optional string `values` supplies
@@ -261,29 +263,93 @@ Other server integrations:
   outside a Next request; a user-only server conversion can supply `userId`.
   Record revenue through trusted server code or a metric source, never a client
   beacon.
-- `linkIdentity(serverRuntime, { headers, userId })` belongs on **every**
-  user-creation and sign-in path. It links the signed anonymous cookie to the
-  opaque user id using insert-ignore on its own connection and never throws into
-  sign-in. For visitor experiments using user-keyed outcomes, credit goes to the
-  earliest-exposed linked visitor; cross-group links are reported separately.
+- `linkIdentity(serverRuntime, { headers, userId, verified })` belongs on every
+  user-creation and proving sign-in path; see Identity linking below.
 - `eraseUser(payload, { userId })` deletes the user's identities and associated
   events transactionally and returns `{ identities, events }` counts. Call it
   from the host's authorized user-erasure flow.
 - `recordContentChange({ doc, previousDoc, req, collection })` can be called
   from a host collection's `afterChange` for content tagged with the arm-select
   field; for a collection with drafts, also add `captureContentBaseline` to its
-  `beforeChange` hooks so the previously published content can be compared. Supply that hook's transaction request. Puck collections get the hook
+  `beforeChange` hooks so the previously published content can be compared.
+  Collections without drafts can also add `captureContentBaseline` for an exact
+  raw comparison; without it, the helper normalises relationships to ids. No host
+  change is required for collections without drafts. Supply that hook's transaction request. Puck collections get the hook
   automatically when enabled. Markers compare published content; drafts and
   autosaves do not add markers (collections without drafts record on save).
   Neither the helper nor the installed hooks ever throw into your save: a
   failure is logged and the content save proceeds.
 - `experimentMetadata(serverRuntime, path)` returns metadata to merge into the
   page's Next metadata: live variant targets get noindex and an Original
-  canonical; archived losing paths point canonically to the winner.
+  canonical; archived losing paths point canonically to the winner. It reads
+  the signed request context via `headers()`, so it decides from the URL the
+  visitor requested, not the rewritten route, and the Original's URL is never
+  noindex. While a test is live the Original's URL also gets a canonical to
+  itself, even when it is served with a variant's route: merge this result after
+  your own metadata so its `alternates.canonical` wins over one derived from the
+  rendered route. Call it only from a dynamic `generateMetadata`, never inside
+  `'use cache'`; `path` is the fallback when the proxy did not run.
   `experimentSitemapFilter(path, manifest)` is a synchronous build-time filter:
   pass a privately server-loaded manifest and keep paths for which it returns
   `true`, excluding live variants and archived losing paths.
 
+## Identity linking
+
+Call `linkIdentity(serverRuntime, { headers, userId, verified })` on every
+user-creation and proving sign-in path. It links the signed anonymous cookie
+to an opaque user id on its own connection and never throws into sign-in.
+`verified` is required. Pass `true` only when **this request proves, in this
+browser, that the person owns the account**: a magic-link or one-time-code
+callback, password sign-in, or OAuth callback. It does not mean "a session
+exists" or "the account's email is verified". Registration alone uses `false`.
+
+```ts
+// Registration, before proof of account ownership:
+await linkIdentity(serverRuntime, { headers, userId, verified: false })
+// After successful magic-link/code verification in this browser:
+await linkIdentity(serverRuntime, { headers, userId, verified: true })
+// After a successful password sign-in (or OAuth callback):
+await linkIdentity(serverRuntime, { headers, userId, verified: true })
+```
+
+An auth "after" hook that fires whenever a new session cookie is set also
+fires on routine session refreshes; pass `verified: true` only from proving
+sign-in paths, never from a refresh.
+
+The result is `'linked'` for an inserted link, `'verified'` for an upgraded
+unverified link, `'skipped'` for no consent, no valid unit cookie or an existing
+link with nothing to change, and `'failed'` for an error (safely logged).
+First link per browser wins; a different account never replaces it.
+
+For visitor experiments with user-keyed events, verified links are always
+eligible. Unverified links are eligible only before the first verification
+across all of that user's browsers, or indefinitely if nobody ever verifies.
+Each event goes to the earliest exposed eligible visitor. Eligible links
+spanning groups are reported as a diagnostic. User-keyed CUPED covariates use
+the same eligibility rule at each visitor's first exposure and credit one
+aggregate value to the earliest eligible visitor per user.
+
+Accepted limits:
+
+- Cross-device: register in A, verify on B, keep using A. A's events after B's
+  proof are not credited until a proving sign-in on A upgrades A. Events before
+  B's proof, including the registration funnel, remain credited. Once A is
+  upgraded, its link is verified and eligible for all its events, so the next
+  results run also credits A's events from between B's proof and the upgrade.
+- Password set before verification: a later password sign-in is proof the
+  plugin accepts. Whether unverified registration may set a password is a host
+  authentication concern.
+- Shared device: a different user's verified sign-in never relinks an already
+  linked browser and returns `'skipped'`; that user's events get no credit
+  through that browser.
+- If someone registers another person's email from their own browser and that
+  person never signs in, a later user-keyed event the host records against that
+  account (for example an offline gift) is credited to the registering browser.
+  It needs a deliberate registration of someone else's email plus an event
+  matched to that account, and it moves one unit's value, never assignment.
+  There is no time window, so recurring gifts from never-signed-in donors keep
+  counting.
+
 ## Results
 
 The host owns scheduling and authentication. Call
@@ -368,7 +434,8 @@ a user-keyed source. Visitor experiments can use a user-keyed source through
 linked identities.
 
 Optional `covariate({ units: [{ unitId, firstExposureTs }], lookbackDays })`
-returns `[{ unitId, value }]` for pre-exposure CUPED adjustment (the plugin asks for a 14-day lookback). Percentile
+returns `[{ unitId, value }]`, one aggregate value per unit for pre-exposure
+CUPED adjustment (the plugin asks for a 14-day lookback). Percentile
 revenue capping reuses `query` over the 90 days before Start; the resolved cap
 is frozen for the round. Capping affects decision statistics, while results
 also show uncapped totals. Count and binomial metrics are never capped.
@@ -389,16 +456,61 @@ are recorded with `trackConversion`. Beacon conversions for a non-writable name
 are dropped silently. Only listed names become `events:<name>` sources. They
 support visitor and user units and do not represent counts or revenue in v1.
 
+### Whole-page options
+
+| Option | Purpose |
+| --- | --- |
+| `pages.collections` | Optional array of `{ slug, pathOf, findByPath }`. `pathOf(doc)` derives a public path from the slug (used to protect live test paths); `findByPath(path, req)` resolves a published page to `{ id, published: true }`, or returns `null`. Start and readiness check every non-holdout version, including the Original. Registered collections must exist when this plugin runs. Localized publication status and field-level `beforeChange` hooks that rewrite paths after the collection guard are unsupported; derive paths from fields finalized before collection hooks. Without this option, readiness warns that pages are unchecked and Start keeps its existing behaviour. |
+
+Do not swallow errors from nested updates of guarded page collections inside host hooks: a swallowed error leaves the guard's operation frame in place for the rest of the request because Payload's Local API does not call `afterError`.
+
+For a collection with drafts enabled, use a published-only slug lookup with the
+manager's request and `overrideAccess: false`:
+
+```ts
+pages: {
+  collections: [{
+    slug: 'pages',
+    pathOf: doc => typeof doc.slug === 'string' ? `/${doc.slug}` : null,
+    findByPath: async (path, req) => {
+      const { docs } = await req.payload.find({
+        collection: 'pages', req, overrideAccess: false, draft: false, depth: 0, limit: 1,
+        where: { and: [
+          { slug: { equals: path.slice(1) } },
+          { _status: { equals: 'published' } },
+        ] },
+      })
+      return docs[0] ? { id: docs[0].id, published: true } : null
+    },
+  }],
+},
+```
+
+Adapt both callbacks for the site's homepage path. These checks run before Start;
+they do not protect pages from later deletion, unpublishing or renaming. Keep an
+archived winner published at the same address: archived test URLs still point to it.
+No migration is needed.
+
 ### `admin` options
 
 Host links and kind descriptions are optional. For a host using payload-puck's
-editor route and a `puck-templates` collection, pass:
+editor route, public pages and a `puck-templates` collection:
+
+| Option | Signature and behavior |
+| --- | --- |
+| `admin.previewUrl` | `(placement: PreviewPlacement) => string \| null \| undefined \| Promise<string \| null \| undefined>`. Receives `collection`, `id`, `title`, `published`, and optional `slug`. May be async; return `null` to hide preview for that placement. Each call has a 2 s limit, with at most 8 lookups awaited concurrently. |
+
+For example, pass:
 
 ```ts
 admin: {
   editorUrl: ({ collection, id, draft }) =>
     `/admin/puck-editor/${encodeURIComponent(collection)}/${encodeURIComponent(String(id))}`,
-  previewUrl: ({ id }) => `/preview/template/${encodeURIComponent(String(id))}`,
+  previewUrl: async ({ collection, slug }) => {
+    if (collection !== 'pages' || slug === undefined) return null
+    // Match the site's public page routing, including its homepage convention.
+    return slug === 'home' ? '/' : `/${slug}`
+  },
   descriptions: {
     block: 'Try different content on a landing page.',
     document: 'Try different versions of a donation ask.',
@@ -407,11 +519,21 @@ admin: {
 }
 ```
 
-Use your host's preview route for `previewUrl`. Preflight resolves both URL
-callbacks on the server for block placements; it passes only URL strings to the
-client. Missing or failed callbacks leave the URL unset, and the client uses the
-admin collection route when no editor URL is supplied. Kind descriptions are
-available to the server views through `config.custom.xp`.
+`PreviewPlacement` is exported from the package root and `/rsc`. Its `slug` is
+the published main row's string slug, including for a placement found in a newer
+draft with a different slug. It is absent for unpublished documents. `published`
+describes the placement's version; the callback receives no other document fields.
+An async callback can also look up the public path by `collection` and `id` using
+the host's own data access. The old `({ collection, id }) => string` form still works.
+
+Preflight resolves the URL callbacks on the server for block placements and sends
+only URL strings to the client. A null/undefined result, a non-string, an exception,
+a rejected promise or a timeout leaves preview hidden for that placement. When an
+editor URL exists, the admin offers "Edit on page"; when neither URL exists, it
+omits the row. Timed-out host work cannot be cancelled and keeps its concurrency
+slot until it settles; if all 8 slots remain occupied, further preview lookups
+are skipped for that preflight. Kind descriptions are available to the server views through
+`config.custom.xp`.
 
 A flag point can also declare `description?: string` to explain the setting below its label.
 
@@ -471,7 +593,8 @@ dashboard also includes **Tests** and **Goals** cards above its widgets. Goals
 appears only for users who can manage tests. Both additions follow the host's
 admin route and keep host-provided navigation and dashboard entries first.
 After upgrading, run `payload generate:importmap` to register the dashboard card
-component.
+component and the server nav wrapper.
+A host nav group also labelled "Tests" shares Payload's NavGroup DOM id and open-state preference with the plugin's group, so pick another label.
 
 ### Client assignments and exposure
 
@@ -524,19 +647,41 @@ configured to allow client writes; record revenue on the server.
 
 ### Puck render and editor configs
 
-Register `ExperimentSlot` from `@delmaredigital/payload-experiments/puck` in
-your render config and `ExperimentSlotEditor` from
-`@delmaredigital/payload-experiments/puck/editor` in your editor config, using
-the same `ExperimentSlot` component name:
+Register the factories from `/puck` and `/puck/editor`, using the same
+`ExperimentSlot` component name in both configs. Pass the host's Payload routes:
 
 ```ts
+import { createExperimentSlot } from '@delmaredigital/payload-experiments/puck'
+import { createExperimentSlotEditor, experimentOutlinePlugin } from '@delmaredigital/payload-experiments/puck/editor'
+
+const routes = { admin: '/manage', api: '/content-api' } // Same as Payload config.routes
+const accent = '#1f3a68' // Same as the plugin's theme.accent
+
 // Render config
-components: { ExperimentSlot }
+components: { ExperimentSlot: createExperimentSlot({ routes }) }
 
 // Editor config
-components: { ExperimentSlot: ExperimentSlotEditor }
+components: { ExperimentSlot: createExperimentSlotEditor({ routes, accent }) }
+```
+
+Add `experimentOutlinePlugin()` to Puck's `plugins` alongside this block config
+(with payload-puck, use its `plugins` option). It composes with the host's outline
+override and hides empty version slots that the block's test doesn't use. Content
+in an unused slot stays visible in the outline and gets a warning in the block.
+
+```ts
+plugins: [experimentOutlinePlugin()]
 ```
 
+Routes default to Payload's `/admin` and `/api` when omitted. The existing
+`ExperimentSlot` and `ExperimentSlotEditor` exports remain available with these
+defaults. Hosts on non-default routes switch to the factories. The standalone
+`createExperimentSlotField({ routes })` also accepts the host's routes.
+
+Pass the same accent as the plugin's `theme.accent`: the editor cannot read
+Payload config. Both use the same validator and report the same error for an
+invalid colour. Without an accent the editor uses the neutral palette.
+
 Load the editor stylesheet inside the Puck iframe. With Tailwind v4, add the
 package import **after** Tailwind in the CSS that builds your editor iframe
 stylesheet (the stylesheet listed in payload-puck's `editorStylesheets`):
@@ -552,6 +697,15 @@ The editor CSS includes its own light and dark palette, selected by
 `data-theme="dark"` or the `dark` class on the iframe's `<html>`. The client component's CSS import alone
 does not reach an iframe configured with `syncHostStyles: false`.
 
+If Puck runs outside Payload's admin, the page needs Payload's admin styles (the plugin's tokens map onto them) and the plugin's admin styles, plus `data-theme="light"` or `data-theme="dark"` on `<html>`. Inside the admin all of this is already in place:
+
+```css
+@import "@payloadcms/next/css";
+@import "@payloadcms/ui/styles.css";
+@import "@delmaredigital/payload-experiments/admin/theme.css";
+@import "@delmaredigital/payload-experiments/admin/ui.css";
+```
+
 Resolve the Puck data on the server with `resolveExperimentData` before
 rendering it. The render component shows only the selected group's slot. The
 editor lets you select a block experiment, edit each group's content, copy the
@@ -632,10 +786,32 @@ changes block actions**; save first so they use the saved experiment.
 - The Puck picker lists 100 experiments.
 - The Puck experiment picker always calls `/api`; hosts with a custom API route
   see an empty picker.
-- Making a winner permanent skips pages in collections that are localized,
-  have hidden fields, or contain fields the manager cannot read or update. Those pages
-  are listed with a note to update them by hand.
 - Content-change markers are recorded on publish only (on save for collections
   without drafts).
 - Results cover the whole round; breakdowns by segment or dimension are not
   available in v1.
+
+### Limits
+
+| Capability | Limit |
+|---|---|
+| Make it the content: localized collections | not supported yet |
+| Make it the content: field defaults | An update-denied field empty in both versions may publish its default because Payload applies defaults when reading. |
+
+## Theme
+
+The admin inherits Payload's neutral palette by default. Set one host accent:
+
+```ts
+theme: { accent: '#1f3a68' }
+```
+
+`theme.accent` accepts `#rgb` or `#rrggbb`; empty or undefined uses the default.
+The plugin derives darker hover/active fills (clamped to keep text contrast), a
+contrasting focus ring, and an opaque 8% subtle tint.
+In dark mode it raises HSL lightness when needed to reach 3:1 against the
+reference dark surface, then selects white or near-black foreground text.
+The WCAG 2.x gate requires at least 4.5:1 **foreground text on the accent fill**
+in each theme (the accent is used as a fill, not as page text). A failing
+colour is refused with its ratio. Status chips retain Payload semantic colours.
+`deriveAccent` is also exported from `/puck/editor` for editor integrations.

package/dist/admin/PreviewLinks.d.ts

@@ -1,6 +1,7 @@
 import type { DocId } from '../types/contracts.js';
 import type { PreflightResult } from '../types/endpoints.js';
-export declare function PreviewLinks({ experimentId, placements, arms }: {
+export declare function PreviewLinks({ experimentId, placements, arms, flagPoint }: {
+    flagPoint?: string;
     experimentId: DocId;
     placements: PreflightResult['placements'];
     arms: readonly {

package/dist/admin/PreviewLinks.js

@@ -1,13 +1,14 @@
 'use client';
 import { jsx as _jsx, jsxs as _jsxs } from "react/jsx-runtime";
 import { useRef, useState } from 'react';
-import { Button } from '@payloadcms/ui';
+import { Button } from '../ui/Button.js';
 import { armLabel } from './labels.js';
 import { useApi } from './useApi.js';
 import { REVIEW_COPY as C } from './copy/review.js';
-import { API_MESSAGES } from './copy/common.js';
-import { previewPages } from './previewLinks.model.js';
-export function PreviewLinks({ experimentId, placements, arms }) {
+import { TEST_PAGE } from './copy/testPage.js';
+import { API_MESSAGES, refusalMessage } from './copy/common.js';
+import { actionablePreviewPages } from './previewLinks.model.js';
+export function PreviewLinks({ experimentId, placements, arms, flagPoint }) {
     const api = useApi();
     const inFlight = useRef(false);
     const [busy, setBusy] = useState(false);
@@ -17,20 +18,37 @@ export function PreviewLinks({ experimentId, placements, arms }) {
         inFlight.current = true;
         setBusy(true);
         setError(null);
+        let win = null;
         try {
+            win = window.open('', '_blank');
+            if (!win) {
+                setError(C.previewBlocked);
+                return;
+            }
+            // Clearing opener immediately gives the same opener protection as noopener: the
+            // previewed page cannot reach window.opener. The admin URL may be sent as
+            // referrer to the host's own site (resolved against location.origin); accepted.
+            try {
+                win.opener = null;
+            } catch  {}
             const result = await api.previewToken(experimentId, arm);
+            // The person closed the blank tab while the token loaded: nothing to show or report.
+            if (win.closed) return;
             if (!result.ok) {
-                setError(result.message);
+                win.close();
+                setError(result.status === 0 ? API_MESSAGES.network : refusalMessage(result.code, result.status));
                 return;
             }
-            if (!result.data?.token) {
+            if (typeof result.data?.token !== 'string' || !result.data.token) {
+                win.close();
                 setError(API_MESSAGES.unreadable);
                 return;
             }
             const target = new URL(url, window.location.origin);
             target.searchParams.set('xp_preview', result.data.token);
-            window.open(target.href, '_blank', 'noopener,noreferrer');
+            win.location.replace(target.href);
         } catch  {
+            win?.close();
             setError(API_MESSAGES.failed);
         } finally{
             inFlight.current = false;
@@ -40,7 +58,8 @@ export function PreviewLinks({ experimentId, placements, arms }) {
     return /*#__PURE__*/ _jsxs("div", {
         className: "xp-preview-links",
         children: [
-            previewPages(placements).map((placement)=>/*#__PURE__*/ _jsxs("div", {
+            actionablePreviewPages(placements, flagPoint).map((placement)=>/*#__PURE__*/ _jsxs("div", {
+                    className: "xp-preview-row",
                     children: [
                         /*#__PURE__*/ _jsxs("p", {
                             children: [
@@ -48,19 +67,29 @@ export function PreviewLinks({ experimentId, placements, arms }) {
                                 placement.slotCount > 1 ? ` ${C.placedTimes(placement.slotCount)}` : null
                             ]
                         }),
-                        placement.url ? arms.filter((arm)=>arm.kind !== 'holdout' && arm.key !== 'holdout').map((arm)=>/*#__PURE__*/ _jsxs(Button, {
-                                disabled: busy,
-                                onClick: ()=>{
-                                    void preview(placement.url, arm.key);
-                                },
-                                children: [
-                                    C.preview,
-                                    " ",
-                                    arm.label?.trim() || armLabel(arm.key)
-                                ]
-                            }, arm.key)) : /*#__PURE__*/ _jsx(Button, {
-                            disabled: true,
-                            children: C.noPreview
+                        /*#__PURE__*/ _jsxs("div", {
+                            className: "xp-ui-button-group",
+                            children: [
+                                placement.url ? arms.filter((arm)=>arm.kind !== 'holdout' && arm.key !== 'holdout').map((arm)=>/*#__PURE__*/ _jsxs(Button, {
+                                        variant: "secondary",
+                                        size: "sm",
+                                        disabled: busy,
+                                        onClick: ()=>{
+                                            void preview(placement.url, arm.key);
+                                        },
+                                        children: [
+                                            C.preview,
+                                            " ",
+                                            arm.label?.trim() || armLabel(arm.key)
+                                        ]
+                                    }, arm.key)) : null,
+                                placement.editorUrl ? /*#__PURE__*/ _jsx(Button, {
+                                    variant: "link",
+                                    size: "sm",
+                                    href: placement.editorUrl,
+                                    children: TEST_PAGE.editOnPage
+                                }) : null
+                            ]
                         })
                     ]
                 }, placement.key)),

package/dist/admin/TestsNavLink.d.ts

@@ -1,3 +1,5 @@
 import './theme.css';
-declare function TestsNav(): import("react").JSX.Element | null;
+declare function TestsNav({ isOpen }: {
+    isOpen?: boolean;
+}): import("react").JSX.Element | null;
 export { TestsNav as TestsNavLink };

package/dist/admin/TestsNavLink.js

@@ -5,7 +5,7 @@ import { Link, NavGroup, useAuth, useConfig } from '@payloadcms/ui';
 import { formatAdminURL } from 'payload/shared';
 import { usePathname } from 'next/navigation.js';
 import { SURFACE_COPY } from './copy/surface.js';
-function TestsNav() {
+function TestsNav({ isOpen = true }) {
     const { config } = useConfig();
     const { permissions } = useAuth();
     const pathname = usePathname();
@@ -38,9 +38,9 @@ function TestsNav() {
             active: goalsActive
         }
     ];
-    // getPreference is async and can fetch; let NavGroup default to open.
     return /*#__PURE__*/ _jsx(NavGroup, {
         label: SURFACE_COPY.title,
+        isOpen: isOpen,
         children: entries.map((entry)=>{
             const props = {
                 className: 'nav__link',

package/dist/admin/actions/ConfirmDialog.d.ts

@@ -1,13 +1,16 @@
-import '../theme.css';
 import type { ReactNode } from 'react';
 import type { DocId } from '../../types/contracts.js';
 import type { ExperimentAction } from './actions.model.js';
-export declare function DialogFrame({ title, onClose, children }: {
+export declare function DialogFrame({ title, onClose, children, initialFocus, dismissible }: {
+    dismissible?: boolean;
+    initialFocus?: 'cancel';
     title: string;
     onClose: () => void;
     children: ReactNode;
-}): import("react").ReactPortal;
-export declare function ConfirmDialog({ id, action, winnerLabel, onClose }: {
+}): import("react").JSX.Element;
+export declare function ConfirmDialog({ id, action, winnerLabel, onClose, canManage, isDecided }: {
+    canManage?: boolean;
+    isDecided?: boolean;
     id: DocId;
     action: ExperimentAction;
     winnerLabel: string;