@delmaredigital/payload-experiments 0.0.0-stage → 0.1.0-rc.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +590 -2
- package/dist/access.d.ts +1 -0
- package/dist/access.js +2 -0
- package/dist/admin/AllocationField.d.ts +2 -0
- package/dist/admin/AllocationField.js +80 -0
- package/dist/admin/FlagPointSelect.d.ts +2 -0
- package/dist/admin/FlagPointSelect.js +54 -0
- package/dist/admin/LayerUsage.d.ts +2 -0
- package/dist/admin/LayerUsage.js +24 -0
- package/dist/admin/LockedFieldNotice.d.ts +2 -0
- package/dist/admin/LockedFieldNotice.js +26 -0
- package/dist/admin/MetricSourceSelect.d.ts +2 -0
- package/dist/admin/MetricSourceSelect.js +32 -0
- package/dist/admin/PhaseHistory.d.ts +2 -0
- package/dist/admin/PhaseHistory.js +76 -0
- package/dist/admin/PreviewLinks.d.ts +11 -0
- package/dist/admin/PreviewLinks.js +69 -0
- package/dist/admin/StatusCell.d.ts +3 -0
- package/dist/admin/StatusCell.js +19 -0
- package/dist/admin/SummaryCell.d.ts +2 -0
- package/dist/admin/SummaryCell.js +32 -0
- package/dist/admin/actions/AddMetricDialog.d.ts +6 -0
- package/dist/admin/actions/AddMetricDialog.js +138 -0
- package/dist/admin/actions/AllocateDialog.d.ts +6 -0
- package/dist/admin/actions/AllocateDialog.js +81 -0
- package/dist/admin/actions/ConfirmDialog.d.ts +14 -0
- package/dist/admin/actions/ConfirmDialog.js +192 -0
- package/dist/admin/actions/DecideDialog.d.ts +7 -0
- package/dist/admin/actions/DecideDialog.js +173 -0
- package/dist/admin/actions/ExperimentActions.d.ts +3 -0
- package/dist/admin/actions/ExperimentActions.js +110 -0
- package/dist/admin/actions/MakeContentResult.d.ts +6 -0
- package/dist/admin/actions/MakeContentResult.js +50 -0
- package/dist/admin/actions/actions.model.d.ts +37 -0
- package/dist/admin/actions/actions.model.js +71 -0
- package/dist/admin/actions/decide.model.d.ts +21 -0
- package/dist/admin/actions/decide.model.js +38 -0
- package/dist/admin/allocation.model.d.ts +20 -0
- package/dist/admin/allocation.model.js +38 -0
- package/dist/admin/api.d.ts +32 -0
- package/dist/admin/api.js +95 -0
- package/dist/admin/armSelect/ArmSelectField.d.ts +2 -0
- package/dist/admin/armSelect/ArmSelectField.js +185 -0
- package/dist/admin/armSelect/armSelect.model.d.ts +30 -0
- package/dist/admin/armSelect/armSelect.model.js +20 -0
- package/dist/admin/arms/ArmsField.d.ts +2 -0
- package/dist/admin/arms/ArmsField.js +310 -0
- package/dist/admin/arms/ParamsEditor.d.ts +16 -0
- package/dist/admin/arms/ParamsEditor.js +79 -0
- package/dist/admin/arms/arms.model.d.ts +21 -0
- package/dist/admin/arms/arms.model.js +153 -0
- package/dist/admin/arms/pageTarget.model.d.ts +4 -0
- package/dist/admin/arms/pageTarget.model.js +19 -0
- package/dist/admin/arms/params.model.d.ts +7 -0
- package/dist/admin/arms/params.model.js +35 -0
- package/dist/admin/copy/actions.d.ts +38 -0
- package/dist/admin/copy/actions.js +38 -0
- package/dist/admin/copy/armSelect.d.ts +11 -0
- package/dist/admin/copy/armSelect.js +11 -0
- package/dist/admin/copy/arms.d.ts +37 -0
- package/dist/admin/copy/arms.js +47 -0
- package/dist/admin/copy/common.d.ts +20 -0
- package/dist/admin/copy/common.js +27 -0
- package/dist/admin/copy/history.d.ts +9 -0
- package/dist/admin/copy/history.js +9 -0
- package/dist/admin/copy/metrics.d.ts +9 -0
- package/dist/admin/copy/metrics.js +9 -0
- package/dist/admin/copy/results.d.ts +78 -0
- package/dist/admin/copy/results.js +81 -0
- package/dist/admin/copy/review.d.ts +33 -0
- package/dist/admin/copy/review.js +127 -0
- package/dist/admin/copy/status.d.ts +8 -0
- package/dist/admin/copy/status.js +8 -0
- package/dist/admin/copy/tabs.d.ts +7 -0
- package/dist/admin/copy/tabs.js +7 -0
- package/dist/admin/copy/who.d.ts +36 -0
- package/dist/admin/copy/who.js +36 -0
- package/dist/admin/labels.d.ts +45 -0
- package/dist/admin/labels.js +53 -0
- package/dist/admin/layerUsage.model.d.ts +15 -0
- package/dist/admin/layerUsage.model.js +18 -0
- package/dist/admin/lockedNotice.model.d.ts +3 -0
- package/dist/admin/lockedNotice.model.js +18 -0
- package/dist/admin/metricSource.model.d.ts +7 -0
- package/dist/admin/metricSource.model.js +11 -0
- package/dist/admin/paths.d.ts +16 -0
- package/dist/admin/paths.js +16 -0
- package/dist/admin/permissions.model.d.ts +10 -0
- package/dist/admin/permissions.model.js +4 -0
- package/dist/admin/phaseHistory.model.d.ts +19 -0
- package/dist/admin/phaseHistory.model.js +19 -0
- package/dist/admin/reload.d.ts +1 -0
- package/dist/admin/reload.js +1 -0
- package/dist/admin/results/ArmTable.d.ts +8 -0
- package/dist/admin/results/ArmTable.js +112 -0
- package/dist/admin/results/CappedCell.d.ts +4 -0
- package/dist/admin/results/CappedCell.js +24 -0
- package/dist/admin/results/CiChart.d.ts +10 -0
- package/dist/admin/results/CiChart.js +178 -0
- package/dist/admin/results/DataQualityPanel.d.ts +4 -0
- package/dist/admin/results/DataQualityPanel.js +40 -0
- package/dist/admin/results/ResultsView.d.ts +3 -0
- package/dist/admin/results/ResultsView.js +312 -0
- package/dist/admin/results/VerdictCard.d.ts +9 -0
- package/dist/admin/results/VerdictCard.js +49 -0
- package/dist/admin/results/chart.model.d.ts +32 -0
- package/dist/admin/results/chart.model.js +52 -0
- package/dist/admin/results/format.d.ts +10 -0
- package/dist/admin/results/format.js +55 -0
- package/dist/admin/results/results.model.d.ts +105 -0
- package/dist/admin/results/results.model.js +175 -0
- package/dist/admin/review/ReviewPanel.d.ts +2 -0
- package/dist/admin/review/ReviewPanel.js +268 -0
- package/dist/admin/review/review.model.d.ts +31 -0
- package/dist/admin/review/review.model.js +40 -0
- package/dist/admin/status.model.d.ts +16 -0
- package/dist/admin/status.model.js +17 -0
- package/dist/admin/styles.css +10 -0
- package/dist/admin/styles.d.ts +1 -0
- package/dist/admin/styles.js +2 -0
- package/dist/admin/summaryBatcher.d.ts +8 -0
- package/dist/admin/summaryBatcher.js +33 -0
- package/dist/admin/targeting/TargetingAttributeField.d.ts +2 -0
- package/dist/admin/targeting/TargetingAttributeField.js +31 -0
- package/dist/admin/targeting/TargetingOpField.d.ts +2 -0
- package/dist/admin/targeting/TargetingOpField.js +45 -0
- package/dist/admin/targeting/TargetingValueField.d.ts +2 -0
- package/dist/admin/targeting/TargetingValueField.js +142 -0
- package/dist/admin/targeting/targeting.model.d.ts +18 -0
- package/dist/admin/targeting/targeting.model.js +48 -0
- package/dist/admin/useApi.d.ts +20 -0
- package/dist/admin/useApi.js +10 -0
- package/dist/admin/useLayerUsage.d.ts +15 -0
- package/dist/admin/useLayerUsage.js +92 -0
- package/dist/client/ExperimentPageview.d.ts +1 -0
- package/dist/client/ExperimentPageview.js +27 -0
- package/dist/client/ExperimentsProvider.d.ts +6 -0
- package/dist/client/ExperimentsProvider.js +22 -0
- package/dist/client/ViewportExposure.d.ts +9 -0
- package/dist/client/ViewportExposure.js +69 -0
- package/dist/client/beacon.d.ts +17 -0
- package/dist/client/beacon.js +71 -0
- package/dist/client/context.d.ts +2 -0
- package/dist/client/context.js +2 -0
- package/dist/client/idempotency.d.ts +2 -0
- package/dist/client/idempotency.js +13 -0
- package/dist/client/trackConversionBeacon.d.ts +3 -0
- package/dist/client/trackConversionBeacon.js +16 -0
- package/dist/client/useExperiment.d.ts +5 -0
- package/dist/client/useExperiment.js +27 -0
- package/dist/client/useFlagPoint.d.ts +9 -0
- package/dist/client/useFlagPoint.js +32 -0
- package/dist/collections/access.d.ts +21 -0
- package/dist/collections/access.js +25 -0
- package/dist/collections/events.d.ts +3 -0
- package/dist/collections/events.js +117 -0
- package/dist/collections/experiments/armInvariants.d.ts +12 -0
- package/dist/collections/experiments/armInvariants.js +89 -0
- package/dist/collections/experiments/config.d.ts +3 -0
- package/dist/collections/experiments/config.js +574 -0
- package/dist/collections/experiments/createDefaults.d.ts +2 -0
- package/dist/collections/experiments/createDefaults.js +7 -0
- package/dist/collections/experiments/deleteGuard.d.ts +3 -0
- package/dist/collections/experiments/deleteGuard.js +8 -0
- package/dist/collections/experiments/editability.d.ts +4 -0
- package/dist/collections/experiments/editability.js +35 -0
- package/dist/collections/experiments/flagControlParams.d.ts +2 -0
- package/dist/collections/experiments/flagControlParams.js +32 -0
- package/dist/collections/experiments/lockHook.d.ts +2 -0
- package/dist/collections/experiments/lockHook.js +134 -0
- package/dist/collections/identities.d.ts +3 -0
- package/dist/collections/identities.js +30 -0
- package/dist/collections/index.d.ts +6 -0
- package/dist/collections/index.js +24 -0
- package/dist/collections/layers.d.ts +3 -0
- package/dist/collections/layers.js +77 -0
- package/dist/collections/metrics.d.ts +3 -0
- package/dist/collections/metrics.js +115 -0
- package/dist/collections/phases.d.ts +3 -0
- package/dist/collections/phases.js +59 -0
- package/dist/collections/serialize.d.ts +5 -0
- package/dist/collections/serialize.js +40 -0
- package/dist/collections/snapshots.d.ts +3 -0
- package/dist/collections/snapshots.js +47 -0
- package/dist/constants.d.ts +7 -0
- package/dist/constants.js +7 -0
- package/dist/db/adapter.d.ts +3 -0
- package/dist/db/adapter.js +9 -0
- package/dist/db/appends.d.ts +58 -0
- package/dist/db/appends.js +116 -0
- package/dist/db/delete.d.ts +17 -0
- package/dist/db/delete.js +35 -0
- package/dist/db/erase.d.ts +12 -0
- package/dist/db/erase.js +55 -0
- package/dist/db/events.d.ts +21 -0
- package/dist/db/events.js +100 -0
- package/dist/db/executor.d.ts +10 -0
- package/dist/db/executor.js +37 -0
- package/dist/db/identities.d.ts +11 -0
- package/dist/db/identities.js +29 -0
- package/dist/db/locks.d.ts +15 -0
- package/dist/db/locks.js +23 -0
- package/dist/db/manifest.d.ts +8 -0
- package/dist/db/manifest.js +8 -0
- package/dist/db/names.d.ts +271 -0
- package/dist/db/names.js +270 -0
- package/dist/db/pgExecutor.d.ts +41 -0
- package/dist/db/pgExecutor.js +250 -0
- package/dist/db/results.d.ts +53 -0
- package/dist/db/results.js +103 -0
- package/dist/db/slices.d.ts +18 -0
- package/dist/db/slices.js +51 -0
- package/dist/db/snapshots.d.ts +47 -0
- package/dist/db/snapshots.js +89 -0
- package/dist/db/sql.d.ts +5 -0
- package/dist/db/sql.js +5 -0
- package/dist/db/state.d.ts +12 -0
- package/dist/db/state.js +27 -0
- package/dist/db/sweep.d.ts +13 -0
- package/dist/db/sweep.js +90 -0
- package/dist/db/txid.d.ts +7 -0
- package/dist/db/txid.js +12 -0
- package/dist/define.d.ts +3 -0
- package/dist/define.js +6 -0
- package/dist/endpoints/events.d.ts +7 -0
- package/dist/endpoints/events.js +98 -0
- package/dist/endpoints/index.d.ts +18 -0
- package/dist/endpoints/index.js +160 -0
- package/dist/endpoints/manifest.d.ts +7 -0
- package/dist/endpoints/manifest.js +30 -0
- package/dist/endpoints/preflight.d.ts +8 -0
- package/dist/endpoints/preflight.js +25 -0
- package/dist/endpoints/previewToken.d.ts +7 -0
- package/dist/endpoints/previewToken.js +54 -0
- package/dist/endpoints/recompute.d.ts +7 -0
- package/dist/endpoints/recompute.js +41 -0
- package/dist/endpoints/results.d.ts +24 -0
- package/dist/endpoints/results.js +136 -0
- package/dist/endpoints/summary.d.ts +7 -0
- package/dist/endpoints/summary.js +40 -0
- package/dist/endpoints/transitions.d.ts +42 -0
- package/dist/endpoints/transitions.js +154 -0
- package/dist/errors.d.ts +27 -0
- package/dist/errors.js +33 -0
- package/dist/events/body.d.ts +7 -0
- package/dist/events/body.js +43 -0
- package/dist/events/dimensions.d.ts +4 -0
- package/dist/events/dimensions.js +51 -0
- package/dist/events/preview.d.ts +2 -0
- package/dist/events/preview.js +9 -0
- package/dist/events/request.d.ts +3 -0
- package/dist/events/request.js +14 -0
- package/dist/events/rows.d.ts +7 -0
- package/dist/events/rows.js +65 -0
- package/dist/exports/adminClient.d.ts +16 -0
- package/dist/exports/adminClient.js +17 -0
- package/dist/exports/client.d.ts +7 -0
- package/dist/exports/client.js +7 -0
- package/dist/exports/index.d.ts +4 -0
- package/dist/exports/index.js +3 -0
- package/dist/exports/proxy.d.ts +4 -0
- package/dist/exports/proxy.js +3 -0
- package/dist/exports/puck.d.ts +4 -0
- package/dist/exports/puck.js +3 -0
- package/dist/exports/puckEditor.d.ts +1 -0
- package/dist/exports/puckEditor.js +2 -0
- package/dist/exports/rsc.d.ts +13 -0
- package/dist/exports/rsc.js +13 -0
- package/dist/exports/runtimeProxy.d.ts +4 -0
- package/dist/exports/runtimeProxy.js +2 -0
- package/dist/exports/runtimeServer.d.ts +2 -0
- package/dist/exports/runtimeServer.js +1 -0
- package/dist/fields/armSelect.d.ts +7 -0
- package/dist/fields/armSelect.js +33 -0
- package/dist/globals/state.d.ts +3 -0
- package/dist/globals/state.js +31 -0
- package/dist/hooks/contentChanges.d.ts +3 -0
- package/dist/hooks/contentChanges.js +10 -0
- package/dist/hooks/hostHook.d.ts +5 -0
- package/dist/hooks/hostHook.js +20 -0
- package/dist/index.d.ts +1 -0
- package/dist/index.js +4 -0
- package/dist/manifest/cache.d.ts +10 -0
- package/dist/manifest/cache.js +51 -0
- package/dist/manifest/compile.d.ts +63 -0
- package/dist/manifest/compile.js +237 -0
- package/dist/onInit.d.ts +5 -0
- package/dist/onInit.js +13 -0
- package/dist/options/fromReq.d.ts +3 -0
- package/dist/options/fromReq.js +5 -0
- package/dist/options.d.ts +9 -0
- package/dist/options.js +107 -0
- package/dist/plugin.d.ts +7 -0
- package/dist/plugin.js +62 -0
- package/dist/proxy/apply.d.ts +3 -0
- package/dist/proxy/apply.js +5 -0
- package/dist/proxy/consentFallback.d.ts +5 -0
- package/dist/proxy/consentFallback.js +27 -0
- package/dist/proxy/control.d.ts +6 -0
- package/dist/proxy/control.js +36 -0
- package/dist/proxy/createProxy.d.ts +3 -0
- package/dist/proxy/createProxy.js +15 -0
- package/dist/proxy/pageRewrite.d.ts +4 -0
- package/dist/proxy/pageRewrite.js +41 -0
- package/dist/proxy/resolve.d.ts +13 -0
- package/dist/proxy/resolve.js +163 -0
- package/dist/proxy/skip.d.ts +1 -0
- package/dist/proxy/skip.js +9 -0
- package/dist/puck/ExperimentPicker.d.ts +5 -0
- package/dist/puck/ExperimentPicker.js +88 -0
- package/dist/puck/ExperimentSlot.d.ts +3 -0
- package/dist/puck/ExperimentSlot.js +16 -0
- package/dist/puck/copy.d.ts +14 -0
- package/dist/puck/copy.js +14 -0
- package/dist/puck/editor/CreateExperimentDrawer.d.ts +5 -0
- package/dist/puck/editor/CreateExperimentDrawer.js +120 -0
- package/dist/puck/editor/ExperimentSlotEditor.d.ts +3 -0
- package/dist/puck/editor/ExperimentSlotEditor.js +225 -0
- package/dist/puck/editor/cloneSlot.d.ts +3 -0
- package/dist/puck/editor/cloneSlot.js +41 -0
- package/dist/puck/editor/copy.d.ts +25 -0
- package/dist/puck/editor/copy.js +25 -0
- package/dist/puck/editor/editor.model.d.ts +16 -0
- package/dist/puck/editor/editor.model.js +33 -0
- package/dist/puck/experimentSlotField.d.ts +5 -0
- package/dist/puck/experimentSlotField.js +13 -0
- package/dist/puck/slot.model.d.ts +2 -0
- package/dist/puck/slot.model.js +11 -0
- package/dist/puck/slotConfig.d.ts +5 -0
- package/dist/puck/slotConfig.js +25 -0
- package/dist/results/aggregate.d.ts +34 -0
- package/dist/results/aggregate.js +66 -0
- package/dist/results/analyse.d.ts +79 -0
- package/dist/results/analyse.js +378 -0
- package/dist/results/cuped.d.ts +16 -0
- package/dist/results/cuped.js +31 -0
- package/dist/results/decideGuard.d.ts +3 -0
- package/dist/results/decideGuard.js +8 -0
- package/dist/results/guardrails.d.ts +17 -0
- package/dist/results/guardrails.js +10 -0
- package/dist/results/history.d.ts +13 -0
- package/dist/results/history.js +39 -0
- package/dist/results/identityCredit.d.ts +16 -0
- package/dist/results/identityCredit.js +46 -0
- package/dist/results/power.d.ts +15 -0
- package/dist/results/power.js +60 -0
- package/dist/results/rolloutUnits.d.ts +17 -0
- package/dist/results/rolloutUnits.js +26 -0
- package/dist/results/snapshot.d.ts +9 -0
- package/dist/results/snapshot.js +154 -0
- package/dist/results/srm.d.ts +9 -0
- package/dist/results/srm.js +22 -0
- package/dist/results/verdict.d.ts +82 -0
- package/dist/results/verdict.js +85 -0
- package/dist/runtime/assign.d.ts +23 -0
- package/dist/runtime/assign.js +104 -0
- package/dist/runtime/bot.d.ts +1 -0
- package/dist/runtime/bot.js +7 -0
- package/dist/runtime/cookie.d.ts +17 -0
- package/dist/runtime/cookie.js +35 -0
- package/dist/runtime/eligibility.d.ts +22 -0
- package/dist/runtime/eligibility.js +20 -0
- package/dist/runtime/header.d.ts +17 -0
- package/dist/runtime/header.js +97 -0
- package/dist/runtime/loaders.d.ts +17 -0
- package/dist/runtime/loaders.js +29 -0
- package/dist/runtime/normalisePath.d.ts +7 -0
- package/dist/runtime/normalisePath.js +15 -0
- package/dist/runtime/previewToken.d.ts +9 -0
- package/dist/runtime/previewToken.js +23 -0
- package/dist/runtime/proxyRuntime.d.ts +2 -0
- package/dist/runtime/proxyRuntime.js +22 -0
- package/dist/runtime/seed.d.ts +2 -0
- package/dist/runtime/seed.js +4 -0
- package/dist/runtime/signing.d.ts +3 -0
- package/dist/runtime/signing.js +16 -0
- package/dist/runtime/targeting.d.ts +6 -0
- package/dist/runtime/targeting.js +9 -0
- package/dist/server/context.d.ts +10 -0
- package/dist/server/context.js +59 -0
- package/dist/server/eraseUser.d.ts +7 -0
- package/dist/server/eraseUser.js +4 -0
- package/dist/server/getAssignments.d.ts +9 -0
- package/dist/server/getAssignments.js +227 -0
- package/dist/server/getFlagPoint.d.ts +11 -0
- package/dist/server/getFlagPoint.js +36 -0
- package/dist/server/headerRevision.d.ts +6 -0
- package/dist/server/headerRevision.js +10 -0
- package/dist/server/linkIdentity.d.ts +9 -0
- package/dist/server/linkIdentity.js +34 -0
- package/dist/server/metadata.d.ts +7 -0
- package/dist/server/metadata.js +24 -0
- package/dist/server/previewDraft.d.ts +7 -0
- package/dist/server/previewDraft.js +51 -0
- package/dist/server/recordContentChange.d.ts +7 -0
- package/dist/server/recordContentChange.js +150 -0
- package/dist/server/recordExposures.d.ts +7 -0
- package/dist/server/recordExposures.js +56 -0
- package/dist/server/request.d.ts +7 -0
- package/dist/server/request.js +39 -0
- package/dist/server/resolveExperimentData.d.ts +6 -0
- package/dist/server/resolveExperimentData.js +64 -0
- package/dist/server/runResults.d.ts +22 -0
- package/dist/server/runResults.js +307 -0
- package/dist/server/serverRuntime.d.ts +7 -0
- package/dist/server/serverRuntime.js +20 -0
- package/dist/server/sitemap.d.ts +3 -0
- package/dist/server/sitemap.js +6 -0
- package/dist/server/toClientAssignments.d.ts +7 -0
- package/dist/server/toClientAssignments.js +56 -0
- package/dist/server/trackConversion.d.ts +12 -0
- package/dist/server/trackConversion.js +69 -0
- package/dist/targeting/ops.d.ts +8 -0
- package/dist/targeting/ops.js +58 -0
- package/dist/transitions/addMetric.d.ts +16 -0
- package/dist/transitions/addMetric.js +126 -0
- package/dist/transitions/allocate.d.ts +9 -0
- package/dist/transitions/allocate.js +84 -0
- package/dist/transitions/archive.d.ts +9 -0
- package/dist/transitions/archive.js +46 -0
- package/dist/transitions/checkStart.d.ts +61 -0
- package/dist/transitions/checkStart.js +104 -0
- package/dist/transitions/confirmPhrases.d.ts +5 -0
- package/dist/transitions/confirmPhrases.js +18 -0
- package/dist/transitions/decide.d.ts +9 -0
- package/dist/transitions/decide.js +138 -0
- package/dist/transitions/deleteExperiment.d.ts +5 -0
- package/dist/transitions/deleteExperiment.js +66 -0
- package/dist/transitions/kernel.d.ts +31 -0
- package/dist/transitions/kernel.js +69 -0
- package/dist/transitions/makeContent.d.ts +20 -0
- package/dist/transitions/makeContent.js +411 -0
- package/dist/transitions/phaseRecord.d.ts +27 -0
- package/dist/transitions/phaseRecord.js +81 -0
- package/dist/transitions/placements.d.ts +26 -0
- package/dist/transitions/placements.js +89 -0
- package/dist/transitions/preflightData.d.ts +21 -0
- package/dist/transitions/preflightData.js +317 -0
- package/dist/transitions/releaseHoldout.d.ts +8 -0
- package/dist/transitions/releaseHoldout.js +30 -0
- package/dist/transitions/resolveCaps.d.ts +28 -0
- package/dist/transitions/resolveCaps.js +68 -0
- package/dist/transitions/start.d.ts +26 -0
- package/dist/transitions/start.js +220 -0
- package/dist/transitions/stop.d.ts +8 -0
- package/dist/transitions/stop.js +25 -0
- package/dist/types/admin.d.ts +20 -0
- package/dist/types/admin.js +1 -0
- package/dist/types/assignments.d.ts +57 -0
- package/dist/types/assignments.js +1 -0
- package/dist/types/contracts.d.ts +141 -0
- package/dist/types/contracts.js +1 -0
- package/dist/types/endpoints.d.ts +135 -0
- package/dist/types/endpoints.js +1 -0
- package/dist/types/index.d.ts +8 -0
- package/dist/types/index.js +8 -0
- package/dist/types/manifest.d.ts +42 -0
- package/dist/types/manifest.js +1 -0
- package/dist/types/puck.d.ts +23 -0
- package/dist/types/puck.js +1 -0
- package/dist/types/runtime.d.ts +59 -0
- package/dist/types/runtime.js +1 -0
- package/dist/types/snapshot.d.ts +117 -0
- package/dist/types/snapshot.js +1 -0
- package/package.json +138 -3
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Delmare Digital
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,591 @@
|
|
|
1
|
-
#
|
|
1
|
+
# @delmaredigital/payload-experiments
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A/B testing for Payload CMS and Next.js, with results inside the admin and data
|
|
4
|
+
in your own Postgres database. Editors can test blocks, whole pages and
|
|
5
|
+
registered flag points, choose a winner, and make it the permanent content.
|
|
6
|
+
Results count people who actually saw the content, with checks for traffic
|
|
7
|
+
imbalances and metrics that must not get worse. Puck support is optional.
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
pnpm add @delmaredigital/payload-experiments
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## Setup
|
|
14
|
+
|
|
15
|
+
Requires Node.js `>=22.23.4`, Next.js App Router, and Postgres through
|
|
16
|
+
`@payloadcms/db-postgres` with transactions enabled. Other database adapters
|
|
17
|
+
are not supported. Install the peers used by your host:
|
|
18
|
+
|
|
19
|
+
| Peer | Range |
|
|
20
|
+
| --- | --- |
|
|
21
|
+
| `payload`, `@payloadcms/db-postgres`, `@payloadcms/next`, `@payloadcms/ui` | `>=3.90.0 <4` |
|
|
22
|
+
| `next` | `>=16.4.1 <17` |
|
|
23
|
+
| `react`, `react-dom` | `>=19.2.1 <20` |
|
|
24
|
+
| `pg` | `>=8.13.0 <9` |
|
|
25
|
+
| `@puckeditor/core` (optional, for Puck integration) | `>=0.24.0` |
|
|
26
|
+
|
|
27
|
+
Create one proxy runtime, then a server runtime shared by your Payload config
|
|
28
|
+
and server readers. Keep Payload imports lazy so the proxy stays independent
|
|
29
|
+
of Payload, React and Puck:
|
|
30
|
+
|
|
31
|
+
```ts
|
|
32
|
+
// experiments/proxyRuntime.ts
|
|
33
|
+
import {
|
|
34
|
+
defineExperimentsProxyRuntime,
|
|
35
|
+
createPostgresManifestLoader,
|
|
36
|
+
} from '@delmaredigital/payload-experiments/runtime/proxy'
|
|
37
|
+
import { manifestPool } from '@/db/manifestPool' // host-owned pg Pool
|
|
38
|
+
|
|
39
|
+
export const proxyRuntime = defineExperimentsProxyRuntime({
|
|
40
|
+
secret: process.env.XP_SECRET!,
|
|
41
|
+
apiRoute: '/api', // match Payload's config.routes.api
|
|
42
|
+
loadManifest: createPostgresManifestLoader({ pool: manifestPool }),
|
|
43
|
+
})
|
|
44
|
+
|
|
45
|
+
// experiments/serverRuntime.ts
|
|
46
|
+
import { defineExperimentsServerRuntime } from
|
|
47
|
+
'@delmaredigital/payload-experiments/runtime/server'
|
|
48
|
+
import { proxyRuntime } from './proxyRuntime'
|
|
49
|
+
|
|
50
|
+
export const serverRuntime = defineExperimentsServerRuntime(proxyRuntime, {
|
|
51
|
+
getPayload: async () => {
|
|
52
|
+
const { getPayload } = await import('payload')
|
|
53
|
+
const { default: config } = await import('@payload-config')
|
|
54
|
+
return getPayload({ config })
|
|
55
|
+
},
|
|
56
|
+
})
|
|
57
|
+
|
|
58
|
+
// In payload.config.ts
|
|
59
|
+
import { createExperimentsPlugin } from '@delmaredigital/payload-experiments'
|
|
60
|
+
import { serverRuntime } from '@/experiments/serverRuntime'
|
|
61
|
+
import { canManageExperiments } from '@/access/experiments' // host permission check
|
|
62
|
+
|
|
63
|
+
// Add to buildConfig({ plugins: [...] })
|
|
64
|
+
createExperimentsPlugin({
|
|
65
|
+
runtime: serverRuntime,
|
|
66
|
+
access: { manage: canManageExperiments },
|
|
67
|
+
puck: {
|
|
68
|
+
enabled: true,
|
|
69
|
+
collections: ['pages', { slug: 'templates', field: 'content' }],
|
|
70
|
+
},
|
|
71
|
+
})
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
`secret` is required on `defineExperimentsProxyRuntime`, not a top-level plugin
|
|
75
|
+
option. Use a stable, private secret shared by all instances. For rotation, pass
|
|
76
|
+
an array with the newest secret first; older entries verify existing signatures.
|
|
77
|
+
Set a finite `connectionTimeoutMillis` on the host's manifest pool to bound
|
|
78
|
+
connection acquisition. `createPostgresManifestLoader` also accepts `schemaName`
|
|
79
|
+
when Payload uses a custom Postgres schema. It reads the private compiled
|
|
80
|
+
manifest directly, avoiding HTTP calls back into your deployment.
|
|
81
|
+
|
|
82
|
+
`createExperimentsPlugin` requires `runtime` and `access.manage`. The permission
|
|
83
|
+
callback receives `{ req }` and must return a boolean (or a promise of one);
|
|
84
|
+
permission is granted only for `true`. Optional `access.admin` and
|
|
85
|
+
`access.readResults` default to `manage`; `access.readIdentities` defaults to
|
|
86
|
+
`admin`. Use `admin` for code experiments and changing the analysis method.
|
|
87
|
+
Other plugin options are `flagPoints`, `metricSources`, `eventSources`,
|
|
88
|
+
`attributeKeys`, `puck` and `events`:
|
|
89
|
+
|
|
90
|
+
- `attributeKeys` declares targeting attributes as `{ type, values? }`, with
|
|
91
|
+
`type` one of `string`, `number`, `boolean`; optional string `values` supplies
|
|
92
|
+
an admin select. Pass only non-identifying attributes.
|
|
93
|
+
- `puck.collections` registers existing host collections for placement discovery,
|
|
94
|
+
content-change markers and making winners permanent. A slug string uses the
|
|
95
|
+
`puckData` JSON field; `{ slug, field }` names another field. `puck.enabled`
|
|
96
|
+
defaults to `true` when this list is non-empty; set it explicitly in TypeScript
|
|
97
|
+
config (the current public type requires it). `false` disables the installed
|
|
98
|
+
content-change hook.
|
|
99
|
+
- `events.retentionDays` defaults to `180`; `events.maxBodyBytes` to `8192`.
|
|
100
|
+
Both must be positive integers. `events.rateLimit` is an optional host adapter,
|
|
101
|
+
called with `{ req, unitId }`; return `true` to permit or `false` to drop.
|
|
102
|
+
- `events.dimensions` (optional, default none) names up to 8 request attributes
|
|
103
|
+
to record on each exposure for later analysis, for example `['loggedIn']`.
|
|
104
|
+
Values come only from the attributes the server computes for that request
|
|
105
|
+
(`path`, `loggedIn`, `staff` and your `getUnit` attributes), never from what a
|
|
106
|
+
browser sends; only strings (first 64 characters), numbers and booleans are
|
|
107
|
+
kept. Names that identify a person (such as `userId`, `email` or `ip`) are
|
|
108
|
+
refused at startup, but whether an attribute you add yourself identifies
|
|
109
|
+
someone is your responsibility. For client-reported exposures `path` is the
|
|
110
|
+
page path from the request's `Referer`. Reporting by dimension is not yet
|
|
111
|
+
available.
|
|
112
|
+
Supply a shared limiter suitable for your deployment; there is no built-in
|
|
113
|
+
in-memory limiter.
|
|
114
|
+
|
|
115
|
+
The plugin adds `xp-experiments`, `xp-phases`, `xp-layers`, `xp-metrics`,
|
|
116
|
+
`xp-events`, `xp-identities`, `xp-results-snapshots` and the `xp-state` global.
|
|
117
|
+
The host owns their migrations. After registering the plugin, generate and
|
|
118
|
+
review a migration, then apply it in the host:
|
|
119
|
+
|
|
120
|
+
```sh
|
|
121
|
+
pnpm exec payload migrate:create
|
|
122
|
+
pnpm exec payload migrate
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## Next.js proxy
|
|
126
|
+
|
|
127
|
+
For a standalone proxy, export the wrapper from your host's `proxy.ts`:
|
|
128
|
+
|
|
129
|
+
```ts
|
|
130
|
+
import { createExperimentProxy } from '@delmaredigital/payload-experiments/proxy'
|
|
131
|
+
import { proxyRuntime } from '@/experiments/proxyRuntime'
|
|
132
|
+
|
|
133
|
+
export const proxy = createExperimentProxy(proxyRuntime)
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
To compose with other proxy logic, call `resolveExperiments` and forward its
|
|
137
|
+
request headers on **both** the normal response and any page rewrite. Then call
|
|
138
|
+
`applyExperiments` to attach response cookies and headers:
|
|
139
|
+
|
|
140
|
+
```ts
|
|
141
|
+
import { NextResponse, type NextRequest } from 'next/server'
|
|
142
|
+
import { resolveExperiments, applyExperiments } from
|
|
143
|
+
'@delmaredigital/payload-experiments/proxy'
|
|
144
|
+
import { proxyRuntime } from '@/experiments/proxyRuntime'
|
|
145
|
+
|
|
146
|
+
export async function proxy(request: NextRequest) {
|
|
147
|
+
const result = await resolveExperiments(request, proxyRuntime)
|
|
148
|
+
const options = { request: { headers: result.requestHeaders } }
|
|
149
|
+
const response = result.rewrite
|
|
150
|
+
? NextResponse.rewrite(result.rewrite, options)
|
|
151
|
+
: NextResponse.next(options)
|
|
152
|
+
return applyExperiments(response, result)
|
|
153
|
+
}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
The default `xp_id` cookie is a signed opaque visitor id: HttpOnly,
|
|
157
|
+
SameSite=Lax, Secure on HTTPS, path `/`, lifetime 365 days. Runtime `cookie`
|
|
158
|
+
options are `name`, `maxAgeDays`, `domain` and `consent`. A newly minted cookie
|
|
159
|
+
is forwarded into the same request, so server readers see it immediately.
|
|
160
|
+
Inbound `x-xp-*` headers are removed before any skip check. The proxy writes
|
|
161
|
+
signed `x-xp-assignments` (at most 1 KB) and `x-xp-request` (method, path/search
|
|
162
|
+
and consent, at most 2048 bytes). Server readers verify these; unsigned or
|
|
163
|
+
changed values are ignored, with a safe fallback when context is missing.
|
|
164
|
+
|
|
165
|
+
`cookie.consent(request)` must read **cookies and headers only**: server
|
|
166
|
+
conversion and identity helpers call it with a placeholder URL. When it returns
|
|
167
|
+
`false`, nobody is enrolled, including signed-in users; no visitor id is minted
|
|
168
|
+
or honoured, and an existing unit cookie is removed from the forwarded request.
|
|
169
|
+
Running tests show the Original. Decided winners without an active holdout
|
|
170
|
+
round still serve the decided content subject to targeting. Archived page
|
|
171
|
+
rewrites serve shipped content by path alone. Neither records exposure.
|
|
172
|
+
`linkIdentity` and `trackConversion`
|
|
173
|
+
also become no-ops when consent is declined. Omitting `consent` permits cookies
|
|
174
|
+
and enrolment.
|
|
175
|
+
|
|
176
|
+
Optional runtime `getUnit(request)` returns `{ userId?, attributes,
|
|
177
|
+
sessionUnknown? }`, synchronously or asynchronously. Use opaque user ids. With
|
|
178
|
+
no session token, return known logged-out attributes such as
|
|
179
|
+
`{ loggedIn: false, staff: false }`. With a verified session cache, return the
|
|
180
|
+
known user and attributes; with an unresolved session, set `sessionUnknown:
|
|
181
|
+
true`. User-unit experiments and session-dependent targeting can then be
|
|
182
|
+
resolved on the server through `getUser`. Server runtime options `userIdOf`
|
|
183
|
+
(default `String(user.id)`) and `attributesOf` map the host user to the unit id
|
|
184
|
+
and targeting attributes; server attributes take precedence. `attributesOf`
|
|
185
|
+
defaults to `{ loggedIn: Boolean(user) }`. If you declare `staff` in
|
|
186
|
+
`attributeKeys`, pass your own `attributesOf` that identifies staff to
|
|
187
|
+
`defineExperimentsServerRuntime`; the plugin refuses to start otherwise. The plugin owns the
|
|
188
|
+
normalized `path` attribute (no search, no trailing slash except `/`).
|
|
189
|
+
|
|
190
|
+
Staff, bots detected by the built-in user-agent list, preview renders and `HEAD`
|
|
191
|
+
requests produce no exposures. A signed `xp_preview` token can select a group
|
|
192
|
+
for review without counting it; preview responses are not cached. The plugin
|
|
193
|
+
skips admin, API, Next internals and asset paths. Host editor renders must also
|
|
194
|
+
remain outside exposure recording.
|
|
195
|
+
|
|
196
|
+
Whole-page experiments use visitor units only. Requests to any target path are
|
|
197
|
+
rewritten to the assigned group's page; a holdout sees the Original page. After
|
|
198
|
+
a winner is made permanent, losing paths keep rewriting to the winner while
|
|
199
|
+
the archived experiment record exists. A live experiment targeting a path takes
|
|
200
|
+
precedence. Decided winners remain subject to targeting.
|
|
201
|
+
|
|
202
|
+
## Server components (`/rsc`)
|
|
203
|
+
|
|
204
|
+
Import server helpers from `@delmaredigital/payload-experiments/rsc`.
|
|
205
|
+
`getAssignments(serverRuntime, { user? })` reads and verifies request context,
|
|
206
|
+
resolves deferred users, and recomputes stale or missing assignments using the
|
|
207
|
+
signed cookie and manifest. It is memoised per request. It reads dynamic Next
|
|
208
|
+
headers: call it inside `<Suspense>` under Cache Components, outside
|
|
209
|
+
`'use cache'`.
|
|
210
|
+
|
|
211
|
+
For Puck content, resolve before rendering and record only the returned
|
|
212
|
+
`rendered` list after the response. Capture all request values before `after()`:
|
|
213
|
+
|
|
214
|
+
```tsx
|
|
215
|
+
import { after } from 'next/server'
|
|
216
|
+
import { getAssignments, resolveExperimentData, recordExposures } from
|
|
217
|
+
'@delmaredigital/payload-experiments/rsc'
|
|
218
|
+
import { serverRuntime } from '@/experiments/serverRuntime'
|
|
219
|
+
|
|
220
|
+
// Inside your async Server Component, after loading the host page:
|
|
221
|
+
const assignments = await getAssignments(serverRuntime)
|
|
222
|
+
const { data, rendered } = resolveExperimentData(page.puckData, assignments)
|
|
223
|
+
after(() => recordExposures(serverRuntime, rendered, assignments))
|
|
224
|
+
// Render `data` with your host's Puck render config.
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
**Exposure is the denominator.** Reading an assignment is not an exposure.
|
|
228
|
+
`resolveExperimentData` removes unselected content and returns only block
|
|
229
|
+
experiments actually present with the `render` policy. Pass that list to
|
|
230
|
+
`recordExposures(serverRuntime, rendered, assignments)`, never the full
|
|
231
|
+
assignment map. Only schedule it when you render that resolved content.
|
|
232
|
+
Viewport and pageview exposures come from the client; manual flag/code
|
|
233
|
+
exposures require an explicit host call. Non-enrolled units see the Original
|
|
234
|
+
without being counted. Same-unit exposures to multiple groups in a round are
|
|
235
|
+
excluded and reported.
|
|
236
|
+
|
|
237
|
+
Next's default prefetch does not render the dynamic subtree in `<Suspense>`.
|
|
238
|
+
Full prefetch (`<Link prefetch={true}>` or full `router.prefetch`) of pages with
|
|
239
|
+
`render`-policy slots is unsupported: it would record content before a visit.
|
|
240
|
+
|
|
241
|
+
`toClientAssignments(assignments, { pathname, apiRoute?, flagPoints?,
|
|
242
|
+
experiments? })` creates the provider value without visitor identifiers. Both
|
|
243
|
+
allow-lists default to empty. Use the served page's pathname for pageview
|
|
244
|
+
logging; see the client integration below.
|
|
245
|
+
|
|
246
|
+
`getFlagPoint(serverRuntime, id, { user? })` returns `arm`, `params`, `source`
|
|
247
|
+
(`experiment` or `default`), optional `experiment`/`experimentId`, and
|
|
248
|
+
`expose(renderedArm)`. Bind host code to the registered point id. It uses the
|
|
249
|
+
live experiment, then saved winner defaults, then the point's `defaultParams`.
|
|
250
|
+
Render the returned settings and call `await point.expose(point.arm)` only
|
|
251
|
+
when that arm's content is actually delivered; this records only the manual
|
|
252
|
+
exposure policy. Nothing is exposed by reading the point.
|
|
253
|
+
|
|
254
|
+
Other server integrations:
|
|
255
|
+
|
|
256
|
+
- `trackConversion(serverRuntime, { eventName, value?, idempotencyKey, userId?,
|
|
257
|
+
headers? })` records a server event (default value `1`). Use a stable, globally
|
|
258
|
+
unique idempotency key for each business event. Pass request headers when
|
|
259
|
+
outside a Next request; a user-only server conversion can supply `userId`.
|
|
260
|
+
Record revenue through trusted server code or a metric source, never a client
|
|
261
|
+
beacon.
|
|
262
|
+
- `linkIdentity(serverRuntime, { headers, userId })` belongs on **every**
|
|
263
|
+
user-creation and sign-in path. It links the signed anonymous cookie to the
|
|
264
|
+
opaque user id using insert-ignore on its own connection and never throws into
|
|
265
|
+
sign-in. For visitor experiments using user-keyed outcomes, credit goes to the
|
|
266
|
+
earliest-exposed linked visitor; cross-group links are reported separately.
|
|
267
|
+
- `eraseUser(payload, { userId })` deletes the user's identities and associated
|
|
268
|
+
events transactionally and returns `{ identities, events }` counts. Call it
|
|
269
|
+
from the host's authorized user-erasure flow.
|
|
270
|
+
- `recordContentChange({ doc, previousDoc, req, collection })` can be called
|
|
271
|
+
from a host collection's `afterChange` for content tagged with the arm-select
|
|
272
|
+
field; for a collection with drafts, also add `captureContentBaseline` to its
|
|
273
|
+
`beforeChange` hooks so the previously published content can be compared. Supply that hook's transaction request. Puck collections get the hook
|
|
274
|
+
automatically when enabled. Markers compare published content; drafts and
|
|
275
|
+
autosaves do not add markers (collections without drafts record on save).
|
|
276
|
+
Neither the helper nor the installed hooks ever throw into your save: a
|
|
277
|
+
failure is logged and the content save proceeds.
|
|
278
|
+
- `experimentMetadata(serverRuntime, path)` returns metadata to merge into the
|
|
279
|
+
page's Next metadata: live variant targets get noindex and an Original
|
|
280
|
+
canonical; archived losing paths point canonically to the winner.
|
|
281
|
+
`experimentSitemapFilter(path, manifest)` is a synchronous build-time filter:
|
|
282
|
+
pass a privately server-loaded manifest and keep paths for which it returns
|
|
283
|
+
`true`, excluding live variants and archived losing paths.
|
|
284
|
+
|
|
285
|
+
## Results
|
|
286
|
+
|
|
287
|
+
The host owns scheduling and authentication. Call
|
|
288
|
+
`runResults(payload, { budgetMs })` in-process from your cron route; no Payload
|
|
289
|
+
jobs queue or HTTP call back into the application is needed:
|
|
290
|
+
|
|
291
|
+
```ts
|
|
292
|
+
// app/api/cron/xp-results/route.ts
|
|
293
|
+
import { runResults } from '@delmaredigital/payload-experiments/rsc'
|
|
294
|
+
import { serverRuntime } from '@/experiments/serverRuntime'
|
|
295
|
+
import { isAuthorizedCronRequest } from '@/auth/cron' // host-owned auth
|
|
296
|
+
|
|
297
|
+
export async function GET(request: Request) {
|
|
298
|
+
if (!(await isAuthorizedCronRequest(request))) {
|
|
299
|
+
return new Response('Unauthorized', { status: 401 })
|
|
300
|
+
}
|
|
301
|
+
const payload = await serverRuntime.getPayload()
|
|
302
|
+
const result = await runResults(payload, { budgetMs: 20_000 })
|
|
303
|
+
return Response.json(result, { headers: { 'Cache-Control': 'no-store' } })
|
|
304
|
+
}
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
The default budget is 20 seconds. Choose a budget below the host route's timeout.
|
|
308
|
+
The runner prioritizes the oldest snapshots, checks the remaining budget before
|
|
309
|
+
metrics, bounds waiting for host `query`/`covariate` promises and database
|
|
310
|
+
connections, and applies a remaining-budget SQL statement timeout. Hung work
|
|
311
|
+
cannot keep the runner waiting beyond that budget. Source failures mark that
|
|
312
|
+
metric unavailable. A timed-out source is reported in `unavailableMetrics`;
|
|
313
|
+
that experiment's new snapshots are not written, its previous snapshots remain,
|
|
314
|
+
and `budgetExhausted` signals the cutoff. Racing a host promise does not cancel
|
|
315
|
+
its underlying work: give your own source clients appropriate timeouts too.
|
|
316
|
+
The returned summary also includes `ran`, `skipped`, `swept`, `sweepAttempted` and `processedIds`.
|
|
317
|
+
Optional `experimentId` limits analysis to one experiment; `sweep: false`
|
|
318
|
+
disables retention for that call.
|
|
319
|
+
|
|
320
|
+
Analysis uses each round's frozen config and actual exposures. Its window ends
|
|
321
|
+
at `endedAt`; later exposures and conversions are excluded. Finished rounds
|
|
322
|
+
have a **24-hour late-beacon grace** before their final snapshot freezes: once
|
|
323
|
+
a snapshot has been computed after that grace, later runs leave it unchanged.
|
|
324
|
+
This does not extend the attribution window beyond the round's end.
|
|
325
|
+
|
|
326
|
+
Event retention defaults to **180 days after the experiment's final round
|
|
327
|
+
ends**, configurable with `events.retentionDays`. No experiment events are
|
|
328
|
+
swept while any of its rounds is open, including a long-lived holdout round.
|
|
329
|
+
The runner reserves part of its budget for the batched retention sweep.
|
|
330
|
+
|
|
331
|
+
Routes below are relative to Payload's `config.routes.api` (normally `/api`):
|
|
332
|
+
|
|
333
|
+
| Endpoint | Permission and response |
|
|
334
|
+
| --- | --- |
|
|
335
|
+
| `GET /xp/experiments/:id/results?phase=1&series=1` | `access.readResults`; round summaries, newest selected-round `snapshot` (`null` before a run), optional `decisionSnapshot` and requested time series. Without `phase`, a decided experiment opens on its decision round. |
|
|
336
|
+
| `GET /xp/experiments/summary?ids=...` | `access.readResults`; map of experiment ids to exposures, days and verdict summary. Encode each id individually, comma-join, and request at most 100 ids. |
|
|
337
|
+
| `POST /xp/experiments/:id/recompute` | `access.manage`; runs results for one experiment without sweeping and returns `{ snapshotId }`. Failed or budget-exhausted work returns an error. Frozen rounds stay frozen. |
|
|
338
|
+
|
|
339
|
+
## Flag points and metric sources
|
|
340
|
+
|
|
341
|
+
Register stable host integration points in `flagPoints`. `defineFlagPoint` and
|
|
342
|
+
`defineMetricSource` are optional typed identity helpers from the root export;
|
|
343
|
+
the root also exports the `FlagPoint`, `ParamField` and `MetricSource` types.
|
|
344
|
+
|
|
345
|
+
A `FlagPoint` has `id`, `label`, `params` and `defaultParams`. `params` maps
|
|
346
|
+
setting names to `ParamField` definitions: `{ type, label, required?, options?,
|
|
347
|
+
min?, max? }`. Supported types are `text`, `textarea`, `number`, `checkbox`,
|
|
348
|
+
`select`; select options are `{ label, value }`. Keep defaults valid for that
|
|
349
|
+
schema. Optional callbacks integrate host-owned content:
|
|
350
|
+
|
|
351
|
+
- `placements({ experimentId, arms, req })` returns
|
|
352
|
+
`[{ arm, ok, detail? }]` for the required content groups at Start. Holdouts
|
|
353
|
+
use Original content and are not passed as required placements.
|
|
354
|
+
- `makeContent({ experimentId, winner: { arm, params }, req })` runs when the
|
|
355
|
+
chosen settings are made permanent. Match host tags on both experiment id
|
|
356
|
+
and arm. Use the manager's `req` and `overrideAccess: false` for host-document
|
|
357
|
+
writes. The plugin saves the winning settings as defaults for the point.
|
|
358
|
+
|
|
359
|
+
A `MetricSource`, registered in `metricSources`, has `id`, `label`,
|
|
360
|
+
`kind: 'binomial' | 'count' | 'revenue'`, and `keyedBy: 'visitor' | 'user'`.
|
|
361
|
+
`clientWritable` optionally permits client conversions (revenue is always
|
|
362
|
+
server-only). `query({ from, to, unitIds? })` returns a promise of
|
|
363
|
+
`[{ unitId, value, ts }]`, **one row per event**, with `ts` a `Date`; use the
|
|
364
|
+
same opaque ids as assignment or identity linking. A user experiment requires
|
|
365
|
+
a user-keyed source. Visitor experiments can use a user-keyed source through
|
|
366
|
+
linked identities.
|
|
367
|
+
|
|
368
|
+
Optional `covariate({ units: [{ unitId, firstExposureTs }], lookbackDays })`
|
|
369
|
+
returns `[{ unitId, value }]` for pre-exposure CUPED adjustment (the plugin asks for a 14-day lookback). Percentile
|
|
370
|
+
revenue capping reuses `query` over the 90 days before Start; the resolved cap
|
|
371
|
+
is frozen for the round. Capping affects decision statistics, while results
|
|
372
|
+
also show uncapped totals. Count and binomial metrics are never capped.
|
|
373
|
+
|
|
374
|
+
For simple yes/no outcomes, list names in `eventSources`, for example
|
|
375
|
+
`eventSources: ['signup']`. A bare string allows client writes; the object form
|
|
376
|
+
sets that permission explicitly:
|
|
377
|
+
|
|
378
|
+
```ts
|
|
379
|
+
eventSources: ['signup', { name: 'donation_completed', clientWritable: false }]
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
Names must match `/^[a-z0-9_]+$/` and be unique across both forms. Editors can
|
|
383
|
+
choose `events:signup` or `events:donation_completed` as binomial sources backed
|
|
384
|
+
by stored conversions with the corresponding `eventName`. Writable sources
|
|
385
|
+
accept both `trackConversion` and `trackConversionBeacon`; server-only sources
|
|
386
|
+
are recorded with `trackConversion`. Beacon conversions for a non-writable name
|
|
387
|
+
are dropped silently. Only listed names become `events:<name>` sources. They
|
|
388
|
+
support visitor and user units and do not represent counts or revenue in v1.
|
|
389
|
+
|
|
390
|
+
## Security
|
|
391
|
+
|
|
392
|
+
The unit cookie, assignments header, request-context header and preview tokens
|
|
393
|
+
are HMAC-SHA256 signed with purpose-separated inputs and constant-time
|
|
394
|
+
verification. Signing prevents a browser from choosing a trusted identity,
|
|
395
|
+
assigned arm, request context or preview group by editing a value. The manifest
|
|
396
|
+
is private. Store no PII beyond opaque unit ids: keep names, emails and other
|
|
397
|
+
identifying values out of attributes, params and event data.
|
|
398
|
+
|
|
399
|
+
Every collection/global access operation is explicit: create, read, update,
|
|
400
|
+
delete, readVersions and unlock. Managers operate experiments, layers and
|
|
401
|
+
metrics; results and identity reads have separate permissions. Internal event,
|
|
402
|
+
identity, round, snapshot and state writes are denied through ordinary
|
|
403
|
+
collection/global access. Every Payload Local API call declares
|
|
404
|
+
`overrideAccess`; request handlers and views pass `req` and
|
|
405
|
+
`overrideAccess: false`. The enumerated privileged operations are the events
|
|
406
|
+
insert and assignment lookup, `recordExposures`, server `trackConversion`,
|
|
407
|
+
`linkIdentity`, manifest reads, transition writes to plugin collections/state,
|
|
408
|
+
content-change markers, results writes and retention sweep, and `eraseUser`.
|
|
409
|
+
Host content writes when making a winner permanent still use the manager's
|
|
410
|
+
access. Privileged side effects follow their authorized operation in the same
|
|
411
|
+
transaction; identity linking deliberately uses its own insert-ignore connection
|
|
412
|
+
so duplicate links cannot abort sign-in.
|
|
413
|
+
|
|
414
|
+
`POST /xp/events` requires a valid signed unit cookie, reads a streamed body
|
|
415
|
+
with an **8192-byte default limit**, and accepts at most **20 events** per body.
|
|
416
|
+
It parses JSON regardless of content type for beacon compatibility. Exposure
|
|
417
|
+
beacons must name the current round, policy and the arm actually rendered;
|
|
418
|
+
the server recomputes assignment and drops mismatches while counting them
|
|
419
|
+
separately. Only manual, viewport and pageview exposure policies are accepted.
|
|
420
|
+
Conversions require a client-writable source, never revenue; client values are
|
|
421
|
+
stored as `1`, attributes are allow-listed, and the host's `events.rateLimit`
|
|
422
|
+
adapter runs when supplied. Accepted and dropped events both return `202 {}`;
|
|
423
|
+
malformed requests can return errors.
|
|
424
|
+
|
|
425
|
+
**The real bound on slow or method-override bodies is the host firewall/platform
|
|
426
|
+
timeout.** A stream under the byte limit can remain open, and Payload can read
|
|
427
|
+
an `X-HTTP-Method-Override` body before the plugin's streamed cap runs. Apply
|
|
428
|
+
host firewall rules and platform request timeouts at that boundary.
|
|
429
|
+
|
|
430
|
+
Static checks enforce explicit access and keep raw SQL in the database modules,
|
|
431
|
+
but have accepted gaps: extracting a Payload method with a computed property
|
|
432
|
+
in destructuring; reflection through `Object.getOwnPropertyNames`,
|
|
433
|
+
`Object.getOwnPropertyDescriptor` or `Reflect.ownKeys`; capturing a computed
|
|
434
|
+
method from an unknown receiver and calling it later; `payload.jobs.*` calls
|
|
435
|
+
(the jobs queue is not used or supported here); lowercase SQL through `pg`;
|
|
436
|
+
and aliases changed through `let` reassignment. **Code review is the backstop**
|
|
437
|
+
for these patterns.
|
|
438
|
+
|
|
439
|
+
## Admin and client
|
|
440
|
+
|
|
441
|
+
### Client assignments and exposure
|
|
442
|
+
|
|
443
|
+
Read assignments on the server and pass them to `ExperimentsProvider` from
|
|
444
|
+
`@delmaredigital/payload-experiments/client`. Build its `value` with
|
|
445
|
+
`toClientAssignments` from `@delmaredigital/payload-experiments/rsc`:
|
|
446
|
+
|
|
447
|
+
```tsx
|
|
448
|
+
<ExperimentsProvider value={toClientAssignments(assignments, {
|
|
449
|
+
pathname,
|
|
450
|
+
flagPoints: ['signup-form'],
|
|
451
|
+
experiments: ['welcome-copy'],
|
|
452
|
+
})}>
|
|
453
|
+
{children}
|
|
454
|
+
</ExperimentsProvider>
|
|
455
|
+
```
|
|
456
|
+
|
|
457
|
+
`flagPoints` and `experiments` are allow-lists: both default to none. Include
|
|
458
|
+
only the flag points and experiments that the client needs. The provider value
|
|
459
|
+
contains no visitor identifier. Use **one `apiRoute` for every provider on a
|
|
460
|
+
page**, including nested providers; it is the API base, such as `/api`. When the
|
|
461
|
+
API isn't at `/api`, set that shared route through
|
|
462
|
+
`toClientAssignments(assignments, { pathname, apiRoute })`.
|
|
463
|
+
|
|
464
|
+
`useFlagPoint(id)` reads the current group and settings at a registered flag
|
|
465
|
+
point. `useExperiment(slug)` reads an allow-listed experiment. An unlisted flag
|
|
466
|
+
point or experiment slug throws in development and returns the Original with
|
|
467
|
+
one console error in production. Neither hook records exposure automatically.
|
|
468
|
+
For manual exposure, call
|
|
469
|
+
`expose(renderedArm)` only when that group's content actually showed, passing
|
|
470
|
+
the group key you rendered. Reading an assignment alone does not count as
|
|
471
|
+
seeing content. `expose()` sends only for experiments using the manual exposure
|
|
472
|
+
policy.
|
|
473
|
+
|
|
474
|
+
On each whole-page experiment target, render a nested `ExperimentsProvider`
|
|
475
|
+
whose value is built for **that page's pathname**, and put
|
|
476
|
+
`<ExperimentPageview />` inside it. This component reads from the nearest
|
|
477
|
+
provider and records the page that appeared, including after navigation.
|
|
478
|
+
|
|
479
|
+
Every page that renders Puck viewport slots also needs a provider. A
|
|
480
|
+
layout-level provider with empty allow-lists is enough:
|
|
481
|
+
`toClientAssignments(assignments, { pathname })`. Without a provider, viewport
|
|
482
|
+
exposures cannot be sent.
|
|
483
|
+
|
|
484
|
+
For a client-writable conversion, call
|
|
485
|
+
`trackConversionBeacon(eventName, { idempotencyKey })`. Pass a stable key for
|
|
486
|
+
conversions that matter, so retries count as the same conversion. It sends
|
|
487
|
+
nothing without a mounted provider. The server accepts only conversion sources
|
|
488
|
+
configured to allow client writes; record revenue on the server.
|
|
489
|
+
|
|
490
|
+
### Puck render and editor configs
|
|
491
|
+
|
|
492
|
+
Register `ExperimentSlot` from `@delmaredigital/payload-experiments/puck` in
|
|
493
|
+
your render config and `ExperimentSlotEditor` from
|
|
494
|
+
`@delmaredigital/payload-experiments/puck/editor` in your editor config, using
|
|
495
|
+
the same `ExperimentSlot` component name:
|
|
496
|
+
|
|
497
|
+
```ts
|
|
498
|
+
// Render config
|
|
499
|
+
components: { ExperimentSlot }
|
|
500
|
+
|
|
501
|
+
// Editor config
|
|
502
|
+
components: { ExperimentSlot: ExperimentSlotEditor }
|
|
503
|
+
```
|
|
504
|
+
|
|
505
|
+
Resolve the Puck data on the server with `resolveExperimentData` before
|
|
506
|
+
rendering it. The render component shows only the selected group's slot. The
|
|
507
|
+
editor lets you select a block experiment, edit each group's content, copy the
|
|
508
|
+
Original into a variant, or create an experiment. It warns when edits would
|
|
509
|
+
change content during a running test. `createExperimentSlotField` is available
|
|
510
|
+
from `/puck` if you need the experiment picker in another custom component.
|
|
511
|
+
|
|
512
|
+
The picker lists the 100 most recently updated block experiments.
|
|
513
|
+
Viewport exposure watches a fixed, invisible **100 px sentinel**
|
|
514
|
+
at the top of the slot and counts it when at least half is visible. The
|
|
515
|
+
sentinel stays the same size for short and tall content.
|
|
516
|
+
|
|
517
|
+
### Tag host content with a group
|
|
518
|
+
|
|
519
|
+
Add `createArmSelectField({ flagPoint: 'signup-form' })` from
|
|
520
|
+
`@delmaredigital/payload-experiments` to a host collection's fields. By default
|
|
521
|
+
the field is named `experimentArm`; it stores
|
|
522
|
+
`{ experiment: String(experimentId), arm }`.
|
|
523
|
+
|
|
524
|
+
Match host content on **`String(experimentId)` plus the arm key**, never on
|
|
525
|
+
the experiment slug. The field shows group labels, leaves out the holdout,
|
|
526
|
+
warns about tags from an earlier experiment, and lets you clear them. A running
|
|
527
|
+
experiment shows a warning before you change its content.
|
|
528
|
+
|
|
529
|
+
### Inside the admin
|
|
530
|
+
|
|
531
|
+
The experiment's edit tabs guide setup:
|
|
532
|
+
|
|
533
|
+
- **What:** what you are testing and why, including the flag point when needed.
|
|
534
|
+
- **Who:** who takes part, targeting, allocation, and space in a layer.
|
|
535
|
+
- **Groups:** the Original, variants and optional holdout, their percentages,
|
|
536
|
+
settings, or page paths.
|
|
537
|
+
- **Metrics:** the main outcome, supporting metrics, guardrails, and planned
|
|
538
|
+
duration.
|
|
539
|
+
- **Review:** saved setup checks, placements and preview links, recent traffic,
|
|
540
|
+
and a duration estimate when enough information is available. Apply copies
|
|
541
|
+
the suggested duration and sensitivity into the form; save before you start.
|
|
542
|
+
|
|
543
|
+
The **Results** view lets you select a round, read the verdict, compare groups,
|
|
544
|
+
check data quality, and see results over time with content-change markers.
|
|
545
|
+
Holdouts are measured without being compared during the test. Revenue results
|
|
546
|
+
show uncapped totals alongside any capped decision statistic. Recompute
|
|
547
|
+
requests fresh results. The experiment list shows status and a results summary;
|
|
548
|
+
round history and layer usage provide context in the admin.
|
|
549
|
+
|
|
550
|
+
Actions depend on the experiment's status and your permissions. **Unsaved
|
|
551
|
+
changes block actions**; save first so they use the saved experiment.
|
|
552
|
+
|
|
553
|
+
- **Start** runs a draft. **Start a new round** after stopping warns:
|
|
554
|
+
“Starting again begins a new round; results restart.”
|
|
555
|
+
- **Stop** says: “Everyone will see the Original. Starting again begins a new
|
|
556
|
+
round; results restart.”
|
|
557
|
+
- **Raise allocation** adds people without changing the group of anyone
|
|
558
|
+
already taking part. Lowering allocation needs a new round.
|
|
559
|
+
- **Add a metric** adds a supporting metric or guardrail to a running test.
|
|
560
|
+
- **Decide** starts with the recommended group. Choosing the Original says
|
|
561
|
+
“Everyone will see the Original.” Choosing a variant says everyone will see
|
|
562
|
+
it, except an existing holdout, which keeps seeing the Original so you can
|
|
563
|
+
measure the shipped change. When results need caution or you choose against
|
|
564
|
+
the recommendation, enter a note and type the chosen group's label.
|
|
565
|
+
- **Release holdout** says the holdout will see the winner too and measuring
|
|
566
|
+
since shipping ends.
|
|
567
|
+
- **Make it the content** copies the winner into page content or saves its
|
|
568
|
+
settings as defaults, then archives the experiment. Failed page updates are
|
|
569
|
+
listed with Retry. **Finish archiving** completes archiving when the content
|
|
570
|
+
has already been updated.
|
|
571
|
+
- **Archive** closes a stopped experiment. **Discard winner** says: “Everyone
|
|
572
|
+
goes back to the Original. The decision note is kept.” It requires typed
|
|
573
|
+
confirmation. For code experiments, the action confirming that the developer
|
|
574
|
+
has hard-coded the winner also requires typed confirmation before archiving.
|
|
575
|
+
|
|
576
|
+
### Known limitations (v1)
|
|
577
|
+
|
|
578
|
+
- Preview links open after a token request, so Safari's popup blocker may block
|
|
579
|
+
them. Allow popups for the admin.
|
|
580
|
+
- “Raise allocation” is offered even at 100%.
|
|
581
|
+
- The metric picker in “Add a metric” lists 100 metrics.
|
|
582
|
+
- The Puck picker lists 100 experiments.
|
|
583
|
+
- The Puck experiment picker always calls `/api`; hosts with a custom API route
|
|
584
|
+
see an empty picker.
|
|
585
|
+
- Making a winner permanent skips pages in collections that are localized,
|
|
586
|
+
have hidden fields, or contain fields the manager cannot read or update. Those pages
|
|
587
|
+
are listed with a note to update them by hand.
|
|
588
|
+
- Content-change markers are recorded on publish only (on save for collections
|
|
589
|
+
without drafts).
|
|
590
|
+
- Results cover the whole round; breakdowns by segment or dimension are not
|
|
591
|
+
available in v1.
|
package/dist/access.d.ts
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export { deny, isTrue, managedAccess, readOnlyAccess } from './collections/access.js';
|
package/dist/access.js
ADDED