@decocms/blocks 8.20.2 → 8.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/v8/analytics/client.d.ts +3 -1
- package/dist/v8/analytics/client.d.ts.map +1 -1
- package/dist/v8/analytics/client.js +12 -3
- package/dist/v8/analytics/client.js.map +1 -1
- package/dist/v8/builtins/experiments.d.ts +7 -2
- package/dist/v8/builtins/experiments.d.ts.map +1 -1
- package/dist/v8/builtins/experiments.js +22 -5
- package/dist/v8/builtins/experiments.js.map +1 -1
- package/dist/v8/cli/check/index.d.ts.map +1 -1
- package/dist/v8/cli/check/index.js +16 -0
- package/dist/v8/cli/check/index.js.map +1 -1
- package/dist/v8/cli/schema/builtinSchemas.js +1 -1
- package/dist/v8/cli/schema/builtinSchemas.js.map +1 -1
- package/dist/v8/errorReporting.d.ts +23 -0
- package/dist/v8/errorReporting.d.ts.map +1 -0
- package/dist/v8/errorReporting.js +140 -0
- package/dist/v8/errorReporting.js.map +1 -0
- package/dist/v8/experiments.d.ts +14 -3
- package/dist/v8/experiments.d.ts.map +1 -1
- package/dist/v8/experiments.js +11 -3
- package/dist/v8/experiments.js.map +1 -1
- package/dist/v8/nested.d.ts +23 -0
- package/dist/v8/nested.d.ts.map +1 -0
- package/dist/v8/nested.js +40 -0
- package/dist/v8/nested.js.map +1 -0
- package/dist/v8/telemetry.d.ts +6 -1
- package/dist/v8/telemetry.d.ts.map +1 -1
- package/dist/v8/telemetry.js +27 -18
- package/dist/v8/telemetry.js.map +1 -1
- package/dist/v8/telemetryText.d.ts +3 -0
- package/dist/v8/telemetryText.d.ts.map +1 -0
- package/dist/v8/telemetryText.js +21 -0
- package/dist/v8/telemetryText.js.map +1 -0
- package/package.json +7 -2
- package/skills/deco-variants/SKILL.md +1 -1
- package/skills/deco-variants/references/experiments-and-campaigns.md +32 -3
- package/src/v8/analytics/client.ts +14 -3
- package/src/v8/builtins/experiments.ts +21 -5
- package/src/v8/cli/check/index.ts +21 -0
- package/src/v8/cli/schema/builtinSchemas.ts +1 -1
- package/src/v8/errorReporting.ts +155 -0
- package/src/v8/experiments.ts +16 -4
- package/src/v8/nested.ts +40 -0
- package/src/v8/telemetry.ts +111 -90
- package/src/v8/telemetryText.ts +22 -0
|
@@ -28,7 +28,7 @@ Put the control last, under `always`.
|
|
|
28
28
|
|
|
29
29
|
- A visitor has an id: 16 random bytes as 22 base64url characters, stored in the `deco_vid` cookie for a year. The cookie is not `HttpOnly`, so an analytics script can read it.
|
|
30
30
|
- For each test, the visitor's roll is a hash of `<test name>:<id>` (MurmurHash3, in `bucketing.ts`), a fixed number from 0 to 99.99. The visitor is in the test when the roll is below the percentage.
|
|
31
|
-
- Nothing per test is stored. One id serves every test, a new test needs no new state, and two tests are independent of each other.
|
|
31
|
+
- Nothing per test is stored. One id serves every test, a new test needs no new state, and two tests are independent of each other, unless one is named within the other (see [A test within a test](#a-test-within-a-test)).
|
|
32
32
|
- The decision is the same on a server, in a browser and in an app: `assign(experiments, visitor)` is a pure function.
|
|
33
33
|
|
|
34
34
|
What follows from that:
|
|
@@ -38,6 +38,35 @@ What follows from that:
|
|
|
38
38
|
- **0 is off, 100 is everyone.** A fractional percentage is rounded. An entry that is not a finite number is left out.
|
|
39
39
|
- **No visitor, no test.** A client created without a visitor has every test off, unless the test is given to it.
|
|
40
40
|
|
|
41
|
+
### A test within a test
|
|
42
|
+
|
|
43
|
+
A name written `<parent>/<child>` is drawn only among the visitors in `<parent>`, so a test can split its own arm without touching the other one. For 50% control and the other 50% split evenly between two variants:
|
|
44
|
+
|
|
45
|
+
```json
|
|
46
|
+
{ "__resolveType": "split-traffic", "plp-optimizer": 50, "plp-optimizer/b": 50 }
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
```json
|
|
50
|
+
{ "__resolveType": "multivariate",
|
|
51
|
+
"variants": [
|
|
52
|
+
{ "rule": { "__resolveType": "experiment", "name": "plp-optimizer" },
|
|
53
|
+
"value": { "__resolveType": "lazy", "value": {
|
|
54
|
+
"__resolveType": "multivariate",
|
|
55
|
+
"variants": [
|
|
56
|
+
{ "rule": { "__resolveType": "experiment", "name": "plp-optimizer/b" },
|
|
57
|
+
"value": { "__resolveType": "lazy", "value": "…variant B…" } },
|
|
58
|
+
{ "rule": { "__resolveType": "always" },
|
|
59
|
+
"value": { "__resolveType": "lazy", "value": "…variant A…" } } ] } } },
|
|
60
|
+
{ "rule": { "__resolveType": "always" },
|
|
61
|
+
"value": { "__resolveType": "lazy", "value": "…control…" } } ] }
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
- The child's percentage is a share of the parent's visitors: `plp-optimizer/b: 50` is 25% of all traffic. The child keeps its own roll, so ramping `plp-optimizer` to 80 gives 20/40/40 and keeps everyone already in A or B where they were.
|
|
65
|
+
- The child is off for every visitor outside the parent, whatever it is given: switching on only `plp-optimizer/b` in a preview shows the control to a visitor outside `plp-optimizer`. Switch both on to see B, the parent on and the child off to see A. The parent alone shows the visitor's own arm of the child.
|
|
66
|
+
- **A name already saved with a `/`** whose prefix is also a test now runs within it. `deco check` warns on every test it reads that way: rename one if the two are separate tests.
|
|
67
|
+
- The parent is the nearest prefix before a `/` that is a test of the same block (`a/b/c` falls back to `a` when there is no `a/b`). A `/` in a name whose prefix is no test changes nothing.
|
|
68
|
+
- Nothing else changes: the header, the cache key and the preview link carry the child as one more test.
|
|
69
|
+
|
|
41
70
|
### Who reads the visitor
|
|
42
71
|
|
|
43
72
|
The binding does it, once per request. On `@decocms/tanstack`, `createDecoWorker` calls `readVisitor(request)` (the cookie, or a new visitor), decides every test and campaign, and hands both to the app on the `x-deco-visitor` and `x-deco-assignments` headers. It removes any copy of those headers sent from outside. `contentClient(cms)` from `@decocms/tanstack/server` builds the client from them, so the page decides nothing again. On `@decocms/nextjs`, the proxy does the same with `decideVisitor` and `rewriteToAssignmentsRoute` (or `decideAssignments`) from `@decocms/nextjs/proxy`. The cookie goes out only on a first visit, and never from a preview.
|
|
@@ -95,9 +124,9 @@ v7's cron matcher has no v8 counterpart. Write each occurrence as its own window
|
|
|
95
124
|
|
|
96
125
|
## Reading the results
|
|
97
126
|
|
|
98
|
-
v8 decides who sees what. It does not record conversions
|
|
127
|
+
v8 decides who sees what. It does not record conversions. Send the arm with every event:
|
|
99
128
|
|
|
100
|
-
-
|
|
129
|
+
- With `startAnalytics({ experiments })` from `@decocms/blocks/analytics/client`, pass the map that decided the page (on TanStack, `requestAssignments(request)?.experiments` from `@decocms/tanstack/server`; never on a preview). Every event then carries each test as `on` or `off` in `exp`. A child test is sent only while its parent is on, so `plp-optimizer` compares control with both variants, and `plp-optimizer/b` compares B with A with no filter to remember.
|
|
101
130
|
- On any binding, the page's own client gives the same answer: `const [tests] = await client.resolve("experiments")` gives each test's name and `true` or `false`, never a percentage.
|
|
102
131
|
- In the browser, the `deco_vid` cookie holds the visitor; `parseVisitor` and `assign` from `@decocms/blocks/experiments` give the same answer as the server for a given `experiments` block.
|
|
103
132
|
|
|
@@ -9,6 +9,8 @@
|
|
|
9
9
|
* failure is "analytics stopped", never "the page broke".
|
|
10
10
|
*/
|
|
11
11
|
|
|
12
|
+
import { parentTest } from "../nested.ts";
|
|
13
|
+
|
|
12
14
|
/** The optional modules, by the names the collector's site config uses. */
|
|
13
15
|
export type AnalyticsModule =
|
|
14
16
|
| "click"
|
|
@@ -49,20 +51,29 @@ export interface AnalyticsClientOptions {
|
|
|
49
51
|
* arm, `on` or `off`, in `exp`, the first page view included, and the collector stores it as
|
|
50
52
|
* it stores any `exp`. Passing the decision rather than an id keeps what is measured equal to
|
|
51
53
|
* what was shown, a preview's switched tests and a release change included. Leave it unset
|
|
52
|
-
* on a preview, whose switched tests are no visitor's.
|
|
54
|
+
* on a preview, whose switched tests are no visitor's. A test within a test
|
|
55
|
+
* (`plp-optimizer/b`) is sent only while its parent is on, so its `on`/`off` compares the
|
|
56
|
+
* parent's own arms (B against A) and the control carries the parent alone.
|
|
53
57
|
*/
|
|
54
58
|
experiments?: Readonly<Record<string, boolean>>;
|
|
55
59
|
}
|
|
56
60
|
|
|
57
|
-
/**
|
|
61
|
+
/**
|
|
62
|
+
* The decided tests as the collector's `exp` values, `on` or `off`; non-boolean entries left
|
|
63
|
+
* out, and so is a `<parent>/<child>` test whose parent is off: that visitor never saw either arm.
|
|
64
|
+
*/
|
|
58
65
|
function armsOf(
|
|
59
66
|
experiments: Readonly<Record<string, boolean>> | undefined,
|
|
60
67
|
): Record<string, string> {
|
|
61
68
|
const arms = Object.create(null) as Record<string, string>;
|
|
62
69
|
if (typeof experiments !== "object" || experiments === null) return arms;
|
|
70
|
+
const has = (name: string) => typeof experiments[name] === "boolean";
|
|
63
71
|
for (const name of Object.keys(experiments)) {
|
|
64
72
|
const on = experiments[name];
|
|
65
|
-
if (typeof on
|
|
73
|
+
if (typeof on !== "boolean") continue;
|
|
74
|
+
const parent = parentTest(name, has);
|
|
75
|
+
if (parent !== undefined && experiments[parent] !== true) continue;
|
|
76
|
+
arms[name] = on ? "on" : "off";
|
|
66
77
|
}
|
|
67
78
|
return arms;
|
|
68
79
|
}
|
|
@@ -12,10 +12,16 @@
|
|
|
12
12
|
* read it through `currentClient()`, before any `await`. A test the client
|
|
13
13
|
* is given (`cms.client({ experiments })`) is on or off as given, whatever
|
|
14
14
|
* the visitor and its percentage (a preview may switch on a test at 0%); a
|
|
15
|
-
* name the block doesn't have stays out.
|
|
15
|
+
* name the block doesn't have stays out. A test `<parent>/<child>` is on only
|
|
16
|
+
* while `<parent>` is, given or not, and is otherwise the child's own roll or
|
|
17
|
+
* what it was given: a preview switching on `plp-optimizer` alone shows the
|
|
18
|
+
* visitor's own arm of `plp-optimizer/b` (A without a visitor), and one
|
|
19
|
+
* switching on `plp-optimizer/b` alone shows B only to visitors in
|
|
20
|
+
* `plp-optimizer`.
|
|
16
21
|
*/
|
|
17
22
|
import { currentClient, experimentOverrides } from "../current.ts";
|
|
18
23
|
import { type Assignments, assign } from "../experiments.ts";
|
|
24
|
+
import { nestTests } from "../nested.ts";
|
|
19
25
|
|
|
20
26
|
/** The saved block every `experiment` rule reads, and its built-in type. */
|
|
21
27
|
export const EXPERIMENTS_BLOCK = "experiments";
|
|
@@ -27,19 +33,29 @@ export const EXPERIMENT_RULE = "experiment";
|
|
|
27
33
|
/**
|
|
28
34
|
* `split-traffic`: each test and whether the current client's visitor is in
|
|
29
35
|
* it (`assign` from `@decocms/blocks/experiments`), with the tests the
|
|
30
|
-
* client was given applied on top
|
|
31
|
-
*
|
|
36
|
+
* client was given applied on top, then each `<parent>/<child>` turned off
|
|
37
|
+
* where its parent is off. Without a visitor every test it wasn't given is
|
|
38
|
+
* off.
|
|
39
|
+
*
|
|
40
|
+
* Each test is drawn on its own first, so the nesting sees the given parent
|
|
41
|
+
* rather than the visitor's: a child's roll still counts when a preview
|
|
42
|
+
* switches its parent on.
|
|
32
43
|
*/
|
|
33
44
|
export const splitTraffic = (allocations: Record<string, number> = {}): Assignments => {
|
|
34
45
|
const client = currentClient();
|
|
35
|
-
const assignments =
|
|
46
|
+
const assignments = Object.create(null) as Assignments;
|
|
47
|
+
for (const name of Object.keys(allocations).sort()) {
|
|
48
|
+
// One test per call: nothing to nest within, so this is the test's own roll.
|
|
49
|
+
const own = assign({ [name]: allocations[name] as number }, client.visitor);
|
|
50
|
+
if (Object.hasOwn(own, name)) assignments[name] = own[name] === true;
|
|
51
|
+
}
|
|
36
52
|
const given = experimentOverrides(client);
|
|
37
53
|
if (given !== undefined) {
|
|
38
54
|
for (const name of Object.keys(given)) {
|
|
39
55
|
if (Object.hasOwn(assignments, name)) assignments[name] = given[name] === true;
|
|
40
56
|
}
|
|
41
57
|
}
|
|
42
|
-
return assignments;
|
|
58
|
+
return nestTests(assignments);
|
|
43
59
|
};
|
|
44
60
|
|
|
45
61
|
/** Inputs of the built-in `experiment` rule. */
|
|
@@ -34,6 +34,7 @@ import {
|
|
|
34
34
|
} from "../../builtins/experiments.ts";
|
|
35
35
|
import { isBlock, own } from "../../json.ts";
|
|
36
36
|
import { findRouteConflicts } from "../../matchRoute.ts";
|
|
37
|
+
import { parentTest } from "../../nested.ts";
|
|
37
38
|
import { BUILT_IN_BLOCKS, BUILT_IN_USES, isBuiltIn, storesPlainVariants } from "../builtins.ts";
|
|
38
39
|
import { paint } from "../color.ts";
|
|
39
40
|
import { readSavedBlocks, type SavedBlocks } from "../content.ts";
|
|
@@ -1045,6 +1046,26 @@ class Checker {
|
|
|
1045
1046
|
"warning",
|
|
1046
1047
|
);
|
|
1047
1048
|
}
|
|
1049
|
+
if (
|
|
1050
|
+
name === EXPERIMENTS_BLOCK &&
|
|
1051
|
+
this.canonicalType(entry.__resolveType) === SPLIT_TRAFFIC_TYPE
|
|
1052
|
+
) {
|
|
1053
|
+
// Before 8.x every name was its own test: say which ones now run within another.
|
|
1054
|
+
const has = (test: string) => test !== "__resolveType" && Object.hasOwn(entry, test);
|
|
1055
|
+
for (const test of Object.keys(entry)) {
|
|
1056
|
+
if (test === "__resolveType") continue;
|
|
1057
|
+
const parent = parentTest(test, has);
|
|
1058
|
+
if (parent === undefined) continue;
|
|
1059
|
+
this.report(
|
|
1060
|
+
ctx,
|
|
1061
|
+
test,
|
|
1062
|
+
`"${test}" is a test within "${parent}": it runs only for the visitors in "${parent}", ` +
|
|
1063
|
+
`and its percentage is a share of them; if the two are separate tests, rename it so ` +
|
|
1064
|
+
`it doesn't start with "${parent}/"`,
|
|
1065
|
+
"warning",
|
|
1066
|
+
);
|
|
1067
|
+
}
|
|
1068
|
+
}
|
|
1048
1069
|
if (name === CAMPAIGNS_BLOCK && this.isOwnBuiltIn(CALENDAR_TYPE)) {
|
|
1049
1070
|
if (this.canonicalType(entry.__resolveType) !== CALENDAR_TYPE) {
|
|
1050
1071
|
this.report(
|
|
@@ -55,7 +55,7 @@ const BUILT_IN_DOCS: Record<string, { title: string; description: string; icon?:
|
|
|
55
55
|
"split-traffic": {
|
|
56
56
|
title: "Split traffic",
|
|
57
57
|
description:
|
|
58
|
-
"A/B tests: each name with the percentage of visitors in it (0 off, 100 everyone); resolves to whether the visitor is in each. The type of the saved block experiments",
|
|
58
|
+
"A/B tests: each name with the percentage of visitors in it (0 off, 100 everyone); a name parent/child takes that percentage of parent's visitors; resolves to whether the visitor is in each. The type of the saved block experiments",
|
|
59
59
|
icon: "flask",
|
|
60
60
|
},
|
|
61
61
|
calendar: {
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Server errors on the CMS's existing sampled OTLP pipeline. Framework
|
|
3
|
+
* bindings call this at their own error boundaries; reporting never replaces
|
|
4
|
+
* an exception, changes a response, or installs a process-wide handler.
|
|
5
|
+
*/
|
|
6
|
+
import { isResolutionError } from "./errors.ts";
|
|
7
|
+
import { currentTelemetry } from "./telemetry.ts";
|
|
8
|
+
|
|
9
|
+
export interface ErrorReportContext {
|
|
10
|
+
framework?: "tanstack" | "nextjs";
|
|
11
|
+
source?: "request" | "render" | "action" | "route" | "proxy";
|
|
12
|
+
/** A matched route template, never the request URL or path. */
|
|
13
|
+
route?: string | (() => string | undefined | Promise<string | undefined>);
|
|
14
|
+
method?: string;
|
|
15
|
+
/** Omit when the response status is not known (for example a streamed render). */
|
|
16
|
+
status?: number;
|
|
17
|
+
/** A static component identifier, never its props or a component stack. */
|
|
18
|
+
component?: string;
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
const SOURCES = new Set(["request", "render", "action", "route", "proxy"]);
|
|
22
|
+
const METHODS = new Set([
|
|
23
|
+
"GET",
|
|
24
|
+
"HEAD",
|
|
25
|
+
"POST",
|
|
26
|
+
"PUT",
|
|
27
|
+
"DELETE",
|
|
28
|
+
"CONNECT",
|
|
29
|
+
"OPTIONS",
|
|
30
|
+
"TRACE",
|
|
31
|
+
"PATCH",
|
|
32
|
+
]);
|
|
33
|
+
const FLUSH_TIMEOUT_MS = 2_000;
|
|
34
|
+
|
|
35
|
+
/** Reads only a bounded string field, including from objects with hostile getters. */
|
|
36
|
+
function field(value: unknown, key: string, max: number): string | undefined {
|
|
37
|
+
try {
|
|
38
|
+
const item = (value as Record<string, unknown> | null | undefined)?.[key];
|
|
39
|
+
return typeof item === "string" ? item.slice(0, max) : undefined;
|
|
40
|
+
} catch {
|
|
41
|
+
return undefined;
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
function alreadyReported(value: unknown): boolean {
|
|
46
|
+
// The CMS reports its ResolutionError at the point of failure. A framework
|
|
47
|
+
// wrapping it does not make it a new failure. No process-long identity cache:
|
|
48
|
+
// another request may legitimately throw the same Error object again.
|
|
49
|
+
const seen = new Set<unknown>();
|
|
50
|
+
for (let depth = 0; depth < 5 && value !== null && typeof value === "object"; depth++) {
|
|
51
|
+
if (seen.has(value)) break;
|
|
52
|
+
seen.add(value);
|
|
53
|
+
try {
|
|
54
|
+
if (isResolutionError(value)) return true;
|
|
55
|
+
value = (value as { cause?: unknown }).cause;
|
|
56
|
+
} catch {
|
|
57
|
+
break;
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
return false;
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
function errorText(value: unknown): string {
|
|
64
|
+
const message = field(value, "message", 1_000);
|
|
65
|
+
if (message !== undefined) return message;
|
|
66
|
+
if (typeof value === "string") return value.slice(0, 1_000);
|
|
67
|
+
if (typeof value === "number" || typeof value === "boolean" || typeof value === "bigint") {
|
|
68
|
+
return String(value);
|
|
69
|
+
}
|
|
70
|
+
return "unknown error";
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
function component(value: unknown): string | undefined {
|
|
74
|
+
return typeof value === "string" && /^[\w./:-]{1,200}$/.test(value) ? value : undefined;
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
async function recordError(
|
|
78
|
+
error: unknown,
|
|
79
|
+
context: ErrorReportContext,
|
|
80
|
+
telemetry = currentTelemetry(),
|
|
81
|
+
): Promise<void> {
|
|
82
|
+
if (!telemetry?.logging() || alreadyReported(error)) return;
|
|
83
|
+
const name = field(error, "name", 80) || "Error";
|
|
84
|
+
let route: unknown;
|
|
85
|
+
try {
|
|
86
|
+
route = typeof context.route === "function" ? await context.route() : context.route;
|
|
87
|
+
} catch {
|
|
88
|
+
// A failed route lookup must not hide the original failure.
|
|
89
|
+
}
|
|
90
|
+
const source = SOURCES.has(context.source ?? "") ? context.source! : "request";
|
|
91
|
+
const method = typeof context.method === "string" ? context.method.toUpperCase() : undefined;
|
|
92
|
+
const routeTemplate =
|
|
93
|
+
typeof route === "string" && route.startsWith("/")
|
|
94
|
+
? route.split(/[?#]/, 1)[0]?.slice(0, 200)
|
|
95
|
+
: undefined;
|
|
96
|
+
const framework =
|
|
97
|
+
context.framework === "tanstack" || context.framework === "nextjs"
|
|
98
|
+
? context.framework
|
|
99
|
+
: undefined;
|
|
100
|
+
telemetry.error(`${source} failed: ${name}: ${errorText(error)}`, {
|
|
101
|
+
"error.code": "UNCAUGHT_EXCEPTION",
|
|
102
|
+
"exception.type": name,
|
|
103
|
+
"exception.stacktrace": field(error, "stack", 4_000),
|
|
104
|
+
"error.source": source,
|
|
105
|
+
"error.runtime": "server",
|
|
106
|
+
"deco.framework": framework,
|
|
107
|
+
"deco.component": component(context.component),
|
|
108
|
+
"http.route": routeTemplate,
|
|
109
|
+
"http.request.method":
|
|
110
|
+
method === undefined ? undefined : METHODS.has(method) ? method : "_OTHER",
|
|
111
|
+
"http.response.status_code":
|
|
112
|
+
Number.isInteger(context.status) && context.status! >= 100 && context.status! <= 599
|
|
113
|
+
? context.status
|
|
114
|
+
: undefined,
|
|
115
|
+
});
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
/**
|
|
119
|
+
* Queue a server error using the CMS's current sampling and scrubbing.
|
|
120
|
+
* Safe for a catch handler: no error or rejected promise reaches its caller.
|
|
121
|
+
*/
|
|
122
|
+
export async function reportError(error: unknown, context: ErrorReportContext = {}): Promise<void> {
|
|
123
|
+
try {
|
|
124
|
+
await recordError(error, context);
|
|
125
|
+
} catch {
|
|
126
|
+
// Observability never becomes an application failure.
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
/**
|
|
131
|
+
* Bounded delivery for framework hooks that are awaited before a serverless
|
|
132
|
+
* request ends. Drains the pipeline's metrics/traces too; never waits over 2s.
|
|
133
|
+
* Ordinary background batching is unchanged.
|
|
134
|
+
*/
|
|
135
|
+
export async function flushTelemetry(): Promise<void> {
|
|
136
|
+
const telemetry = currentTelemetry();
|
|
137
|
+
if (telemetry === undefined) return;
|
|
138
|
+
const controller = new AbortController();
|
|
139
|
+
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
140
|
+
try {
|
|
141
|
+
await Promise.race([
|
|
142
|
+
telemetry.flush({ signal: controller.signal, waitForPending: true }),
|
|
143
|
+
new Promise<void>((resolve) => {
|
|
144
|
+
timer = setTimeout(() => {
|
|
145
|
+
controller.abort();
|
|
146
|
+
resolve();
|
|
147
|
+
}, FLUSH_TIMEOUT_MS);
|
|
148
|
+
}),
|
|
149
|
+
]);
|
|
150
|
+
} catch {
|
|
151
|
+
// A collector failure must not escape a framework's error hook.
|
|
152
|
+
} finally {
|
|
153
|
+
clearTimeout(timer);
|
|
154
|
+
}
|
|
155
|
+
}
|
package/src/v8/experiments.ts
CHANGED
|
@@ -6,7 +6,9 @@
|
|
|
6
6
|
* type `split-traffic`); a decision is arithmetic over the visitor's id. What
|
|
7
7
|
* a visitor gets is never stored: it is recomputed from the percentage and the
|
|
8
8
|
* id, so one id serves every test (each roll hashes `name:id`), a new test
|
|
9
|
-
* needs no new state and two tests are independent.
|
|
9
|
+
* needs no new state and two tests are independent. A test named
|
|
10
|
+
* `<parent>/<child>` is drawn only among the visitors in `<parent>`, so a test
|
|
11
|
+
* can split its own arm (50% control, 25% A, 25% B).
|
|
10
12
|
*
|
|
11
13
|
* Pure functions over values and strings: no React, no framework, no module
|
|
12
14
|
* state, no environment reads, no network. Nothing here is called for you:
|
|
@@ -21,8 +23,13 @@
|
|
|
21
23
|
*/
|
|
22
24
|
|
|
23
25
|
import { newId, roll } from "./bucketing.ts";
|
|
26
|
+
import { nestTests } from "./nested.ts";
|
|
24
27
|
|
|
25
|
-
/**
|
|
28
|
+
/**
|
|
29
|
+
* The saved block `experiments`: each test's name and the whole percentage of
|
|
30
|
+
* visitors in it, 0–100. For `<parent>/<child>`, the percentage of the
|
|
31
|
+
* visitors in `<parent>`.
|
|
32
|
+
*/
|
|
26
33
|
export type Experiments = Record<string, number>;
|
|
27
34
|
|
|
28
35
|
/** What a visitor got: each test's name and whether they are in it. */
|
|
@@ -151,7 +158,12 @@ function roundPct(allocation: number): number {
|
|
|
151
158
|
* nobody in, 100 everybody, and raising a percentage only adds people. A
|
|
152
159
|
* fractional percentage is rounded. Without a visitor, or with an id that
|
|
153
160
|
* isn't 22 base64url characters, every test is `false`. Entries that aren't
|
|
154
|
-
* finite numbers are left out.
|
|
161
|
+
* finite numbers are left out.
|
|
162
|
+
*
|
|
163
|
+
* A test named `<parent>/<child>`, where `<parent>` is a test of the same
|
|
164
|
+
* block, is on only for visitors in `<parent>`: its own roll decides among
|
|
165
|
+
* them, so `{ a: 50, "a/b": 50 }` gives 50% neither, 25% `a` alone and 25%
|
|
166
|
+
* both. Pure; never throws.
|
|
155
167
|
*/
|
|
156
168
|
export function assign(
|
|
157
169
|
experiments: Experiments | null | undefined,
|
|
@@ -166,5 +178,5 @@ export function assign(
|
|
|
166
178
|
if (typeof allocation !== "number" || !Number.isFinite(allocation)) continue;
|
|
167
179
|
assignments[name] = known && roll(name, id as string) < roundPct(allocation);
|
|
168
180
|
}
|
|
169
|
-
return assignments;
|
|
181
|
+
return nestTests(assignments);
|
|
170
182
|
}
|
package/src/v8/nested.ts
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Tests within a test: a name written `<parent>/<child>` is drawn only among
|
|
3
|
+
* the visitors in `<parent>`, so a test can split its own arm. With
|
|
4
|
+
* `{ "plp-optimizer": 50, "plp-optimizer/b": 50 }` half the visitors see the
|
|
5
|
+
* control, a quarter `plp-optimizer` alone (A) and a quarter
|
|
6
|
+
* `plp-optimizer/b` too (B); ramping `plp-optimizer` keeps its arm split 50/50.
|
|
7
|
+
*
|
|
8
|
+
* The child keeps its own roll (`<parent>/<child>:id`), independent of the
|
|
9
|
+
* parent's; nesting only turns it off for the visitors outside the parent. The
|
|
10
|
+
* parent is the nearest prefix ending before a `/` that names a test of the
|
|
11
|
+
* same map, so a `/` in a name whose prefix is no test changes nothing.
|
|
12
|
+
* Internal: `assign`, the built-in `split-traffic` and the analytics client
|
|
13
|
+
* apply it; the tests import it by path.
|
|
14
|
+
*/
|
|
15
|
+
|
|
16
|
+
/** The test `name` is drawn within: the nearest `/`-prefix that `has` a test, or `undefined`. */
|
|
17
|
+
export function parentTest(name: string, has: (name: string) => boolean): string | undefined {
|
|
18
|
+
for (let slash = name.lastIndexOf("/"); slash > 0; slash = name.lastIndexOf("/", slash - 1)) {
|
|
19
|
+
const prefix = name.slice(0, slash);
|
|
20
|
+
if (has(prefix)) return prefix;
|
|
21
|
+
}
|
|
22
|
+
return undefined;
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
/**
|
|
26
|
+
* Turns off, in place, every test whose parent is off, so a child is on only
|
|
27
|
+
* when the whole chain above it is. A parent is a prefix of its child, so it
|
|
28
|
+
* sorts first and is final by the time its child is read.
|
|
29
|
+
*/
|
|
30
|
+
export function nestTests(assignments: Record<string, boolean>): Record<string, boolean> {
|
|
31
|
+
const has = (name: string) => Object.hasOwn(assignments, name);
|
|
32
|
+
for (const name of Object.keys(assignments).sort(byCodeUnit)) {
|
|
33
|
+
const parent = parentTest(name, has);
|
|
34
|
+
if (parent !== undefined && assignments[parent] !== true) assignments[name] = false;
|
|
35
|
+
}
|
|
36
|
+
return assignments;
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
/** Plain code-unit order: a prefix before every name it starts. */
|
|
40
|
+
const byCodeUnit = (a: string, b: string): number => (a < b ? -1 : a > b ? 1 : 0);
|