@variantlab/core 0.1.5 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1195 -75
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -2,14 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
> The framework-agnostic A/B testing and feature-flag engine. Zero runtime dependencies, runs anywhere.
|
|
4
4
|
|
|
5
|
-

|
|
6
6
|

|
|
7
7
|

|
|
8
8
|
|
|
9
9
|
## Install
|
|
10
10
|
|
|
11
11
|
```bash
|
|
12
|
-
npm install @variantlab/core
|
|
12
|
+
npm install @variantlab/core
|
|
13
13
|
```
|
|
14
14
|
|
|
15
15
|
## Quick start
|
|
@@ -24,6 +24,7 @@ Create an `experiments.json` file:
|
|
|
24
24
|
"experiments": [
|
|
25
25
|
{
|
|
26
26
|
"id": "cta-copy",
|
|
27
|
+
"name": "CTA button copy",
|
|
27
28
|
"type": "value",
|
|
28
29
|
"default": "buy-now",
|
|
29
30
|
"variants": [
|
|
@@ -33,6 +34,7 @@ Create an `experiments.json` file:
|
|
|
33
34
|
},
|
|
34
35
|
{
|
|
35
36
|
"id": "hero-layout",
|
|
37
|
+
"name": "Hero layout",
|
|
36
38
|
"type": "render",
|
|
37
39
|
"default": "centered",
|
|
38
40
|
"variants": [
|
|
@@ -120,43 +122,6 @@ const trace = explain(experiments, "hero-layout", {
|
|
|
120
122
|
// Returns step-by-step targeting trace with pass/fail per field
|
|
121
123
|
```
|
|
122
124
|
|
|
123
|
-
## Assignment strategies
|
|
124
|
-
|
|
125
|
-
The engine supports multiple assignment strategies per experiment:
|
|
126
|
-
|
|
127
|
-
| Strategy | Description |
|
|
128
|
-
|----------|-------------|
|
|
129
|
-
| `default` | Always assigns the default variant |
|
|
130
|
-
| `random` | Random assignment on each evaluation |
|
|
131
|
-
| `sticky-hash` | Deterministic hash-based assignment (requires `userId`) |
|
|
132
|
-
| `weighted` | Weighted random distribution |
|
|
133
|
-
|
|
134
|
-
```json
|
|
135
|
-
{
|
|
136
|
-
"id": "pricing",
|
|
137
|
-
"type": "value",
|
|
138
|
-
"default": "low",
|
|
139
|
-
"assignment": { "strategy": "sticky-hash" },
|
|
140
|
-
"variants": [
|
|
141
|
-
{ "id": "low", "value": 9.99, "weight": 50 },
|
|
142
|
-
{ "id": "high", "value": 14.99, "weight": 50 }
|
|
143
|
-
]
|
|
144
|
-
}
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
## Targeting operators
|
|
148
|
-
|
|
149
|
-
Built-in targeting predicates:
|
|
150
|
-
|
|
151
|
-
- `platform` — `ios`, `android`, `web`
|
|
152
|
-
- `appVersion` — semver ranges (`>=1.2.0`, `^2.0.0`)
|
|
153
|
-
- `locale` — locale codes (`en`, `bn`, `fr`)
|
|
154
|
-
- `screenSize` — `small`, `medium`, `large`
|
|
155
|
-
- `routes` — glob patterns (`/settings/*`, `/dashboard`)
|
|
156
|
-
- `userId` — exact match or list
|
|
157
|
-
- `attributes` — custom key-value matching
|
|
158
|
-
- `predicate` — compound `and`/`or`/`not` logic
|
|
159
|
-
|
|
160
125
|
## Key features
|
|
161
126
|
|
|
162
127
|
- Zero runtime dependencies
|
|
@@ -181,53 +146,1208 @@ Use `@variantlab/core` directly for vanilla JS/TS, or pair it with a framework a
|
|
|
181
146
|
|
|
182
147
|
---
|
|
183
148
|
|
|
184
|
-
|
|
149
|
+
# Config format (`experiments.json`)
|
|
150
|
+
|
|
151
|
+
The canonical specification for the `experiments.json` file.
|
|
152
|
+
|
|
153
|
+
## File structure
|
|
154
|
+
|
|
155
|
+
```json
|
|
156
|
+
{
|
|
157
|
+
"$schema": "https://variantlab.dev/schemas/experiments.schema.json",
|
|
158
|
+
"version": 1,
|
|
159
|
+
"signature": "base64url-hmac-optional",
|
|
160
|
+
"enabled": true,
|
|
161
|
+
"experiments": [
|
|
162
|
+
{
|
|
163
|
+
"id": "example",
|
|
164
|
+
"name": "Example experiment",
|
|
165
|
+
"variants": [
|
|
166
|
+
{ "id": "a" },
|
|
167
|
+
{ "id": "b" }
|
|
168
|
+
],
|
|
169
|
+
"default": "a"
|
|
170
|
+
}
|
|
171
|
+
]
|
|
172
|
+
}
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
## Top-level fields
|
|
176
|
+
|
|
177
|
+
| Field | Type | Required | Description |
|
|
178
|
+
|---|---|:-:|---|
|
|
179
|
+
| `$schema` | string | No | JSON Schema reference for IDE support. Ignored by the engine. |
|
|
180
|
+
| `version` | integer | Yes | Schema version. Must be `1`. The engine rejects unknown versions. |
|
|
181
|
+
| `signature` | string | No | Base64url-encoded HMAC-SHA256 of the canonical form of `experiments`. Verified via Web Crypto API when an `hmacKey` is configured. |
|
|
182
|
+
| `enabled` | boolean | No | Global kill switch. When `false`, all experiments return their default variant. Defaults to `true`. |
|
|
183
|
+
| `experiments` | array | Yes | Array of experiment definitions. Max 1000 entries. |
|
|
184
|
+
|
|
185
|
+
## Experiment fields
|
|
186
|
+
|
|
187
|
+
| Field | Type | Required | Default | Description |
|
|
188
|
+
|---|---|:-:|---|---|
|
|
189
|
+
| `id` | string | Yes | — | Unique identifier. `/^[a-z0-9][a-z0-9-]{0,63}$/` |
|
|
190
|
+
| `name` | string | Yes | — | Human-readable. Max 128 chars. |
|
|
191
|
+
| `description` | string | No | — | Shown in debug overlay. Max 512 chars. |
|
|
192
|
+
| `type` | enum | No | `"render"` | `"render"` for component swaps, `"value"` for returned values. |
|
|
193
|
+
| `variants` | array | Yes | — | At least 2, at most 100. |
|
|
194
|
+
| `default` | string | Yes | — | Must match one of `variants[].id`. |
|
|
195
|
+
| `routes` | array | No | — | Glob patterns. Max 100. |
|
|
196
|
+
| `targeting` | object | No | — | Targeting predicate. |
|
|
197
|
+
| `assignment` | enum | No | `"default"` | Strategy: `default | random | sticky-hash | weighted`. |
|
|
198
|
+
| `split` | object | No | — | Traffic split for `weighted` strategy. |
|
|
199
|
+
| `mutex` | string | No | — | Mutual exclusion group. |
|
|
200
|
+
| `rollback` | object | No | — | Crash-rollback configuration. |
|
|
201
|
+
| `status` | enum | No | `"active"` | `draft | active | archived`. |
|
|
202
|
+
| `startDate` | ISO 8601 | No | — | Inactive before this. |
|
|
203
|
+
| `endDate` | ISO 8601 | No | — | Inactive after this. |
|
|
204
|
+
| `owner` | string | No | — | Free text. Max 128 chars. |
|
|
205
|
+
| `overridable` | boolean | No | `false` | Whether deep link overrides are accepted. |
|
|
206
|
+
|
|
207
|
+
### Experiment `id`
|
|
208
|
+
|
|
209
|
+
Case-sensitive, lowercase. Allowed characters: `a-z`, `0-9`, `-`. Max 64 characters. Must not start with a hyphen.
|
|
210
|
+
|
|
211
|
+
- `cta-copy` — valid
|
|
212
|
+
- `news-card-layout` — valid
|
|
213
|
+
- `checkout-v2` — valid
|
|
214
|
+
- `CTA_copy` — invalid (uppercase + underscore)
|
|
215
|
+
- `-leading-dash` — invalid
|
|
216
|
+
|
|
217
|
+
### Experiment `type`
|
|
218
|
+
|
|
219
|
+
- `"render"`: designed for `<Variant>` component-swap usage. Variants don't need a `value`.
|
|
220
|
+
- `"value"`: designed for `useVariantValue` usage. Each variant has a `value` field.
|
|
221
|
+
|
|
222
|
+
### Experiment `default`
|
|
223
|
+
|
|
224
|
+
Required. Must reference a valid variant ID. Used when:
|
|
225
|
+
|
|
226
|
+
- Targeting fails
|
|
227
|
+
- Kill switch is on
|
|
228
|
+
- `startDate` is in the future or `endDate` is in the past
|
|
229
|
+
- Engine is in fail-open mode and an error occurs
|
|
230
|
+
- Deep link override is not allowed
|
|
231
|
+
- Crash rollback has triggered
|
|
232
|
+
|
|
233
|
+
### Experiment `routes`
|
|
234
|
+
|
|
235
|
+
Glob patterns matching current route/pathname. Supported patterns:
|
|
236
|
+
|
|
237
|
+
- Exact: `/`, `/about`
|
|
238
|
+
- Wildcard segment: `/blog/*`
|
|
239
|
+
- Wildcard deep: `/docs/**`
|
|
240
|
+
- Parameter: `/user/:id`
|
|
241
|
+
- Trailing-slash insensitive
|
|
242
|
+
|
|
243
|
+
### Assignment strategies
|
|
244
|
+
|
|
245
|
+
- `"default"` — always return the default variant. Useful for pre-launch experiments.
|
|
246
|
+
- `"random"` — uniform random across variants, assigned once per user and cached.
|
|
247
|
+
- `"sticky-hash"` — deterministic hash of `(userId, experimentId)` mapped to a variant. Stable across devices for the same `userId`.
|
|
248
|
+
- `"weighted"` — traffic split via `split` field. Uses sticky-hash for determinism.
|
|
249
|
+
|
|
250
|
+
### Traffic split
|
|
251
|
+
|
|
252
|
+
Required when `assignment: "weighted"`. Object mapping variant IDs to integer percentages summing to 100.
|
|
253
|
+
|
|
254
|
+
```json
|
|
255
|
+
{
|
|
256
|
+
"id": "pricing",
|
|
257
|
+
"type": "value",
|
|
258
|
+
"default": "low",
|
|
259
|
+
"assignment": "weighted",
|
|
260
|
+
"split": { "control": 50, "treatment-a": 25, "treatment-b": 25 },
|
|
261
|
+
"variants": [
|
|
262
|
+
{ "id": "control", "value": 9.99 },
|
|
263
|
+
{ "id": "treatment-a", "value": 12.99 },
|
|
264
|
+
{ "id": "treatment-b", "value": 14.99 }
|
|
265
|
+
]
|
|
266
|
+
}
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
### Mutex groups
|
|
270
|
+
|
|
271
|
+
Experiments with the same `mutex` cannot co-run on the same user. When two mutex'd experiments both target a user, the engine picks one by stable hash and excludes the others.
|
|
272
|
+
|
|
273
|
+
### Experiment `status`
|
|
274
|
+
|
|
275
|
+
- `"draft"` — visible in debug overlay (with a draft badge), returns default in production
|
|
276
|
+
- `"active"` — normal operation
|
|
277
|
+
- `"archived"` — hidden from debug overlay, returns default
|
|
278
|
+
|
|
279
|
+
### Time gates (`startDate` / `endDate`)
|
|
280
|
+
|
|
281
|
+
ISO 8601 timestamps. Inclusive start, exclusive end. Useful for time-boxed rollouts.
|
|
282
|
+
|
|
283
|
+
## Variant fields
|
|
284
|
+
|
|
285
|
+
| Field | Type | Required | Description |
|
|
286
|
+
|---|---|:-:|---|
|
|
287
|
+
| `id` | string | Yes | Unique within the experiment. Same regex as experiment ID. |
|
|
288
|
+
| `label` | string | No | Human-readable. Shown in debug overlay. Max 128 chars. |
|
|
289
|
+
| `description` | string | No | Shown in debug overlay. Max 512 chars. |
|
|
290
|
+
| `value` | any | No | For `type: "value"` experiments, the value returned by `getVariantValue`. |
|
|
291
|
+
|
|
292
|
+
### Variant `value`
|
|
293
|
+
|
|
294
|
+
Any JSON-serializable value. Strings, numbers, booleans, arrays, and objects are all supported. Type safety on the JS/TS side comes via codegen or explicit generic arguments.
|
|
295
|
+
|
|
296
|
+
## Rollback fields
|
|
297
|
+
|
|
298
|
+
| Field | Type | Required | Default | Description |
|
|
299
|
+
|---|---|:-:|---|---|
|
|
300
|
+
| `threshold` | integer | Yes | `3` | Crashes that trigger rollback. 1-100. |
|
|
301
|
+
| `window` | integer | Yes | `60000` | Time window in ms. 1000-3600000. |
|
|
302
|
+
| `persistent` | boolean | No | `false` | Persist rollback across sessions. |
|
|
303
|
+
|
|
304
|
+
When enabled, if a variant crashes `threshold` times within `window` milliseconds, the engine:
|
|
305
|
+
|
|
306
|
+
1. Clears the user's assignment for that experiment
|
|
307
|
+
2. Forces the `default` variant
|
|
308
|
+
3. Emits an `onRollback` event
|
|
309
|
+
4. If `persistent`, stores the rollback in Storage
|
|
310
|
+
|
|
311
|
+
## Validation rules
|
|
312
|
+
|
|
313
|
+
The engine validates configs at load time and rejects:
|
|
314
|
+
|
|
315
|
+
- Unknown version (`version !== 1`)
|
|
316
|
+
- Config larger than 1 MB
|
|
317
|
+
- Duplicate experiment IDs
|
|
318
|
+
- Duplicate variant IDs within an experiment
|
|
319
|
+
- `default` that doesn't match any variant
|
|
320
|
+
- `split` sum != 100 when assignment is `weighted`
|
|
321
|
+
- Invalid route globs (unsupported patterns)
|
|
322
|
+
- Invalid semver ranges
|
|
323
|
+
- Targeting nesting deeper than 10 levels
|
|
324
|
+
- Invalid ISO 8601 timestamps
|
|
325
|
+
- Reserved keys (`__proto__`, `constructor`, `prototype`)
|
|
326
|
+
|
|
327
|
+
Errors are collected and thrown as a `ConfigValidationError` with an `issues` array. In fail-open mode (default), the engine logs the error and falls back to returning defaults. In fail-closed mode, it throws.
|
|
328
|
+
|
|
329
|
+
## Config examples
|
|
330
|
+
|
|
331
|
+
### Simple value experiment
|
|
332
|
+
|
|
333
|
+
```json
|
|
334
|
+
{
|
|
335
|
+
"version": 1,
|
|
336
|
+
"experiments": [
|
|
337
|
+
{
|
|
338
|
+
"id": "cta-copy",
|
|
339
|
+
"name": "CTA button copy",
|
|
340
|
+
"type": "value",
|
|
341
|
+
"default": "buy-now",
|
|
342
|
+
"variants": [
|
|
343
|
+
{ "id": "buy-now", "value": "Buy now" },
|
|
344
|
+
{ "id": "get-started", "value": "Get started" },
|
|
345
|
+
{ "id": "try-free", "value": "Try it free" }
|
|
346
|
+
]
|
|
347
|
+
}
|
|
348
|
+
]
|
|
349
|
+
}
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
### Render experiment with route scope
|
|
353
|
+
|
|
354
|
+
```json
|
|
355
|
+
{
|
|
356
|
+
"version": 1,
|
|
357
|
+
"experiments": [
|
|
358
|
+
{
|
|
359
|
+
"id": "news-card-layout",
|
|
360
|
+
"name": "News card layout",
|
|
361
|
+
"routes": ["/", "/feed"],
|
|
362
|
+
"targeting": { "screenSize": ["small"] },
|
|
363
|
+
"default": "responsive",
|
|
364
|
+
"variants": [
|
|
365
|
+
{ "id": "responsive", "label": "Responsive image" },
|
|
366
|
+
{ "id": "scale-to-fit", "label": "Scale to fit" },
|
|
367
|
+
{ "id": "pip-thumbnail", "label": "PIP thumbnail" }
|
|
368
|
+
]
|
|
369
|
+
}
|
|
370
|
+
]
|
|
371
|
+
}
|
|
372
|
+
```
|
|
373
|
+
|
|
374
|
+
### Weighted rollout with rollback
|
|
375
|
+
|
|
376
|
+
```json
|
|
377
|
+
{
|
|
378
|
+
"version": 1,
|
|
379
|
+
"experiments": [
|
|
380
|
+
{
|
|
381
|
+
"id": "new-checkout",
|
|
382
|
+
"name": "New checkout flow",
|
|
383
|
+
"assignment": "weighted",
|
|
384
|
+
"split": { "control": 90, "new": 10 },
|
|
385
|
+
"default": "control",
|
|
386
|
+
"variants": [
|
|
387
|
+
{ "id": "control" },
|
|
388
|
+
{ "id": "new" }
|
|
389
|
+
],
|
|
390
|
+
"rollback": {
|
|
391
|
+
"threshold": 5,
|
|
392
|
+
"window": 120000,
|
|
393
|
+
"persistent": true
|
|
394
|
+
}
|
|
395
|
+
}
|
|
396
|
+
]
|
|
397
|
+
}
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
### Time-boxed experiment
|
|
401
|
+
|
|
402
|
+
```json
|
|
403
|
+
{
|
|
404
|
+
"version": 1,
|
|
405
|
+
"experiments": [
|
|
406
|
+
{
|
|
407
|
+
"id": "black-friday-banner",
|
|
408
|
+
"name": "Black Friday banner",
|
|
409
|
+
"type": "render",
|
|
410
|
+
"startDate": "2026-11-24T00:00:00Z",
|
|
411
|
+
"endDate": "2026-12-01T00:00:00Z",
|
|
412
|
+
"default": "hidden",
|
|
413
|
+
"variants": [
|
|
414
|
+
{ "id": "hidden" },
|
|
415
|
+
{ "id": "shown" }
|
|
416
|
+
]
|
|
417
|
+
}
|
|
418
|
+
]
|
|
419
|
+
}
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
### Targeted beta
|
|
423
|
+
|
|
424
|
+
```json
|
|
425
|
+
{
|
|
426
|
+
"version": 1,
|
|
427
|
+
"experiments": [
|
|
428
|
+
{
|
|
429
|
+
"id": "ai-assistant",
|
|
430
|
+
"name": "AI assistant beta",
|
|
431
|
+
"targeting": {
|
|
432
|
+
"platform": ["ios", "android"],
|
|
433
|
+
"appVersion": ">=2.0.0",
|
|
434
|
+
"attributes": { "betaOptIn": true }
|
|
435
|
+
},
|
|
436
|
+
"default": "disabled",
|
|
437
|
+
"variants": [
|
|
438
|
+
{ "id": "disabled" },
|
|
439
|
+
{ "id": "enabled" }
|
|
440
|
+
]
|
|
441
|
+
}
|
|
442
|
+
]
|
|
443
|
+
}
|
|
444
|
+
```
|
|
445
|
+
|
|
446
|
+
---
|
|
447
|
+
|
|
448
|
+
# Targeting DSL
|
|
449
|
+
|
|
450
|
+
How targeting predicates work and the semantics of each operator.
|
|
451
|
+
|
|
452
|
+
## The predicate shape
|
|
453
|
+
|
|
454
|
+
```ts
|
|
455
|
+
interface Targeting {
|
|
456
|
+
platform?: Array<"ios" | "android" | "web" | "node">;
|
|
457
|
+
appVersion?: string; // semver range
|
|
458
|
+
locale?: string[]; // IETF language tags
|
|
459
|
+
screenSize?: Array<"small" | "medium" | "large">;
|
|
460
|
+
routes?: string[]; // glob patterns
|
|
461
|
+
userId?: string[] | { hash: "sha256"; mod: number };
|
|
462
|
+
attributes?: Record<string, string | number | boolean>;
|
|
463
|
+
predicate?: (context: VariantContext) => boolean; // escape hatch, code-only
|
|
464
|
+
}
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
A `Targeting` object is an **implicit AND** of all specified fields. If no fields are specified, the predicate matches every user.
|
|
468
|
+
|
|
469
|
+
## Evaluation semantics
|
|
470
|
+
|
|
471
|
+
```
|
|
472
|
+
match(targeting, context) =
|
|
473
|
+
platform_match(targeting.platform, context.platform)
|
|
474
|
+
AND appVersion_match(targeting.appVersion, context.appVersion)
|
|
475
|
+
AND locale_match(targeting.locale, context.locale)
|
|
476
|
+
AND screenSize_match(targeting.screenSize, context.screenSize)
|
|
477
|
+
AND routes_match(targeting.routes, context.route)
|
|
478
|
+
AND userId_match(targeting.userId, context.userId)
|
|
479
|
+
AND attributes_match(targeting.attributes, context.attributes)
|
|
480
|
+
AND predicate(context)
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
Each sub-match is:
|
|
484
|
+
|
|
485
|
+
- **True if the field is not specified in targeting** (open by default)
|
|
486
|
+
- **True if the specified predicate matches**
|
|
487
|
+
- **False otherwise**
|
|
488
|
+
|
|
489
|
+
An unspecified field in the context does **not** match a specified targeting field. For example, if `targeting.platform` is `["ios"]` and `context.platform` is `undefined`, the targeting fails.
|
|
490
|
+
|
|
491
|
+
## Targeting operators
|
|
492
|
+
|
|
493
|
+
### `platform`
|
|
494
|
+
|
|
495
|
+
```ts
|
|
496
|
+
platform?: Array<"ios" | "android" | "web" | "node">;
|
|
497
|
+
```
|
|
498
|
+
|
|
499
|
+
Set membership. Matches if `context.platform` is in the array.
|
|
500
|
+
|
|
501
|
+
- `"ios"` — iOS, iPadOS
|
|
502
|
+
- `"android"` — Android
|
|
503
|
+
- `"web"` — any browser environment (desktop web, mobile web, PWA)
|
|
504
|
+
- `"node"` — server-side (SSR, edge runtimes)
|
|
505
|
+
|
|
506
|
+
### `appVersion`
|
|
507
|
+
|
|
508
|
+
```ts
|
|
509
|
+
appVersion?: string; // semver range
|
|
510
|
+
```
|
|
511
|
+
|
|
512
|
+
Semver range matching. Supported syntax (subset of npm semver):
|
|
513
|
+
|
|
514
|
+
- Comparators: `=`, `<`, `<=`, `>`, `>=`
|
|
515
|
+
- Caret: `^1.2.0` (>= 1.2.0 < 2.0.0)
|
|
516
|
+
- Tilde: `~1.2.0` (>= 1.2.0 < 1.3.0)
|
|
517
|
+
- Range: `1.2.0 - 2.0.0`
|
|
518
|
+
- Compound: `>=1.0.0 <2.0.0`
|
|
519
|
+
- OR ranges: `>=1.0.0 <2.0.0 || >=3.0.0`
|
|
520
|
+
|
|
521
|
+
### `locale`
|
|
522
|
+
|
|
523
|
+
```ts
|
|
524
|
+
locale?: string[]; // IETF language tags
|
|
525
|
+
```
|
|
526
|
+
|
|
527
|
+
Two match modes:
|
|
528
|
+
|
|
529
|
+
- **Exact**: `"en-US"` matches `"en-US"` only
|
|
530
|
+
- **Prefix**: `"en"` matches `"en"`, `"en-US"`, `"en-GB"`, etc.
|
|
531
|
+
|
|
532
|
+
### `screenSize`
|
|
533
|
+
|
|
534
|
+
```ts
|
|
535
|
+
screenSize?: Array<"small" | "medium" | "large">;
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
Set membership on pre-bucketed screen sizes:
|
|
539
|
+
|
|
540
|
+
- `"small"`: `max(width, height) < 700 px`
|
|
541
|
+
- `"medium"`: `700 <= max(width, height) < 1200 px`
|
|
542
|
+
- `"large"`: `max(width, height) >= 1200 px`
|
|
543
|
+
|
|
544
|
+
Thresholds are configurable at engine creation.
|
|
545
|
+
|
|
546
|
+
### `routes`
|
|
547
|
+
|
|
548
|
+
```ts
|
|
549
|
+
routes?: string[]; // glob patterns
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
Matches if `context.route` matches any pattern. Supported:
|
|
553
|
+
|
|
554
|
+
- Exact: `/about`
|
|
555
|
+
- Wildcard segment: `/blog/*`
|
|
556
|
+
- Wildcard deep: `/docs/**`
|
|
557
|
+
- Parameter: `/user/:id`
|
|
558
|
+
- Trailing slash insensitive
|
|
559
|
+
|
|
560
|
+
### `userId`
|
|
561
|
+
|
|
562
|
+
```ts
|
|
563
|
+
userId?: string[] | { hash: "sha256"; mod: number };
|
|
564
|
+
```
|
|
565
|
+
|
|
566
|
+
Two modes:
|
|
567
|
+
|
|
568
|
+
**Explicit list:**
|
|
569
|
+
|
|
570
|
+
```json
|
|
571
|
+
"userId": ["alice", "bob", "charlie"]
|
|
572
|
+
```
|
|
573
|
+
|
|
574
|
+
Matches if `context.userId` is in the list. Max 10,000 entries.
|
|
575
|
+
|
|
576
|
+
**Hash bucket:**
|
|
577
|
+
|
|
578
|
+
```json
|
|
579
|
+
"userId": { "hash": "sha256", "mod": 10 }
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
Matches if `sha256(userId) % 100 < mod`. In this example, 10% of users match. Uses Web Crypto API for uniform distribution.
|
|
583
|
+
|
|
584
|
+
### `attributes`
|
|
585
|
+
|
|
586
|
+
```ts
|
|
587
|
+
attributes?: Record<string, string | number | boolean>;
|
|
588
|
+
```
|
|
589
|
+
|
|
590
|
+
Exact-match predicate on `context.attributes`. Every specified key must match exactly.
|
|
591
|
+
|
|
592
|
+
```json
|
|
593
|
+
"targeting": {
|
|
594
|
+
"attributes": {
|
|
595
|
+
"plan": "premium",
|
|
596
|
+
"region": "us-west",
|
|
597
|
+
"betaOptIn": true
|
|
598
|
+
}
|
|
599
|
+
}
|
|
600
|
+
```
|
|
601
|
+
|
|
602
|
+
### The `predicate` escape hatch
|
|
603
|
+
|
|
604
|
+
```ts
|
|
605
|
+
targeting: {
|
|
606
|
+
predicate: (context) => context.daysSinceInstall > 7 && context.isPremium
|
|
607
|
+
}
|
|
608
|
+
```
|
|
609
|
+
|
|
610
|
+
The `predicate` field is a function available **only in application code**, never in JSON configs. It is ANDed with the other targeting fields. Use it for complex logic not covered by built-in operators.
|
|
611
|
+
|
|
612
|
+
## Evaluation order
|
|
613
|
+
|
|
614
|
+
The engine evaluates predicates in this order for fast short-circuiting:
|
|
615
|
+
|
|
616
|
+
1. `enabled` kill switch (O(1))
|
|
617
|
+
2. `startDate` / `endDate` (O(1))
|
|
618
|
+
3. `platform` (O(n), n <= 4)
|
|
619
|
+
4. `screenSize` (O(n), n <= 3)
|
|
620
|
+
5. `locale` (O(n))
|
|
621
|
+
6. `appVersion` (O(n), n = range tokens)
|
|
622
|
+
7. `routes` (O(n x m), n = patterns, m = path segments)
|
|
623
|
+
8. `attributes` (O(n))
|
|
624
|
+
9. `userId` (O(n) for list; O(hash) for bucket)
|
|
625
|
+
10. `predicate` (O(?) — unknown, runs last)
|
|
626
|
+
|
|
627
|
+
## Targeting examples
|
|
628
|
+
|
|
629
|
+
### Target iOS users on small screens running the latest version
|
|
630
|
+
|
|
631
|
+
```json
|
|
632
|
+
"targeting": {
|
|
633
|
+
"platform": ["ios"],
|
|
634
|
+
"screenSize": ["small"],
|
|
635
|
+
"appVersion": ">=2.0.0"
|
|
636
|
+
}
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
### Target 10% of users deterministically
|
|
640
|
+
|
|
641
|
+
```json
|
|
642
|
+
"targeting": {
|
|
643
|
+
"userId": { "hash": "sha256", "mod": 10 }
|
|
644
|
+
}
|
|
645
|
+
```
|
|
646
|
+
|
|
647
|
+
### Target premium users in Bengali locale
|
|
648
|
+
|
|
649
|
+
```json
|
|
650
|
+
"targeting": {
|
|
651
|
+
"locale": ["bn"],
|
|
652
|
+
"attributes": { "plan": "premium" }
|
|
653
|
+
}
|
|
654
|
+
```
|
|
655
|
+
|
|
656
|
+
### Time-based targeting (application code)
|
|
657
|
+
|
|
658
|
+
```ts
|
|
659
|
+
const targeting = {
|
|
660
|
+
platform: ["ios", "android"],
|
|
661
|
+
predicate: (ctx) => {
|
|
662
|
+
const installDate = new Date(ctx.attributes.installDate as string);
|
|
663
|
+
const daysSinceInstall = (Date.now() - installDate.getTime()) / 86400000;
|
|
664
|
+
return daysSinceInstall >= 7 && daysSinceInstall <= 30;
|
|
665
|
+
}
|
|
666
|
+
};
|
|
667
|
+
```
|
|
668
|
+
|
|
669
|
+
---
|
|
670
|
+
|
|
671
|
+
# API Reference
|
|
672
|
+
|
|
673
|
+
Complete TypeScript API surface for all packages.
|
|
674
|
+
|
|
675
|
+
## Core types
|
|
676
|
+
|
|
677
|
+
```ts
|
|
678
|
+
/** Top-level config file shape. */
|
|
679
|
+
export interface ExperimentsConfig {
|
|
680
|
+
version: 1;
|
|
681
|
+
signature?: string;
|
|
682
|
+
enabled?: boolean;
|
|
683
|
+
experiments: Experiment[];
|
|
684
|
+
}
|
|
685
|
+
|
|
686
|
+
/** A single experiment definition. */
|
|
687
|
+
export interface Experiment {
|
|
688
|
+
id: string;
|
|
689
|
+
name: string;
|
|
690
|
+
description?: string;
|
|
691
|
+
type?: "render" | "value";
|
|
692
|
+
variants: Variant[];
|
|
693
|
+
default: string;
|
|
694
|
+
routes?: string[];
|
|
695
|
+
targeting?: Targeting;
|
|
696
|
+
assignment?: AssignmentStrategy;
|
|
697
|
+
split?: Record<string, number>;
|
|
698
|
+
mutex?: string;
|
|
699
|
+
rollback?: RollbackConfig;
|
|
700
|
+
status?: "draft" | "active" | "archived";
|
|
701
|
+
startDate?: string;
|
|
702
|
+
endDate?: string;
|
|
703
|
+
owner?: string;
|
|
704
|
+
}
|
|
705
|
+
|
|
706
|
+
/** A variant of an experiment. */
|
|
707
|
+
export interface Variant {
|
|
708
|
+
id: string;
|
|
709
|
+
label?: string;
|
|
710
|
+
description?: string;
|
|
711
|
+
value?: unknown;
|
|
712
|
+
}
|
|
713
|
+
|
|
714
|
+
/** Runtime context used for targeting and assignment. */
|
|
715
|
+
export interface VariantContext {
|
|
716
|
+
userId?: string;
|
|
717
|
+
route?: string;
|
|
718
|
+
platform?: string;
|
|
719
|
+
appVersion?: string;
|
|
720
|
+
locale?: string;
|
|
721
|
+
screenSize?: "small" | "medium" | "large";
|
|
722
|
+
attributes?: Record<string, string | number | boolean>;
|
|
723
|
+
}
|
|
724
|
+
|
|
725
|
+
/** Targeting predicate. All fields are optional; all specified fields must match. */
|
|
726
|
+
export interface Targeting {
|
|
727
|
+
platform?: Array<"ios" | "android" | "web" | "node">;
|
|
728
|
+
appVersion?: string;
|
|
729
|
+
locale?: string[];
|
|
730
|
+
screenSize?: Array<"small" | "medium" | "large">;
|
|
731
|
+
routes?: string[];
|
|
732
|
+
userId?: string[] | { hash: string; mod: number };
|
|
733
|
+
attributes?: Record<string, unknown>;
|
|
734
|
+
predicate?: (context: VariantContext) => boolean;
|
|
735
|
+
}
|
|
736
|
+
|
|
737
|
+
/** Assignment strategy. */
|
|
738
|
+
export type AssignmentStrategy =
|
|
739
|
+
| "default" // always return the default variant
|
|
740
|
+
| "random" // uniform random on first eligibility
|
|
741
|
+
| "sticky-hash" // deterministic hash of (userId, experimentId)
|
|
742
|
+
| "weighted"; // weighted by split config, sticky-hashed
|
|
743
|
+
|
|
744
|
+
/** Crash-rollback configuration. */
|
|
745
|
+
export interface RollbackConfig {
|
|
746
|
+
threshold: number;
|
|
747
|
+
window: number;
|
|
748
|
+
persistent?: boolean;
|
|
749
|
+
}
|
|
750
|
+
```
|
|
751
|
+
|
|
752
|
+
## Engine API
|
|
753
|
+
|
|
754
|
+
```ts
|
|
755
|
+
/** Options passed to createEngine. */
|
|
756
|
+
export interface EngineOptions {
|
|
757
|
+
storage: Storage;
|
|
758
|
+
fetcher?: Fetcher;
|
|
759
|
+
telemetry?: Telemetry;
|
|
760
|
+
hmacKey?: Uint8Array | CryptoKey;
|
|
761
|
+
context?: VariantContext;
|
|
762
|
+
errorMode?: "fail-open" | "fail-closed";
|
|
763
|
+
timeTravel?: boolean;
|
|
764
|
+
}
|
|
765
|
+
|
|
766
|
+
/** The runtime engine. */
|
|
767
|
+
export class VariantEngine {
|
|
768
|
+
constructor(config: ExperimentsConfig, options: EngineOptions);
|
|
769
|
+
|
|
770
|
+
/** Get the current variant ID for an experiment. Synchronous, O(1) after warmup. */
|
|
771
|
+
getVariant(experimentId: string, context?: VariantContext): string;
|
|
772
|
+
|
|
773
|
+
/** Get the variant value (for "value" experiments). */
|
|
774
|
+
getVariantValue<T = unknown>(experimentId: string, context?: VariantContext): T;
|
|
775
|
+
|
|
776
|
+
/** Force a variant. Used by debug overlay and deep links. */
|
|
777
|
+
setVariant(experimentId: string, variantId: string): void;
|
|
778
|
+
|
|
779
|
+
/** Clear a forced variant, falling back to assignment. */
|
|
780
|
+
clearVariant(experimentId: string): void;
|
|
781
|
+
|
|
782
|
+
/** Clear all forced variants. */
|
|
783
|
+
resetAll(): void;
|
|
784
|
+
|
|
785
|
+
/** Get all experiments, optionally filtered by route. */
|
|
786
|
+
getExperiments(route?: string): Experiment[];
|
|
787
|
+
|
|
788
|
+
/** Subscribe to variant changes. Returns unsubscribe function. */
|
|
789
|
+
subscribe(listener: (event: EngineEvent) => void): () => void;
|
|
790
|
+
|
|
791
|
+
/** Update runtime context. Triggers re-evaluation of all experiments. */
|
|
792
|
+
updateContext(patch: Partial<VariantContext>): void;
|
|
793
|
+
|
|
794
|
+
/** Replace the config (e.g., after a remote fetch). Validates + verifies signature. */
|
|
795
|
+
loadConfig(config: ExperimentsConfig): Promise<void>;
|
|
796
|
+
|
|
797
|
+
/** Report a crash for rollback tracking. */
|
|
798
|
+
reportCrash(experimentId: string, error: Error): void;
|
|
799
|
+
|
|
800
|
+
/** Track an arbitrary event. Forwarded to telemetry. */
|
|
801
|
+
track(eventName: string, properties?: Record<string, unknown>): void;
|
|
802
|
+
|
|
803
|
+
/** Get time-travel history (if enabled). */
|
|
804
|
+
getHistory(): EngineEvent[];
|
|
805
|
+
|
|
806
|
+
/** Clean up subscriptions, timers, and listeners. */
|
|
807
|
+
dispose(): void;
|
|
808
|
+
}
|
|
809
|
+
|
|
810
|
+
/** Factory — preferred over `new VariantEngine()`. */
|
|
811
|
+
export function createEngine(
|
|
812
|
+
config: ExperimentsConfig,
|
|
813
|
+
options: EngineOptions
|
|
814
|
+
): VariantEngine;
|
|
815
|
+
|
|
816
|
+
/** Events emitted by the engine. */
|
|
817
|
+
export type EngineEvent =
|
|
818
|
+
| { type: "ready"; config: ExperimentsConfig }
|
|
819
|
+
| { type: "assignment"; experimentId: string; variantId: string; context: VariantContext }
|
|
820
|
+
| { type: "exposure"; experimentId: string; variantId: string }
|
|
821
|
+
| { type: "variantChanged"; experimentId: string; variantId: string; source: "user" | "system" | "deeplink" | "qr" }
|
|
822
|
+
| { type: "rollback"; experimentId: string; variantId: string; reason: string }
|
|
823
|
+
| { type: "configLoaded"; config: ExperimentsConfig }
|
|
824
|
+
| { type: "error"; error: Error };
|
|
825
|
+
```
|
|
826
|
+
|
|
827
|
+
## Storage interface
|
|
828
|
+
|
|
829
|
+
```ts
|
|
830
|
+
/** Pluggable storage adapter. All methods may be sync or async. */
|
|
831
|
+
export interface Storage {
|
|
832
|
+
getItem(key: string): string | null | Promise<string | null>;
|
|
833
|
+
setItem(key: string, value: string): void | Promise<void>;
|
|
834
|
+
removeItem(key: string): void | Promise<void>;
|
|
835
|
+
keys?(): string[] | Promise<string[]>;
|
|
836
|
+
}
|
|
837
|
+
|
|
838
|
+
/** In-memory storage, useful for tests and SSR. */
|
|
839
|
+
export function createMemoryStorage(): Storage;
|
|
840
|
+
```
|
|
841
|
+
|
|
842
|
+
Adapter packages provide concrete implementations:
|
|
843
|
+
|
|
844
|
+
- `@variantlab/react-native` exports `AsyncStorageAdapter`, `MMKVStorageAdapter`, `SecureStoreAdapter`
|
|
845
|
+
- `@variantlab/react` exports `LocalStorageAdapter`, `SessionStorageAdapter`, `CookieStorageAdapter`
|
|
846
|
+
- `@variantlab/next` exports `NextCookieAdapter` (SSR-aware)
|
|
847
|
+
|
|
848
|
+
## Fetcher interface
|
|
849
|
+
|
|
850
|
+
```ts
|
|
851
|
+
/** Optional remote config fetcher. */
|
|
852
|
+
export interface Fetcher {
|
|
853
|
+
fetch(): Promise<ExperimentsConfig>;
|
|
854
|
+
pollInterval?: number;
|
|
855
|
+
}
|
|
856
|
+
|
|
857
|
+
/** Simple HTTP fetcher helper. */
|
|
858
|
+
export function createHttpFetcher(options: {
|
|
859
|
+
url: string;
|
|
860
|
+
headers?: Record<string, string>;
|
|
861
|
+
pollInterval?: number;
|
|
862
|
+
signal?: AbortSignal;
|
|
863
|
+
}): Fetcher;
|
|
864
|
+
```
|
|
865
|
+
|
|
866
|
+
## Telemetry interface
|
|
867
|
+
|
|
868
|
+
```ts
|
|
869
|
+
/** Optional telemetry sink. Called for every engine event. */
|
|
870
|
+
export interface Telemetry {
|
|
871
|
+
track(event: EngineEvent): void;
|
|
872
|
+
}
|
|
873
|
+
|
|
874
|
+
/** Helper to combine multiple telemetry sinks. */
|
|
875
|
+
export function combineTelemetry(...sinks: Telemetry[]): Telemetry;
|
|
876
|
+
```
|
|
877
|
+
|
|
878
|
+
## Targeting API
|
|
879
|
+
|
|
880
|
+
```ts
|
|
881
|
+
/** Evaluate a targeting predicate against a context. */
|
|
882
|
+
export function evaluate(
|
|
883
|
+
targeting: EvaluableTargeting,
|
|
884
|
+
context: VariantContext | EvalContext,
|
|
885
|
+
): TargetingResult;
|
|
886
|
+
|
|
887
|
+
/** Thin wrapper returning evaluate(...).matched. */
|
|
888
|
+
export function matchTargeting(
|
|
889
|
+
targeting: EvaluableTargeting,
|
|
890
|
+
context: VariantContext | EvalContext,
|
|
891
|
+
): boolean;
|
|
892
|
+
|
|
893
|
+
/** Walk an experiment's targeting and return a full trace. */
|
|
894
|
+
export function explain(
|
|
895
|
+
experiment: Experiment,
|
|
896
|
+
context: VariantContext | EvalContext,
|
|
897
|
+
): ExplainResult;
|
|
898
|
+
|
|
899
|
+
/** Match a route pattern. Supports /foo, /foo/*, /foo/**, /user/:id. */
|
|
900
|
+
export function matchRoute(pattern: string, route: string): boolean;
|
|
901
|
+
|
|
902
|
+
/** Match a semver range. Supports >=1.2.0, ^1.2.0, 1.2.0 - 2.0.0. */
|
|
903
|
+
export function matchSemver(range: string, version: string): boolean;
|
|
904
|
+
|
|
905
|
+
/** Compute a sha256 bucket (0..99) for a userId. Async (Web Crypto). */
|
|
906
|
+
export function hashUserId(userId: string): Promise<number>;
|
|
907
|
+
```
|
|
908
|
+
|
|
909
|
+
## Assignment API
|
|
910
|
+
|
|
911
|
+
```ts
|
|
912
|
+
/** Deterministic hash of (userId + experimentId) to a [0, 1) float. */
|
|
913
|
+
export function stickyHash(userId: string, experimentId: string): number;
|
|
914
|
+
|
|
915
|
+
/** Evaluate an assignment strategy. */
|
|
916
|
+
export function assignVariant(
|
|
917
|
+
experiment: Experiment,
|
|
918
|
+
context: VariantContext
|
|
919
|
+
): string;
|
|
920
|
+
```
|
|
921
|
+
|
|
922
|
+
## Errors
|
|
923
|
+
|
|
924
|
+
```ts
|
|
925
|
+
/** A single validation issue surfaced by validateConfig. */
|
|
926
|
+
export interface ConfigIssue {
|
|
927
|
+
readonly path: string; // RFC 6901 JSON Pointer
|
|
928
|
+
readonly code: IssueCode; // machine-readable
|
|
929
|
+
readonly message: string; // human-readable
|
|
930
|
+
}
|
|
931
|
+
|
|
932
|
+
/** Thrown when config validation fails. */
|
|
933
|
+
export class ConfigValidationError extends Error {
|
|
934
|
+
readonly issues: ReadonlyArray<ConfigIssue>;
|
|
935
|
+
}
|
|
936
|
+
|
|
937
|
+
/** Thrown when HMAC verification fails. */
|
|
938
|
+
export class SignatureVerificationError extends Error {}
|
|
939
|
+
|
|
940
|
+
/** Thrown when an experiment ID is unknown (fail-closed mode). */
|
|
941
|
+
export class UnknownExperimentError extends Error {
|
|
942
|
+
readonly experimentId: string;
|
|
943
|
+
}
|
|
944
|
+
```
|
|
945
|
+
|
|
946
|
+
## React API (`@variantlab/react`)
|
|
947
|
+
|
|
948
|
+
### Provider
|
|
949
|
+
|
|
950
|
+
```tsx
|
|
951
|
+
export interface VariantLabProviderProps {
|
|
952
|
+
config: ExperimentsConfig;
|
|
953
|
+
options?: Omit<EngineOptions, "storage"> & { storage?: Storage };
|
|
954
|
+
context?: Partial<VariantContext>;
|
|
955
|
+
children: React.ReactNode;
|
|
956
|
+
}
|
|
957
|
+
|
|
958
|
+
export const VariantLabProvider: React.FC<VariantLabProviderProps>;
|
|
959
|
+
```
|
|
960
|
+
|
|
961
|
+
### Hooks
|
|
962
|
+
|
|
963
|
+
```ts
|
|
964
|
+
/** Returns the current variant ID for an experiment. */
|
|
965
|
+
export function useVariant(experimentId: string): string;
|
|
966
|
+
|
|
967
|
+
/** Returns the variant value (for "value" experiments). */
|
|
968
|
+
export function useVariantValue<T = unknown>(experimentId: string): T;
|
|
969
|
+
|
|
970
|
+
/** Returns { variant, value, track }. */
|
|
971
|
+
export function useExperiment<T = unknown>(experimentId: string): {
|
|
972
|
+
variant: string;
|
|
973
|
+
value: T;
|
|
974
|
+
track: (eventName: string, properties?: Record<string, unknown>) => void;
|
|
975
|
+
};
|
|
976
|
+
|
|
977
|
+
/** Imperative variant setter. Dev-only by default. */
|
|
978
|
+
export function useSetVariant(): (experimentId: string, variantId: string) => void;
|
|
979
|
+
|
|
980
|
+
/** Low-level engine access. */
|
|
981
|
+
export function useVariantLabEngine(): VariantEngine;
|
|
982
|
+
|
|
983
|
+
/** Returns experiments applicable to the current route. */
|
|
984
|
+
export function useRouteExperiments(route?: string): Experiment[];
|
|
985
|
+
```
|
|
986
|
+
|
|
987
|
+
### Components
|
|
988
|
+
|
|
989
|
+
```tsx
|
|
990
|
+
/** Render-prop switch for "render" experiments. */
|
|
991
|
+
export const Variant: React.FC<{
|
|
992
|
+
experimentId: string;
|
|
993
|
+
children: Record<string, React.ReactNode>;
|
|
994
|
+
fallback?: React.ReactNode;
|
|
995
|
+
}>;
|
|
996
|
+
|
|
997
|
+
/** Render-prop for "value" experiments. */
|
|
998
|
+
export function VariantValue<T>(props: {
|
|
999
|
+
experimentId: string;
|
|
1000
|
+
children: (value: T) => React.ReactNode;
|
|
1001
|
+
}): React.ReactElement;
|
|
1002
|
+
|
|
1003
|
+
/** Error boundary that reports crashes to the engine. */
|
|
1004
|
+
export const VariantErrorBoundary: React.ComponentType<{
|
|
1005
|
+
experimentId: string;
|
|
1006
|
+
fallback?: React.ReactNode | ((error: Error) => React.ReactNode);
|
|
1007
|
+
children: React.ReactNode;
|
|
1008
|
+
}>;
|
|
1009
|
+
```
|
|
1010
|
+
|
|
1011
|
+
## React Native API (`@variantlab/react-native`)
|
|
1012
|
+
|
|
1013
|
+
Re-exports everything from `@variantlab/react` plus:
|
|
1014
|
+
|
|
1015
|
+
### Storage adapters
|
|
1016
|
+
|
|
1017
|
+
```ts
|
|
1018
|
+
export function createAsyncStorageAdapter(): Storage;
|
|
1019
|
+
export function createMMKVStorageAdapter(): Storage;
|
|
1020
|
+
export function createSecureStoreAdapter(): Storage;
|
|
1021
|
+
```
|
|
1022
|
+
|
|
1023
|
+
### Debug overlay
|
|
1024
|
+
|
|
1025
|
+
```tsx
|
|
1026
|
+
export const VariantDebugOverlay: React.FC<{
|
|
1027
|
+
shakeToOpen?: boolean;
|
|
1028
|
+
routeFilter?: boolean;
|
|
1029
|
+
position?: "top-left" | "top-right" | "bottom-left" | "bottom-right";
|
|
1030
|
+
hideButton?: boolean;
|
|
1031
|
+
}>;
|
|
1032
|
+
|
|
1033
|
+
export function openDebugOverlay(): void;
|
|
1034
|
+
```
|
|
1035
|
+
|
|
1036
|
+
### Deep link handler
|
|
1037
|
+
|
|
1038
|
+
```ts
|
|
1039
|
+
export function registerDeepLinkHandler(
|
|
1040
|
+
engine: VariantEngine,
|
|
1041
|
+
options?: { scheme?: string; host?: string }
|
|
1042
|
+
): () => void;
|
|
1043
|
+
```
|
|
1044
|
+
|
|
1045
|
+
## Next.js API (`@variantlab/next`)
|
|
1046
|
+
|
|
1047
|
+
### Server
|
|
1048
|
+
|
|
1049
|
+
```ts
|
|
1050
|
+
export function createVariantLabServer(
|
|
1051
|
+
config: ExperimentsConfig,
|
|
1052
|
+
options?: Omit<EngineOptions, "storage"> & { storage?: Storage }
|
|
1053
|
+
): VariantEngine;
|
|
1054
|
+
|
|
1055
|
+
export function getVariantSSR(
|
|
1056
|
+
experimentId: string,
|
|
1057
|
+
request: Request | NextApiRequest,
|
|
1058
|
+
config: ExperimentsConfig
|
|
1059
|
+
): string;
|
|
1060
|
+
|
|
1061
|
+
export function getVariantValueSSR<T = unknown>(
|
|
1062
|
+
experimentId: string,
|
|
1063
|
+
request: Request | NextApiRequest,
|
|
1064
|
+
config: ExperimentsConfig
|
|
1065
|
+
): T;
|
|
1066
|
+
```
|
|
1067
|
+
|
|
1068
|
+
### Middleware
|
|
1069
|
+
|
|
1070
|
+
```ts
|
|
1071
|
+
export function variantLabMiddleware(config: ExperimentsConfig): (
|
|
1072
|
+
req: NextRequest
|
|
1073
|
+
) => NextResponse;
|
|
1074
|
+
```
|
|
1075
|
+
|
|
1076
|
+
## CLI commands
|
|
1077
|
+
|
|
1078
|
+
```
|
|
1079
|
+
variantlab init Scaffold experiments.json + install adapter
|
|
1080
|
+
variantlab generate [--out <path>] Generate .d.ts from experiments.json
|
|
1081
|
+
variantlab validate [--config <path>] Validate config + check for orphaned IDs
|
|
1082
|
+
variantlab eval <experiment> [context] Evaluate a single experiment with a context
|
|
1083
|
+
```
|
|
1084
|
+
|
|
1085
|
+
---
|
|
1086
|
+
|
|
1087
|
+
# Architecture
|
|
1088
|
+
|
|
1089
|
+
## Design goals
|
|
1090
|
+
|
|
1091
|
+
1. **Core runs anywhere.** Any ECMAScript 2020 environment — Node 18+, Deno, Bun, browsers, React Native Hermes, Cloudflare Workers, Vercel Edge, AWS Lambda@Edge.
|
|
1092
|
+
2. **Adapters are trivially small.** Each framework adapter is < 200 source LOC and < 2 KB gzipped.
|
|
1093
|
+
3. **Tree-shakeable everything.** Every export lives in its own module file. Unused code is eliminated at build time.
|
|
1094
|
+
4. **No implicit IO.** Core never reads from disk, network, or global storage. All IO happens through injected adapters (Storage, Fetcher, Telemetry).
|
|
1095
|
+
5. **Deterministic at hash boundaries.** Same `userId` + experiment = same variant across every runtime.
|
|
1096
|
+
6. **Forward-compatible config schema.** `experiments.json` has a `version` field. The engine refuses unknown versions.
|
|
1097
|
+
|
|
1098
|
+
## Runtime data flow
|
|
1099
|
+
|
|
1100
|
+
```
|
|
1101
|
+
+-------------------------------------------------------------------+
|
|
1102
|
+
| Application code (framework) |
|
|
1103
|
+
| |
|
|
1104
|
+
| useVariant("x") <Variant experimentId="x"> track(...) |
|
|
1105
|
+
+---------+------------------------+------------------------+--------+
|
|
1106
|
+
| | |
|
|
1107
|
+
+---------v------------------------v------------------------v--------+
|
|
1108
|
+
| Framework adapter |
|
|
1109
|
+
| (@variantlab/react, /next, ...) |
|
|
1110
|
+
| |
|
|
1111
|
+
| React Context | Hooks | SSR helpers | Debug overlay |
|
|
1112
|
+
+---------+----------------------------------------------------------+
|
|
1113
|
+
|
|
|
1114
|
+
| subscribe / getVariant / setVariant / trackEvent
|
|
1115
|
+
|
|
|
1116
|
+
+---------v----------------------------------------------------------+
|
|
1117
|
+
| @variantlab/core |
|
|
1118
|
+
| |
|
|
1119
|
+
| +------------+ +------------+ +------------+ +----------+ |
|
|
1120
|
+
| | Engine |--| Targeting |--| Assignment |--| Schema | |
|
|
1121
|
+
| | | | | | | | validator| |
|
|
1122
|
+
| +-----+------+ +------------+ +------------+ +----------+ |
|
|
1123
|
+
| | |
|
|
1124
|
+
| +-----v------+ +------------+ +------------+ +----------+ |
|
|
1125
|
+
| | Storage | | Fetcher | | Telemetry | | Crypto | |
|
|
1126
|
+
| | (injected) | | (injected) | | (injected) | | (WebAPI) | |
|
|
1127
|
+
| +------------+ +------------+ +------------+ +----------+ |
|
|
1128
|
+
+--------------------------------------------------------------------+
|
|
1129
|
+
```
|
|
1130
|
+
|
|
1131
|
+
### Resolve variant (hot path)
|
|
1132
|
+
|
|
1133
|
+
Called on every `getVariant()` read. Must be O(1).
|
|
1134
|
+
|
|
1135
|
+
1. Engine checks in-memory override map (dev/debug overrides win)
|
|
1136
|
+
2. Engine checks Storage for a persisted assignment
|
|
1137
|
+
3. If none, engine evaluates targeting predicates against `context`
|
|
1138
|
+
4. If targeting passes, engine runs the assignment strategy
|
|
1139
|
+
5. Engine writes the result to Storage and memoizes
|
|
1140
|
+
6. Engine emits an `onAssignment` event to Telemetry (first time only per session)
|
|
1141
|
+
7. Returns variant ID
|
|
1142
|
+
|
|
1143
|
+
### Load config (cold start)
|
|
1144
|
+
|
|
1145
|
+
1. App mounts provider with inline config or async `Fetcher`
|
|
1146
|
+
2. Engine validates config (hand-rolled validator, no zod)
|
|
1147
|
+
3. If HMAC signature is present, engine verifies via Web Crypto
|
|
1148
|
+
4. Engine hydrates Storage — reads all previously persisted assignments
|
|
1149
|
+
5. Engine emits `onReady` event
|
|
1150
|
+
|
|
1151
|
+
### Override flow (dev / QA)
|
|
1152
|
+
|
|
1153
|
+
1. User taps a variant in debug overlay, or deep link fires, or QR is scanned
|
|
1154
|
+
2. Adapter calls `engine.setVariant(experimentId, variantId)`
|
|
1155
|
+
3. Engine writes override to Storage with priority flag
|
|
1156
|
+
4. Engine emits `onVariantChanged`
|
|
1157
|
+
5. All subscribed components re-render via `useSyncExternalStore`
|
|
1158
|
+
|
|
1159
|
+
### Crash rollback flow
|
|
1160
|
+
|
|
1161
|
+
1. `<VariantErrorBoundary>` catches an error
|
|
1162
|
+
2. Adapter calls `engine.reportCrash(experimentId, error)`
|
|
1163
|
+
3. Engine increments crash counter in Storage
|
|
1164
|
+
4. If counter exceeds threshold within window, engine forces the default variant and emits `onRollback`
|
|
1165
|
+
|
|
1166
|
+
## Package boundaries
|
|
1167
|
+
|
|
1168
|
+
| Package | Size budget (gzip) | Runtime deps | Description |
|
|
1169
|
+
|---|---:|---|---|
|
|
1170
|
+
| `@variantlab/core` | **3.0 KB** | 0 | Engine, targeting, assignment |
|
|
1171
|
+
| `@variantlab/react` | **1.5 KB** | core | React 18/19 hooks + components |
|
|
1172
|
+
| `@variantlab/react-native` | **4.0 KB** | core, react | RN bindings + debug overlay |
|
|
1173
|
+
| `@variantlab/next` | **2.0 KB** | core, react | Next.js 14/15 SSR + Edge |
|
|
1174
|
+
| `@variantlab/cli` | — | core | CLI tool (dev dependency) |
|
|
1175
|
+
|
|
1176
|
+
## Build tooling
|
|
1177
|
+
|
|
1178
|
+
| Tool | Purpose |
|
|
1179
|
+
|---|---|
|
|
1180
|
+
| **pnpm** | Package manager + workspace |
|
|
1181
|
+
| **tsup** | Bundle (ESM+CJS+DTS via esbuild) |
|
|
1182
|
+
| **TypeScript 5.6+** | Type checking (strict mode) |
|
|
1183
|
+
| **Vitest** | Unit + integration tests |
|
|
1184
|
+
| **size-limit** | Bundle size enforcement in CI |
|
|
1185
|
+
| **Changesets** | Per-package semver versioning |
|
|
1186
|
+
| **Biome** | Lint + format (30x faster than ESLint) |
|
|
1187
|
+
|
|
1188
|
+
## Dependency policy
|
|
1189
|
+
|
|
1190
|
+
- **Core**: zero runtime dependencies, forever. Enforced by CI.
|
|
1191
|
+
- **Adapters**: `@variantlab/core` only. Everything else is peer.
|
|
1192
|
+
- Every runtime dependency is a supply-chain attack vector. By refusing all runtime deps in core, the audit surface is our own code.
|
|
1193
|
+
|
|
1194
|
+
---
|
|
1195
|
+
|
|
1196
|
+
# Design principles
|
|
1197
|
+
|
|
1198
|
+
The 8 principles that govern every design decision in variantlab.
|
|
1199
|
+
|
|
1200
|
+
### 1. Framework-agnostic core, thin adapters
|
|
185
1201
|
|
|
186
|
-
|
|
1202
|
+
The engine runs in any ECMAScript environment. Every framework binding is a thin wrapper (< 200 LOC). `@variantlab/core` never imports `react`, `vue`, `svelte`, or any framework.
|
|
187
1203
|
|
|
188
|
-
###
|
|
1204
|
+
### 2. Zero runtime dependencies
|
|
189
1205
|
|
|
190
|
-
-
|
|
191
|
-
- [API.md](./docs/API.md) — complete TypeScript API surface
|
|
192
|
-
- [SECURITY.md](./docs/SECURITY.md) — threat model, mitigations, reporting
|
|
193
|
-
- [ROADMAP.md](./docs/ROADMAP.md) — phased feature rollout
|
|
194
|
-
- [CONTRIBUTING.md](./docs/CONTRIBUTING.md) — how to contribute
|
|
1206
|
+
`@variantlab/core` has zero runtime dependencies. We hand-roll our schema validator (400 bytes vs zod's 12 KB), semver matcher (250 bytes vs `semver`'s 6 KB), route glob matcher (150 bytes vs `minimatch`'s 4 KB), and hash function (80 bytes vs `murmurhash`'s 500 bytes).
|
|
195
1207
|
|
|
196
|
-
###
|
|
1208
|
+
### 3. ESM-first, tree-shakeable, edge-compatible
|
|
197
1209
|
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
1210
|
+
All packages ship ES modules with `"sideEffects": false`. ES2020 target. Dual ESM+CJS output. No Node built-ins in core. Runs in Node 18+, Deno, Bun, browsers, React Native Hermes, Cloudflare Workers, Vercel Edge, AWS Lambda@Edge.
|
|
1211
|
+
|
|
1212
|
+
### 4. Security by construction
|
|
1213
|
+
|
|
1214
|
+
No `eval`, no `Function()`, no dynamic `import()` on config data. Prototype pollution blocked via `Object.create(null)` and key allow-lists. Constant-time HMAC via Web Crypto. Hard limits on config size, nesting, and iteration. Config frozen after load via `Object.freeze`.
|
|
1215
|
+
|
|
1216
|
+
### 5. Declarative JSON as the contract
|
|
1217
|
+
|
|
1218
|
+
`experiments.json` is the single source of truth. JSON configs are version-controllable, reviewable, toolable, portable, and safe.
|
|
1219
|
+
|
|
1220
|
+
### 6. SSR correct everywhere
|
|
1221
|
+
|
|
1222
|
+
The engine is deterministic. Same config + context = same variant, every time. No `Math.random()` in hot paths. No hydration mismatches in Next.js App Router, Remix, SvelteKit, SolidStart, or Nuxt.
|
|
1223
|
+
|
|
1224
|
+
### 7. Privacy by default
|
|
1225
|
+
|
|
1226
|
+
Zero data collection. No phone-home on import. No analytics, error tracking, or usage stats. Every network call is explicit and user-provided. GDPR/CCPA/LGPD compliant out of the box.
|
|
1227
|
+
|
|
1228
|
+
### 8. Docs-first development
|
|
1229
|
+
|
|
1230
|
+
Every public API is specified in markdown before code is written. `API.md` is authoritative. Features have specs before implementation. Every phase has a plan with exit criteria.
|
|
1231
|
+
|
|
1232
|
+
### Priority order when principles conflict
|
|
1233
|
+
|
|
1234
|
+
Security > Privacy > Zero-dependency > SSR correctness > Framework-agnostic > Bundle size
|
|
1235
|
+
|
|
1236
|
+
---
|
|
204
1237
|
|
|
205
|
-
|
|
1238
|
+
# Security
|
|
206
1239
|
|
|
207
|
-
|
|
208
|
-
- [config-format.md](./docs/design/config-format.md) — `experiments.json` specification
|
|
209
|
-
- [targeting-dsl.md](./docs/design/targeting-dsl.md) — targeting predicate language
|
|
210
|
-
- [api-philosophy.md](./docs/design/api-philosophy.md) — why the API looks the way it does
|
|
1240
|
+
## Threat model
|
|
211
1241
|
|
|
212
|
-
|
|
1242
|
+
| Threat | Description | Mitigation |
|
|
1243
|
+
|---|---|---|
|
|
1244
|
+
| **Malicious remote config** | Compromised CDN injects bad variants | Optional HMAC-SHA256 signed configs. Verify with Web Crypto before applying. |
|
|
1245
|
+
| **Tampered local storage** | Malicious app writes arbitrary keys | Stored variants validated against config. Unknown IDs discarded. |
|
|
1246
|
+
| **Config-based XSS** | Executable code in config | No `eval`, no `Function()`, no dynamic `import()` on config data. |
|
|
1247
|
+
| **Prototype pollution** | Crafted JSON with `__proto__` keys | `Object.create(null)` for parsed objects. Reserved keys rejected. |
|
|
1248
|
+
| **Large/malicious config DoS** | Exponential patterns, huge arrays | Hard limits: 1 MB config, 100 variants, 1000 experiments, depth 10. |
|
|
1249
|
+
| **HMAC timing attack** | Observe timing to guess bytes | `crypto.subtle.verify` is constant-time by spec. |
|
|
1250
|
+
| **Supply chain attack** | Compromised npm package | Zero runtime deps in core. Signed releases via provenance + Sigstore. |
|
|
1251
|
+
| **Debug overlay in production** | End users see internal debug UI | Overlay tree-shaken in production. Throws unless `NODE_ENV === "development"`. |
|
|
1252
|
+
| **Deep link abuse** | Force users into broken variants | Deep links off by default. Only `overridable: true` experiments. Session-scoped. |
|
|
1253
|
+
| **Storage key collision** | Another library writes same keys | All keys prefixed with `variantlab:v1:`. Corrupted values discarded. |
|
|
213
1254
|
|
|
214
|
-
|
|
215
|
-
- [targeting.md](./docs/features/targeting.md) — segmentation predicates
|
|
216
|
-
- [value-experiments.md](./docs/features/value-experiments.md) — non-render variant values
|
|
217
|
-
- [debug-overlay.md](./docs/features/debug-overlay.md) — runtime picker UX
|
|
218
|
-
- [crash-rollback.md](./docs/features/crash-rollback.md) — error-boundary-driven auto-rollback
|
|
219
|
-
- [qr-sharing.md](./docs/features/qr-sharing.md) — state sharing via QR codes
|
|
220
|
-
- [hmac-signing.md](./docs/features/hmac-signing.md) — config integrity verification
|
|
221
|
-
- [time-travel.md](./docs/features/time-travel.md) — record + replay debugging
|
|
222
|
-
- [multivariate.md](./docs/features/multivariate.md) — crossed experiments
|
|
223
|
-
- [killer-features.md](./docs/features/killer-features.md) — the 10 differentiators
|
|
1255
|
+
## Security commitments
|
|
224
1256
|
|
|
225
|
-
|
|
1257
|
+
1. **Never add telemetry** that reports to a server we control.
|
|
1258
|
+
2. **Never add auto-update** mechanisms that fetch new code at runtime.
|
|
1259
|
+
3. **Never phone home** on import. The engine does nothing on module load.
|
|
1260
|
+
4. **Publish a full SBOM** with every release.
|
|
1261
|
+
5. **Sign every release** via Sigstore and npm provenance.
|
|
1262
|
+
6. **Respond to security reports within 48 hours.**
|
|
226
1263
|
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
1264
|
+
## Privacy commitments
|
|
1265
|
+
|
|
1266
|
+
1. Zero data collection about users, developers, or their apps.
|
|
1267
|
+
2. Zero network requests on its own. Every call from user-provided adapters.
|
|
1268
|
+
3. GDPR / CCPA / LGPD compliant out of the box — no data to collect.
|
|
1269
|
+
4. User IDs hashed client-side before any network call.
|
|
1270
|
+
5. Debug overlay state stored locally only.
|
|
1271
|
+
|
|
1272
|
+
## CSP compatibility
|
|
1273
|
+
|
|
1274
|
+
Works under the most restrictive Content Security Policies:
|
|
1275
|
+
|
|
1276
|
+
```
|
|
1277
|
+
Content-Security-Policy: default-src 'self'; script-src 'self'; object-src 'none'
|
|
1278
|
+
```
|
|
1279
|
+
|
|
1280
|
+
No inline scripts, no inline styles, no `unsafe-eval`, no `unsafe-inline`.
|
|
1281
|
+
|
|
1282
|
+
## Reporting a vulnerability
|
|
1283
|
+
|
|
1284
|
+
**Do not file public GitHub issues for security vulnerabilities.**
|
|
1285
|
+
|
|
1286
|
+
Use GitHub Security Advisories or email `security@variantlab.dev`.
|
|
1287
|
+
|
|
1288
|
+
We will:
|
|
1289
|
+
1. Acknowledge receipt within 48 hours
|
|
1290
|
+
2. Provide initial assessment within 7 days
|
|
1291
|
+
3. Follow a 90-day coordinated disclosure window
|
|
1292
|
+
|
|
1293
|
+
---
|
|
1294
|
+
|
|
1295
|
+
# Roadmap
|
|
1296
|
+
|
|
1297
|
+
## Phase 1: MVP (v0.1) — Complete
|
|
1298
|
+
|
|
1299
|
+
- `@variantlab/core` — engine, targeting, assignment
|
|
1300
|
+
- `@variantlab/react` — hooks, components
|
|
1301
|
+
- `@variantlab/react-native` — RN bindings, storage adapters, debug overlay
|
|
1302
|
+
- `@variantlab/next` — App Router + Pages Router SSR
|
|
1303
|
+
- `@variantlab/cli` — `init`, `generate`, `validate`, `eval`
|
|
1304
|
+
|
|
1305
|
+
## Phase 2: Expansion (v0.2)
|
|
1306
|
+
|
|
1307
|
+
- `@variantlab/remix` — Remix loaders, actions, cookie stickiness
|
|
1308
|
+
- `@variantlab/vue` — Vue 3 composables + components
|
|
1309
|
+
- `@variantlab/vanilla` — plain JS/TS helpers
|
|
1310
|
+
- `@variantlab/devtools` — Chrome/Firefox browser extension
|
|
1311
|
+
- React web `VariantDebugOverlay`
|
|
1312
|
+
|
|
1313
|
+
## Phase 3: Ecosystem (v0.3)
|
|
1314
|
+
|
|
1315
|
+
- `@variantlab/svelte` — Svelte 5 stores + SvelteKit
|
|
1316
|
+
- `@variantlab/solid` — SolidJS signals + SolidStart
|
|
1317
|
+
- `@variantlab/astro` — Astro integration
|
|
1318
|
+
- `@variantlab/nuxt` — Nuxt module
|
|
1319
|
+
- `@variantlab/storybook` — Storybook 8 addon
|
|
1320
|
+
- `@variantlab/eslint-plugin` — lint rules
|
|
1321
|
+
- `@variantlab/test-utils` — Jest/Vitest/Playwright helpers
|
|
1322
|
+
|
|
1323
|
+
## Phase 4: Advanced (v0.4)
|
|
1324
|
+
|
|
1325
|
+
- HMAC signing GA with CLI tooling
|
|
1326
|
+
- Crash-triggered rollback GA
|
|
1327
|
+
- Time-travel debugger
|
|
1328
|
+
- QR code state sharing
|
|
1329
|
+
- Multivariate crossed experiments
|
|
1330
|
+
- Holdout groups
|
|
1331
|
+
- Mutual exclusion groups GA
|
|
1332
|
+
|
|
1333
|
+
## Phase 5: v1.0 Stable
|
|
1334
|
+
|
|
1335
|
+
- API freeze with semver strict
|
|
1336
|
+
- Third-party security audit
|
|
1337
|
+
- Reproducible builds
|
|
1338
|
+
- Long-term support policy
|
|
1339
|
+
- Migration guides from Firebase Remote Config, GrowthBook, Statsig, LaunchDarkly
|
|
1340
|
+
|
|
1341
|
+
## Versioning commitments
|
|
1342
|
+
|
|
1343
|
+
| Version range | Stability | Breaking changes |
|
|
1344
|
+
|---|---|---|
|
|
1345
|
+
| 0.0.x | Experimental | Any time |
|
|
1346
|
+
| 0.1.x - 0.4.x | Beta | Minor versions can break |
|
|
1347
|
+
| 0.5.x - 0.9.x | Release candidate | Patch only for security |
|
|
1348
|
+
| 1.0.0+ | Stable | Semver strict — major version required |
|
|
1349
|
+
|
|
1350
|
+
---
|
|
231
1351
|
|
|
232
1352
|
## License
|
|
233
1353
|
|