@variantlab/core 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1209 -39
- package/docs/API.md +692 -0
- package/docs/ARCHITECTURE.md +430 -0
- package/docs/CONTRIBUTING.md +264 -0
- package/docs/ROADMAP.md +292 -0
- package/docs/SECURITY.md +323 -0
- package/docs/design/api-philosophy.md +347 -0
- package/docs/design/config-format.md +442 -0
- package/docs/design/design-principles.md +212 -0
- package/docs/design/targeting-dsl.md +433 -0
- package/docs/features/codegen.md +351 -0
- package/docs/features/crash-rollback.md +399 -0
- package/docs/features/debug-overlay.md +328 -0
- package/docs/features/hmac-signing.md +330 -0
- package/docs/features/killer-features.md +308 -0
- package/docs/features/multivariate.md +339 -0
- package/docs/features/qr-sharing.md +372 -0
- package/docs/features/targeting.md +481 -0
- package/docs/features/time-travel.md +306 -0
- package/docs/features/value-experiments.md +487 -0
- package/docs/phases/phase-2-expansion.md +307 -0
- package/docs/phases/phase-3-ecosystem.md +289 -0
- package/docs/phases/phase-4-advanced.md +306 -0
- package/docs/phases/phase-5-v1-stable.md +350 -0
- package/docs/research/bundle-size-analysis.md +279 -0
- package/docs/research/competitors.md +327 -0
- package/docs/research/framework-ssr-quirks.md +394 -0
- package/docs/research/naming-rationale.md +238 -0
- package/docs/research/origin-story.md +179 -0
- package/docs/research/security-threats.md +312 -0
- package/package.json +2 -1
package/docs/ROADMAP.md
ADDED
|
@@ -0,0 +1,292 @@
|
|
|
1
|
+
# Roadmap
|
|
2
|
+
|
|
3
|
+
variantlab ships in five phases. This document describes what each phase contains, what success looks like, and why the order was chosen. Detailed per-phase documents live in [`docs/phases/`](./docs/phases/).
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [Phase 0: Foundation](#phase-0-foundation) — you are here
|
|
8
|
+
- [Phase 1: MVP (v0.1)](#phase-1-mvp-v01)
|
|
9
|
+
- [Phase 2: Expansion (v0.2)](#phase-2-expansion-v02)
|
|
10
|
+
- [Phase 3: Ecosystem (v0.3)](#phase-3-ecosystem-v03)
|
|
11
|
+
- [Phase 4: Advanced (v0.4)](#phase-4-advanced-v04)
|
|
12
|
+
- [Phase 5: v1.0 Stable](#phase-5-v10-stable)
|
|
13
|
+
- [Versioning commitments](#versioning-commitments)
|
|
14
|
+
- [How priorities can change](#how-priorities-can-change)
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Phase 0: Foundation
|
|
19
|
+
|
|
20
|
+
**Status**: In progress.
|
|
21
|
+
|
|
22
|
+
**Goal**: Lock the public API surface, document the architecture, define the threat model, and validate every design decision with writing *before* a single line of production code is written.
|
|
23
|
+
|
|
24
|
+
**Deliverables**:
|
|
25
|
+
|
|
26
|
+
- [x] `README.md` — vision and positioning
|
|
27
|
+
- [x] `ARCHITECTURE.md` — monorepo layout and runtime data flow
|
|
28
|
+
- [x] `API.md` — complete TypeScript API surface
|
|
29
|
+
- [x] `SECURITY.md` — threat model and mitigations
|
|
30
|
+
- [x] `ROADMAP.md` — this document
|
|
31
|
+
- [ ] `CONTRIBUTING.md` — contribution guide
|
|
32
|
+
- [ ] `LICENSE` — MIT
|
|
33
|
+
- [ ] `experiments.schema.json` — JSON Schema for the config file
|
|
34
|
+
- [ ] `docs/research/` — competitor analysis, bundle-size research, SSR quirks, naming rationale, security threats, origin story
|
|
35
|
+
- [ ] `docs/design/` — design principles, config format, targeting DSL, API philosophy
|
|
36
|
+
- [ ] `docs/features/` — detailed specs for all killer features
|
|
37
|
+
- [ ] `docs/phases/` — per-phase detailed plans
|
|
38
|
+
- [ ] GitHub repo created, branch protection configured, issue templates
|
|
39
|
+
- [ ] Design review with at least 3 external reviewers
|
|
40
|
+
- [ ] Naming finalized (see [`docs/research/naming-rationale.md`](./docs/research/naming-rationale.md))
|
|
41
|
+
- [ ] First 10 "design partners" identified — developers willing to try the MVP
|
|
42
|
+
|
|
43
|
+
**Exit criteria**:
|
|
44
|
+
|
|
45
|
+
1. Every public API in `API.md` has been reviewed by at least 2 maintainers
|
|
46
|
+
2. `SECURITY.md` has been reviewed by someone with security experience
|
|
47
|
+
3. At least 3 framework adapter specs exist (`react`, `react-native`, `next`)
|
|
48
|
+
4. A demo of the Drishtikon card resize use case has been sketched using the planned API
|
|
49
|
+
5. Naming is locked
|
|
50
|
+
6. GitHub repo exists with labels, milestones, and issue templates
|
|
51
|
+
|
|
52
|
+
**Non-goals**:
|
|
53
|
+
|
|
54
|
+
- Writing production code
|
|
55
|
+
- Setting up CI
|
|
56
|
+
- Publishing anything to npm
|
|
57
|
+
|
|
58
|
+
See [`docs/phases/phase-0-foundation.md`](./docs/phases/phase-0-foundation.md) for the detailed plan.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Phase 1: MVP (v0.1)
|
|
63
|
+
|
|
64
|
+
**Goal**: Ship the smallest possible version that a real product can use. Prove the thesis: one core, multiple adapters, real type safety.
|
|
65
|
+
|
|
66
|
+
**Packages shipped**:
|
|
67
|
+
|
|
68
|
+
- `@variantlab/core` — engine, targeting, assignment, HMAC (but off by default)
|
|
69
|
+
- `@variantlab/react` — hooks, components, debug overlay
|
|
70
|
+
- `@variantlab/react-native` — RN bindings, AsyncStorage adapter, native debug overlay
|
|
71
|
+
- `@variantlab/next` — App Router + Pages Router SSR support
|
|
72
|
+
- `@variantlab/cli` — `init`, `generate`, `validate`
|
|
73
|
+
|
|
74
|
+
**Features shipped**:
|
|
75
|
+
|
|
76
|
+
- Inline JSON config
|
|
77
|
+
- `useVariant`, `useVariantValue`, `useExperiment` hooks
|
|
78
|
+
- `<Variant>` and `<VariantValue>` components
|
|
79
|
+
- Debug overlay with route-aware filtering
|
|
80
|
+
- Screen-size targeting
|
|
81
|
+
- Platform targeting
|
|
82
|
+
- App-version targeting (semver)
|
|
83
|
+
- Locale targeting
|
|
84
|
+
- Default + sticky-hash assignment strategies
|
|
85
|
+
- Deep link override (opt-in)
|
|
86
|
+
- Codegen for type-safe experiment IDs
|
|
87
|
+
- Example apps: `next-app-router`, `expo-router`, `vite-react`
|
|
88
|
+
|
|
89
|
+
**Features NOT shipped in v0.1** (deliberate):
|
|
90
|
+
|
|
91
|
+
- Remote config fetching (users can plug it themselves via `Fetcher` interface)
|
|
92
|
+
- HMAC signing (the API exists, but the UX + CLI tooling lands in v0.4)
|
|
93
|
+
- Crash-triggered rollback (the infrastructure exists, but the UX ships in v0.4)
|
|
94
|
+
- Time-travel replay
|
|
95
|
+
- QR code sharing
|
|
96
|
+
- Multivariate / crossed experiments
|
|
97
|
+
- Traffic splits beyond simple weights
|
|
98
|
+
- Analytics integrations
|
|
99
|
+
|
|
100
|
+
**Exit criteria**:
|
|
101
|
+
|
|
102
|
+
1. All packages shipped to npm with provenance
|
|
103
|
+
2. Docs site deployed at `variantlab.dev` with interactive playground
|
|
104
|
+
3. Drishtikon Mobile has migrated from its hand-rolled context to `@variantlab/react-native`
|
|
105
|
+
4. At least 5 external projects have adopted v0.1
|
|
106
|
+
5. Bundle sizes within budget (see [`ARCHITECTURE.md`](./ARCHITECTURE.md))
|
|
107
|
+
6. 95%+ test coverage on core
|
|
108
|
+
7. Zero runtime dependencies in core verified in CI
|
|
109
|
+
|
|
110
|
+
**Success metric**: 100 stars on GitHub within 30 days of release, 10 external integrations within 60 days.
|
|
111
|
+
|
|
112
|
+
See [`docs/phases/phase-1-mvp.md`](./docs/phases/phase-1-mvp.md).
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## Phase 2: Expansion (v0.2)
|
|
117
|
+
|
|
118
|
+
**Goal**: Broaden framework support to cover the major meta-frameworks beyond React, and ship the browser devtools extension.
|
|
119
|
+
|
|
120
|
+
**Packages shipped**:
|
|
121
|
+
|
|
122
|
+
- `@variantlab/remix` — Remix loaders, actions, cookie stickiness
|
|
123
|
+
- `@variantlab/vue` — Vue 3 composables + components
|
|
124
|
+
- `@variantlab/vanilla` — plain JS/TS helpers, no framework
|
|
125
|
+
- `@variantlab/devtools` — Chrome/Firefox browser extension
|
|
126
|
+
|
|
127
|
+
**Features shipped**:
|
|
128
|
+
|
|
129
|
+
- Full Remix integration including nested route experiments
|
|
130
|
+
- Vue 3 composables matching the React API 1:1
|
|
131
|
+
- Browser devtools extension:
|
|
132
|
+
- Inspect current variant assignments
|
|
133
|
+
- Override variants from DevTools panel
|
|
134
|
+
- Visualize targeting evaluation (why was this variant chosen)
|
|
135
|
+
- Time-travel through variant changes in the current session
|
|
136
|
+
- Route-aware filtering in the devtools panel (matches current URL)
|
|
137
|
+
|
|
138
|
+
**Exit criteria**:
|
|
139
|
+
|
|
140
|
+
1. Example apps for Remix, Vue 3, and Nuxt (via Vue adapter) working end-to-end
|
|
141
|
+
2. Browser devtools extension published to Chrome Web Store and Firefox Add-ons
|
|
142
|
+
3. At least 10 external integrations total across all adapters
|
|
143
|
+
|
|
144
|
+
See [`docs/phases/phase-2-expansion.md`](./docs/phases/phase-2-expansion.md).
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
## Phase 3: Ecosystem (v0.3)
|
|
149
|
+
|
|
150
|
+
**Goal**: Fill out the remaining frameworks and ship the developer experience layer (lint, test utils, Storybook).
|
|
151
|
+
|
|
152
|
+
**Packages shipped**:
|
|
153
|
+
|
|
154
|
+
- `@variantlab/svelte` — Svelte 5 stores + SvelteKit hooks
|
|
155
|
+
- `@variantlab/solid` — SolidJS signals + SolidStart
|
|
156
|
+
- `@variantlab/astro` — Astro integration
|
|
157
|
+
- `@variantlab/nuxt` — Nuxt module
|
|
158
|
+
- `@variantlab/storybook` — Storybook 8 addon
|
|
159
|
+
- `@variantlab/eslint-plugin` — lint rules for config correctness and safe overlay imports
|
|
160
|
+
- `@variantlab/test-utils` — Jest/Vitest/Playwright helpers
|
|
161
|
+
|
|
162
|
+
**Features shipped**:
|
|
163
|
+
|
|
164
|
+
- Full framework parity across the JavaScript ecosystem
|
|
165
|
+
- Storybook addon with per-story variant override
|
|
166
|
+
- ESLint rules:
|
|
167
|
+
- No unknown experiment IDs
|
|
168
|
+
- No debug overlay in production imports
|
|
169
|
+
- No missing defaults
|
|
170
|
+
- No duplicate variant IDs
|
|
171
|
+
- Test utilities:
|
|
172
|
+
- `<TestVariantProvider>` for Jest/RNTL
|
|
173
|
+
- `setVariant()` imperative helper for tests
|
|
174
|
+
- Playwright fixture for E2E variant overrides
|
|
175
|
+
|
|
176
|
+
**Exit criteria**:
|
|
177
|
+
|
|
178
|
+
1. All 10+ planned adapters shipped
|
|
179
|
+
2. Every adapter has an example app
|
|
180
|
+
3. ESLint plugin installed in 25+ projects
|
|
181
|
+
4. Compat matrix green across Node 18/20/22 and all supported framework versions
|
|
182
|
+
|
|
183
|
+
See [`docs/phases/phase-3-ecosystem.md`](./docs/phases/phase-3-ecosystem.md).
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## Phase 4: Advanced (v0.4)
|
|
188
|
+
|
|
189
|
+
**Goal**: Ship the "killer features" that differentiate variantlab from every paid competitor.
|
|
190
|
+
|
|
191
|
+
**Features shipped**:
|
|
192
|
+
|
|
193
|
+
- **HMAC signing GA** — CLI tooling for signing + verifying configs, docs for key management, reference Cloudflare Worker remote config server
|
|
194
|
+
- **Crash-triggered rollback GA** — `<VariantErrorBoundary>` in all framework adapters, rollback metrics dashboard in debug overlay
|
|
195
|
+
- **Time-travel debugger** — record every variant change + context update, scrubber UI in devtools, export replay as JSON
|
|
196
|
+
- **QR code state sharing** — generate QR from debug overlay encoding current variants + context, scan via native QR reader
|
|
197
|
+
- **Deep link override UX** — toast notifications when deep link overrides are applied, programmatic API
|
|
198
|
+
- **Multivariate crossed experiments** — `layout × theme × copy` combinations, deterministic assignment
|
|
199
|
+
- **Weighted traffic splits** — `{ A: 25, B: 50, C: 25 }` with sticky hash
|
|
200
|
+
- **Holdout groups** — always-default percentage for clean measurement
|
|
201
|
+
- **Mutual exclusion groups** — enforce that co-running experiments don't conflict
|
|
202
|
+
|
|
203
|
+
**Exit criteria**:
|
|
204
|
+
|
|
205
|
+
1. HMAC signing works end-to-end from CLI sign → CDN deploy → client verify
|
|
206
|
+
2. Crash rollback has been field-tested in the Drishtikon app
|
|
207
|
+
3. Time-travel debugger records + replays in all major frameworks
|
|
208
|
+
4. Reference Cloudflare Worker template for remote config published
|
|
209
|
+
5. One published case study showing a crash rollback saving a production incident
|
|
210
|
+
|
|
211
|
+
See [`docs/phases/phase-4-advanced.md`](./docs/phases/phase-4-advanced.md).
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## Phase 5: v1.0 Stable
|
|
216
|
+
|
|
217
|
+
**Goal**: Declare the API stable. Commit to semver strict. Welcome enterprise adoption.
|
|
218
|
+
|
|
219
|
+
**Deliverables**:
|
|
220
|
+
|
|
221
|
+
- **API freeze** — no breaking changes in v1.x without a 30-day RFC and major version bump
|
|
222
|
+
- **Third-party security audit** published
|
|
223
|
+
- **Reproducible builds** verified by an independent party
|
|
224
|
+
- **Long-term support policy** documented
|
|
225
|
+
- **Enterprise support tier** (community-maintained, still free — but with SLAs from a funded maintainer pool if we raise funding)
|
|
226
|
+
- **Case studies** from at least 10 production users across different industries
|
|
227
|
+
- **Benchmarks** vs paid competitors published
|
|
228
|
+
- **Comprehensive migration guides** from Firebase Remote Config, GrowthBook, Statsig, LaunchDarkly
|
|
229
|
+
|
|
230
|
+
**Exit criteria**:
|
|
231
|
+
|
|
232
|
+
1. Zero P0 bugs open
|
|
233
|
+
2. Zero known security vulnerabilities
|
|
234
|
+
3. Test coverage ≥ 95% across all packages
|
|
235
|
+
4. All documentation pages have been reviewed
|
|
236
|
+
5. Benchmarks show variantlab outperforms or matches paid alternatives on core metrics
|
|
237
|
+
6. At least 500 GitHub stars and 50 production integrations
|
|
238
|
+
7. Sustainable maintenance model in place
|
|
239
|
+
|
|
240
|
+
**Post-v1.0 priorities** (not a promise, a wishlist):
|
|
241
|
+
|
|
242
|
+
- Hosted docs search (Algolia DocSearch)
|
|
243
|
+
- Interactive learning track — "A/B testing fundamentals with variantlab"
|
|
244
|
+
- Conference talk submissions
|
|
245
|
+
- Community plugin registry
|
|
246
|
+
- Reference dashboards for common observability stacks (Grafana, Datadog, PostHog)
|
|
247
|
+
|
|
248
|
+
See [`docs/phases/phase-5-v1-stable.md`](./docs/phases/phase-5-v1-stable.md).
|
|
249
|
+
|
|
250
|
+
---
|
|
251
|
+
|
|
252
|
+
## Versioning commitments
|
|
253
|
+
|
|
254
|
+
| Version range | Stability | Breaking changes |
|
|
255
|
+
|---|---|---|
|
|
256
|
+
| 0.0.x | Experimental | Any time |
|
|
257
|
+
| 0.1.x - 0.4.x | Beta | Minor versions can break |
|
|
258
|
+
| 0.5.x - 0.9.x | Release candidate | Patch versions can break only for security |
|
|
259
|
+
| 1.0.0+ | Stable | Semver strict — major version required for breaks |
|
|
260
|
+
|
|
261
|
+
**From v0.5 onward** we commit to never breaking without a deprecation warning shipped in at least one prior minor version.
|
|
262
|
+
|
|
263
|
+
**From v1.0 onward** we commit to:
|
|
264
|
+
|
|
265
|
+
- Semver strict
|
|
266
|
+
- Minimum 30-day RFC for breaking changes
|
|
267
|
+
- Minimum 12-month deprecation window for removed APIs
|
|
268
|
+
- Migration codemods for major-version upgrades
|
|
269
|
+
|
|
270
|
+
---
|
|
271
|
+
|
|
272
|
+
## How priorities can change
|
|
273
|
+
|
|
274
|
+
This roadmap is a plan, not a contract. Priorities shift based on:
|
|
275
|
+
|
|
276
|
+
1. **User feedback** — if the community tells us a framework is critical, we'll reshuffle
|
|
277
|
+
2. **Security issues** — critical vulnerabilities always jump the queue
|
|
278
|
+
3. **Funding** — if funding lands, paid features from Phase 4 may ship earlier
|
|
279
|
+
4. **Blocking issues** — some features depend on upstream framework changes
|
|
280
|
+
|
|
281
|
+
We will document every roadmap change in a public GitHub discussion with justification.
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## See also
|
|
286
|
+
|
|
287
|
+
- [`docs/phases/phase-0-foundation.md`](./docs/phases/phase-0-foundation.md)
|
|
288
|
+
- [`docs/phases/phase-1-mvp.md`](./docs/phases/phase-1-mvp.md)
|
|
289
|
+
- [`docs/phases/phase-2-expansion.md`](./docs/phases/phase-2-expansion.md)
|
|
290
|
+
- [`docs/phases/phase-3-ecosystem.md`](./docs/phases/phase-3-ecosystem.md)
|
|
291
|
+
- [`docs/phases/phase-4-advanced.md`](./docs/phases/phase-4-advanced.md)
|
|
292
|
+
- [`docs/phases/phase-5-v1-stable.md`](./docs/phases/phase-5-v1-stable.md)
|
package/docs/SECURITY.md
ADDED
|
@@ -0,0 +1,323 @@
|
|
|
1
|
+
# Security
|
|
2
|
+
|
|
3
|
+
variantlab is designed with security as a first-class concern, not an afterthought. This document describes the threat model, mitigations, reporting process, and our commitments.
|
|
4
|
+
|
|
5
|
+
## Table of contents
|
|
6
|
+
|
|
7
|
+
- [Threat model](#threat-model)
|
|
8
|
+
- [Mitigations](#mitigations)
|
|
9
|
+
- [Security commitments](#security-commitments)
|
|
10
|
+
- [Dependency policy](#dependency-policy)
|
|
11
|
+
- [Supply chain integrity](#supply-chain-integrity)
|
|
12
|
+
- [Privacy commitments](#privacy-commitments)
|
|
13
|
+
- [Reporting a vulnerability](#reporting-a-vulnerability)
|
|
14
|
+
- [Disclosure policy](#disclosure-policy)
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Threat model
|
|
19
|
+
|
|
20
|
+
We consider the following threat actors and attack scenarios:
|
|
21
|
+
|
|
22
|
+
### T1 — Malicious CDN / compromised remote config
|
|
23
|
+
|
|
24
|
+
**Actor**: An attacker who can modify the bytes returned by a user's remote config endpoint (compromised CDN, MITM, DNS hijack, compromised cloud storage bucket).
|
|
25
|
+
|
|
26
|
+
**Attack**: Inject a variant config that redirects users to a malicious UI, disables security features behind flags, or exposes private data.
|
|
27
|
+
|
|
28
|
+
**Mitigation**: Optional HMAC-SHA256 signed configs. Users sign their config with a secret key at build time, ship the public key with the app, and the engine verifies the signature before applying any config. See [`docs/features/hmac-signing.md`](./docs/features/hmac-signing.md).
|
|
29
|
+
|
|
30
|
+
### T2 — Tampered local storage
|
|
31
|
+
|
|
32
|
+
**Actor**: A malicious app on the same device, a user trying to cheat an experiment, or a compromised browser extension.
|
|
33
|
+
|
|
34
|
+
**Attack**: Write arbitrary keys to AsyncStorage / localStorage to force a variant.
|
|
35
|
+
|
|
36
|
+
**Mitigation**:
|
|
37
|
+
1. Variants read from storage are validated against the config. If the stored variant ID is not in the experiment's variant list, the engine discards it and re-assigns.
|
|
38
|
+
2. `@variantlab/react-native/secure-store` adapter uses encrypted keychain storage for sensitive experiments.
|
|
39
|
+
3. In fail-closed mode, the engine throws on unknown variants instead of silently accepting them.
|
|
40
|
+
|
|
41
|
+
### T3 — Config-based XSS / code injection
|
|
42
|
+
|
|
43
|
+
**Actor**: An attacker who controls the config file (e.g., a compromised Git repo, a malicious PR merged by mistake).
|
|
44
|
+
|
|
45
|
+
**Attack**: Inject executable code via the config — e.g., a predicate string that gets `eval()`'d.
|
|
46
|
+
|
|
47
|
+
**Mitigation**: **variantlab never uses `eval`, `Function()`, or dynamic `import()` on config data.** Targeting predicates are JSON-shaped data structures interpreted by a pure function, not code. Custom predicates (the `targeting.predicate` field) can only be supplied *in application code*, not in the JSON config.
|
|
48
|
+
|
|
49
|
+
### T4 — Prototype pollution
|
|
50
|
+
|
|
51
|
+
**Actor**: An attacker who can feed crafted JSON to the engine.
|
|
52
|
+
|
|
53
|
+
**Attack**: Prototype pollution via keys like `__proto__` or `constructor.prototype`.
|
|
54
|
+
|
|
55
|
+
**Mitigation**: The hand-rolled schema validator uses `Object.create(null)` for all parsed objects and explicitly rejects `__proto__`, `constructor`, and `prototype` as object keys. Never spreads untrusted objects without allow-listing.
|
|
56
|
+
|
|
57
|
+
### T5 — Denial of service via large / malicious config
|
|
58
|
+
|
|
59
|
+
**Actor**: An attacker who can feed crafted JSON.
|
|
60
|
+
|
|
61
|
+
**Attack**: Configs with exponential regex targets, deeply nested objects, or huge arrays to exhaust CPU/memory.
|
|
62
|
+
|
|
63
|
+
**Mitigation**:
|
|
64
|
+
- Hard limit: configs larger than 1 MB are rejected
|
|
65
|
+
- Hard limit: experiments with more than 100 variants are rejected
|
|
66
|
+
- Hard limit: nesting depth in targeting trees is capped at 10
|
|
67
|
+
- Route glob matching uses a linear-time matcher, not RegExp
|
|
68
|
+
- Semver matching uses a purpose-built parser, not RegExp backtracking
|
|
69
|
+
|
|
70
|
+
### T6 — Timing attacks on HMAC verification
|
|
71
|
+
|
|
72
|
+
**Actor**: A local attacker who can observe timing differences.
|
|
73
|
+
|
|
74
|
+
**Attack**: Guess HMAC bytes by measuring verification timing.
|
|
75
|
+
|
|
76
|
+
**Mitigation**: HMAC verification uses `crypto.subtle.verify` (Web Crypto API), which is constant-time by spec in all major implementations.
|
|
77
|
+
|
|
78
|
+
### T7 — Supply chain attack on variantlab itself
|
|
79
|
+
|
|
80
|
+
**Actor**: A compromised maintainer account, a compromised npm registry, or a malicious transitive dependency.
|
|
81
|
+
|
|
82
|
+
**Attack**: Ship malicious code to users via npm.
|
|
83
|
+
|
|
84
|
+
**Mitigation**:
|
|
85
|
+
1. **Zero runtime dependencies in `@variantlab/core`**. There is no transitive supply chain to compromise.
|
|
86
|
+
2. **Per-release SBOM** published as a CycloneDX document attached to every GitHub release.
|
|
87
|
+
3. **Signed releases** via `npm publish --provenance` and Sigstore — publicly verifiable.
|
|
88
|
+
4. **Branch protection + required reviews** on the main branch.
|
|
89
|
+
5. **Scoped npm tokens** with 2FA enforced on all maintainer accounts.
|
|
90
|
+
6. **Automated `npm audit` + `socket.dev` checks** on every PR.
|
|
91
|
+
|
|
92
|
+
### T8 — Debug overlay exposing sensitive experiments in production
|
|
93
|
+
|
|
94
|
+
**Actor**: A developer who forgot to tree-shake the debug overlay.
|
|
95
|
+
|
|
96
|
+
**Attack**: End users see an internal debug UI, potentially revealing unreleased features or exposing a variant-override surface.
|
|
97
|
+
|
|
98
|
+
**Mitigation**:
|
|
99
|
+
1. `VariantDebugOverlay` is exported from a dedicated entry point so bundlers can tree-shake it cleanly.
|
|
100
|
+
2. The overlay component throws in production builds unless `process.env.NODE_ENV === "development"` or an explicit `__forceDevOverlay` prop is passed.
|
|
101
|
+
3. Documentation strongly recommends gating the overlay behind `__DEV__` or `NODE_ENV`.
|
|
102
|
+
4. A linter rule (phase 3) warns when the overlay is imported without a production guard.
|
|
103
|
+
|
|
104
|
+
### T9 — Deep link abuse
|
|
105
|
+
|
|
106
|
+
**Actor**: A malicious website or app that opens a deep link on an installed variantlab-enabled app.
|
|
107
|
+
|
|
108
|
+
**Attack**: Force users into a broken experiment variant, or flip experiments to hide security warnings.
|
|
109
|
+
|
|
110
|
+
**Mitigation**:
|
|
111
|
+
1. Deep link handling is **off by default**. Users must explicitly call `registerDeepLinkHandler`.
|
|
112
|
+
2. Deep links only work for experiments explicitly marked `overridable: true` in config.
|
|
113
|
+
3. Deep links are session-scoped by default; app restart reverts.
|
|
114
|
+
4. Deep-link overrides emit a visible toast (opt-out) so users notice.
|
|
115
|
+
|
|
116
|
+
### T10 — Storage key collision with another library
|
|
117
|
+
|
|
118
|
+
**Actor**: Accidentally malicious third-party library.
|
|
119
|
+
|
|
120
|
+
**Attack**: Another library writes to the same storage keys and corrupts state.
|
|
121
|
+
|
|
122
|
+
**Mitigation**: All variantlab storage keys are prefixed with `variantlab:v1:` and validated on read. Unknown keys are ignored. Corrupted values are discarded and re-assigned.
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
126
|
+
## Mitigations
|
|
127
|
+
|
|
128
|
+
Summarized here in priority order.
|
|
129
|
+
|
|
130
|
+
### 1. Zero `eval` / zero dynamic code execution
|
|
131
|
+
|
|
132
|
+
variantlab contains no `eval`, no `new Function()`, no dynamic `import()` of config data, no `setTimeout("string")`, no `setInterval("string")`. This is enforced by an ESLint rule in CI.
|
|
133
|
+
|
|
134
|
+
### 2. CSP-strict compatible
|
|
135
|
+
|
|
136
|
+
variantlab works under the most restrictive Content Security Policies:
|
|
137
|
+
|
|
138
|
+
```
|
|
139
|
+
Content-Security-Policy: default-src 'self'; script-src 'self'; object-src 'none'
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
No inline scripts, no inline styles, no `unsafe-eval`, no `unsafe-inline`.
|
|
143
|
+
|
|
144
|
+
### 3. SSR-safe
|
|
145
|
+
|
|
146
|
+
The core engine is deterministic — the same config + context always produces the same variant. This means hydration never mismatches between server and client, preventing a common attack vector where hydration mismatches can reveal internal state.
|
|
147
|
+
|
|
148
|
+
### 4. No globals pollution
|
|
149
|
+
|
|
150
|
+
The engine never writes to `window`, `globalThis`, or `process`. All state is encapsulated in the `VariantEngine` instance.
|
|
151
|
+
|
|
152
|
+
### 5. Bounded memory
|
|
153
|
+
|
|
154
|
+
The engine has hard limits on:
|
|
155
|
+
|
|
156
|
+
- Config size (1 MB)
|
|
157
|
+
- Variants per experiment (100)
|
|
158
|
+
- Experiments per config (1000)
|
|
159
|
+
- Time-travel history (last 1000 events)
|
|
160
|
+
- Targeting nesting depth (10)
|
|
161
|
+
|
|
162
|
+
### 6. Constant-time HMAC verification
|
|
163
|
+
|
|
164
|
+
Uses `crypto.subtle.verify` which is specified to be constant-time in conforming Web Crypto implementations.
|
|
165
|
+
|
|
166
|
+
### 7. Origin validation for remote configs
|
|
167
|
+
|
|
168
|
+
The built-in `createHttpFetcher` supports an allow-list of origins. The engine refuses to load configs fetched from disallowed origins (useful when `fetch` is intercepted).
|
|
169
|
+
|
|
170
|
+
### 8. Read-only config after load
|
|
171
|
+
|
|
172
|
+
Once loaded, the `ExperimentsConfig` is frozen via `Object.freeze` recursively. Attempts to mutate it fail silently in loose mode and throw in strict mode.
|
|
173
|
+
|
|
174
|
+
---
|
|
175
|
+
|
|
176
|
+
## Security commitments
|
|
177
|
+
|
|
178
|
+
1. **We will never add telemetry that reports to a server we control.** Not anonymized, not opt-out, never. Telemetry is always user-provided, user-directed, and user-configurable.
|
|
179
|
+
2. **We will never add auto-update mechanisms** that fetch new code at runtime. Code changes ship via npm only, with the user's explicit consent.
|
|
180
|
+
3. **We will never phone home on import.** The engine does nothing on module load except define classes and functions.
|
|
181
|
+
4. **We will publish a full SBOM** with every release.
|
|
182
|
+
5. **We will sign every release** via Sigstore and npm provenance.
|
|
183
|
+
6. **We will respond to security reports within 48 hours** and publish advisories via GitHub Security Advisories.
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## Dependency policy
|
|
188
|
+
|
|
189
|
+
### `@variantlab/core`
|
|
190
|
+
|
|
191
|
+
- **Runtime dependencies**: **zero**, enforced by CI.
|
|
192
|
+
- **Dev dependencies**: allowed, but reviewed.
|
|
193
|
+
- **Peer dependencies**: none.
|
|
194
|
+
|
|
195
|
+
### Adapter packages
|
|
196
|
+
|
|
197
|
+
- **Runtime dependencies**: exactly one — `@variantlab/core`.
|
|
198
|
+
- **Peer dependencies**: the framework (React, Vue, etc.) and optional integrations.
|
|
199
|
+
- **Optional peer dependencies** (for storage adapters): marked explicitly as `peerDependenciesMeta.optional = true`.
|
|
200
|
+
|
|
201
|
+
### Dev tooling
|
|
202
|
+
|
|
203
|
+
- Dev dependencies are allowed at the workspace root and per package.
|
|
204
|
+
- Every dev dependency is audited at install time by `pnpm audit`.
|
|
205
|
+
- No dev dependency that has had a CVE in the last 12 months is accepted.
|
|
206
|
+
|
|
207
|
+
### Rationale
|
|
208
|
+
|
|
209
|
+
Every runtime dependency is a potential vulnerability. By refusing all runtime dependencies in core, we reduce the audit surface to our own code. For adapters, the single allowed dependency (`@variantlab/core`) is under our direct control.
|
|
210
|
+
|
|
211
|
+
---
|
|
212
|
+
|
|
213
|
+
## Supply chain integrity
|
|
214
|
+
|
|
215
|
+
### Signed releases
|
|
216
|
+
|
|
217
|
+
Every published package includes:
|
|
218
|
+
|
|
219
|
+
1. **npm provenance attestation** — ties the published tarball to a specific GitHub Actions workflow run
|
|
220
|
+
2. **Sigstore signature** — publicly verifiable via the Sigstore transparency log
|
|
221
|
+
3. **CycloneDX SBOM** — complete bill of materials including transitive dev dependencies
|
|
222
|
+
4. **SLSA provenance** — level 3 target
|
|
223
|
+
|
|
224
|
+
Users can verify with:
|
|
225
|
+
|
|
226
|
+
```bash
|
|
227
|
+
npm audit signatures
|
|
228
|
+
npx @variantlab/cli verify-release @variantlab/core@0.1.0
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
### Reproducible builds
|
|
232
|
+
|
|
233
|
+
We commit to making every release reproducible from source:
|
|
234
|
+
|
|
235
|
+
- Fixed Node version per release (`.nvmrc` committed)
|
|
236
|
+
- Lockfile committed (`pnpm-lock.yaml`)
|
|
237
|
+
- Build environment documented in `RELEASE.md`
|
|
238
|
+
- Anyone can rebuild and compare tarball hashes
|
|
239
|
+
|
|
240
|
+
### Maintainer security
|
|
241
|
+
|
|
242
|
+
- All maintainers must use **hardware security keys** (YubiKey or equivalent) for GitHub and npm
|
|
243
|
+
- **2FA enforced** on all maintainer accounts
|
|
244
|
+
- npm tokens are **scoped and short-lived** (CI-issued, never stored locally)
|
|
245
|
+
- GitHub Actions OIDC is used for npm publishing — no long-lived tokens on CI
|
|
246
|
+
|
|
247
|
+
---
|
|
248
|
+
|
|
249
|
+
## Privacy commitments
|
|
250
|
+
|
|
251
|
+
1. **variantlab collects zero data about users, developers, or their apps.** There is no analytics, no anonymous ID, no "just counting downloads", nothing.
|
|
252
|
+
2. **variantlab makes zero network requests on its own.** Every network call comes from user-provided `Fetcher` adapters.
|
|
253
|
+
3. **variantlab is GDPR / CCPA / LGPD compliant out of the box** — because it has no data to collect.
|
|
254
|
+
4. **User IDs passed for sticky hashing are never stored in plaintext** in remote locations. They are hashed client-side before any network call.
|
|
255
|
+
5. **Debug overlay state is stored locally only.** QR sharing generates a QR locally and displays it on-device — no upload.
|
|
256
|
+
|
|
257
|
+
---
|
|
258
|
+
|
|
259
|
+
## Reporting a vulnerability
|
|
260
|
+
|
|
261
|
+
**Do not file public GitHub issues for security vulnerabilities.**
|
|
262
|
+
|
|
263
|
+
Instead, use GitHub Security Advisories: `https://github.com/variantlab/variantlab/security/advisories/new`
|
|
264
|
+
|
|
265
|
+
Alternatively, email: `security@variantlab.dev` (PGP key in [`SECURITY.asc`](./SECURITY.asc) once published).
|
|
266
|
+
|
|
267
|
+
Please include:
|
|
268
|
+
|
|
269
|
+
1. A description of the vulnerability
|
|
270
|
+
2. Steps to reproduce
|
|
271
|
+
3. Affected package(s) and version(s)
|
|
272
|
+
4. The impact (what can an attacker do)
|
|
273
|
+
5. (Optional) A proposed fix
|
|
274
|
+
|
|
275
|
+
We will:
|
|
276
|
+
|
|
277
|
+
1. Acknowledge receipt within **48 hours**
|
|
278
|
+
2. Provide an initial assessment within **7 days**
|
|
279
|
+
3. Work with you on a coordinated disclosure timeline
|
|
280
|
+
4. Credit you in the advisory (unless you prefer anonymity)
|
|
281
|
+
|
|
282
|
+
---
|
|
283
|
+
|
|
284
|
+
## Disclosure policy
|
|
285
|
+
|
|
286
|
+
We follow a **90-day disclosure window** by default:
|
|
287
|
+
|
|
288
|
+
- Day 0: vulnerability reported privately
|
|
289
|
+
- Day 1-7: triage, assessment, CVSS scoring
|
|
290
|
+
- Day 7-60: fix developed, tested, and prepared
|
|
291
|
+
- Day 60-80: coordinated disclosure with reporter, pre-announcement to major downstream users
|
|
292
|
+
- Day 80-90: public advisory published, patched release shipped
|
|
293
|
+
|
|
294
|
+
Exceptions:
|
|
295
|
+
|
|
296
|
+
- **Critical severity + active exploitation**: emergency release within 7 days
|
|
297
|
+
- **Low severity**: may be rolled into the next regular release
|
|
298
|
+
|
|
299
|
+
Advisories are published via:
|
|
300
|
+
|
|
301
|
+
- GitHub Security Advisories
|
|
302
|
+
- CVE assignment (for medium+ severity)
|
|
303
|
+
- npm audit database
|
|
304
|
+
- Our public changelog
|
|
305
|
+
|
|
306
|
+
---
|
|
307
|
+
|
|
308
|
+
## Security audit history
|
|
309
|
+
|
|
310
|
+
This section will list third-party security audits once we commission them.
|
|
311
|
+
|
|
312
|
+
| Date | Auditor | Scope | Report |
|
|
313
|
+
|---|---|---|---|
|
|
314
|
+
| *(planned for post-v1.0)* | TBD | Core engine + HMAC | TBD |
|
|
315
|
+
|
|
316
|
+
---
|
|
317
|
+
|
|
318
|
+
## Related documents
|
|
319
|
+
|
|
320
|
+
- [`ARCHITECTURE.md`](./ARCHITECTURE.md) — runtime architecture and data flow
|
|
321
|
+
- [`docs/research/security-threats.md`](./docs/research/security-threats.md) — detailed threat research
|
|
322
|
+
- [`docs/features/hmac-signing.md`](./docs/features/hmac-signing.md) — HMAC signing implementation
|
|
323
|
+
- [`docs/design/config-format.md`](./docs/design/config-format.md) — config schema and validation
|