@variantlab/core 0.1.4 → 0.1.6

package/docs/research/bundle-size-analysis.md

@@ -0,0 +1,279 @@
+# Bundle size analysis
+
+How we plan to hit **< 3 KB gzipped for `@variantlab/core`** and the other budgets in [`ARCHITECTURE.md`](../../ARCHITECTURE.md).
+
+## Table of contents
+
+- [Why bundle size matters](#why-bundle-size-matters)
+- [The budgets](#the-budgets)
+- [Techniques we use](#techniques-we-use)
+- [Techniques we avoid](#techniques-we-avoid)
+- [Measurement methodology](#measurement-methodology)
+- [Per-feature cost estimates](#per-feature-cost-estimates)
+- [What happens when we exceed the budget](#what-happens-when-we-exceed-the-budget)
+
+---
+
+## Why bundle size matters
+
+1. **First-contentful-paint on mobile web.** Every KB in your JS bundle delays FCP by ~10-20 ms on median 4G connections. A 30 KB A/B testing SDK costs half a second before your users see anything.
+2. **React Native cold-start time.** Hermes parses JS on every cold launch. Larger bundles = slower launches. Users notice 100 ms delays.
+3. **Edge runtime limits.** Cloudflare Workers have a 1 MB unzipped bundle limit. Every KB matters.
+4. **Dependency hell avoidance.** Small bundles can be copy-pasted as a file. Large bundles force you to dependency-manage.
+5. **Honesty.** Most A/B testing SDKs bloat over time. Committing to a hard budget at the start keeps us honest.
+
+---
+
+## The budgets
+
+| Package | Budget (gzipped) | Rationale |
+|---|---:|---|
+| `@variantlab/core` | **3 KB** | The entire engine. This is the number people will benchmark us on. |
+| `@variantlab/react` | **1.5 KB** | Just hooks + context + render-prop components. |
+| `@variantlab/react-native` | **4 KB** | Includes debug overlay; tree-shaken in prod. |
+| `@variantlab/next` | **2 KB** | Server + middleware + client re-exports. |
+| `@variantlab/remix` | **1.5 KB** | Loader helpers + cookie stickiness. |
+| `@variantlab/vue` | **1.5 KB** | Composables + components. |
+| `@variantlab/svelte` | **1 KB** | Svelte compiles most of the runtime away. |
+| `@variantlab/solid` | **1 KB** | Same reason. |
+| `@variantlab/vanilla` | **0.5 KB** | Just a hook over the engine. |
+| `@variantlab/astro` | **1 KB** | Astro integration is very thin. |
+| `@variantlab/nuxt` | **1.5 KB** | Nuxt module wrapping Vue adapter. |
+
+All budgets are **enforced in CI** by `size-limit`. A PR that exceeds any budget is blocked.
+
+---
+
+## Techniques we use
+
+### 1. Zero runtime dependencies
+
+Every runtime dependency adds at minimum a few hundred bytes of import overhead and often much more due to transitive bloat. We ship zero.
+
+**What we would pull in if we were careless**:
+
+| Dep | Purpose | Cost (gzipped) |
+|---|---|---:|
+| `zod` | Schema validation | ~12 KB |
+| `valibot` | Schema validation | ~2 KB |
+| `lodash.merge` | Deep merge | ~1 KB |
+| `nanoid` | ID generation | ~130 B |
+| `ms` | Duration parsing | ~250 B |
+| `semver` | Version range matching | ~6 KB |
+| `glob` | Pattern matching | ~4 KB |
+
+Total if we used all of these: ~25 KB.
+
+**What we do instead**: Write hand-rolled replacements. Each is 100-400 bytes and purpose-built for our use case. Total cost: ~1.5 KB for all replacements combined.
+
+### 2. Pure ESM with tree-shaking
+
+- Every module exports only what it needs
+- No barrel files that re-export everything
+- No side effects marked at package.json level: `"sideEffects": false`
+- Bundlers (esbuild, Rollup, Vite, Webpack 5+) automatically drop unused code
+
+### 3. No class inheritance hierarchies
+
+Classes with inheritance force bundlers to include the whole chain. We use a single `VariantEngine` class with composition via injected interfaces (`Storage`, `Fetcher`, `Telemetry`).
+
+### 4. No generics at runtime
+
+TypeScript generics are compile-time only. We never use reflection, `Object.keys`, or runtime type checks beyond simple `typeof` guards.
+
+### 5. Hand-rolled schema validator
+
+The schema validator is ~400 bytes. It's a switch statement over primitive types plus an allow-list check for object keys. That's it. No recursive descent parsers, no grammar, no regex. It rejects anything it doesn't understand.
+
+### 6. Hand-rolled semver matcher
+
+We support only the operators that matter for targeting: `>=`, `<`, `<=`, `>`, `=`, `^`, `~`, range dashes. Total implementation: ~250 bytes. Full `semver` package is 6 KB. We lose nothing practical.
+
+### 7. Hand-rolled glob matcher
+
+We support `/foo`, `/foo/*`, `/foo/**`, and `/foo/:param`. No character classes, no negation, no brace expansion. Total implementation: ~150 bytes. The `glob` or `minimatch` packages are 4+ KB.
+
+### 8. Hand-rolled hash function
+
+For sticky assignment we need a stable hash of `(userId, experimentId)`. We use a simplified 32-bit FNV-1a implementation: ~80 bytes. `murmurhash` is 500 bytes.
+
+### 9. Web Crypto API for HMAC
+
+`crypto.subtle.verify` is available in every modern runtime — browsers, Node 18+, Deno, Bun, Cloudflare Workers, React Native Hermes (via `react-native-quick-crypto` peer dep). We use the platform, not a polyfill. Zero bytes for HMAC.
+
+### 10. Dead code elimination via `__DEV__`
+
+Debug overlay code and extended error messages are gated behind `typeof __DEV__ !== "undefined"` or `process.env.NODE_ENV !== "production"`. Bundlers replace these at build time with constants and drop unreachable branches.
+
+Example:
+
+```ts
+function validateExperiment(exp) {
+  if (typeof __DEV__ !== "undefined" && __DEV__) {
+    if (!exp.id) {
+      throw new Error(
+        `Experiment missing ID. Experiments need a unique string ID to track assignments. Example: { id: "my-experiment", ... }`
+      );
+    }
+  } else {
+    if (!exp.id) throw new Error("no id");
+  }
+}
+```
+
+In production, the verbose error message is eliminated entirely.
+
+### 11. Minification-friendly APIs
+
+- Short internal names
+- Exported names are the only ones that survive minification
+- No computed property names in hot paths
+- No `arguments` object usage
+
+### 12. Separate entry points
+
+Instead of one big `index.ts`, each feature is a separate entry point:
+
+```json
+{
+  "exports": {
+    ".": "./dist/index.js",
+    "./debug": "./dist/debug.js",
+    "./crypto": "./dist/crypto.js",
+    "./storage/memory": "./dist/storage-memory.js"
+  }
+}
+```
+
+Users import only what they need. `@variantlab/core/debug` is separate from `@variantlab/core` so debug code never ships to production.
+
+---
+
+## Techniques we avoid
+
+### Class transforms and polyfills
+
+No `Object.entries` polyfills (required since ES2017). No `class` polyfills (required since ES2015). Target ES2020 baseline — Hermes, Node 18, every modern browser.
+
+### JSON Schema libraries
+
+JSON Schema validators are 20-100+ KB. We use our hand-rolled validator for runtime + publish the `experiments.schema.json` only for IDE tooling.
+
+### Runtime TypeScript features
+
+- No `enum` (produces runtime object)
+- No `namespace` (produces runtime object)
+- No decorators
+- No reflect-metadata
+
+### Prototype chains
+
+A single class. No inheritance. No mixins.
+
+### Eager initialization
+
+Storage is lazy. Fetcher polling is lazy. Crash-rollback counters allocate on first crash. Nothing runs until you call into the engine.
+
+---
+
+## Measurement methodology
+
+### Tooling
+
+- **`size-limit`** with the `@size-limit/esbuild` plugin — matches what users actually ship
+- **Bundlephobia** referenced for competitive comparison
+- **Webpack stats-analyzer** for identifying unexpected bloat
+
+### Automation
+
+- Every PR runs `size-limit` as a required check
+- Size changes are reported as a PR comment
+- A release cannot ship if any package exceeds budget
+
+### Real-world measurement
+
+We test bundle impact in real apps:
+
+1. **Drishtikon Mobile** (React Native, Expo)
+2. **Next.js App Router example**
+3. **SvelteKit example**
+4. **Vite + React example**
+
+Each example publishes its production bundle size to a tracking doc so we see *actual user impact*, not just synthetic benchmarks.
+
+---
+
+## Per-feature cost estimates
+
+Rough estimates based on our hand-rolled implementations:
+
+| Feature | Estimated cost (gzipped) |
+|---|---:|
+| `VariantEngine` class + constructor | ~500 B |
+| Schema validator | ~400 B |
+| Targeting matcher | ~300 B |
+| Semver matcher | ~250 B |
+| Glob route matcher | ~150 B |
+| Sticky hash function | ~80 B |
+| Assignment strategies | ~200 B |
+| Storage interface + memory impl | ~150 B |
+| Fetcher interface | ~80 B |
+| Telemetry interface | ~50 B |
+| Crash rollback logic | ~200 B |
+| Time-travel recording | ~150 B (off by default) |
+| HMAC verification (WebCrypto wrapper) | ~150 B |
+| Error classes | ~150 B |
+| Type exports (zero runtime) | 0 B |
+| Boilerplate / glue | ~200 B |
+| **Total core** | **~3000 B** |
+
+Fits within the 3 KB budget with ~100 B to spare. Any new feature must either stay under that headroom or displace an existing feature.
+
+---
+
+## What happens when we exceed the budget
+
+A PR that exceeds the budget is blocked by CI. The author must either:
+
+1. **Optimize** the change to fit
+2. **Remove** an existing feature to make room
+3. **Move** the change to a separate entry point (e.g., `@variantlab/core/extra`)
+4. **Request a budget increase** via a GitHub discussion, requiring 2 maintainer approvals
+
+We expect to reject the majority of budget-increase requests. The budget is the contract.
+
+---
+
+## Comparison: what we avoid
+
+If we used common conveniences, our core would be:
+
+| Luxury | Cost | Running total |
+|---|---:|---:|
+| Starting point (hand-rolled) | 3.0 KB | 3.0 KB |
+| Add `zod` for schema | +12 KB | 15 KB |
+| Add `semver` | +6 KB | 21 KB |
+| Add `minimatch` | +4 KB | 25 KB |
+| Add `lodash.merge` | +1 KB | 26 KB |
+| Add `eventemitter3` | +1 KB | 27 KB |
+| Add `nanoid` | +0.5 KB | 27.5 KB |
+
+**We would be nearly 10x our budget** before we wrote a line of actual logic. This is why zero-dependency is the founding principle, not an optimization we add later.
+
+---
+
+## Future work
+
+- **Brotli-compressed size as a secondary metric.** Browsers support Brotli; npm does not measure it. We should publish both numbers.
+- **ESBuild minification tuning.** Terser may squeeze out a few more bytes in corner cases.
+- **Constant folding for literal configs.** If a user passes a literal JSON config, we can inline it at build time via a plugin.
+- **Subsetting for fixed-feature builds.** If someone uses only value experiments and never render experiments, we could ship a smaller build.
+
+---
+
+## See also
+
+- [`ARCHITECTURE.md`](../../ARCHITECTURE.md#size-budgets) — the budget table
+- [`docs/design/api-philosophy.md`](../design/api-philosophy.md) — how API choices affect bundle size
+- Bundlephobia: https://bundlephobia.com (for competitor measurements)
+- size-limit: https://github.com/ai/size-limit

package/docs/research/competitors.md

@@ -0,0 +1,327 @@
+# Competitor analysis
+
+A detailed comparison of variantlab to every existing A/B testing and feature-flagging solution we evaluated. The goal of this document is to be honest: where competitors do something better, we say so, and we plan to match or exceed them.
+
+## Table of contents
+
+- [Methodology](#methodology)
+- [Summary matrix](#summary-matrix)
+- [Firebase Remote Config](#firebase-remote-config)
+- [GrowthBook](#growthbook)
+- [Statsig](#statsig)
+- [LaunchDarkly](#launchdarkly)
+- [Amplitude Experiment](#amplitude-experiment)
+- [PostHog Feature Flags](#posthog-feature-flags)
+- [Flagsmith](#flagsmith)
+- [Unleash](#unleash)
+- [ConfigCat](#configcat)
+- [react-native-ab](#react-native-ab)
+- [Why a new package](#why-a-new-package)
+
+---
+
+## Methodology
+
+For each competitor we evaluated:
+
+1. **Pricing** — free tier, paid tier, what you give up for free
+2. **Framework support** — which frameworks have official SDKs, which are community-maintained
+3. **SSR support** — does it work in Next.js App Router, Remix, SvelteKit without hydration mismatches
+4. **Bundle size** — minified + gzipped for the primary SDK
+5. **Runtime dependencies** — transitive supply chain exposure
+6. **Self-hosting** — can you run the whole thing on your own infra
+7. **Debug tooling** — what do you get for local development
+8. **Security model** — HMAC, encryption, CSP compatibility
+9. **Privacy** — what telemetry does the SDK send
+10. **Type safety** — are experiment IDs typed
+
+Sources: official documentation, npm package analysis, GitHub repos, pricing pages, community forums (as of early 2026).
+
+---
+
+## Summary matrix
+
+| Tool | Free forever | Self-host | Bundle (gz) | Deps | Multi-framework | SSR | Type safety | Privacy |
+|---|:-:|:-:|---:|---:|:-:|:-:|:-:|:-:|
+| **variantlab** | ✅ | ✅ | ~3 KB | 0 | 10+ | ✅ | ✅ codegen | ✅ zero telemetry |
+| Firebase Remote Config | Limited | ❌ | ~18 KB | many | 3 | Partial | ❌ | ❌ Google telemetry |
+| GrowthBook | ✅ OSS | ✅ | ~7 KB | few | 5 | Partial | Partial | Partial |
+| Statsig | Limited free | ❌ | ~14 KB | some | 4 | Partial | Partial | ❌ |
+| LaunchDarkly | ❌ $$$ | ❌ | ~25 KB | some | 8 | ✅ | Partial | ❌ |
+| Amplitude Experiment | Enterprise | ❌ | ~20 KB | many | 3 | Partial | ❌ | ❌ |
+| PostHog Feature Flags | ✅ OSS | ✅ | ~30 KB | many | 4 | Partial | ❌ | Partial |
+| Flagsmith | ✅ OSS | ✅ | ~8 KB | few | 4 | Partial | ❌ | Partial |
+| Unleash | ✅ OSS | ✅ | ~10 KB | some | 5 | Partial | ❌ | Partial |
+| ConfigCat | Limited | ❌ | ~12 KB | few | 5 | Partial | ❌ | ❌ |
+| react-native-ab | ✅ OSS | N/A | ~2 KB | 0 | 1 | N/A | ❌ | ✅ |
+
+Numbers are approximate and reflect the primary SDK (`firebase/remote-config`, `@growthbook/growthbook-react`, etc.) at the time of research.
+
+---
+
+## Firebase Remote Config
+
+**Pricing**: Free under the Spark plan with limits; paid under Blaze.
+
+**Strengths**:
+- Mature, battle-tested, used by billions of users
+- Tight integration with other Firebase products (Analytics, Crashlytics)
+- A/B Testing Console has a decent UI
+- Works offline with local caching
+
+**Weaknesses for us to beat**:
+- **Lock-in** — tied to Google Cloud. You can't migrate away without rewriting.
+- **Bundle size** — the Web SDK pulls in 18 KB+ of Firebase app initialization code even for a single `getValue()` call.
+- **Telemetry-by-default** — sends analytics events on every config fetch.
+- **No true SSR support** — Firebase Web SDK assumes a browser global.
+- **No screen-size targeting** — you have to implement it yourself.
+- **No type safety** — values are stringly-typed; you cast on read.
+- **No debug picker UI** — you build your own.
+- **No self-host option** — you cannot run Remote Config without Google Cloud.
+- **Config signing is proprietary** — verified by Google's infrastructure, not cryptographically auditable by the client.
+
+**Verdict**: Great if you're already all-in on Firebase. Wrong choice for anyone who wants portability, small bundles, or strong privacy.
+
+---
+
+## GrowthBook
+
+**Pricing**: Free open-source + paid SaaS.
+
+**Strengths**:
+- Open source core, self-hostable
+- Well-designed A/B test result analysis (Bayesian + frequentist)
+- React, Vue, and Node SDKs
+- Visual editor (paid)
+- Feature flag concept is well-modeled
+
+**Weaknesses for us to beat**:
+- **Web-first, React Native is an afterthought** — the RN SDK is a wrapper around the JS SDK with known issues around offline mode.
+- **No Svelte, Solid, or Astro support.**
+- **Bundle size is ~7 KB gzipped** for the React SDK, which is better than Firebase but still 2x our target.
+- **No crash-triggered rollback.**
+- **No debug overlay out of the box** — you get a "debug panel" in the dashboard, not on-device.
+- **No deep link override.**
+- **SSR works but is manual** — you must hydrate context yourself.
+- **Type safety is partial** — you can type feature keys in TS but it's not auto-generated from config.
+
+**Verdict**: The best existing OSS option, and a worthy benchmark. variantlab's differentiation is (a) wider framework support, (b) smaller bundle, (c) better RN story, (d) on-device debug tooling, (e) HMAC signing.
+
+---
+
+## Statsig
+
+**Pricing**: Free tier up to a modest event volume, then paid.
+
+**Strengths**:
+- Fastest-growing in the space
+- Excellent real-time experiment dashboards
+- Native SDKs for iOS and Android (not just React Native)
+- Pulse analysis for automated decisioning
+
+**Weaknesses for us to beat**:
+- **Telemetry is mandatory** — Statsig's core value prop is sending events back to their dashboard
+- **No self-host** in the free tier
+- **Bundle size** ~14 KB
+- **Framework support** is limited to React, React Native, Node, and browsers
+- **Target audience is teams with data scientists** — overkill for a solo developer
+
+**Verdict**: Excellent product for a funded startup. Overkill and underfit for our audience of developers who want a free, lightweight, self-contained toolkit.
+
+---
+
+## LaunchDarkly
+
+**Pricing**: Starts at ~$8.33/seat/month, enterprise features much higher.
+
+**Strengths**:
+- Industry standard for feature flags in regulated enterprises
+- Best-in-class approval workflows and audit logs
+- Wide framework support (8+ SDKs)
+- Excellent SSR support including streaming
+- Crash-triggered rollback (in enterprise tier)
+
+**Weaknesses for us to beat**:
+- **Not free.** At all. For anyone.
+- **Bundle size ~25 KB** gzipped for the browser SDK
+- **Telemetry mandatory** — core value prop is shipping events to the dashboard
+- **Vendor lock-in** — their proprietary targeting DSL does not map cleanly to anything else
+- **Adoption cost is high** — requires org-wide buy-in
+
+**Verdict**: The feature set we want to match for free. LaunchDarkly's crash-rollback, approval workflows, and SSR streaming are the north star.
+
+---
+
+## Amplitude Experiment
+
+**Pricing**: Enterprise tier of Amplitude Analytics. Not sold standalone.
+
+**Strengths**:
+- Tight integration with Amplitude's behavioral analytics
+- Sophisticated cohort-based targeting
+
+**Weaknesses for us to beat**:
+- **Enterprise-only**
+- **Bundle size ~20 KB** plus Amplitude's core SDK
+- **Telemetry-heavy**
+- **React Native support is limited**
+
+**Verdict**: Not relevant to our audience. Listed for completeness.
+
+---
+
+## PostHog Feature Flags
+
+**Pricing**: Free tier generous; paid for high volume.
+
+**Strengths**:
+- Open source and self-hostable
+- Part of the broader PostHog analytics platform
+- Good React and RN SDKs
+
+**Weaknesses for us to beat**:
+- **Bundle size ~30 KB** for the primary SDK because it's coupled to PostHog's analytics
+- **Telemetry coupling** — feature flags require PostHog's event pipeline
+- **No type safety from config**
+- **No on-device debug picker**
+- **No SSR story for Next.js App Router**
+
+**Verdict**: Good choice if you already use PostHog for analytics. Heavy if you don't.
+
+---
+
+## Flagsmith
+
+**Pricing**: Free OSS + paid SaaS.
+
+**Strengths**:
+- Open source and self-hostable
+- Multi-environment support built in
+- Segment-based targeting
+
+**Weaknesses for us to beat**:
+- **Bundle size ~8 KB**
+- **SSR requires manual hydration**
+- **React Native SDK has network-request-per-flag antipattern**
+- **No type safety**
+- **No on-device debug tooling**
+
+**Verdict**: Solid for server-side feature flags. Weak on mobile.
+
+---
+
+## Unleash
+
+**Pricing**: Free OSS + paid SaaS.
+
+**Strengths**:
+- Open source and self-hostable
+- Focus on enterprise feature flags (as opposed to A/B testing)
+- Well-designed API
+
+**Weaknesses for us to beat**:
+- **Weak A/B testing story** — designed as a flagging tool, not an experiment tool
+- **No React Native SDK** (as of early 2026)
+- **No debug overlay**
+- **No type safety**
+
+**Verdict**: Enterprise-flag focused. Different problem space.
+
+---
+
+## ConfigCat
+
+**Pricing**: Free tier up to 10 feature flags.
+
+**Strengths**:
+- Simple, focused on feature flags
+- Wide SDK coverage
+
+**Weaknesses for us to beat**:
+- **10-flag limit on free tier**
+- **Bundle size ~12 KB**
+- **No A/B test analysis**
+- **No debug picker**
+- **Telemetry by default**
+
+**Verdict**: Too limited for our audience.
+
+---
+
+## react-native-ab
+
+**Pricing**: Free OSS.
+
+**Strengths**:
+- Tiny (~2 KB)
+- Zero dependencies
+- Simple API
+
+**Weaknesses for us to beat**:
+- **Unmaintained** — last release 2+ years ago
+- **React Native only**
+- **No targeting beyond random split**
+- **No persistence**
+- **No debug picker**
+- **No SSR**
+- **No type safety**
+
+**Verdict**: The closest thing to what we want, and the direct inspiration for variantlab. But it hasn't grown up — variantlab is what `react-native-ab` could have been.
+
+---
+
+## Why a new package
+
+Looking at the matrix, every existing option fails at least one of the following:
+
+1. **Free forever with no usage caps**
+2. **Zero runtime dependencies**
+3. **Smaller than 5 KB gzipped**
+4. **Full multi-framework support**
+5. **Strong SSR correctness**
+6. **On-device debug tooling**
+7. **Type safety from codegen**
+8. **Zero telemetry by default**
+
+variantlab's thesis: it is possible to do all of these at once, and no one has. That's the opportunity.
+
+### What we should copy
+
+- **LaunchDarkly's crash rollback** — best-in-class, bring it to free
+- **GrowthBook's Bayesian analysis story** — reuse for post-v1.0 analytics integrations
+- **Statsig's real-time targeting evaluation UI** — bring that to our debug overlay
+- **Firebase's offline caching** — our Storage adapters already do this
+- **PostHog's self-host-first ethos** — we do the same, but lighter
+
+### What we should avoid
+
+- **Firebase's bundle bloat** from an over-engineered SDK initialization story
+- **Statsig's telemetry coupling** — we separate engine from analytics
+- **LaunchDarkly's proprietary DSL** — we use plain JSON
+- **GrowthBook's manual SSR hydration** — we automate it
+- **Everyone's lack of on-device debug pickers** — this is our wedge
+
+### Where we will lose (at first)
+
+- **Dashboard UX** — we don't ship a hosted dashboard in v0.1. GrowthBook, Statsig, LaunchDarkly all do. We bet that the CLI + JSON config workflow is enough for early adopters.
+- **Analytics integration depth** — PostHog ships with full event tracking. We only call `onExposure` hooks; integrators wire the rest.
+- **Marketing** — we're a 1-person OSS project vs funded startups.
+
+These are acceptable losses for v0.1. Post-v1.0 we can address the first two.
+
+---
+
+## Sources
+
+- Firebase Remote Config docs: https://firebase.google.com/docs/remote-config
+- GrowthBook docs: https://docs.growthbook.io
+- Statsig docs: https://docs.statsig.com
+- LaunchDarkly docs: https://docs.launchdarkly.com
+- Amplitude Experiment docs: https://www.docs.developers.amplitude.com/experiment/
+- PostHog docs: https://posthog.com/docs/feature-flags
+- Flagsmith docs: https://docs.flagsmith.com
+- Unleash docs: https://docs.getunleash.io
+- ConfigCat docs: https://configcat.com/docs
+- react-native-ab on npm: https://www.npmjs.com/package/react-native-ab
+
+Bundle sizes measured via Bundlephobia and direct npm tarball inspection.