next-evolve 0.3.3 → 0.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +113 -86
- package/dist/cli/detect.js +1 -1
- package/dist/cli/files.js +4 -3
- package/dist/cli/init.js +6 -2
- package/dist/cli/load-app.js +1 -1
- package/dist/client/beacon.d.ts +7 -0
- package/dist/client/beacon.js +28 -1
- package/dist/client/evolve-script.js +2 -15
- package/dist/react/region.d.ts +12 -0
- package/dist/react/region.js +23 -11
- package/dist/store/memory.js +1 -0
- package/docs/api.md +238 -0
- package/docs/concepts.md +70 -0
- package/docs/configuration.md +74 -0
- package/docs/goals-and-attribution.md +81 -0
- package/docs/production.md +75 -0
- package/docs/stripe.md +91 -0
- package/docs/troubleshooting.md +90 -0
- package/docs/what-leaves-your-server.md +38 -0
- package/docs/without-an-account.md +28 -0
- package/package.json +3 -2
package/docs/api.md
ADDED
|
@@ -0,0 +1,238 @@
|
|
|
1
|
+
# API reference
|
|
2
|
+
|
|
3
|
+
The package has six entry points. Server-only ones import `server-only`, so importing them from a client component fails the build.
|
|
4
|
+
|
|
5
|
+
| Import | Where it runs |
|
|
6
|
+
| ----------------------------- | --------------------------------------------------------- |
|
|
7
|
+
| `next-evolve` | Server components: regions and static-mode pages |
|
|
8
|
+
| `next-evolve/config` | Anywhere: `defineConfig`, `defineGoals` |
|
|
9
|
+
| `next-evolve/server` | Server: route handlers, `evolve.track`, `evolve.identify` |
|
|
10
|
+
| `next-evolve/client` | Browser: `<EvolveScript />`, `track`, `identify` |
|
|
11
|
+
| `next-evolve/proxy` | `proxy.ts` / `middleware.ts` (Node runtime) |
|
|
12
|
+
| `next-evolve/instrumentation` | `instrumentation.ts` |
|
|
13
|
+
|
|
14
|
+
## `next-evolve`
|
|
15
|
+
|
|
16
|
+
### `optimizable(Component, schema)`
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
function optimizable<P, S extends z.ZodObject>(
|
|
20
|
+
component: ComponentType<P>,
|
|
21
|
+
schema: S,
|
|
22
|
+
): OptimizableComponent<P, S>;
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Marks a component as optimizable. `schema` is a Zod 4 object listing the props Evolve may change. Every schema key must be a prop the component accepts, with a compatible type, or it fails to compile. Constraints (`min`, `max`, `enum`, `url()`…) become the dashboard's form validation. `.meta({ label: "Headline" })` labels a field. Returns a thin wrapper; your component isn't modified.
|
|
26
|
+
|
|
27
|
+
### `<Optimizable id component props />`
|
|
28
|
+
|
|
29
|
+
```tsx
|
|
30
|
+
<Optimizable id="hero" component={Hero} props={{ title: "Ship faster", cta: "Start free" }} />;
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
| Prop | |
|
|
34
|
+
| ----------- | ------------------------------------------------------------------------------------------------------- |
|
|
35
|
+
| `id` | Region id, unique per site: `/^[a-z0-9][a-z0-9-]{0,39}$/` |
|
|
36
|
+
| `component` | A component returned by `optimizable()` |
|
|
37
|
+
| `props` | Control props. Fields with schema defaults are optional; non-schema props pass through to every variant |
|
|
38
|
+
|
|
39
|
+
A server component (async). It renders the variant the proxy chose, or control. It never throws because of Evolve: a missing bundle, an unknown variant or props that fail the schema all render control. In development it warns once when it renders without Evolve context (the page isn't under `app/[evolve]/` and the route isn't a dynamic-mode route in `evolve.config.ts`).
|
|
40
|
+
|
|
41
|
+
On a page's first render in a process it registers the region (id, JSON Schema of the props, control props, path) with the cloud, in the background.
|
|
42
|
+
|
|
43
|
+
With `cacheComponents: true`, wrap it in `<Suspense>`.
|
|
44
|
+
|
|
45
|
+
### `withEvolve(Page, { path })`
|
|
46
|
+
|
|
47
|
+
Wraps a static-mode page under `app/[evolve]/<route>/page.tsx`. Reads the variant code from the `[evolve]` segment (a malformed code is a 404) and makes it available to every `<Optimizable>` on the page. `path` is the public path (`"/"`, `"/pricing"`). `init` writes this for you.
|
|
48
|
+
|
|
49
|
+
### `evolveStaticParams()`
|
|
50
|
+
|
|
51
|
+
`generateStaticParams` for static-mode pages. Prerenders control only (`ev_0`), so builds need no network. Variants render on first request and are then cached.
|
|
52
|
+
|
|
53
|
+
### `evolveMetadata(path, base?)`
|
|
54
|
+
|
|
55
|
+
`generateMetadata` for static-mode pages. `base` is your metadata object or function. Sets `alternates.canonical` to `path` and `robots: { index: false, follow: true }` on variant renders.
|
|
56
|
+
|
|
57
|
+
A static-mode page as `init` writes it:
|
|
58
|
+
|
|
59
|
+
```tsx
|
|
60
|
+
export const dynamicParams = true;
|
|
61
|
+
export const generateStaticParams = evolveStaticParams;
|
|
62
|
+
export const revalidate = 3600;
|
|
63
|
+
export const generateMetadata = evolveMetadata("/", { title: "Acme" });
|
|
64
|
+
export default withEvolve(Home, { path: "/" });
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### `getOptimizableMeta(component)`
|
|
68
|
+
|
|
69
|
+
Returns `{ schema, jsonSchema, schemaHash, keys }` for an optimizable component, or `null`. For tooling and tests.
|
|
70
|
+
|
|
71
|
+
## `next-evolve/config`
|
|
72
|
+
|
|
73
|
+
### `defineConfig(config)`
|
|
74
|
+
|
|
75
|
+
```ts
|
|
76
|
+
import { defineConfig } from "next-evolve/config";
|
|
77
|
+
|
|
78
|
+
export default defineConfig({
|
|
79
|
+
routes: { "/": { mode: "static" }, "/pricing": { mode: "dynamic" } },
|
|
80
|
+
});
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Validates and returns the config; throws on mistakes. All options are in [configuration.md](configuration.md).
|
|
84
|
+
|
|
85
|
+
### `defineGoals(goals)`
|
|
86
|
+
|
|
87
|
+
Validates and returns the goals. See [goals-and-attribution.md](goals-and-attribution.md).
|
|
88
|
+
|
|
89
|
+
## `next-evolve/server`
|
|
90
|
+
|
|
91
|
+
### `createEvolveHandler({ config, goals })`
|
|
92
|
+
|
|
93
|
+
```ts
|
|
94
|
+
// app/api/evolve/[...evolve]/route.ts
|
|
95
|
+
import { createEvolveHandler } from "next-evolve/server";
|
|
96
|
+
import evolveConfig from "@/evolve.config";
|
|
97
|
+
import goals from "@/evolve.goals";
|
|
98
|
+
|
|
99
|
+
export const { GET, POST } = createEvolveHandler({ config: evolveConfig, goals });
|
|
100
|
+
export const maxDuration = 60;
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Serves the routes under `/api/evolve/` (see [HTTP routes](#http-routes)). Keep `maxDuration = 60` so aggregation fits on serverless hosts.
|
|
104
|
+
|
|
105
|
+
### `evolve.track(name, options?)`
|
|
106
|
+
|
|
107
|
+
```ts
|
|
108
|
+
import { evolve } from "next-evolve/server";
|
|
109
|
+
|
|
110
|
+
await evolve.track("project_created", { userId: user.id });
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Writes an event to your database. Never throws. Options:
|
|
114
|
+
|
|
115
|
+
| Option | Default | |
|
|
116
|
+
| ----------- | --------------------------------------------------- | ----------------------------------------------------------------------------------- |
|
|
117
|
+
| `userId` | — | Your user id. Links the event to experiments through `identify` |
|
|
118
|
+
| `anonId` | the visitor's `evolve_id` cookie (inside a request) | The visitor id |
|
|
119
|
+
| `value` | — | A number, used by goals without their own `value` |
|
|
120
|
+
| `props` | — | Any JSON object up to 2 KB (larger props are dropped, the event is kept) |
|
|
121
|
+
| `source` | `"server"` | A label goals can filter on, e.g. `"stripe"` |
|
|
122
|
+
| `ts` | now | Event time (`Date`, ISO string or ms) |
|
|
123
|
+
| `dedupeKey` | — | Stored once; a second event with the same key is ignored (use the webhook event id) |
|
|
124
|
+
|
|
125
|
+
### `evolve.identify(userId, { anonId? })`
|
|
126
|
+
|
|
127
|
+
Links the visitor (the `evolve_id` cookie, or `anonId`) to your user id, so events with that `userId` (on other devices, or from your server) count for that visitor's experiments. The first link for a visitor wins. Never throws. Call it after sign-up or sign-in.
|
|
128
|
+
|
|
129
|
+
## `next-evolve/client`
|
|
130
|
+
|
|
131
|
+
### `<EvolveScript />`
|
|
132
|
+
|
|
133
|
+
Put it once in the root layout's `<body>`. About 1.6 KB gzipped, no dependencies, not needed to show a variant. On every route it sends a page view and one exposure per counted region; on clicks (forms: on submit) of elements with `data-evolve-goal="<event>"` it sends that event. It batches every 2 seconds and on page hide, to `/api/evolve/collect` on your own origin. It does nothing under automation (`navigator.webdriver`).
|
|
134
|
+
|
|
135
|
+
### `track(name, props?)` and `identify(userId)`
|
|
136
|
+
|
|
137
|
+
Browser versions of the server calls. Prefer `evolve.identify()` on the server.
|
|
138
|
+
|
|
139
|
+
## `next-evolve/proxy`
|
|
140
|
+
|
|
141
|
+
### `createEvolveProxy(config, { buildBundle })`
|
|
142
|
+
|
|
143
|
+
```ts
|
|
144
|
+
// proxy.ts (Next 16)
|
|
145
|
+
import { createEvolveProxy } from "next-evolve/proxy";
|
|
146
|
+
import buildBundle from "./.evolve/bundle.generated";
|
|
147
|
+
import evolveConfig from "./evolve.config";
|
|
148
|
+
|
|
149
|
+
export const proxy = createEvolveProxy(evolveConfig, { buildBundle });
|
|
150
|
+
export const config = { matcher: ["/", "/pricing", "/(ev_.*)"] };
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
On Next 15.5 use `middleware.ts`, `export const middleware = …`, and add `runtime: "nodejs"` to `config`. The matcher must cover every route in `evolve.config.ts` plus `/(ev_.*)`.
|
|
154
|
+
|
|
155
|
+
### `applyEvolve(config, req, event, { buildBundle })`
|
|
156
|
+
|
|
157
|
+
For an existing proxy. Returns a response for Evolve routes and `/ev_*` paths, or `null` for everything else:
|
|
158
|
+
|
|
159
|
+
```ts
|
|
160
|
+
import { applyEvolve, evolveReady } from "next-evolve/proxy";
|
|
161
|
+
|
|
162
|
+
export async function proxy(req: NextRequest, event: NextFetchEvent) {
|
|
163
|
+
await evolveReady(evolveConfig); // only matters with Vercel Global Config
|
|
164
|
+
const evolved = applyEvolve(evolveConfig, req, event, { buildBundle });
|
|
165
|
+
if (evolved) return evolved;
|
|
166
|
+
// …your own logic
|
|
167
|
+
}
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
### `evolveReady(config)`
|
|
171
|
+
|
|
172
|
+
With Vercel Global Config, a fresh instance's first request waits up to 20 ms for one read of the store. Returns `null` when there's nothing to wait for.
|
|
173
|
+
|
|
174
|
+
## `next-evolve/instrumentation`
|
|
175
|
+
|
|
176
|
+
### `registerEvolve({ config, goals })`
|
|
177
|
+
|
|
178
|
+
```ts
|
|
179
|
+
// instrumentation.ts
|
|
180
|
+
export async function register() {
|
|
181
|
+
if (process.env.NEXT_RUNTIME === "nodejs") {
|
|
182
|
+
const [{ registerEvolve }, { default: config }, { default: goals }] = await Promise.all([
|
|
183
|
+
import("next-evolve/instrumentation"),
|
|
184
|
+
import("./evolve.config"),
|
|
185
|
+
import("./evolve.goals"),
|
|
186
|
+
]);
|
|
187
|
+
await registerEvolve({ config, goals });
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Registers your goals for the process (goal `value` functions need this), sends the manifest (routes, goals, versions, app URL) to the cloud once per process and, on long-running Node, polls the bundle in the background and keeps it on disk. Never throws.
|
|
193
|
+
|
|
194
|
+
## CLI
|
|
195
|
+
|
|
196
|
+
Run with `npx next-evolve <command>`. Reads `.env.local` and `.env` (never overriding variables already set).
|
|
197
|
+
|
|
198
|
+
| Command | What it does |
|
|
199
|
+
| ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
200
|
+
| `init [--key] [--webhook-secret] [--db <url>] [--app-url <url>] [--mode static\|dynamic] [--routes "/ /pricing"] [--yes] [--dry-run]` | Sets Evolve up in this app (see the README). Re-running changes nothing that's already done. `--dry-run` prints the diff and writes, installs, migrates and sends nothing. `--yes` answers every prompt with the default. Without `--db` it offers `DATABASE_URL` / `POSTGRES_URL` if set |
|
|
201
|
+
| `migrate [--url <postgres-url>]` | Creates or updates the `evolve` schema (default `$EVOLVE_DATABASE_URL`). Run it on every deploy, or after upgrading `next-evolve` |
|
|
202
|
+
| `pull` | Writes the verified bundle to `.evolve/bundle.generated.ts`. Runs as `prebuild`. Never fails the build: without a key or the cloud it keeps the previous file or writes an empty one |
|
|
203
|
+
| `push` | Sends the manifest (routes and goals) now. It's also sent when the app starts |
|
|
204
|
+
| `connect [--test] [--app-url <url>]` | Checks the key; `--app-url` registers where the app lives; `--test` asks the cloud to call your sync route (ping) and aggregate route (signed dry run) |
|
|
205
|
+
| `doctor` | Checks the setup: Next.js version, env vars, key, database (reachable, migrated), bundle (valid, more than 7 days from expiry, build copy present), proxy matcher, pages, `<EvolveScript />`, cloud → app reachability, CDN. Exits 1 when a check fails |
|
|
206
|
+
| `aggregate [--upload]` | Asks the cloud what to aggregate, runs it on your local database and prints the result; `--upload` sends it to the cloud. For local development, where the cloud can't reach your app |
|
|
207
|
+
|
|
208
|
+
## HTTP routes
|
|
209
|
+
|
|
210
|
+
All served by `createEvolveHandler` under `/api/evolve/`.
|
|
211
|
+
|
|
212
|
+
| Route | Caller | Auth | Answers |
|
|
213
|
+
| ---------------------------- | -------------------------------------------------------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
214
|
+
| `POST /api/evolve/collect` | `<EvolveScript />` | Same origin only | `204`. Batches over 32 KB or 50 events, or cross-origin, are refused. Bot user agents, preview sessions and visitors without an `evolve_id` cookie are dropped |
|
|
215
|
+
| `POST /api/evolve/track` | Your servers, Segment, a CRM | `Authorization: Bearer $EVOLVE_TRACK_SECRET` | `{ accepted: n }`; `404` when `EVOLVE_TRACK_SECRET` isn't set; `401` wrong bearer; `400` bad body |
|
|
216
|
+
| `POST /api/evolve/sync` | Evolve Cloud (bundle changed, or a connection-test ping) | HMAC with `EVOLVE_WEBHOOK_SECRET`, 5-minute clock window, replay check | `202`, then fetches the bundle |
|
|
217
|
+
| `POST /api/evolve/aggregate` | Evolve Cloud | HMAC with `EVOLVE_WEBHOOK_SECRET`; the response is signed too | Counts and sums. `503` when the database isn't configured or is down; `409` for a replay; `429` while another aggregation runs |
|
|
218
|
+
| `GET /api/evolve/health` | You, monitoring | None | `{ ok, protocol, db }` (`db`: whether a database URL is set) |
|
|
219
|
+
|
|
220
|
+
`/api/evolve/track` body:
|
|
221
|
+
|
|
222
|
+
```json
|
|
223
|
+
{
|
|
224
|
+
"events": [
|
|
225
|
+
{
|
|
226
|
+
"name": "subscription_started",
|
|
227
|
+
"userId": "u_123",
|
|
228
|
+
"value": 49,
|
|
229
|
+
"props": { "plan": "pro" },
|
|
230
|
+
"source": "segment",
|
|
231
|
+
"ts": "2026-11-01T12:00:00Z",
|
|
232
|
+
"dedupeKey": "evt_1"
|
|
233
|
+
}
|
|
234
|
+
]
|
|
235
|
+
}
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
At most 500 events per call. `name` is required; every other field is optional (as in `evolve.track`). The default `source` is `"webhook"`. The events are written before the response, so a sender that retries on failure won't lose any.
|
package/docs/concepts.md
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# Concepts
|
|
2
|
+
|
|
3
|
+
## The pieces
|
|
4
|
+
|
|
5
|
+
- **Region:** a component on a page that Evolve may change, rendered with `<Optimizable id="…">`. Its id is unique per site (`a-z`, `0-9`, `-`, up to 40 characters). The component is wrapped with `optimizable(Component, schema)`; the Zod schema lists the props Evolve may set and their limits.
|
|
6
|
+
- **Control:** the props you pass to `<Optimizable props={…}>` in your code. Schema defaults apply.
|
|
7
|
+
- **Variant:** a set of prop overrides made in the dashboard. Variant props are merged over control and validated against the schema; a variant that fails the schema renders control and isn't counted. A started variant can't change (visitors already saw it); pause it and add a new one instead.
|
|
8
|
+
- **Experiment:** one region on one or more paths, 2–6 variants including control, a primary goal (and up to 5 secondary ones), and a traffic share. Two kinds:
|
|
9
|
+
- **A/B/n test:** a fixed split. Read the result when it's decided.
|
|
10
|
+
- **Bandit:** Evolve Cloud shifts traffic toward the better arm as data arrives (recomputed at most hourly, on the primary goal).
|
|
11
|
+
- **Goal:** what counts as success, defined in `evolve.goals.ts` (an event or a page view, a window, optional value). See [goals-and-attribution.md](goals-and-attribution.md).
|
|
12
|
+
- **Exposure:** the first time a visitor sees an experiment. The beacon reports it; one row per visitor per experiment in your database.
|
|
13
|
+
|
|
14
|
+
Limits enforced by the cloud: at most 3 running experiments per path and one live experiment per region.
|
|
15
|
+
|
|
16
|
+
## How a request is served
|
|
17
|
+
|
|
18
|
+
1. The **proxy** (`proxy.ts`, or `middleware.ts` on Next 15.5) runs for the routes in `evolve.config.ts`. It reads the **bundle** from memory, picks a variant per running experiment and rewrites (static mode) or tags (dynamic mode) the request. It makes no network call; Evolve's part takes well under a millisecond.
|
|
19
|
+
2. The page renders. `<Optimizable>` looks up the chosen variant's props and renders the component with them, inside a `<div style="display:contents" data-evolve-…>` marker.
|
|
20
|
+
3. `<EvolveScript />` in the browser sends a page view and one exposure per marked region to `/api/evolve/collect` **on your own origin**. Your app writes them to your Postgres.
|
|
21
|
+
4. On a schedule, Evolve Cloud sends a signed **aggregate request** to `/api/evolve/aggregate`. Your app runs SQL on its own database and answers with counts and sums per experiment × variant × goal × day. The cloud computes results (Bayesian; P(beat control), lift, risk) and, for bandits, new weights.
|
|
22
|
+
5. When anything changes (start, pause, new weights), the cloud signs a new bundle and calls `/api/evolve/sync`; your app fetches the bundle. Apps also poll for it every 60 seconds.
|
|
23
|
+
|
|
24
|
+
## The bundle
|
|
25
|
+
|
|
26
|
+
The bundle is your site's serving configuration: which experiments run on which paths, their traffic share and weights, and each variant's props. It contains no results, priors, goal logic or plan rules.
|
|
27
|
+
|
|
28
|
+
- It's signed by Evolve Cloud (Ed25519). The SDK ships the public key and rejects anything that doesn't verify, belongs to another site, or is older than the one it has.
|
|
29
|
+
- It expires 14 days after it was signed. The cloud re-signs every active site's bundle daily. If your app can't reach the cloud, it keeps serving the last bundle until it expires, then serves control.
|
|
30
|
+
- Sources, freshest first: memory, then (Vercel + Global Config) the store, then (long-running Node) a disk cache, then the build-time copy `next-evolve pull` writes to `.evolve/bundle.generated.ts` during `prebuild`.
|
|
31
|
+
|
|
32
|
+
## Static and dynamic mode
|
|
33
|
+
|
|
34
|
+
Each route in `evolve.config.ts` has a mode.
|
|
35
|
+
|
|
36
|
+
| | Static (default) | Dynamic |
|
|
37
|
+
| ------------------- | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
|
|
38
|
+
| Page file | Moved to `app/[evolve]/<route>/page.tsx` and wrapped with `withEvolve()` | Stays where it is |
|
|
39
|
+
| What the proxy does | Rewrites `/pricing` to `/<code>/pricing` (the code names the variants) | Passes the code in a request header |
|
|
40
|
+
| Caching | One cached (ISR) render per variant | Rendered per request |
|
|
41
|
+
| Use for | Marketing pages | Pages that are already dynamic, apps with a top-level `[slug]` route, `cacheComponents: true` |
|
|
42
|
+
|
|
43
|
+
`init` picks static unless moving the page isn't safe: a top-level dynamic segment (`app/[slug]`), a route group, `force-dynamic`, its own `generateStaticParams`, a `"use client"` page, or a layout shared with child routes. It says which reason applied.
|
|
44
|
+
|
|
45
|
+
**`cacheComponents: true`** (on in new `create-next-app` projects since Next 16.4) rejects the segment config static mode uses, so `init` uses dynamic mode for every route. Wrap each `<Optimizable>` in `<Suspense>`: the region then streams in after the static shell, and the rest of the page stays prerendered.
|
|
46
|
+
|
|
47
|
+
Direct requests to `/ev_…` paths return 404; they exist only as rewrite targets. In static mode, `evolveMetadata()` sets the canonical URL to the original path and marks variant renders `noindex`.
|
|
48
|
+
|
|
49
|
+
`<Optimizable>` is for pages; inside layouts it isn't supported yet.
|
|
50
|
+
|
|
51
|
+
## Who gets a variant
|
|
52
|
+
|
|
53
|
+
- **New visitors** on a route with a running experiment get two cookies: `evolve_id` (a random id) and `evolve_a` (their assignments). Both are `HttpOnly`, `SameSite=Lax`, `Secure` on https, and last 365 days by default. A visitor keeps their variant on later visits.
|
|
54
|
+
- **Traffic share:** visitors outside the share see control and aren't counted.
|
|
55
|
+
- **Bots** (by user agent: search engines, AI crawlers, link unfurlers, Lighthouse, headless browsers) always get control, no cookies, and their events are dropped. Add your own patterns with `bots.extraPatterns`.
|
|
56
|
+
- **Consent:** set `consent: (req) => boolean` in `evolve.config.ts`. When it returns `false` the visitor gets control and no cookies.
|
|
57
|
+
- **No bundle** (new site with nothing started, or an expired bundle): control and no cookies.
|
|
58
|
+
- **Preview links** from the dashboard force a variant for one browser session and record nothing.
|
|
59
|
+
- Paused experiments are removed from the bundle; their visitors see control. Completed experiments serve the winner to everyone, uncounted, until you ship it in code and archive the experiment.
|
|
60
|
+
|
|
61
|
+
## Where things run
|
|
62
|
+
|
|
63
|
+
| In your app | In Evolve Cloud |
|
|
64
|
+
| --------------------------------------- | ----------------------------------------------- |
|
|
65
|
+
| Variant assignment, cookies, bot filter | Experiments, variants, lifecycle |
|
|
66
|
+
| Event writes to your Postgres | Statistics, decisions, bandit weights |
|
|
67
|
+
| Aggregation SQL on your database | Scheduling aggregates, storing aggregate counts |
|
|
68
|
+
| Bundle verification | Signing bundles |
|
|
69
|
+
|
|
70
|
+
The cloud never connects to your database. See [what-leaves-your-server.md](what-leaves-your-server.md).
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Configuration and environment variables
|
|
2
|
+
|
|
3
|
+
## `evolve.config.ts`
|
|
4
|
+
|
|
5
|
+
```ts
|
|
6
|
+
import { defineConfig } from "next-evolve/config";
|
|
7
|
+
|
|
8
|
+
export default defineConfig({
|
|
9
|
+
routes: {
|
|
10
|
+
"/": { mode: "static" },
|
|
11
|
+
"/pricing": { mode: "static" },
|
|
12
|
+
"/features": { mode: "dynamic" },
|
|
13
|
+
},
|
|
14
|
+
// everything below is optional; these are the defaults
|
|
15
|
+
store: "auto",
|
|
16
|
+
sync: { pollSeconds: 60 }, // 5 in development
|
|
17
|
+
database: { urlEnv: "EVOLVE_DATABASE_URL" },
|
|
18
|
+
aggregate: { maxMs: 25_000, statementTimeoutMs: 20_000 },
|
|
19
|
+
cookies: { maxAgeDays: 365, domain: undefined },
|
|
20
|
+
consent: undefined, // (req: Request) => boolean
|
|
21
|
+
bots: { extraPatterns: [] },
|
|
22
|
+
debug: process.env.NODE_ENV !== "production",
|
|
23
|
+
});
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
| Option | |
|
|
27
|
+
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
28
|
+
| `routes` | Pages Evolve may test, by exact path (no wildcards or `[params]`, can't start with `/ev_`). `mode`: `"static"` (the page lives under `app/[evolve]/`, cached per variant) or `"dynamic"` (the page stays put, rendered per request). Add a route here **and** to the proxy matcher; `doctor` checks both |
|
|
29
|
+
| `store` | `"auto"` uses Vercel Global Config when its env vars are set, else memory. `"memory"` or `"global-config"` force one |
|
|
30
|
+
| `sync.pollSeconds` | How often each instance checks for a new bundle (± 10% jitter) |
|
|
31
|
+
| `database.urlEnv` | Which env var holds the database URL |
|
|
32
|
+
| `aggregate.maxMs` | Time budget for one aggregate request; past it the response is partial and the cloud asks again |
|
|
33
|
+
| `aggregate.statementTimeoutMs` | Postgres `statement_timeout` per aggregation query |
|
|
34
|
+
| `cookies.maxAgeDays` | Lifetime of `evolve_id` and `evolve_a` |
|
|
35
|
+
| `cookies.domain` | Set to `.example.com` to share assignments across subdomains |
|
|
36
|
+
| `consent` | Return `false` to serve control and set no cookies (e.g. before cookie consent) |
|
|
37
|
+
| `bots.extraPatterns` | Extra user-agent patterns (case-insensitive regular expressions) treated as bots |
|
|
38
|
+
| `debug` | Extra logging |
|
|
39
|
+
|
|
40
|
+
## Environment variables
|
|
41
|
+
|
|
42
|
+
All server-only.
|
|
43
|
+
|
|
44
|
+
| Var | Needed | |
|
|
45
|
+
| ------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
46
|
+
| `EVOLVE_SECRET_KEY` | yes (runtime and build) | Site key `evk_live_<site>_<secret>`. Authenticates calls to the cloud |
|
|
47
|
+
| `EVOLVE_WEBHOOK_SECRET` | yes | `whsec_…`. Verifies the cloud's calls to your app and signs aggregate responses |
|
|
48
|
+
| `EVOLVE_DATABASE_URL` | to measure | Your Postgres. Without it variants serve and nothing is recorded. Use the pooled URL on serverless. Never sent anywhere |
|
|
49
|
+
| `EVOLVE_TRACK_SECRET` | optional | Bearer secret for `POST /api/evolve/track`; the route is off (404) without it |
|
|
50
|
+
| `EVOLVE_APP_URL` | off Vercel | Your public base URL, sent with the manifest so the cloud can reach the sync and aggregate routes. On Vercel production it defaults to `https://$VERCEL_PROJECT_PRODUCTION_URL` |
|
|
51
|
+
| `EVOLVE_CACHE_DIR` | optional | Disk cache directory for the bundle on long-running Node (default `os.tmpdir()`) |
|
|
52
|
+
| `EVOLVE_CLOUD_URL` | no | Defaults to `https://api.evolveab.com` |
|
|
53
|
+
| `EVOLVE_GLOBAL_CONFIG` | optional (Vercel) | Global Config connection string (read) |
|
|
54
|
+
| `EVOLVE_GLOBAL_CONFIG_ID` | optional (Vercel) | Store id `ecfg_…` (writes) |
|
|
55
|
+
| `EVOLVE_VERCEL_API_TOKEN` | optional (Vercel) | Vercel token that can write the store |
|
|
56
|
+
| `EVOLVE_VERCEL_TEAM_ID` | optional (Vercel) | For team-owned stores |
|
|
57
|
+
|
|
58
|
+
`init` reuses `DATABASE_URL` or `POSTGRES_URL` if you agree (it writes the value into `EVOLVE_DATABASE_URL`).
|
|
59
|
+
|
|
60
|
+
Test-only variables (`EVOLVE_UNSAFE_TEST_PUBLIC_KEY`, `EVOLVE_UNSAFE_ALLOW_TEST_KEY`) exist for the SDK's own test suite. Never set them in production: they let a test key sign bundles.
|
|
61
|
+
|
|
62
|
+
## Cookies
|
|
63
|
+
|
|
64
|
+
| Cookie | Value | Set when |
|
|
65
|
+
| ----------- | ---------------------------------------------------------- | ------------------------------------------------------- |
|
|
66
|
+
| `evolve_id` | Random visitor id | First visit to a tested route with a running experiment |
|
|
67
|
+
| `evolve_a` | The visitor's assignments (`experiment:variant`, up to 20) | Same, and when an assignment changes |
|
|
68
|
+
| `evolve_pv` | A signed preview token | Opening a dashboard preview link (session cookie) |
|
|
69
|
+
|
|
70
|
+
All are `HttpOnly`, `SameSite=Lax`, `Path=/`, and `Secure` over https. None is set for bots, before consent, or without a valid bundle.
|
|
71
|
+
|
|
72
|
+
## Database
|
|
73
|
+
|
|
74
|
+
`next-evolve migrate` creates a schema named `evolve` with these tables: `exposures` (one row per visitor per experiment), `events`, `identities`, `usage_daily` (tested page views per day), `aggregate_runs` (the last 24 h of aggregate responses, for retries) and `migrations`. Your own tables aren't touched. Rows are kept until you delete them.
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Goals and attribution
|
|
2
|
+
|
|
3
|
+
## Defining goals
|
|
4
|
+
|
|
5
|
+
Goals live in `evolve.goals.ts`, in your repo:
|
|
6
|
+
|
|
7
|
+
```ts
|
|
8
|
+
import { defineGoals } from "next-evolve/config";
|
|
9
|
+
|
|
10
|
+
export default defineGoals({
|
|
11
|
+
cta_click: { event: "cta_click", window: "1d" },
|
|
12
|
+
signup_page: { pageview: "/signup", window: "1d" },
|
|
13
|
+
docs_read: { pageview: "/docs/*" },
|
|
14
|
+
activated: { event: "project_created", window: "7d", unit: "user" },
|
|
15
|
+
pro_signup: { event: "signup", where: { plan: "pro" } },
|
|
16
|
+
paid: {
|
|
17
|
+
event: "subscription_started",
|
|
18
|
+
source: "stripe",
|
|
19
|
+
window: "30d",
|
|
20
|
+
value: (e) => e.props.amount as number,
|
|
21
|
+
},
|
|
22
|
+
});
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Keys are `a-z`, `0-9` and `_`, starting with a letter, up to 48 characters. Each goal is either an **event** goal or a **pageview** goal.
|
|
26
|
+
|
|
27
|
+
| Field | Applies to | Default | |
|
|
28
|
+
| ----------- | ---------- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------- |
|
|
29
|
+
| `event` | event | — | The event name to count |
|
|
30
|
+
| `pageview` | pageview | — | An exact path, or a prefix ending in `*` |
|
|
31
|
+
| `window` | both | `"7d"` | How long after a visitor's first exposure a conversion still counts: `"<n>m"`, `"<n>h"` or `"<n>d"`, at most 365 days |
|
|
32
|
+
| `unit` | both | `"visitor"` | `"visitor"` counts per browser; `"user"` counts per identified user across devices (needs `identify`) |
|
|
33
|
+
| `direction` | both | `"increase"` | `"decrease"` when lower is better |
|
|
34
|
+
| `source` | event | any | Only events with this `source` |
|
|
35
|
+
| `where` | event | — | Only events whose props equal these values |
|
|
36
|
+
| `value` | event | — | Makes it a value goal (revenue per visitor instead of conversion rate): a numeric prop path (`"amount"`, `"order.total"`) or a function |
|
|
37
|
+
|
|
38
|
+
**Value functions run in your app**, when the event is written, and the result is stored with the event. The cloud only learns that the goal has a local value function (`{ kind: "local-fn" }`), never the function. A goal without `value` uses the event's own `value` if it has one.
|
|
39
|
+
|
|
40
|
+
The goal definitions (not the functions) are sent to the cloud when the app starts, so the dashboard can list them. Changing a goal's definition makes the cloud recount it from the experiment's start.
|
|
41
|
+
|
|
42
|
+
## Firing events
|
|
43
|
+
|
|
44
|
+
| From | How |
|
|
45
|
+
| ------------------------------------- | --------------------------------------------------------------------------- |
|
|
46
|
+
| A link or button | `data-evolve-goal="cta_click"` (fires on click) |
|
|
47
|
+
| A form | `data-evolve-goal="signup"` on the `<form>` (fires on submit) |
|
|
48
|
+
| Browser code | `track("cta_click", { plan: "pro" })` from `next-evolve/client` |
|
|
49
|
+
| Server code (actions, route handlers) | `await evolve.track("signup", { userId })` from `next-evolve/server` |
|
|
50
|
+
| Another service (Segment, a CRM) | `POST /api/evolve/track` with the bearer secret ([API](api.md#http-routes)) |
|
|
51
|
+
| Stripe | `evolve.track` in your Stripe webhook handler ([recipe](stripe.md)) |
|
|
52
|
+
|
|
53
|
+
Page views are recorded by `<EvolveScript />` automatically, but stored only when a pageview goal matches the path.
|
|
54
|
+
|
|
55
|
+
## Identity
|
|
56
|
+
|
|
57
|
+
The visitor's id is the `evolve_id` cookie. Browser events and server events inside a request carry it automatically. Events from elsewhere (webhooks, another device, a background job) only have your user id. Call `evolve.identify(userId)` once when a visitor signs up or signs in to link the two; from then on, events with that `userId` count for the visitor's experiments. The first link for a visitor wins.
|
|
58
|
+
|
|
59
|
+
## Attribution
|
|
60
|
+
|
|
61
|
+
For each experiment and goal, Evolve counts, per variant:
|
|
62
|
+
|
|
63
|
+
- **Units:** visitors (or users, for `unit: "user"`) exposed to the variant. A visitor's variant is the one in their first exposure, and that never changes (intention to treat).
|
|
64
|
+
- **Conversions:** units with at least one matching event in **[first exposure, first exposure + window)**. Events before the exposure don't count.
|
|
65
|
+
- **Value:** the sum of matching events' values per unit, for value goals.
|
|
66
|
+
|
|
67
|
+
Events match a unit by visitor id, or by user id through `identify`. Bots, visitors outside the traffic share and preview sessions aren't units.
|
|
68
|
+
|
|
69
|
+
Results are grouped by the day of each unit's first exposure (UTC), so recent days stay "open" until their window has passed. The experiment page says "Window not elapsed for X% of visitors" while conversions may still arrive. Aggregates only count data older than 2 minutes.
|
|
70
|
+
|
|
71
|
+
The SQL runs in your database, triggered by a signed request from the cloud. What comes back is counts and sums only; see [what-leaves-your-server.md](what-leaves-your-server.md).
|
|
72
|
+
|
|
73
|
+
## Reading results
|
|
74
|
+
|
|
75
|
+
The experiment page shows, per goal and variant: units, conversions, rate (or value per unit), lift over control with a 95% interval, P(beat control), P(best) and risk, plus a state:
|
|
76
|
+
|
|
77
|
+
- **insufficient:** below the minimum visitors or conversions; it estimates how many more visitors per arm it needs.
|
|
78
|
+
- **running:** enough data to compare, no decision yet.
|
|
79
|
+
- **winner / loser:** P(beat control) ≥ 99% with expected loss ≤ 1% (winner), or P(beat control) ≤ 1% (loser).
|
|
80
|
+
|
|
81
|
+
Arms with no visitors stay out of the comparison. **Recompute** runs an aggregate now (at most once a minute per site).
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Going to production
|
|
2
|
+
|
|
3
|
+
## Checklist
|
|
4
|
+
|
|
5
|
+
1. **Env vars.** Set these in your host's production environment (they're in `.env.local` after `init`). They're server-only; never prefix them with `NEXT_PUBLIC_`.
|
|
6
|
+
|
|
7
|
+
| Var | |
|
|
8
|
+
| ----------------------- | --------------------------------------------------- |
|
|
9
|
+
| `EVOLVE_SECRET_KEY` | `evk_live_…` from the dashboard |
|
|
10
|
+
| `EVOLVE_WEBHOOK_SECRET` | `whsec_…` from the dashboard |
|
|
11
|
+
| `EVOLVE_DATABASE_URL` | Your production Postgres (pooled URL on serverless) |
|
|
12
|
+
| `EVOLVE_TRACK_SECRET` | Only if you use `POST /api/evolve/track` |
|
|
13
|
+
|
|
14
|
+
`EVOLVE_SECRET_KEY` must also be set **at build time**: `prebuild` runs `next-evolve pull`, which bakes the current bundle into the build so a fresh instance serves it before its first fetch.
|
|
15
|
+
|
|
16
|
+
2. **Migrations.** Run `npx next-evolve migrate` against the production database once, and again after upgrading `next-evolve`. It's safe to run on every deploy, for example in the build command: `next-evolve migrate && next build`.
|
|
17
|
+
|
|
18
|
+
3. **App URL.** The cloud has to know where your app lives to send the sync webhook and aggregate requests. On Vercel production it's detected (`VERCEL_PROJECT_PRODUCTION_URL`); elsewhere set `EVOLVE_APP_URL=https://your-app.com`, or register it once:
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
npx next-evolve connect --app-url https://your-app.com
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
It must be a public `https` URL.
|
|
25
|
+
|
|
26
|
+
4. **Check.** After deploying:
|
|
27
|
+
|
|
28
|
+
```sh
|
|
29
|
+
npx next-evolve connect --test # the cloud pings /api/evolve/sync and dry-runs /api/evolve/aggregate
|
|
30
|
+
npx next-evolve doctor # everything else
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
The site's **Connect** page in the dashboard shows the same status: manifest received, bundle fetched, sync webhook, aggregate route, last aggregate. **Run connection test** there does what `connect --test` does.
|
|
34
|
+
|
|
35
|
+
5. **Firewalls and bot protection.** Evolve Cloud calls `POST /api/evolve/sync` and `POST /api/evolve/aggregate` on your app from servers without fixed IP addresses. If you use a WAF, bot protection or "attack mode" (Vercel Firewall, Cloudflare, …), add a rule that lets `/api/evolve/*` through. The requests are authenticated by their HMAC signature, so the route doesn't need other protection. If the connection test fails with `403` or a challenge page, this is why.
|
|
36
|
+
|
|
37
|
+
## Hosts
|
|
38
|
+
|
|
39
|
+
| | Vercel | Vercel + Global Config | Node server / Docker (`next start`) |
|
|
40
|
+
| --------------------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | --------------------------------------------------------------------------- |
|
|
41
|
+
| Fresh instance serves | the build-time bundle | the bundle in Global Config (waits ≤ 20 ms), then the build-time bundle | the disk cache, then the build-time bundle |
|
|
42
|
+
| Background refresh | poll the cloud every 60 s (± 10%) on traffic | read Global Config (10 s cache) | every 60 s, plus on traffic |
|
|
43
|
+
| When an experiment changes | the instance that gets the webhook updates at once; others within one poll interval | all instances within ~10 s | the instance that gets the webhook at once; others within one poll interval |
|
|
44
|
+
| Events while the DB is down | dropped (one retry) | dropped (one retry) | buffered, up to 10,000 rows |
|
|
45
|
+
|
|
46
|
+
"Within one poll interval" means up to ~66 s after an instance's last check, on its next request (that request is still served the old bundle; the fetch runs in the background).
|
|
47
|
+
|
|
48
|
+
### Vercel + Global Config (recommended on Vercel)
|
|
49
|
+
|
|
50
|
+
Without it, the sync webhook reaches one instance and the rest catch up by polling. With a [Global Config](https://vercel.com/docs/edge-config) store, the sync route writes the bundle there and every instance sees it within ~10 s.
|
|
51
|
+
|
|
52
|
+
1. Create a Global Config store and connect it to the project.
|
|
53
|
+
2. Set `EVOLVE_GLOBAL_CONFIG` (the store's connection string), `EVOLVE_GLOBAL_CONFIG_ID` (`ecfg_…`), `EVOLVE_VERCEL_API_TOKEN` (a Vercel access token that can write the store) and, for a team-owned store, `EVOLVE_VERCEL_TEAM_ID`.
|
|
54
|
+
|
|
55
|
+
Bundles read from the store are verified like any other. Without the token the store is read-only and the sync route logs so.
|
|
56
|
+
|
|
57
|
+
### Node server / Docker
|
|
58
|
+
|
|
59
|
+
`registerEvolve()` (in `instrumentation.ts`) polls in the background and caches the bundle on disk in `os.tmpdir()`, or `EVOLVE_CACHE_DIR`. Mount a volume there if you want restarts to survive a cloud outage. The writer flushes buffered events on `SIGTERM`.
|
|
60
|
+
|
|
61
|
+
### CDNs
|
|
62
|
+
|
|
63
|
+
On Vercel the cache keys on the rewritten path, so each variant is cached separately. Elsewhere, the proxy sets `Cache-Control: private, no-store` on tested routes. If a CDN in front of your app ignores that and caches by URL, every visitor gets whichever variant was cached first. Don't cache tested routes at the CDN, or include the `evolve_a` cookie in the cache key.
|
|
64
|
+
|
|
65
|
+
### Serverless functions
|
|
66
|
+
|
|
67
|
+
The `/api/evolve/[...evolve]` route exports `maxDuration = 60`. Aggregation stops at 25 s by default (`aggregate.maxMs`) and returns a partial result the cloud follows up on.
|
|
68
|
+
|
|
69
|
+
## How often results update
|
|
70
|
+
|
|
71
|
+
The cloud asks your app for aggregates on a schedule set by your plan (every 5–60 minutes, plus a full recount daily; during the alpha the schedule may run only daily) and when you click **Recompute** (at most once a minute). Aggregates count data older than 2 minutes.
|
|
72
|
+
|
|
73
|
+
## Upgrading
|
|
74
|
+
|
|
75
|
+
Upgrade with your package manager, run `npx next-evolve migrate`, deploy. The SDK tells the cloud which protocol versions it speaks; the cloud supports the current and the previous one and shows "upgrade required" in the dashboard at least 90 days before dropping one.
|