e2e 0.15.1 → 0.16.0

package/docs/reference/config.mdx

@@ -0,0 +1,986 @@
+---
+title: E2EConfig
+description: Every configuration key, its default, its valid range, and the environment variables.
+---
+
+Export an object from `e2e.config.ts` and use `satisfies E2EConfig` to check
+its shape without widening its inferred types. The runner also validates the
+config when it loads, including ranges and rules TypeScript cannot check.
+
+```ts title="e2e.config.ts"
+import type { E2EConfig } from 'e2e';
+import { web } from '@e2e-dev/web';
+
+export default {
+  targets: [
+    {
+      name: 'web',
+      engine: web({ browser: 'chromium' }),
+      app: { url: process.env.APP_URL ?? 'http://localhost:3000' },
+    },
+  ],
+} satisfies E2EConfig;
+```
+
+Unknown keys are rejected with `INVALID_CONFIG` at every level. There is no
+top-level `app` key: each target declares the app it tests (see
+[the app under test](#the-app-under-test)).
+
+An optional key takes a value or is left out; `undefined` is not a value, so
+`store: process.env.CACHE_URL && createStore(...)` fails to compile. Spread a
+key that depends on the environment in only when it has a value:
+
+```ts
+export default {
+  targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
+  ...(process.env.CACHE_URL === undefined ? {} : { cache: { store: createStore(process.env.CACHE_URL) } }),
+} satisfies E2EConfig;
+```
+
+## Loading
+
+`--config <path>` selects an explicit file. Otherwise the runner searches the
+working directory and each parent for `e2e.config.ts` or `e2e.config.mts`,
+stopping at the repository root. Both names in one directory is
+`CONFIG_AMBIGUOUS`. The selected file's directory is the project root.
+
+The `tests` globs select files relative to the project root. An entry that
+starts with `!` excludes: `tests: ['tests/**/*.e2e.ts', '!tests/wip/**']`
+selects every test file except those under `tests/wip/`. A file is selected
+when an including glob matches it and no exclusion does, whatever the order of
+the list, and a list of only exclusions is `INVALID_CONFIG`. A glob names
+files, so an entry with no wildcard that names a directory (`'!tests/wip'`)
+is `INVALID_GLOB` naming the glob to write (`'!tests/wip/**'`). A file
+passed to `e2e run` that an exclusion takes out is `NO_TESTS` naming that
+entry. Discovery visits
+only directories that could match, never enters one an exclusion takes whole,
+skips `node_modules`, and ignores symlinks. A dot directory is included only
+when a glob segment explicitly starts with a dot.
+
+e2e loads no `.env` file itself. Load one at the top of `e2e.config.ts` with
+Node's `process.loadEnvFile()`; every worker imports the config again, so the
+variables reach the workers too:
+
+```ts title="e2e.config.ts"
+process.loadEnvFile('.env');
+```
+
+It throws when the file is missing; guard it with `existsSync` from
+`node:fs` when only some machines have one.
+
+Config and test files run as ESM whatever the nearest `package.json` `type`
+says, and so does the TypeScript they import: a helper by path, or a
+workspace package that exports its `.ts` source by name. An installed package
+imported by name keeps its own format. A config or test written with `require`
+or `module.exports` fails to load.
+
+## Top level
+
+```ts
+export interface E2EConfig {
+  /** Stable project id, 1 through 256 characters; defaults to the root `package.json` name. */
+  projectId?: string;
+  /** The surfaces tests run on; required, at least one, each naming its engine. */
+  targets: readonly Target[];
+  /** Test file globs relative to the project root (`*`, `?`, and a whole `**` segment; a leading `!` excludes); default every `*.e2e.ts` under `tests/`. */
+  tests?: string | readonly string[];
+  /** Test attempt deadline in milliseconds; default 120000. */
+  timeout?: number;
+  /** Engine init and attempt start budget in milliseconds; default 60000. */
+  launchTimeout?: number;
+  /** Budget of every engine operation in milliseconds; default 30000. */
+  actionTimeout?: number;
+  /** Default `expect` deadline in milliseconds; default 5000. */
+  assertionTimeout?: number;
+  /** Budget of each `afterEach` hook, fixture teardown, and engine cleanup in milliseconds; default 30000. */
+  cleanupTimeout?: number;
+  /** Retries per test, 0 through 10; default 1 in CI, else 0. */
+  retries?: number;
+  /** Parallel workers, 1 through 1024; default 1 in CI, else half the cores. An engine may cap it lower. */
+  workers?: number;
+  /** `{ store }` hands every artifact to a host store as it is produced. */
+  artifacts?: ArtifactsConfig;
+  /**
+   * Which attempts record a trace; default `on`, `on-first-retry` in CI. A
+   * target's `trace` wins over it, `--trace [mode]` over both, and a test's
+   * own `trace` over all. Applies to the targets whose engine can trace.
+   */
+  trace?: RecordingMode;
+  /**
+   * Which attempts record a video; default `off`. A target's `video` wins
+   * over it, `--video [mode]` over both, and a test's own `video` over all.
+   * Applies to the targets whose engine can record.
+   */
+  video?: RecordingMode;
+  /**
+   * The results directory, relative to the project root; default `.e2e`.
+   * `--output <dir>` overrides it for one run.
+   */
+  output?: string;
+  /**
+   * Output renderers and reporter objects. `junit` writes `<output>/junit.xml`,
+   * `markdown` writes `<output>/summary.md`, `json` prints the report and
+   * excludes `list`; a `Reporter` object runs
+   * beside them and `--reporter` never removes it.
+   */
+  reporters?: readonly (BuiltinReporter | Reporter)[];
+  /**
+   * Named agents, each one plain object: the built-in agent's options, or
+   * `{ executor }` for a custom brain. Tests use `default` unless selected
+   * otherwise. No agent inherits another's values. Each worker constructs
+   * its own agents from this config module.
+   */
+  agents?: Readonly<Record<string, AgentConfig>>;
+  /**
+   * Replay cache mode or options. Defaults to `read-write` locally and
+   * `read-only` in CI. An explicit mode overrides the default;
+   * `--no-cache` disables caching for the run.
+   */
+  cache?: CacheMode | CacheConfig;
+  /**
+   * Named accounts. `credentials.user(name).password` is the handle of a
+   * credential's password, a secret named `<name>.password`; `secrets.get()`
+   * never returns it, so a credential and a secret may share a name.
+   */
+  credentials?: Readonly<Record<string, CredentialConfig>>;
+  /**
+   * Named values the model must never see: API keys, tokens, anything sourced
+   * from the environment. `secrets.get(name)` hands a test the opaque handle;
+   * the value is filled by the runner, masked in every observation, and
+   * redacted from logs, traces, and the report.
+   */
+  secrets?: Readonly<Record<string, SecretConfig>>;
+}
+```
+
+Every numeric value must be a safe integer. `projectId` falls back to
+`unportable-<sha256 of the project root, 32 hex>` when there is no
+`package.json`. `workers` never exceeds what a target's engine declares. A
+targeted agent action is bounded by the smaller of `actionTimeout` and 15 s.
+
+## targets
+
+```ts
+/**
+ * One named test target: the app under test and the engine that drives it.
+ * Without an engine, the target supports agent tools only.
+ */
+export interface Target {
+  /** Label in reports and for `--target`; defaults to the platform. */
+  name?: string;
+  /**
+   * Platform label, inherited from the engine when omitted. Required for a
+   * target without an engine; when both name one, they must agree.
+   */
+  platform?: string;
+  /** The engine driving the surface: `web(...)`, `mobile(...)`, or any `defineEngine` handle. */
+  engine?: EngineHandle;
+  /** The app under test: what it is, where it is served, and the command that starts it. */
+  app?: TargetApp;
+  /**
+   * Which attempts on this target record a trace, in place of the config's
+   * `trace`; `--trace` and a test's own `trace` win over it. A mode set here
+   * is required of the engine: one that cannot trace fails the run with
+   * `UNSUPPORTED_ARTIFACT`.
+   */
+  trace?: RecordingMode;
+  /**
+   * Which attempts on this target record a video, in place of the config's
+   * `video`; `--video` and a test's own `video` win over it. A mode set here
+   * is required of the engine: one that cannot record fails the run with
+   * `UNSUPPORTED_ARTIFACT`.
+   */
+  video?: RecordingMode;
+}
+```
+
+`name` is the id `--target`, reports, sessions, and artifact paths use:
+ASCII letters, digits, `_`, `-`, `.`, not only dots (`.` and `..` are path
+segments), unique across targets.
+
+`{ engine: web(), app: { url } }` is a complete target. Two targets on one
+platform need names. A target without an engine runs through agent tools
+only. Tests whose `requires` names a capability the target lacks are skipped
+for it; a `test.setup` a selected test depends on must be runnable there or
+selection fails with `COLLECTION_ERROR`.
+
+## The app under test
+
+A target declares the app it tests in `app`; the engine only drives it. A
+browser target names the URL it opens, a device target the installed app or
+the build it launches. The runner resolves `app` at config load.
+
+```ts title="e2e.config.ts"
+targets: [
+  {
+    name: 'web',
+    engine: web({ viewport: { width: 1440, height: 900 } }),
+    app: {
+      url: 'http://localhost:3000',
+      command: { executable: 'npm', args: ['run', 'dev'] },
+      readyUrl: 'http://localhost:3000/health',
+    },
+  },
+  {
+    name: 'ios',
+    engine: mobile({ platform: 'ios' }),
+    app: {
+      bundleId: 'com.example.app',
+      appPath: './build/App.app',
+      launchArguments: ['-e2e', 'YES'],
+      permissions: { camera: 'grant' },
+    },
+  },
+],
+```
+
+| Key | Targets | Meaning |
+| --- | --- | --- |
+| `url` | web | Base URL: `app.open()` opens it and relative navigation resolves against it. WHATWG-normalized; no userinfo, query, or fragment; a missing scheme becomes `https://`, or `http://` for a loopback host. Plain HTTP is accepted for loopback hosts only. Port `0` on `127.0.0.1` or `[::1]` asks for a [free port](#free-ports) |
+| `bundleId` | mobile | The installed app a device launches: a bundle id, an Android package name, or a display name the device resolves (`Settings`). A link is `INVALID_CONFIG`: a test opens one with `device.openLink` |
+| `appPath` | mobile | The build the suite runs against, an iOS `.app` bundle or an Android `.apk`, resolved against the project root. Without `bundleId`, the app it installs is the one launched. See [the device engine](/reference/mobile#engine) for when it is installed |
+| `identity` | any | Stable identity keying replay cache and session entries. Defaults to the URL's origin and base path, else `bundleId`, else `appPath`. Set a stable value to reuse entries across preview URLs or versioned build paths. Different apps must have different identities to prevent replay across them |
+| `environment` | any | `test`, `staging`, or `production`. Labels the target in the report and joins the cache and session identity digest; never gates a run. Defaults to `test` for loopback, `.localhost`, and `.test` hosts and for an app without a URL, `production` otherwise |
+| `launchArguments` | mobile | Arguments every fresh launch passes the app: `app.open()`, `app.restart()`, and `app.clearState()`. Arguments that select a build mode change what the app is: give each mode its own `identity` |
+| `permissions` | mobile | Permissions the app holds on every fresh launch, by name, each `grant`, `deny`, or `reset`; see [permissions](/reference/mobile#engine) |
+| `command` | any | Process the runner starts before the first test and stops on every exit path (a dev server, Metro for a debug build); see [command](#command). Targets declaring the same command share one process, probed at the first declaring target's `readyUrl` |
+| `readyUrl` | any | URL polled until `command` is ready (a 200-499 status); defaults to `url`, and is required without one. `{port}` in it is the port of `url` |
+
+Each engine checks the keys at config load, with `INVALID_CONFIG` naming
+the target. A `web()` target needs `app.url` and refuses `bundleId`,
+`appPath`, `launchArguments`, and `permissions`. A `mobile()` target needs
+`app.bundleId` or `app.appPath`; `app.url` on a mobile target is not
+supported yet. An unknown key in `app` or `app.command` fails with a
+did-you-mean. A target with `app` and no engine is
+`INVALID_CONFIG`.
+
+The old spellings are unknown keys and fail at config load with
+`INVALID_CONFIG`: `web({ url })` and the other app options of `web()` and
+`mobile()`, `mobile({ app })` (now `app.bundleId`), an app key directly on the
+target, a top-level `app` or `url`, and `services` anywhere, which this
+version does not have.
+
+Port `0` on any host other than `127.0.0.1` or `[::1]`, `localhost`
+included, is `INVALID_APP_URL`, and a port-0 `url` with nothing to start on
+it (no `app.command`) is `INVALID_CONFIG`. An `app.command` with neither
+`app.url` nor `app.readyUrl` is `APP_URL_REQUIRED` at config load.
+
+### The app's site
+
+A test or the agent may open any http(s) URL; `file:`, `data:`, and
+`javascript:` are `POLICY_DENIED`. The site of `url`, its registrable domain
+(`app.example.com` and `auth.example.com` are one site), scopes two things:
+the browser engine's [`headers`](/reference/web#options) reach the
+site and no other host, and child frames off the site stay out of
+observations.
+
+### Free ports
+
+A `url` on a literal loopback address with port `0` (`http://127.0.0.1:0`,
+`http://[::1]:0`) asks the run for a free TCP port. The runner picks one when
+the config loads and substitutes it wherever the declaration used the port:
+the base URL, the default `readyUrl`, and the report. The command must take
+the port through `{port}` in `args` or `env`.
+
+```ts title="e2e.config.ts"
+engine: web(),
+app: {
+  url: 'http://127.0.0.1:0',
+  command: { executable: 'pnpm', args: ['dev', '--port', '{port}'], env: { PORT: '{port}' } },
+},
+```
+
+`localhost:0` is `INVALID_APP_URL`: it may resolve to an address other than
+the one the command binds. Write the address the command will bind.
+
+Each target asking for port `0` gets its own port and its own process; two
+targets that declare the same command on a fixed port share one. `{port}` in
+`app.command` and `app.readyUrl` is the port of `url`. The default `identity`
+derives from the declared `:0`, so cache and session entries do not churn
+with the port. Tests read the allocated URL from
+[`app.baseUrl`](/reference/app).
+
+### command
+
+```ts
+export interface CommandConfig {
+  /** Resolved with `PATH`; never shell-interpreted. */
+  executable: string;
+  /** Passed verbatim; `{port}` expands to the app's port. */
+  args?: readonly string[];
+  /** Working directory, resolved from the project root. */
+  cwd?: string;
+  /** Added to the inherited set; `{port}` expands in values. */
+  env?: Readonly<Record<string, string>>;
+  /** Ready-probe budget in milliseconds; default 60000. Expiry is `APP_UNREACHABLE`. */
+  startupTimeout?: number;
+  /** Grace period before force-kill in milliseconds; default 10000. */
+  shutdownTimeout?: number;
+  /** File that receives the process's stdout and stderr, appended, resolved from the project root. Omitted discards output. */
+  log?: string;
+  /**
+   * When the readiness URL already answers before the command starts, use that
+   * process instead of spawning: nothing is started and nothing is stopped.
+   * Off by default; CI ignores it and always starts the command. A URL on a
+   * free port (port 0) can never already answer, so the two together are
+   * `INVALID_CONFIG`.
+   */
+  reuseExisting?: boolean;
+}
+```
+
+`log` must stay inside the project root and holds the raw, unredacted
+output, so keep it in an ignored directory. A startup failure quotes the last
+20 lines appended since the command started, with `command.env` values
+replaced by `<secret:NAME>` markers. Without `reuseExisting`, a `readyUrl`
+that already answers is `APP_ALREADY_RUNNING`. `reuseExisting` on an app
+whose `app.url` asks for port 0 is `INVALID_CONFIG`: a process already
+running cannot serve the port the run assigns.
+
+The child inherits only `PATH`, `HOME`, the temp-directory variables, and
+`command.env`, so pass what the app needs explicitly. Every `args` entry and
+`env` value is a string; anything else, a number or a `secrets.get()` handle
+included, is `INVALID_CONFIG`, except an `undefined` `env` value, which is
+dropped. The runner never terminates a process it did not start.
+
+### services
+
+<Note>
+Dependency processes (a database, an API mock, a dev server several targets
+share) are not started by this version, and `services` is an unknown key
+wherever it appears. Start them before the run, or have `app.command` start
+a script that brings them up and serves the app. A services API returns in a
+later release.
+</Note>
+
+## output
+
+```ts
+output: 'test-results/e2e',
+```
+
+Everything a run writes about itself goes under one directory, `.e2e` by
+default, resolved from the project root. `--output <dir>` on
+[`e2e run`](/reference/cli#e2e-run) and `e2e explore` overrides it for one
+run.
+
+| Path under the output | What |
+| --- | --- |
+| `report.json` | The canonical report; `--last-failed` reads the one here |
+| `junit.xml`, `summary.md`, `failures/` | The `junit` and `markdown` reporters' files |
+| `ai-trace.json` | `--ai-trace` |
+| `artifacts/` | Screenshots, traces, videos, downloads, and logs, one directory per attempt. Cleared once a run's tests start, so it holds the latest run's evidence and nothing its report no longer names |
+| `sessions/` | The run's encrypted `test.setup` sessions, deleted when it ends |
+| `videos/<session>/` | [`e2e mcp`](/reference/mcp#recording) session recordings |
+
+A run that stops before its tests start writes nothing here: a selection
+that matches no test (`NO_TESTS`, a mistyped `--grep`), a collection error,
+a target that cannot record what it asks for (`UNSUPPORTED_ARTIFACT`),
+`NO_LAST_RUN`, an app or service that fails to start, or an interrupt
+leaves the previous run's `report.json`, `ai-trace.json`, and `artifacts/`
+as they were, so `--last-failed` still reads the last run that executed. An `e2e mcp` session writes its attempt's artifacts under the same
+`artifacts/` and never clears it.
+
+The replay cache is independent: [`cache.dir`](#cache) defaults to
+`.e2e/cache` and resolves from the project root whatever `output` says, so
+moving the results never moves the recordings. Because a run clears and
+owns part of it, the output directory fails the config load with
+`INVALID_CONFIG`, naming the reason, when it:
+
+- is the project root, or outside it;
+- is a file, or a path under one (a missing directory is created);
+- is `cache.dir` or inside it, or would hold `cache.dir` under `artifacts/`,
+  `sessions/`, or `videos/`;
+- holds a directory a `tests` glob scans (`output: 'tests'` with the default
+  `tests/**/*.e2e.ts`).
+
+Paths are compared with symlinks resolved, so an absolute path spelled
+through a symlink (`/tmp` for `/private/tmp` on macOS) names the same
+directory as its target. A symlink whose target does not exist yet is
+followed too: an output through one that points outside the project root is
+refused.
+
+`e2e init` writes `.gitignore` entries for the default `.e2e/` paths only.
+With `output` set elsewhere, add that directory to `.gitignore` yourself.
+
+The removed `--artifacts <dir>` flag fails with the `--output` it maps to:
+the report used to go to the directory's parent, so `--artifacts
+out/artifacts` is `--output out`. A parent an output cannot be (the project
+root, or outside it) maps to `--output .e2e`.
+
+## artifacts
+
+```ts
+artifacts: {
+  store: {
+    async put(artifact) {
+      const key = await bucket.upload(artifact.path, artifact.bytes);
+      return { ref: key };
+    },
+  },
+},
+```
+
+`artifacts` says where artifacts go, not which are made: [`trace` and
+`video`](#recording) choose the recordings, and a failure's screenshot and
+screen text are captured whenever the engine can (never after a secret
+fill, see [Security](/security)). The removed forms fail with
+`INVALID_CONFIG` naming what replaced them: a kinds list (`artifacts:
+['screenshot', 'trace']` or `artifacts.kinds`) names the `trace` mode it
+meant, `artifacts.trace` names `trace: 'on-all-retries'` for `record:
+'retries'` (read together with a kinds list that holds `trace`, so the pair
+still means retries only), and `artifacts.video` names the `video` option.
+
+`store` receives every artifact once it is complete on disk and returns a
+reference the report records as `ref` beside the local path. A failed `put`
+never fails the run. Each worker constructs its own store from the config
+module. `redaction` is the report's label: a store that exports only what
+the runner vouches for keeps back an `incomplete` artifact, a video or a
+download the runner did not rewrite.
+
+A video a hosted service keeps (a [provider's own recording](#what-a-video-writes))
+is a link with no bytes, so `put` never receives it. A store that implements
+the optional `putLink` gets each one instead, once the attempt stopped the
+recording, and its `ref` lands on the report's artifact beside `url`. A
+passed attempt's link under `retain-on-failure` is never handed over, since
+the report leaves it out too, and a failed `putLink` never fails the run.
+
+```ts
+artifacts: {
+  store: {
+    async put(artifact) {
+      return { ref: await bucket.upload(artifact.path, artifact.bytes) };
+    },
+    async putLink(link) {
+      return { ref: await index.record(link.url, link.attemptId) };
+    },
+  },
+},
+```
+
+```ts
+export interface ArtifactsConfig {
+  /** Host store every produced artifact is handed to; omit it to keep files local only. */
+  store?: ArtifactStore;
+}
+
+export interface ArtifactStore {
+  put(artifact: StoredArtifact): Promise<{ readonly ref: string }>;
+  /** Receives a provider-hosted video as a link; optional. */
+  putLink?(link: StoredArtifactLink): Promise<{ readonly ref: string }>;
+}
+
+/** One produced artifact, handed to `store.put` once it is complete on disk. */
+export interface StoredArtifact {
+  readonly kind: 'screenshot' | 'trace' | 'video' | 'download' | 'log';
+  readonly mediaType: string;
+  readonly bytes: Uint8Array;
+  readonly size: number;
+  readonly sha256: string;
+  /** Report-relative path with `/` separators, stable across runs of the same test. */
+  readonly path: string;
+  /** How much of the file the runner masked, as the report records it. */
+  readonly redaction: 'complete' | 'not-required' | 'incomplete';
+  readonly runId: string;
+  readonly testId: string;
+  readonly attemptId: string;
+  /** The step that produced it, when one was running. */
+  readonly stepId?: string;
+  /** When a time-based artifact (a video segment) began recording. */
+  readonly startedAt?: string;
+}
+
+/** One provider-hosted recording, handed to `store.putLink` once the attempt stopped it. */
+export interface StoredArtifactLink {
+  readonly kind: 'video';
+  /** The `http(s)` URL the service serves the recording from. */
+  readonly url: string;
+  readonly mediaType: string;
+  /** A recording masks nothing, so a link is always `incomplete`. */
+  readonly redaction: 'incomplete';
+  readonly runId: string;
+  readonly testId: string;
+  readonly attemptId: string;
+  /** When the recording started, as an ISO timestamp. */
+  readonly startedAt: string;
+  /** The step that was running when the recording was registered, if any. */
+  readonly stepId?: string;
+}
+```
+
+## Recording
+
+```ts
+trace: 'on-first-retry',
+video: 'retain-on-failure',
+targets: [
+  { name: 'chromium', engine: web(), app: { url } },
+  { name: 'firefox', engine: web({ browser: 'firefox' }), app: { url }, video: 'on-first-retry' },
+],
+```
+
+`trace` and `video` choose which attempts record, with one set of modes:
+
+```ts
+/** Which attempts record a trace or a video, and which recordings are kept. */
+export type RecordingMode = 'off' | 'on' | 'retain-on-failure' | 'on-first-retry' | 'on-all-retries';
+```
+
+| Mode | Records | Keeps |
+| --- | --- | --- |
+| `off` | nothing | nothing |
+| `on` | every attempt | every recording |
+| `retain-on-failure` | every attempt | the recordings of attempts that did not pass; a passed attempt's files are deleted once its verdict is in |
+| `on-first-retry` | the first retry only, the second attempt | that recording. A test that passes first time records nothing |
+| `on-all-retries` | every attempt after the first | every recording. A test that passes first time records nothing |
+
+The names are Playwright's `use.trace` and `use.video` modes. Under a retry
+mode a test with `retries: 0` records nothing, and neither does a failure the
+runner does not retry, such as `LAUNCH_TIMEOUT`. When a test whose engine can
+record runs that way, the run prints one notice per kind at plan time saying
+no traces (or videos) will be recorded for it.
+
+The old trace spellings fail with `INVALID_CONFIG` naming the mode they
+meant, wherever a mode goes (the config, a target, a test, `--trace`):
+`'all'` and `{ record: 'all' }` are `'on'`, and `'retries'` and `{ record:
+'retries' }` are `'on-all-retries'`.
+
+Four places set a mode, lowest first, and each replaces the one before it.
+The same rule holds for `trace` and `video`:
+
+1. `trace` or `video` at the top of the config.
+2. A target's `trace` or `video`.
+3. `--trace [mode]` or `--video [mode]` on [`e2e run`](/reference/cli#e2e-run),
+   over the config and every target. A bare flag is `on`.
+4. A test's or describe's own [`trace` or `video` option](/reference/test#options),
+   innermost wins: `test('checkout', { video: 'on' }, ...)`.
+
+A serial group records one trace and one video per group attempt, so its
+members cannot set `trace` or `video` (a `COLLECTION_ERROR`, like `retries`).
+The value on the serial `describe`, or on a group around it, applies to every
+member.
+
+`e2e explore` and an `e2e mcp` session are one attempt each and use the
+target's mode, `--trace` and `--video` included for `explore`, so a retry
+mode records nothing there. That includes the CI default: `CI=1 e2e explore`
+records no trace unless you pass `--trace on`, and its plan-time notice says
+so. An `e2e mcp` session always closes as passed, so
+it keeps a trace only under `on`. It never records video from the configured
+mode: its recordings are the ones `start_recording` and `stop_recording`
+make.
+
+Neither `trace` nor `video`, at the top or on a target, enters the config
+digest, and neither is part of a replay's
+[cache key](/cache#what-must-match): turning one on or off never
+invalidates cached replays.
+
+### trace
+
+The default is `on` locally and `on-first-retry` in CI (the same `CI`
+detection that makes `retries` default to 1 there). Recording a trace costs
+CPU for the whole attempt: on a 141-test suite, every-attempt traces added
+about a quarter to the run's wall time, which is why CI records only the
+retry a flaky test leaves behind.
+
+The web engine writes a Playwright trace per attempt, `trace/trace.zip` under
+the attempt's artifact directory, which the runner redacts after a secret
+fill; [Debugging](/debugging#artifacts) shows how to open it.
+
+### video
+
+The default is `off`. A video costs more than a trace and holds what the
+screen showed, secrets included, so it is always asked for.
+
+### Engines that cannot record
+
+Before any test starts, the run grades every target. Where a mode was set
+decides what an engine that cannot record that kind does:
+
+- Set at the top of the config or with `--trace` / `--video`: the mode applies
+  to the targets whose engine can record it. The run prints one notice at
+  plan time naming the targets that cannot, and runs them without.
+- Set on a target, or on a test that will run on that target: the mode is
+  required. A target whose engine cannot record it fails the run with
+  `UNSUPPORTED_ARTIFACT`, naming the target and, for a test's, the test.
+
+Only a mode that records on some attempt counts: a run whose tests all
+resolve to `off`, or to a retry mode with `retries: 0`, asks nothing of the
+engine.
+
+### What a video writes
+
+| Recorder | Files under the attempt's artifact directory |
+| --- | --- |
+| Browser engine, Playwright screencast | `video/video.webm`, one segment per page the attempt showed, later ones as `video/video-part<n>.webm`. [`web({ screencast })`](/reference/web#options) sets the frame size and quality |
+| Mobile engine, agent-device | `video/video.mp4` |
+| A provider's own recording ([`BrowserProvider.record`](/browser#recording-on-the-provider), [`DeviceProvider.record`](/mobile#hosted-devices)) | the file the provider wrote, as `video/<file>`: Kernel's is `video/replay.mp4`. Or a link to a recording the service keeps, recorded with `url` in place of `path`; the runner never downloads it |
+
+Each segment carries `startedAt`, so `step.startedAt - artifact.startedAt` is
+a step's offset into it.
+
+A recording masks nothing and is recorded with `redaction: 'incomplete'`.
+Under `retain-on-failure` a link is left out of the report of a passed
+attempt, but the service still holds the recording.
+
+## agents
+
+```ts
+/** A project tool as `defineTool` from `e2e/agent` returns it. */
+export interface AgentTool {
+  readonly tool: object;
+  readonly annotations: { readonly mutates: boolean; readonly platforms?: readonly string[] };
+}
+
+/** The options every agent takes, whichever brain runs its `act` steps. */
+export interface AgentOptions {
+  /** An AI SDK model instance; no implicit default. */
+  model?: ModelInstance;
+  /** Model for `assert`, `waitFor`, and `extract`; defaults to `model`. */
+  judge?: ModelInstance;
+  /**
+   * What the app calls things, told to every model call this agent makes,
+   * act turns and judgments alike. At most 16384 bytes.
+   */
+  context?: string;
+  /** Committed actions per agent call, 1 through 100; default 25. */
+  maxSteps?: number;
+  /** Model requests per agent call, 1 through 100; default 25. */
+  maxModelCalls?: number;
+  /** Deadline of one `assert`, `waitFor`, or `extract` call in milliseconds; default 30000. Raise it for a slow judge; engine operations keep `actionTimeout`. */
+  judgmentTimeout?: number;
+  /** Observation payload ceiling for act turns and judgments, 1024 through 16777216; default 262144. */
+  maxObservationBytes?: number;
+  /** Input tokens per model request, 1 through 1000000; default 64000. A dense screen is cut to fit under it. */
+  maxInputTokens?: number;
+  /**
+   * Provider options every model call carries, e.g. a reasoning effort.
+   * OpenAI and Azure OpenAI calls also carry `store: false` and a prompt
+   * cache key unless set here.
+   */
+  providerOptions?: ProviderOptions;
+}
+
+/** One entry of `agents`: the built-in agent with its options, or a custom brain under `executor`. */
+export type AgentConfig = AgentOptions &
+  (
+    | {
+        /** How the acting agent should work, appended to the built-in rules; only the act loop reads it. */
+        system?: string;
+        /** Project tools from `defineTool`, offered beside the built-in toolset. */
+        tools?: Readonly<Record<string, AgentTool>>;
+      }
+    | {
+        /** A custom brain `agent.act()` dispatches to instead of the built-in agent. */
+        executor: StepExecutor;
+      }
+  );
+```
+
+Tests use the `default` agent unless you select another with
+`e2e run --agent <name>`, a test or group option, or an individual `agent.*`
+call. A test with `{ agent: ['buyer', 'admin'] }` runs once for each agent.
+`default` exists even when the config declares no agents, but has no model.
+
+Names may contain ASCII letters, numbers, `_`, `-`, or `.`, and are never
+only dots: a name is an artifact path segment. An unknown name
+is `INVALID_CONFIG` on the CLI flag, `COLLECTION_ERROR` on a test or group,
+and `INVALID_ARGUMENT` on a call. See [Agents and personas](/agents).
+
+```ts title="e2e.config.ts"
+import type { E2EConfig } from 'e2e';
+import { web } from '@e2e-dev/web';
+import { gateway } from 'ai';
+import { houseBrain } from './house-brain.ts';
+
+export default {
+  targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
+  agents: {
+    default: { model: gateway('openai/gpt-6-luna-fast') },
+    ux: { model: gateway('openai/gpt-6-luna-fast'), system: 'Check labels, contrast, and keyboard access.' },
+    thorough: { model: gateway('anthropic/claude-opus-5'), context: 'Verify every claim on screen.' },
+    buyer: {
+      model: gateway('openai/gpt-6-luna-fast'),
+      system: 'You are a first-time buyer.',
+      context: 'Plans are called tiers. Billing lives under Settings.',
+      maxSteps: 40,
+    },
+    house: { executor: houseBrain, model: gateway('openai/gpt-6-luna-fast'), maxModelCalls: 10 },
+  },
+} satisfies E2EConfig;
+```
+
+Every entry starts from the built-in defaults. Agents do not inherit from
+each other: `ux` above gets no `context` from `default`, and a key left out of
+an entry is the default value, never another entry's.
+
+With `executor`, `model`, `judge`, `context`, and the budgets still apply
+(judgments need a model), while `system` and `tools` are `INVALID_CONFIG`: the
+executor brings its own prompt and tools. `maxModelCalls` bounds the model
+turns of one step. A string in `model` is `INVALID_CONFIG` naming the
+constructor to write, and so is a `model` or `judge` that differs from the one
+the executor brought. See [Custom executors](/executors) and
+[A judge of its own](/models#a-judge-of-its-own).
+
+Loading the config fails with `INVALID_CONFIG` for an unknown key (naming the
+nearest known one), a removed key, and a value in the old shapes:
+
+| Written | Now |
+| --- | --- |
+| a bare `StepExecutor` as the entry | `{ executor }` |
+| `timeout` | `judgmentTimeout` |
+| `maxTurns` (on the entry or `createToolLoopExecutor`) | `maxModelCalls` |
+| top-level `limits.maxModelTokensPerCall` | `maxInputTokens` on each agent |
+| top-level `limits.maxAgentContextBytes`, `maxLedgerBytes`, `maxEventsPerStep` | removed; the runner fixes them at 16384 bytes, 8192 bytes, and 1000 events |
+
+Judgments (`assert`, `waitFor`, `extract`) allow 8192 output tokens per call
+and see the instruction and the current screen only.
+
+### model
+
+```ts
+/**
+ * An AI SDK LanguageModelV2+ instance. Its provider handles transport
+ * and credentials.
+ */
+export interface ModelInstance {
+  /** The AI SDK specification the instance implements. */
+  readonly specificationVersion: string;
+  /** Provider id, recorded per step in the report. */
+  readonly provider: string;
+  /** Model id, recorded per step in the report. */
+  readonly modelId: string;
+  /** The generate call every AI SDK language model implements; what tells a live instance from its three id strings. */
+  readonly doGenerate: (...args: never[]) => unknown;
+}
+```
+
+Construct the model in your config with an AI SDK provider. For subscription
+sign-in, use the [`e2e/oauth/*` models](/subscriptions).
+
+| Gateway | Constructor | Package | Reads |
+| --- | --- | --- | --- |
+| [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) | `gateway('openai/gpt-6-luna-fast')` | `ai` | `AI_GATEWAY_API_KEY`, or a Vercel OIDC token without it |
+| [OpenRouter](https://openrouter.ai/models) | `openrouter('openai/gpt-6-luna-fast')` | `@openrouter/ai-sdk-provider` | `OPENROUTER_API_KEY` |
+| Any OpenAI-compatible endpoint | `createOpenAICompatible({ name, baseURL, apiKey? }).chatModel('llama3.2')` | `@ai-sdk/openai-compatible` | `apiKey`, when the endpoint needs one |
+| A provider directly | `openai('gpt-6-luna')`, `anthropic(...)`, ... | `@ai-sdk/openai`, `@ai-sdk/anthropic`, ... | the provider's own variable |
+
+Reports record `provider` and `modelId` per step. Each worker constructs its
+own instance from the config module; a digest that differs between the
+runner and a worker is `CONFIG_NOT_DETERMINISTIC`. See
+[Models](/models).
+
+## credentials
+
+```ts
+/** One `config.credentials` entry: an account whose password is a secret named `<entry>.password`. */
+export interface CredentialConfig {
+  username: string;
+  password: string | SecretProvider;
+}
+
+type SecretProvider = () => string | Promise<string>;
+```
+
+A `SecretProvider` is called on every authorized fill (a vault lookup, a
+freshly computed TOTP) and recorded in the config digest by name only. An
+`E2E_USER_*` override wins over it. A static password is at least 6
+characters (code points), like a secret, or `INVALID_CONFIG`; a provider
+returning fewer fails the fill with `AUTH_CREDENTIAL_UNAVAILABLE`.
+
+`credentials.user(name)` resolves a configured entry into a `Credential`.
+The username is a plain string; the password is an opaque `Secret` that test
+code cannot read, accepted only by `locator.fill` and `agent.act` params. An
+unconfigured name throws `AUTH_CREDENTIAL_UNAVAILABLE`.
+
+Credentials and secrets are separate namespaces: `secrets.get(name)` returns a
+`secrets` entry only, and a credential's password is reachable only as
+`credentials.user(name).password`. The agent, the replay cache, and the
+reports know that handle as `<name>.password` (`<secret:admin.password>`), so
+`credentials.admin` and `secrets.admin` can both exist; a `secrets` entry
+named exactly `admin.password` beside an `admin` credential is
+`INVALID_CONFIG`.
+
+```ts
+/** Named test identity. The password remains an opaque Secret. */
+export interface Credential {
+  /** The entry's name in `credentials`. */
+  readonly name: string;
+  /** Plain username. */
+  readonly username: string;
+  /** Opaque; test code cannot read it. */
+  readonly password: Secret;
+  readonly [credentialBrand]: true;
+}
+```
+
+See [Signing in](/authentication).
+
+## secrets
+
+```ts
+/** One `config.secrets` entry: the value itself, or a provider computing it at fill time. */
+export type SecretConfig = string | SecretProvider;
+```
+
+An entry is a static value, or a provider called on every authorized fill (a
+vault lookup, a freshly computed TOTP). It is recorded in the config digest by
+name only, and an `E2E_SECRET_<NAME>` override wins over both. Every entry
+must resolve to a string of at least 6 characters (code points) or a
+function; anything else is `INVALID_CONFIG`. The minimum exists for redaction: every occurrence
+of a value becomes `<secret:name>`, and a shorter value would take ordinary
+text in reports and output with it. A provider's value is held to the same
+minimum at fill time, with `SECRET_UNAVAILABLE`.
+
+`secrets.get(name)` returns the opaque `Secret` for a configured entry. It has
+no plaintext accessor and is accepted only by `locator.fill` and `agent.act`
+params; the runner fills the field, masks the value in every observation, and
+redacts it from logs, traces, and the report. An unconfigured name throws
+`SECRET_UNAVAILABLE`.
+
+Called while `e2e.config.ts` evaluates, before any run exists, `secrets.get(name)`
+returns a reference by name that an engine option can hold, such as
+`web({ basicAuth: { username, password: secrets.get('previewPassword') } })`.
+The engine declares it, the config load fails with `INVALID_CONFIG` when no
+`secrets` entry has the name (a credential's password is not one: declare a
+value an engine option needs under `secrets`), and the engine resolves the value
+when each attempt starts (a provider runs again every time). The value joins
+the attempt's redaction then, without the screenshot restrictions a fill
+brings, since it goes to the engine rather than onto the page. The digest
+records the name only. The reference is not the value: turned into a string
+(a template literal, `String()`, `+`, `JSON.stringify`) it throws
+`INVALID_CONFIG`, and so does one placed in an `args` entry or `env` value of
+`app.command`, which takes strings and placeholders only. Read a value a string needs, such as an app's API
+key, from `process.env` yourself.
+
+```ts
+/**
+ * What a secret is for, which decides where it may be filled: a `password`
+ * goes only into a password field; a `generic-secret` (an API key, a token,
+ * any value from the environment) goes into any editable input.
+ */
+export type SecretPurpose = 'password' | 'generic-secret';
+
+/** Opaque host-side value accepted only by sensitive input sinks. */
+export interface Secret {
+  /** The entry's name in `secrets`, or `<credential>.password` for a credential's password. */
+  readonly name: string;
+  /** Where the value may be filled. */
+  readonly purpose: SecretPurpose;
+  readonly [secretBrand]: true;
+}
+```
+
+See [Secrets that are not passwords](/authentication#secrets-that-are-not-passwords).
+
+## cache
+
+In `read-write` mode the replay cache records an `agent.act()` only after a
+later verification passes, such as a locator assertion or `agent.assert()`.
+`read-only` replays existing entries and writes none; `off` disables both.
+See [Caching agent steps](/cache).
+
+```ts
+/**
+ * Replay cache options. Defaults to files under `.e2e/cache/`.
+ * A custom store replaces file storage; each worker constructs its own.
+ */
+export interface CacheConfig {
+  /** Default `read-write`; `read-only` in CI when unset. */
+  mode?: CacheMode;
+  /** Custom entry store; omit it to use the file store at `dir`. */
+  store?: CacheStore;
+  /** File store directory, resolved against the project root; default `.e2e/cache`, whatever `output` says. */
+  dir?: string;
+  /** Fail a step whose recording no longer replays with `REPLAY_STALE`. Default `false`. */
+  strict?: boolean;
+}
+
+/** Where the replay cache keeps its entries: the file store by default, or a host's own through `cache.store`. */
+export interface CacheStore {
+  readonly writable: boolean;
+  read(keyHash: string): Promise<CacheReadResult>;
+  /** Persists one trace, returning its size, or undefined when not written. */
+  write(keyHash: string, payload: ActionTrace): Promise<{ bytes: number } | undefined>;
+  /**
+   * Evicts one entry, best-effort. Called when a replayed or just-recorded
+   * flow is implicated in a failed attempt: the entry re-records on the next
+   * pass instead of replaying a poisoned flow forever. Optional: a store
+   * without eviction merely stays stale until the next confirmed write.
+   */
+  delete?(keyHash: string): Promise<void>;
+}
+
+export type CacheReadResult =
+  | { readonly status: 'hit'; readonly entry: TraceEntry; readonly bytes: number }
+  | { readonly status: 'miss' }
+  | { readonly status: 'invalid'; readonly reason: string; readonly bytes?: number };
+```
+
+`CacheStore`, `CacheReadResult`, `TraceEntry`, and `ActionTrace` are
+exported from `e2e`.
+
+A bare string is shorthand for `{ mode }`. Set `'read-write'` explicitly when
+CI restores the cache from its own cache service; `--no-cache` wins over
+both. A custom `store` is exempt from the CI demotion and states its own
+trust through `writable`. `strict` (or `--strict-cache` for one run) turns a
+recording that exists but no longer replays into a `REPLAY_STALE` failure
+instead of a hand-off to the agent; see
+[Fail on a stale recording](/cache#fail-on-a-stale-recording).
+
+### Commit the replay cache
+
+`e2e init` adds `.e2e/cache/` to `.gitignore`, so committing entries is
+opt-in. Remove that line to share replays with CI and teammates; CI stays
+`read-only` unless `cache: 'read-write'` is set. See
+[Caching agent steps](/cache#commit-the-replay-cache).
+
+## Run limits
+
+The report's `run.limits` block records the ceilings a run enforced. There is
+no `limits` key: `maxObservationBytes` and `maxModelTokensPerCall` are the
+largest `maxObservationBytes` and `maxInputTokens` any configured agent
+sets, and the rest are fixed by the runner.
+
+| Report field | Value |
+| --- | --- |
+| `maxAgentContextBytes` | 16384: an agent's `context` and a test's `agentContext` |
+| `maxLedgerBytes` | 8192: the prior-step ledger an act step reads |
+| `maxEventsPerStep` | 1000: events recorded per step |
+| `maxObservationBytes` | the largest `agents.<name>.maxObservationBytes` |
+| `maxModelTokensPerCall` | the largest `agents.<name>.maxInputTokens` |
+
+Per-call options may only lower an agent's budgets.
+
+## Environment variables
+
+The runner reads a closed set, listed on the
+[Environment variables](/reference/environment) page. `E2E_USER_<NAME>_USERNAME`
+and `E2E_USER_<NAME>_PASSWORD` override a credential, `E2E_SECRET_<NAME>` a
+secret, and `CI` switches the defaults below.
+
+### Resolution order
+
+| Setting | Order |
+| --- | --- |
+| Base URL | the target's `app.url`, else `APP_URL_REQUIRED` at the first navigation. The runner reads no `APP_URL`; the scaffolded config reads it itself |
+| Built-in agent model | `agents.<name>.model`, then `MODEL_UNAVAILABLE` at `agent` acquisition; an agent never falls back to `default`'s model |
+| Custom executor model | `agents.<name>.model`, else the model the executor brought; two different values is `INVALID_CONFIG` |
+| Model credential | the provider package's own variable or the key passed at construction, read by the provider on the first model call |
+| Credential username and password | the `E2E_USER_*` variable, then the config value |
+
+The built-in agent checks for a missing model once per run, when the `agent`
+fixture is first acquired, and stops the run with exit 2. A custom executor
+can implement `act` and `assert` without a model. `waitFor` and `extract`
+still require one.
+
+### CI
+
+CI mode is on unless `CI` is unset, blank, `0`, or `false`.
+
+| Setting | Local | CI |
+| --- | --- | --- |
+| `retries` | `0` | `1` |
+| `workers` | `max(1, floor(availableParallelism() / 2))` | `1` |
+| `cache` | `read-write` | `read-only` when unset |
+| `test.only` | allowed | `ONLY_IN_CI`, exit code 2 |
+| `command.reuseExisting` | honored | ignored, with a notice |