e2e 0.21.1 → 0.21.2

package/docs/reference/environment.mdx

@@ -10,7 +10,15 @@ These variables configure the runner and first-party packages. In the tables,
 `set` means the variable exists, `value` means it is not blank, and `flag`
 means it is not blank, `0`, or `false`. Flag values are case-insensitive.
 
-## Model
+```bash
+export E2E_USER_ADMIN_PASSWORD="$ADMIN_PASSWORD"
+export E2E_CACHE_DIR=~/.cache/e2e
+export E2E_TELEMETRY_DISABLED=1
+```
+
+## Reference
+
+### Model
 
 Configure a model in `e2e.config.ts`. Its provider reads the API key:
 `gateway()` uses `AI_GATEWAY_API_KEY`, `openrouter()` uses
@@ -21,109 +29,173 @@ credential fails on the first model call. Subscription models use a stored
 login instead.
 See [Models](/models) and [Subscriptions](/subscriptions).
 
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `E2E_OAUTH_CREDENTIALS` | value | The stored subscription logins as JSON, the contents of `oauth.json`, for a machine nobody can sign in on. The `e2e/oauth/*` models and `e2e models` read it in place of `$XDG_CONFIG_HOME/e2e/oauth.json`; `e2e login` and `e2e logout` refuse to change it, and refreshed tokens are not written back |
-| `OPENCODE_API_KEY` | value | An OpenCode Console service account API key or an OpenCode Go subscription API key; `opencodeConsole()` and `e2e models opencode-console` use it in place of the stored `opencode-console` login. OpenCode reads the same variable |
-| `XDG_CONFIG_HOME` | value | Directory of the stored logins, `$XDG_CONFIG_HOME/e2e/oauth.json`; default `~/.config/e2e/oauth.json` |
-
-## Credentials and secrets
-
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `E2E_USER_<NAME>_USERNAME` | set | Replaces `credentials.<name>.username` |
-| `E2E_USER_<NAME>_PASSWORD` | set | Replaces `credentials.<name>.password` |
-| `E2E_SECRET_<NAME>` | set | Replaces the value of `secrets.<name>` |
-
-`<NAME>` is the name uppercased with every character outside `A-Z0-9`
-replaced by `_`, so `power-user` becomes `E2E_USER_POWER_USER_USERNAME`. Two
-credentials, or two secrets, whose names map to the same `<NAME>` (`api-key`
-and `api_key` both read `E2E_SECRET_API_KEY`; `svc-a` and `svc_a` both read
-`E2E_USER_SVC_A_USERNAME` and `_PASSWORD`) are `INVALID_CONFIG` at config
-load, whether or not a variable is set, so one rotation never replaces two
-entries; the message names the entries and the variables. A credential and a
-secret may share a name, since their prefixes differ. A
-variable for a name absent from the config is ignored. An empty value is
-`INVALID_CONFIG`, and so is a password or secret shorter than 6 characters
-(code points): redaction rewrites every occurrence of a value, and one that
-short would take ordinary text with it.
-
-## Run mode
-
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `CI` | flag | CI mode: `retries` defaults to 1 and `workers` to 1, `cache` defaults to `read-only`, a focused test is `ONLY_IN_CI`, and `reuseExisting` on an app command is ignored |
-
-## Replay cache
-
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `E2E_CACHE_DIR` | value | Replaces [`cache.dir`](/reference/config#cache) for `e2e run` and `e2e cache`, resolved from the project root; `~` or a leading `~/` is the home directory. `--cache-dir` wins over it, a custom `cache.store` ignores it, the cache mode stays as configured, and `--no-cache` still turns the cache off |
-
-## Diagnostics
-
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `NO_COLOR` | set | Plain CLI output. Colors are otherwise on for a TTY or in CI and off when piped |
-| `FORCE_COLOR` | set | Colored CLI output even when piped |
-| `TERM` | value | `dumb` draws the `e2e init` wordmark at rest, without animation, as in CI |
-| `GITHUB_SHA`, `GITHUB_HEAD_REF`, `GITHUB_REF_NAME` | value | The commit and branch the report records under `run.vcs` when `git` cannot read the project root. `GITHUB_HEAD_REF` wins over `GITHUB_REF_NAME` |
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| [`E2E_OAUTH_CREDENTIALS`](#e2e_oauth_credentials) | `E2E_OAUTH_CREDENTIALS="$(cat oauth.json)"` | value | The stored subscription logins as JSON |
+| [`OPENCODE_API_KEY`](#opencode_api_key) | `OPENCODE_API_KEY=<key>` | value | Replaces the stored `opencode-console` login |
+| `XDG_CONFIG_HOME` | `XDG_CONFIG_HOME=~/.config` | value | Directory of the stored logins, `$XDG_CONFIG_HOME/e2e/oauth.json`; default `~/.config/e2e/oauth.json` |
+
+#### `E2E_OAUTH_CREDENTIALS`
+
+The stored subscription logins as JSON, the contents of `oauth.json`, for a
+machine nobody can sign in on.
+
+- The `e2e/oauth/*` models and `e2e models` read it in place of
+  `$XDG_CONFIG_HOME/e2e/oauth.json`.
+- `e2e login` and `e2e logout` refuse to change it.
+- Refreshed tokens are not written back.
+
+#### `OPENCODE_API_KEY`
+
+An OpenCode Console service account API key or an OpenCode Go subscription
+API key. `opencodeConsole()` and `e2e models opencode-console` use it in place
+of the stored `opencode-console` login. OpenCode reads the same variable.
+
+### Credentials and secrets
+
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| `E2E_USER_<NAME>_USERNAME` | `E2E_USER_ADMIN_USERNAME=admin@example.test` | set | Replaces `credentials.<name>.username` |
+| `E2E_USER_<NAME>_PASSWORD` | `E2E_USER_ADMIN_PASSWORD="$ADMIN_PASSWORD"` | set | Replaces `credentials.<name>.password` |
+| `E2E_SECRET_<NAME>` | `E2E_SECRET_API_KEY="$API_KEY"` | set | Replaces the value of `secrets.<name>` |
+
+- `<NAME>` is the name uppercased with every character outside `A-Z0-9`
+  replaced by `_`, so `power-user` becomes `E2E_USER_POWER_USER_USERNAME`.
+- Two credentials, or two secrets, whose names map to one `<NAME>` are
+  `INVALID_CONFIG` at config load, set or not. So one rotation never
+  replaces two entries. The message names the entries and the variables.
+  For example, `api-key` and `api_key` both read `E2E_SECRET_API_KEY`;
+  `svc-a` and `svc_a` both read `E2E_USER_SVC_A_USERNAME` and `_PASSWORD`.
+- A credential and a secret may share a name, since their prefixes differ.
+- A variable for a name absent from the config is ignored.
+- An empty password or secret is `INVALID_CONFIG`, and so is one shorter
+  than 6 characters (code points). Redaction rewrites every occurrence of a
+  value, and one that short would take ordinary text with it. An empty
+  username is accepted.
+
+### Run mode
+
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| [`CI`](#ci) | `CI=1` | flag | CI mode |
+
+#### `CI`
+
+Turns on CI mode:
+
+- `retries` defaults to 1 and `workers` to 1.
+- `cache` defaults to `read-only`.
+- A focused test is `ONLY_IN_CI`.
+- `reuseExisting` on an app command is ignored.
+
+### Replay cache
+
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| [`E2E_CACHE_DIR`](#e2e_cache_dir) | `E2E_CACHE_DIR=~/.cache/e2e` | value | Replaces [`cache.dir`](/reference/config#cache) |
+
+#### `E2E_CACHE_DIR`
+
+Replaces [`cache.dir`](/reference/config#cache) for `e2e run` and
+`e2e cache`, resolved from the project root.
+
+- `~` or a leading `~/` is the home directory.
+- `--cache-dir` wins over it.
+- A custom `cache.store` ignores it.
+- The cache mode stays as configured, and `--no-cache` still turns the cache
+  off.
+
+### Diagnostics
+
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| `NO_COLOR` | `NO_COLOR=1` | set | Plain CLI output. Without it, colors are on for a TTY or in CI, off when piped |
+| `FORCE_COLOR` | `FORCE_COLOR=1` | set | Colored CLI output even when piped |
+| `TERM` | `TERM=dumb` | value | `dumb` draws the `e2e init` wordmark at rest, without animation, as in CI |
+| `GITHUB_SHA`, `GITHUB_HEAD_REF`, `GITHUB_REF_NAME` | `GITHUB_REF_NAME=main` | value | The commit and branch the report records under `run.vcs` when `git` cannot read the project root. `GITHUB_HEAD_REF` wins over `GITHUB_REF_NAME` |
 
 `--debug` and `--ai-trace` are flags, not variables; see
 [CLI](/reference/cli#e2e-run).
 
-## Telemetry
+### Telemetry
+
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| `E2E_TELEMETRY_DISABLED` | `E2E_TELEMETRY_DISABLED=1` | flag | Turns anonymous usage telemetry off. `e2e telemetry disable` saves the same choice in the preferences file |
+| `DO_NOT_TRACK` | `DO_NOT_TRACK=1` | flag | Same as `E2E_TELEMETRY_DISABLED` |
+| `E2E_TELEMETRY_DEBUG` | `E2E_TELEMETRY_DEBUG=1` | flag | Prints every telemetry event to stderr instead of sending it |
+| [`E2E_TELEMETRY_FLEET`](#e2e_telemetry_fleet) | `E2E_TELEMETRY_FLEET=acme` | value | Names the platform running e2e for its users |
+| `XDG_CONFIG_HOME` | `XDG_CONFIG_HOME=~/.config` | value | Directory of the preferences file, `$XDG_CONFIG_HOME/e2e`; default `~/.config/e2e` |
+| `APPDATA` | `APPDATA=C:\Users\ada\AppData\Roaming` | value | The same on Windows, `%APPDATA%\e2e`; default `~/AppData/Roaming/e2e` |
+
+#### `E2E_TELEMETRY_FLEET`
 
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `E2E_TELEMETRY_DISABLED` | flag | Turns anonymous usage telemetry off. `e2e telemetry disable` saves the same choice in the preferences file |
-| `DO_NOT_TRACK` | flag | Same as `E2E_TELEMETRY_DISABLED` |
-| `E2E_TELEMETRY_DEBUG` | flag | Prints every telemetry event to stderr instead of sending it |
-| `E2E_TELEMETRY_FLEET` | value | Names the platform running e2e for its users; events are attributed to `fleet:<name>` instead of a per-machine id, no preferences file is written, and no notice is printed. A value that is not a plain token is sent as `other` |
-| `XDG_CONFIG_HOME` | value | Directory of the preferences file, `$XDG_CONFIG_HOME/e2e`; default `~/.config/e2e` |
-| `APPDATA` | value | The same on Windows, `%APPDATA%\e2e`; default `~/AppData/Roaming/e2e` |
+Names the platform running e2e for its users.
+
+- Events are attributed to `fleet:<name>` instead of a per-machine id.
+- No preferences file is written, and no notice is printed.
+- A value that is not a plain token is sent as `other`.
+
+#### Detected environments
 
 Telemetry also checks for the markers CI vendors and coding agents set. Only
 the derived name is sent.
 
 | Markers | Reported as |
 | --- | --- |
-| `GITHUB_ACTIONS`, `GITLAB_CI`, `CIRCLECI`, `BUILDKITE`, `JENKINS_URL`, `TRAVIS`, `BITBUCKET_BUILD_NUMBER`, `TF_BUILD`, `TEAMCITY_VERSION`, `CODEBUILD_BUILD_ID`, `VERCEL`, `NETLIFY`, `DRONE`, `SEMAPHORE`, `APPVEYOR`, `WOODPECKER` | `ci_name`: `github-actions`, `gitlab`, `circleci`, `buildkite`, `jenkins`, `travis`, `bitbucket`, `azure-pipelines`, `teamcity`, `aws-codebuild`, `vercel`, `netlify`, `drone`, `semaphore`, `appveyor`, `woodpecker`; `unknown` when only `CI` is set |
+| `GITHUB_ACTIONS`, `GITLAB_CI`, `CIRCLECI`, `BUILDKITE`, `JENKINS_URL`, `TRAVIS`, `BITBUCKET_BUILD_NUMBER`, `TF_BUILD`, `TEAMCITY_VERSION`, `CODEBUILD_BUILD_ID`, `VERCEL`, `NETLIFY`, `DRONE`, `SEMAPHORE`, `APPVEYOR`, `WOODPECKER`, `EAS_BUILD`, `BITRISE_IO`, `CM_BUILD_ID`, `CI_XCODE_CLOUD` | `ci_name`: `github-actions`, `gitlab`, `circleci`, `buildkite`, `jenkins`, `travis`, `bitbucket`, `azure-pipelines`, `teamcity`, `aws-codebuild`, `vercel`, `netlify`, `drone`, `semaphore`, `appveyor`, `woodpecker`, `eas`, `bitrise`, `codemagic`, `xcode-cloud`; `unknown` when only `CI` is set |
 | `CLAUDECODE`, `CLAUDE_CODE`, `CURSOR_AGENT`, `CURSOR_TRACE_ID`, `CODEX_SANDBOX`, `CODEX_THREAD_ID`, `CODEX_CI`, `GEMINI_CLI`, `COPILOT_GITHUB_TOKEN`, `COPILOT_MODEL`, `COPILOT_ALLOW_ALL`, `OPENCODE_CLIENT`, `AUGMENT_AGENT`, `ANTIGRAVITY_AGENT`, `REPL_ID`, `AI_AGENT` | `coding_agent`: `claude-code`, `cursor`, `codex`, `gemini-cli`, `copilot`, `opencode`, `augment`, `antigravity`, `replit`, `other` |
 | `WSL_DISTRO_NAME` | `wsl: true` |
 | `npm_config_user_agent` | `package_manager`: the invoking package manager. `e2e init` reads the same variable to pick the install command |
 
 What is sent and how to read it is on the [Telemetry](/telemetry) page.
 
-## Reporters
+### Reporters
 
 `@e2e-dev/github` reads the variables GitHub Actions sets, when a run finishes:
 
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `GITHUB_ACTIONS` | value | Without it the reporter posts nothing and says so |
-| `GITHUB_REPOSITORY`, `GITHUB_RUN_ID`, `GITHUB_EVENT_NAME`, `GITHUB_WORKFLOW`, `GITHUB_JOB` | value | Always set by the runner; a job missing one fails the reporter with the variable named. The repository and run id build the artifacts link, the event name is in the "not a pull request" row, and the workflow and job join the marker that lets a rerun find its own comment |
-| `GITHUB_TOKEN`, `GH_TOKEN` | value | The token the comment is posted with; the first one set wins. Unset, the job summary is still written |
-| `GITHUB_EVENT_PATH`, `GITHUB_REF` | value | The pull request number and head commit from the event payload, or the number from `refs/pull/N/merge` |
-| `GITHUB_SHA` | value | The commit source links point at when the event is not about a pull request |
-| `GITHUB_WORKSPACE` | value | The checkout root; source links add the project's path inside it, so a suite in a subdirectory links to the right file |
-| `GITHUB_SERVER_URL`, `GITHUB_API_URL` | value | The GitHub host, for Enterprise Server; `https://github.com` and `https://api.github.com` by default |
-| `GITHUB_STEP_SUMMARY` | value | The job summary file the comment text is appended to |
-
-## Engines
-
-| Variable | Reading | Effect |
-| --- | --- | --- |
-| `PLAYWRIGHT_BROWSERS_PATH` | value | Playwright's browser cache, honored by the engine's `prepare` hook |
-| `PLAYWRIGHT_SKIP_BROWSER_GC` | value | Defaults to `1` for the engine's browser install, so it keeps other browsers in the cache. Set `0` to let the install remove them |
-| `KERNEL_API_KEY` | value | The API key `kernel()` from `@e2e-dev/kernel` creates browsers with; unset, the browser lease fails. See [Kernel](/integrations/kernel) |
-| `EXPO_TOKEN` | value | The Expo access token `easSimulators()` from `@e2e-dev/eas` starts and stops simulator sessions with; unset, it uses the eas-cli login in `.expo/state.json` under `HOME` (`USERPROFILE` on Windows), and with neither the device lease fails. See [EAS Simulators](/integrations/eas) |
-| `HOME`, `USERPROFILE` | value | The home `easSimulators()` finds the eas-cli login under when `EXPO_TOKEN` is unset: `USERPROFILE` on Windows, `HOME` elsewhere, the OS user's home when unset |
-| `E2E_AGENT_DEVICE_POOL_<TARGET>_<digest>` | value | Written by `@e2e-dev/mobile` in `prepare` and read by each worker to find its booted device. Not meant to be set by hand |
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| `GITHUB_ACTIONS` | `GITHUB_ACTIONS=true` | value | Without it the reporter posts nothing and says so |
+| `GITHUB_REPOSITORY`, `GITHUB_RUN_ID`, `GITHUB_EVENT_NAME`, `GITHUB_WORKFLOW`, `GITHUB_JOB` | `GITHUB_REPOSITORY=acme/shop` | value | Always set by the runner. A missing one fails the reporter, naming the variable. The repository and run id build the artifacts link. The event name shows in the "not a pull request" row. The workflow and job join the marker a rerun finds its own comment by |
+| `GITHUB_TOKEN`, `GH_TOKEN` | `GITHUB_TOKEN=<token>` | value | Posts the comment; the first one set wins. Unset, the job summary is still written. A fork's `pull_request` token is read-only and cannot post; see [When nothing is posted](/github#when-nothing-is-posted) |
+| `GITHUB_EVENT_PATH`, `GITHUB_REF` | `GITHUB_REF=refs/pull/42/merge` | value | The pull request number and head commit from the event payload, or the number from `refs/pull/N/merge` |
+| `GITHUB_SHA` | `GITHUB_SHA=<commit>` | value | The commit source links point at when the event is not about a pull request |
+| `GITHUB_WORKSPACE` | `GITHUB_WORKSPACE=/home/runner/work/shop/shop` | value | The checkout root. Source links add the project's path inside it, so a suite in a subdirectory links to the right file |
+| `GITHUB_SERVER_URL`, `GITHUB_API_URL` | `GITHUB_SERVER_URL=https://github.example.com` | value | The GitHub host, for Enterprise Server. Default `https://github.com` and `https://api.github.com` |
+| `GITHUB_STEP_SUMMARY` | `GITHUB_STEP_SUMMARY=<file>` | value | The job summary file the comment text is appended to |
+
+### Engines
+
+| Variable | Example | Reading | Effect |
+| --- | --- | --- | --- |
+| `PLAYWRIGHT_BROWSERS_PATH` | `PLAYWRIGHT_BROWSERS_PATH=~/.cache/ms-playwright` | value | Playwright's browser cache, honored by the engine's `prepare` hook |
+| `PLAYWRIGHT_SKIP_BROWSER_GC` | `PLAYWRIGHT_SKIP_BROWSER_GC=0` | value | Defaults to `1` for the engine's browser install, which keeps other browsers in the cache. `0` lets the install remove them |
+| `KERNEL_API_KEY` | `KERNEL_API_KEY=<key>` | value | The API key `kernel()` from `@e2e-dev/kernel` creates browsers with. Unset, the browser lease fails. See [Kernel](/integrations/kernel) |
+| `LIGHTPANDA_PATH` | `LIGHTPANDA_PATH=/opt/lightpanda` | value | The binary `lightpanda()` from `@e2e-dev/lightpanda` starts a CDP server with. Unset, `lightpanda` on `PATH`, `~/.lightpanda`, or `~/.local/bin`. See [Lightpanda](/integrations/lightpanda) |
+| [`EXPO_TOKEN`](#expo_token) | `EXPO_TOKEN=<token>` | value | The Expo access token `easSimulators()` from `@e2e-dev/eas` uses |
+| `HOME`, `USERPROFILE` | `HOME=/home/ada` | value | Where `easSimulators()` finds the eas-cli login without `EXPO_TOKEN`: `USERPROFILE` on Windows, `HOME` elsewhere, the OS user's home when unset |
+| `LIMRUN_API_KEY` | `LIMRUN_API_KEY=<key>` | value | The API key `limrun()` from `@e2e-dev/limrun` leases devices with. Unset, the lease fails with `INVALID_CONFIG`. See [Limrun](/integrations/limrun) |
+| `LIMRUN_BASE_URL` | `LIMRUN_BASE_URL=<url>` | value | The Limrun API host. Default `https://api.limrun.com` |
+| `LT_USERNAME`, `LT_ACCESS_KEY` | `LT_USERNAME=acme` | value | The TestMu AI username and access key `testmu()` from `@e2e-dev/testmu` leases devices with. With either unset, the device lease fails. See [TestMu AI](/integrations/testmu) |
+| `TESTMU_API_ENDPOINT` | `TESTMU_API_ENDPOINT=<url>` | value | The TestMu AI sessions API base URL `testmu()` looks up session recordings at. Default `https://mobile-api.lambdatest.com/mobile-automation/api/v1`. A value that is not an http(s) URL, or carries a username or password, fails the lookup with `INVALID_CONFIG`. agent-device's TestMu AI plugin reads it too |
+| `E2E_WEB_BROWSERS_<TARGET>_<digest>` | - | value | `@e2e-dev/web` writes it in `prepare`; each worker reads it to find its leased browser. Don't set it by hand |
+| `E2E_AGENT_DEVICE_POOL_<TARGET>_<digest>` | - | value | `@e2e-dev/mobile` writes it in `prepare`; each worker reads it to find its booted device. Don't set it by hand |
+
+#### `EXPO_TOKEN`
+
+The Expo access token `easSimulators()` from `@e2e-dev/eas` starts and stops
+simulator sessions with. See [EAS Simulators](/integrations/eas).
+
+- Unset, it uses the eas-cli login in `.expo/state.json` under the home
+  directory ([`HOME`, `USERPROFILE`](#engines)).
+- With neither, the device lease fails.
 
 ## What app processes inherit
 
-An app `command` receives `PATH`, `HOME`, `TMPDIR`, `TMP`,
-`TEMP`, `SystemRoot`, and `COMSPEC`, plus whatever `command.env` sets.
-`APP_URL` is read by the scaffolded config module, not the runner; pass it
-through `command.env` if the dev server needs it too.
+An app `command` receives `PATH`, `HOME`, `TMPDIR`, `TMP`, `TEMP`,
+`SystemRoot`, and `COMSPEC`, plus the variables `command.inheritEnv` matches
+and whatever `command.env` sets.
+
+The scaffolded config module reads `APP_URL`, not the runner. If the dev
+server needs it too, pass it through `command.env`.

package/docs/reference/errors.mdx

@@ -6,8 +6,30 @@ icon: "triangle-exclamation"
 "twitter:image": "https://e2e.tester.army/docs/images/og/reference/errors.png?v=b184354c61"
 ---
 
-Every error carries a class. The class determines the exit code; the code
-identifies the cause. A model never selects either.
+Every runner error carries a class and a code. The class sets the exit code,
+and the code names the cause. A model never selects either. `e2e login`,
+`e2e logout`, and `e2e models` are the exception, under
+[Exit codes](#exit-codes).
+
+```ts title="tests/upgrade.e2e.ts"
+import { isAgentError, test } from 'e2e';
+
+test('upgrade to Pro', async ({ agent }) => {
+  try {
+    await agent.act('upgrade the workspace to Pro');
+  } catch (error) {
+    if (isAgentError(error) && error.code === 'STEP_TIMEOUT') {
+      // the step ran out of time; the message carries the last judgment
+      console.error(error.category, error.details);
+    }
+    throw error;
+  }
+});
+```
+
+## Reference
+
+### Exit codes
 
 | Class | Exit code |
 | --- | ---: |
@@ -17,13 +39,36 @@ identifies the cause. A model never selects either.
 | internal | 4 |
 | interrupted | 130 |
 
-## AgentError
+- In a mixed run the highest-precedence code present wins,
+  `130 > 4 > 3 > 2 > 1 > 0`. The report keeps every individual result.
+- A setup assertion failure is a test failure and skips its dependents.
+  Missing setup infrastructure keeps its configuration or infrastructure
+  class.
+- The `internal` class and exit code 4 are reserved for runner invariant
+  violations. No first-party code raises a code in that class today.
+- `e2e login`, `e2e logout`, and `e2e models` print a failure's message with
+  no code and exit 1, or 130 when a sign-in or picker is cancelled.
+
+| Codes | Description |
+| --- | --- |
+| [Agent codes](#agent-codes) | The `code` of an `AgentError` |
+| [Runner codes](#runner-codes) | Plain strings in output and report |
+| [Test code errors](#test-code-errors) | Thrown from test code |
+
+### AgentError
+
+`AgentError` is the runner-classified agent failure, exported from `e2e`.
 
 ```ts
 /** Runner-classified agent failure. */
 export class AgentError extends Error {
   /** Closed union, assigned by the runner, never taken from model text. */
   readonly code: AgentErrorCode;
+  /** The error class in the table above; it sets the exit code. */
+  readonly category: 'test' | 'configuration' | 'infrastructure' | 'internal' | 'interrupted';
+  readonly retryable: boolean;
+  /** Structured facts: locator, expected, observed, match count, wait time. */
+  readonly details: ErrorDetails | undefined;
   /** Sanitized, bounded prose; the same value as `message`. */
   readonly explanation: string;
   /** Artifact path, present only when evidence was captured and permitted. */
@@ -33,52 +78,47 @@ export class AgentError extends Error {
 }
 ```
 
+<Note>
 Test files load in their own module realm, so `instanceof AgentError` can be
-false. Use `isAgentError` from `e2e` and branch on `code`:
+false. Use `isAgentError` from `e2e` and branch on `code`, as in the example
+at the top of this page.
+</Note>
 
-```ts
-import { isAgentError } from 'e2e';
+#### Agent codes
 
-try {
-  await agent.act('upgrade the workspace to Pro');
-} catch (error) {
-  if (isAgentError(error) && error.code === 'STEP_TIMEOUT') {
-    // the step ran out of time; the message carries the last judgment
-  }
-  throw error;
-}
-```
+The members of the closed `AgentErrorCode` union. `blocked:` names the
+category a blocked verdict reports as the obstacle.
 
 | Code | When it happens | Class | Exit |
 | --- | --- | --- | ---: |
-| `AUTH_CREDENTIAL_UNAVAILABLE` | A `Secret` or `credentials.user()` names a credential absent from `config.credentials`, the call happened outside an active runner, or the password's provider returned something other than a string of at least 6 characters (code points) | configuration | 2 |
-| `AUTH_CREDENTIAL_INVALID` | A configured credential was rejected by the application (blocked: credentials) | configuration | 2 |
-| `SECRET_UNAVAILABLE` | A `Secret` or `secrets.get()` during a run names a secret absent from `config.secrets`, the secret's provider returned something other than a string of at least 6 characters (code points), or an engine resolved a secret it did not declare in its `secrets` | configuration | 2 |
+| `AUTH_CREDENTIAL_UNAVAILABLE` | A `Secret` or `credentials.user()` names a credential missing from `config.credentials`, the call ran outside an active runner, or the password's provider returned no string of at least 6 characters (code points) | configuration | 2 |
+| `AUTH_CREDENTIAL_INVALID` | The application rejected a configured credential (blocked: credentials) | configuration | 2 |
+| `SECRET_UNAVAILABLE` | A `Secret` or `secrets.get()` during a run names a secret missing from `config.secrets`, the secret's provider returned no string of at least 6 characters (code points), or an engine resolved a secret it did not declare in its `secrets` | configuration | 2 |
 | `ENVIRONMENT_UNAVAILABLE` | The environment under test is down or unreachable beyond the app itself (blocked: environment) | infrastructure | 3 |
 | `SEED_DATA_MISSING` | Data the step depends on does not exist in the environment (blocked: seed_data) | configuration | 2 |
-| `TEST_SETUP_FAILED` | Test prerequisites outside the product failed to arrange (blocked: test_setup). Also a `test.extend()` fixture that broke its contract: returned without calling `use()`, called it twice, or redefined a fixture the target's engine contributes | configuration | 2 |
-| `AUTOMATION_UNSUPPORTED` | The step needs an interaction the available toolset cannot perform, e.g. drag-and-drop (blocked: automation) | test | 1 |
-| `MODEL_UNAVAILABLE` | No model configured: the built-in agent's `agents.<name>.model` holds no AI SDK model instance; the message names the agent that was selected. Checked once per run when the `agent` fixture is first acquired; recorded as one run-level error that stops the run. `e2e explore` checks before anything starts, a custom executor's entry included, since explore runs the built-in agent. A custom executor without a model is not checked: its `act` and `assert` steps run, and `waitFor` or `extract` fails with this code on its first call. Evicts no [recording](/cache). | configuration | 2 |
-| `POLICY_DENIED` | A denied destination (a navigation scheme other than `http:` or `https:`, `about:blank` aside, a malformed URL), a forbidden fill (an undeclared secret, a non-editable sink, a password into a field that is not a password field), or pixel evidence after a secret fill (`vision: 'only'` with pixels withheld, or an `act` or `vision: true` step whose semantic capture timed out before the step did). Also reading a value, text, or attribute from a secure field, the getters and the `toHaveValue`, `toHaveText`, `toContainText`, and `toHaveAttribute` matchers alike. An unknown tool or an undeclared field is refused to the model as the call's result and is not this code | configuration | 2 |
-| `APP_UNREACHABLE` | The app URL could not be reached, or an app command exited early or exceeded `startupTimeout`. The startup error includes recent log output with `command.env` values redacted, and suggests adding a `log` if none is configured. Under the agent, also any engine fault with no more specific code: on a device, an iOS automation runner that is busy or wedged reports here with a message that names the runner, the session and device, and the recovery; see [Mobile](/mobile#troubleshooting) | infrastructure | 3 |
-| `APP_ALREADY_RUNNING` | `readyUrl` already answered before the target's `app.command` was spawned and `reuseExisting` was not in effect; stop that process or set `reuseExisting: true` locally | infrastructure | 3 |
-| `MODEL_PROVIDER_FAILED` | Provider transport failure (network error, 5xx, rate limit, or a request with no response within 120 s) after the adapter's transport retries (up to 5 per model call, honoring `retry-after`, bounded by the remaining step timeout). A request that stalls is aborted at 120 s and sent again, and an `agent.act` step's `--debug` transcript notes it. The bound covers the whole request, so a model that needs longer than 120 s for one answer fails here too. An error body the provider package cannot read shows as the HTTP status and the body, secrets redacted, cut at 1 KB. Evicts no [recording](/cache) | infrastructure | 3 |
-| `CANCELLED` | The runner cancelled the operation, without a process signal: an `e2e mcp` `open_session` whose request the client cancelled or whose connection closed, or that the server's shutdown cut short, after stopping what it had started | infrastructure | 3 |
-| `MODEL_OUTPUT_INVALID` | Structured output still failed validation after the repair budget was spent. Under a [decision model](/decision-models), an answer without a valid choice distribution, a choice the request never offered, or a question the provider refused; decisions are not repaired | test | 1 |
-| `APP_NOT_OPEN` | An operation needed an open app; `app.open()` had not run, or the engine reported invalid state | test | 1 |
+| `TEST_SETUP_FAILED` | Test prerequisites outside the product failed to arrange (blocked: test_setup). Also a `test.extend()` fixture that broke its contract: it returned without calling `use()`, called it twice, or redefined a fixture the target's engine contributes | configuration | 2 |
+| `AUTOMATION_UNSUPPORTED` | The step needs an interaction the available toolset cannot perform, such as drag-and-drop (blocked: automation) | test | 1 |
+| `MODEL_UNAVAILABLE` | No model configured: the built-in agent's `agents.<name>.model` holds no AI SDK model instance (the message names the selected agent), or GitHub Copilot does not serve the configured model to this account (`model_not_supported`; the message names the model). Recorded once as a run-level error, and the run stops. In a test run, a missing model is found when the `agent` fixture is first acquired, and an unserved model at its first call. `e2e explore` runs the built-in agent, so it checks for a missing model before anything starts, a custom executor's entry included. A custom executor without a model is not checked: its `act` and `assert` steps run, and `waitFor` or `extract` fails with this code on its first call. Evicts no [recording](/cache). | configuration | 2 |
+| `POLICY_DENIED` | The security policy refused an operation: a denied destination, a forbidden fill, pixels after a secret fill, or a read from a secure field. [Policy denials](/reference/security#policy-denials) lists every case. An unknown tool or an undeclared field goes back to the model as the call's result, never as this code | configuration | 2 |
+| `APP_UNREACHABLE` | The app URL could not be reached, or an app command exited early or exceeded `startupTimeout`. With a `log`, the startup error quotes the log's output since the command started, `command.env` values redacted, or says there was none. Without one, it says to set a `log`. A command that fails to spawn reports the spawn error. Under the agent, also any engine fault with no more specific code. On a device, a busy or wedged iOS automation runner reports here; the message names the runner, the session and device, and the recovery. See [Mobile](/mobile#troubleshooting) | infrastructure | 3 |
+| `APP_ALREADY_RUNNING` | `readyUrl` answered before the target's `app.command` was spawned, and `reuseExisting` was not in effect. Stop that process, or set `reuseExisting: true` locally | infrastructure | 3 |
+| `MODEL_PROVIDER_FAILED` | Provider transport failure (network error, 5xx, rate limit, or no response within 120 s) after the adapter's transport retries: up to 5 per model call, honoring `retry-after`, within the remaining step timeout. A stalled request is aborted at 120 s and sent again; an `agent.act` step's `--debug` transcript notes it. The 120 s covers the whole request, so a model that needs longer for one answer fails here too. An error body the provider package cannot read shows as the HTTP status and the body, secrets redacted, cut at 1 KB. Evicts no [recording](/cache) | infrastructure | 3 |
+| `CANCELLED` | The runner cancelled the operation without a process signal: an `e2e mcp` `open_session` cut short by a client cancel, a closed connection, or the server's shutdown, after it stopped what it had started | infrastructure | 3 |
+| `MODEL_OUTPUT_INVALID` | Structured output still failed validation after the repair budget was spent. Under a [decision model](/decision-models): an answer without a valid choice distribution, a choice the request never offered, or a question the provider refused. Decisions are not repaired | test | 1 |
+| `APP_NOT_OPEN` | An operation needed an open app: `app.open()` had not run, or the engine reported invalid state | test | 1 |
 | `LOCATOR_NOT_FOUND` | No matching node within the deadline, or an agent action refers to a node that is no longer available | test | 1 |
-| `LOCATOR_AMBIGUOUS` | More than one match for a single-node operation. Fails immediately, no polling. | test | 1 |
+| `LOCATOR_AMBIGUOUS` | More than one match for a single-node operation. Fails at once, with no polling. | test | 1 |
 | `ACTION_FAILED` | Actionability failure, operation timeout, or an action that may already have committed | test | 1 |
 | `STEP_BUDGET_EXHAUSTED` | The model-call or action-step budget ran out before a conclusion | test | 1 |
-| `STEP_TIMEOUT` | The invocation deadline expired, an operation timing out within a poll tick of it included. Carries the last judgment where one exists, with the `ASSERTION_INCONCLUSIVE` vision hint when that judgment was inconclusive | test | 1 |
-| `CONTEXT_OVERFLOW` | A request exceeded the model's context window. `act` shortens its history and retries once if that reduces the request; judgments fail directly. Lower `agents.<name>.maxObservationBytes` or split the instruction into shorter steps. Reported as blocked: automation | test | 1 |
+| `STEP_TIMEOUT` | The invocation deadline expired, including an operation that timed out within a poll tick of it. Carries the last judgment where one exists, with the [vision hint](/reference/agent#agent-assert) when that judgment was inconclusive | test | 1 |
+| `CONTEXT_OVERFLOW` | A request exceeded the model's context window. `act` shortens its history and retries once if that makes the request smaller; judgments fail at once. Lower `agents.<name>.maxObservationBytes`, or split the instruction into shorter steps. Reported as blocked: automation | test | 1 |
 | `ASSERTION_FAILED` | `agent.assert` judged false, or a locator, `browser`, or value matcher failed | test | 1 |
-| `ASSERTION_INCONCLUSIVE` | `agent.assert` could not judge: the screen did not show enough to decide either way (the relevant part is not on screen, still loading, or unreadable). `agent.extract` raises it when the model answers that the screen does not show the requested data, instead of returning a placeholder. A failure, never a pass; the explanation says what was missing. Judged from the tree alone with `vision` off, the message adds `pass vision: true when the answer is in pixels`, as a condition (`if the engine captures pixels, ...`) until a step of the attempt has received pixels; no hint when the engine declares no screenshot capture, a secret was filled, or an earlier step's pixel request was degraded | test | 1 |
-| `AUTHENTICATION_FAILED` | The executor concluded that signing in failed inside the app: a product verdict. `AUTH_CREDENTIAL_INVALID` is the blocked counterpart for a configured credential the app rejects | test | 1 |
-| `STEP_NO_CONCLUSION` | The executor used every turn without calling `complete_step`; the step fails with the agent's last summary | test | 1 |
-| `REPLAY_STALE` | Under `cache.strict` or `--strict-cache`, the step's recording exists but no longer replays (`step.cache.reason` is one of `invalid-entry`, `wrong-context`, `target-not-found`, `target-ambiguous`, `viewport-changed`, `action-failed`, `action-uncertain`, `end-mismatch`, or `no-entry` while the cache directory, not a custom `cache.store`, holds a recording of the step under another key, which the message names), so the step fails with no model call instead of handing off to the agent. Not retried: every attempt replays the same recording. Re-record with a read-write run without the flag, and with `cache.strict` off when the config set it, then commit the changed entry; the message names the knob that turned strict on and the configured cache directory (or `cache.store`) | configuration | 2 |
+| `ASSERTION_INCONCLUSIVE` | `agent.assert` could not judge: the screen did not show enough to decide either way (the relevant part is not on screen, still loading, or unreadable). `agent.extract` raises it when the model answers that the screen does not show the requested data, instead of returning a placeholder. A failure, never a pass; the explanation says what was missing. Judged from the tree alone with `vision` off, the message can add a [vision hint](/reference/agent#agent-assert) | test | 1 |
+| `AUTHENTICATION_FAILED` | The executor concluded that signing in failed inside the app: a product verdict. `AUTH_CREDENTIAL_INVALID` is the blocked counterpart, for a configured credential the app rejects | test | 1 |
+| `STEP_NO_CONCLUSION` | The executor used every turn without calling `complete_step`. The step fails with the agent's last summary | test | 1 |
+| `REPLAY_STALE` | Under `cache.strict` or `--strict-cache`, the step's recording exists but no longer replays, so the step fails with no model call instead of handing off to the agent. `step.cache.reason` is one of `invalid-entry`, `wrong-context`, `target-not-found`, `target-ambiguous`, `viewport-changed`, `action-failed`, `action-uncertain`, `end-mismatch`, or `no-entry` while the cache directory (not a custom `cache.store`) holds a recording of the step under another key, which the message names. Not retried: every attempt replays the same recording. To fix it, re-record with a read-write run without the flag, with `cache.strict` off if the config set it, then commit the changed entry. The message names the knob that turned strict on and the configured cache directory (or `cache.store`) | configuration | 2 |
 
-## Runner codes
+### Runner codes
 
 These appear in terminal output and in the report as plain string codes.
 
@@ -86,64 +126,58 @@ These appear in terminal output and in the report as plain string codes.
 | --- | --- | --- | ---: |
 | `CONFIG_NOT_FOUND` | `--config <path>` names a file that does not exist, or no `e2e.config.ts` or `e2e.config.mts` exists from the working directory up to the repository root | configuration | 2 |
 | `CONFIG_AMBIGUOUS` | Both `e2e.config.ts` and `e2e.config.mts` in one directory | configuration | 2 |
-| `CONFIG_LOAD_FAILED` | The config module failed to import: a package that is declared but not installed, an undeclared package, a wrong subpath, a removed export, or any other throw while it evaluates. The message names the fix. A config error raised while it evaluates keeps its own code: an option a factory the config calls (`web()`, `mobile()`, `defineEngine()`, `decisionExecutor()`) refuses, or a `secrets.get()` handle turned into a string, is `INVALID_CONFIG` | configuration | 2 |
-| `INVALID_CONFIG` | A factory the config module calls (`web()`, `mobile()`, `defineEngine()`, `decisionExecutor()`) refused its options, with the factory's own message; or an unknown key, out-of-range value, invalid enum, missing `targets`, a target key outside `{ name, platform, engine, app, trace, video }`, a target `app` key that is unknown or malformed, `app` on a target without an engine, a `web()` target without `app.url` or with a device field, a `mobile()` target without `app.bundleId` or `app.appPath`, with `app.url` (not supported yet), a link in `bundleId`, or an unknown permission name or state (the message names the nearest name), an unknown key in a `web()` or `mobile()` option, including `web({ connect })` and `web({ basicAuth })` (the message names the nearest one), a `trace` or `video` that is not a recording mode, the removed artifact kinds list (`artifacts: [...]` or `artifacts.kinds`) and `artifacts.trace` block (the message names the `trace` mode they meant), the removed `artifacts.video` block or `video` artifact kind (the message names the `video` option), the removed `specVersion` key, a `secrets.get()` in an engine option that names no `secrets` entry (a credential's password is not one), a `secrets.get()` reference turned into a string while the config evaluates (a template literal, `String()`) or given where a string is needed (an `app.command` `args` entry or `env` value; only an engine option that declares secrets takes one), an `app.command` `args` entry or `env` value that is not a string (an `undefined` `env` value is dropped instead), an unknown key in `app.command` (the message names the nearest one), an `engine` that is not a `defineEngine` handle, `json` reporter combined with `list`, duplicate or malformed target name, a secret or credential password shorter than 6 characters (code points), a credential that is not `{ username, password }`, whose `username` is not a string (only `password` may be a function), or that has no username in the config or `E2E_USER_<NAME>_USERNAME`, two credentials or two secrets whose names map to one `E2E_USER_*` or `E2E_SECRET_*` override variable (`api-key` and `api_key`), oversized `agents.<name>.context`, an `expect.poll` `timeout` or `interval` that is not a finite number of milliseconds, a `tests` value that is not a glob or a list of globs, a `tests` list of only `!` exclusions, the removed top-level `limits` key, an agents entry that is a bare `StepExecutor` (the message names `{ executor }`) or a model instance (the message names `{ model }`), an unknown agent key (the message names the nearest one), the removed agent keys `timeout` and `maxTurns` (the message names `judgmentTimeout` and `maxModelCalls`), `system` or `tools` beside `executor`, a `tools` entry `defineTool` did not build or that takes a built-in tool name (a grammar verb, `complete_step`, `locate`, `start_recording`, `stop_recording`, or `report_finding`), `createToolLoopExecutor({ maxTurns })`, `reuseExisting` on an app whose `app.url` asks for a free port, or a port-0 `app.url` with no `app.command` to serve it | configuration | 2 |
+| `CONFIG_LOAD_FAILED` | The config module failed to import: a package that is declared but not installed, an undeclared package, a wrong subpath, a removed export, or any other throw while it evaluates. The message names the fix. A config error raised while it evaluates keeps its own code: options a factory the config calls refuses, or a `secrets.get()` handle turned into a string, is `INVALID_CONFIG` | configuration | 2 |
+| `INVALID_CONFIG` | A factory the config module calls (`web()`, `mobile()`, `defineEngine()`, `decisionExecutor()`) refused its options, with the factory's own message. Or: an unknown key, out-of-range value, invalid enum, or missing `targets`; a target key outside `{ name, platform, engine, app, trace, video }`; a target `app` key that is unknown or malformed; `app` on a target without an engine; a `web()` target without `app.url` or with a device field; a `mobile()` target without `app.bundleId` or `app.appPath`, with `app.url` (not supported yet), with a link in `bundleId`, or with an unknown permission name or state (the message names the nearest name); an unknown key in a `web()` or `mobile()` option, `web({ connect })` and `web({ basicAuth })` included (the message names the nearest one); a `trace` or `video` that is not a recording mode; the removed artifact kinds list (`artifacts: [...]` or `artifacts.kinds`) or `artifacts.trace` block (the message names the `trace` mode they meant); the removed `artifacts.video` block or `video` artifact kind (the message names the `video` option); the removed `specVersion` key; a `secrets.get()` in an engine option that names no `secrets` entry (a credential's password is not one); a `secrets.get()` reference turned into a string while the config evaluates (a template literal, `String()`), or given where a string is needed (an `app.command` `args` entry or `env` value; only an engine option that declares secrets takes one); an `app.command` `args` entry or `env` value that is not a string (an `undefined` `env` value is dropped instead); an unknown key in `app.command` (the message names the nearest one); an `engine` that is not a `defineEngine` handle; the `json` reporter combined with `list`; a duplicate or malformed target name; a secret or credential password shorter than 6 characters (code points); a credential that is not `{ username, password }`, whose `username` is not a string (only `password` may be a function), or that has no username in the config or `E2E_USER_<NAME>_USERNAME`; two credentials or two secrets whose names map to one `E2E_USER_*` or `E2E_SECRET_*` override variable (`api-key` and `api_key`); an oversized `agents.<name>.context`; an `expect.poll` `timeout` or `interval` that is not a finite number of milliseconds; a `tests` value that is not a glob or a list of globs, or a `tests` list of only `!` exclusions; the removed top-level `limits` key; an agents entry that is a bare `StepExecutor` (the message names `{ executor }`) or a model instance (the message names `{ model }`); an unknown agent key (the message names the nearest one); the removed agent keys `timeout` and `maxTurns` (the message names `judgmentTimeout` and `maxModelCalls`); `system` or `tools` beside `executor`; a `tools` entry that `defineTool` did not build, or that takes a built-in tool name (a grammar verb, `complete_step`, `locate`, `start_recording`, `stop_recording`, or `report_finding`); `createToolLoopExecutor({ maxTurns })`; `reuseExisting` on an app whose `app.url` asks for a free port; or a port-0 `app.url` with no `app.command` to serve it | configuration | 2 |
 | `CONFIG_NOT_DETERMINISTIC` | A worker re-resolved the config module and got a different digest than the runner | configuration | 2 |
 | `APP_URL_REQUIRED` | A navigation or relative URL on a target that declares no `app.url`, or an `app.command` with no `app.url` or `app.readyUrl` to poll | configuration | 2 |
-| `INVALID_APP_URL` | Base URL with userinfo, a query, a fragment, a non-HTTP scheme, plain HTTP on a non-loopback host, port 0 on a host other than a literal loopback address (`127.0.0.1`, `[::1]`), a loopback address the run could not bind a free port on, or unparseable | configuration | 2 |
-| `INVALID_GLOB` | Empty pattern, a first segment that starts with `!` beyond the one leading `!` of a `tests` exclusion (`!!tests/**`, `./!tests/**`), a bare `!`, braces, character classes, extglobs, a backslash separator, an absolute path, a `..` segment, a trailing `/` or `.`, a `tests` entry with no wildcard that names a directory (the message names the glob to write), a pattern that names only the project root, or a `**` that is not a whole segment | configuration | 2 |
+| `INVALID_APP_URL` | A base URL that is unparseable or has userinfo, a query, a fragment, a non-HTTP scheme, or plain HTTP on a non-loopback host; port 0 on a host other than a literal loopback address (`127.0.0.1`, `[::1]`); or a loopback address the run could not bind a free port on | configuration | 2 |
+| `INVALID_GLOB` | An empty pattern; a first segment that starts with `!` beyond the one leading `!` of a `tests` exclusion (`!!tests/**`, `./!tests/**`); a bare `!`; braces, character classes, or extglobs; a backslash separator; an absolute path or a `..` segment; a trailing `/` or `.`; a `tests` entry with no wildcard that names a directory (the message names the glob to write); a pattern that names only the project root; or a `**` that is not a whole segment | configuration | 2 |
 | `UNKNOWN_TARGET` | `--target`, `open_session.target`, or `openHost` (`e2e/runner`) named a target that is not configured | configuration | 2 |
 | `TARGET_REQUIRED` | `e2e mcp` or `openHost` (`e2e/runner`): the config declares several targets and none was named | configuration | 2 |
 | `NO_SESSION` | `e2e mcp`: `call` or `tools` before `open_session` or after the session ended (the message names why), or `call`, `tools`, or `close_session` naming a `session` id that is not open or is closing | configuration | 2 |
-| `SESSION_REQUIRED` | `e2e mcp`: `call`, `tools`, or `close_session` without a `session` while several sessions are open; the message lists them | configuration | 2 |
-| `ENGINE_IN_USE` | `e2e mcp`: `open_session` on a target whose engine instance another open session drives, because the config gets it from a package every session shares; create the engine in the config or a file it imports by path | configuration | 2 |
-| `CONFIG_IN_USE` | `e2e mcp`: `open_session` on a config other than the one the open sessions use; credentials and secrets resolve process-wide, so open on that config or close those sessions first | configuration | 2 |
-| `SESSION_OPEN` | `e2e mcp`: `open_session` while every session slot (`--max-sessions`, default 4) holds a session that is open, opening, or closing, or while the server shuts down; close one first | configuration | 2 |
-| `UNKNOWN_TOPIC` | `e2e mcp`: an `e2e://guide/{topic}` resource named a topic the skill does not have; the message lists them | configuration | 2 |
-| `UNKNOWN_TOOL` | `e2e mcp`: `call` or `tools` named a tool the session's catalog does not have; the message lists the catalog | configuration | 2 |
-| `UNSUPPORTED_ARTIFACT` | A [`video` mode](/reference/config#recording) set on a target, or on a test that will run there, records on some attempt, and the target's engine cannot record video. The message names the target and, for a test's mode, the test. A mode from the config root or `--video` never raises it: it applies to the targets that can record, with a notice naming the rest | configuration | 2 |
+| `SESSION_REQUIRED` | `e2e mcp`: `call`, `tools`, or `close_session` without a `session` while several sessions are open. The message lists them | configuration | 2 |
+| `ENGINE_IN_USE` | `e2e mcp`: `open_session` on a target whose engine instance another open session drives, because the config gets it from a package every session shares. Create the engine in the config, or in a file it imports by path | configuration | 2 |
+| `CONFIG_IN_USE` | `e2e mcp`: `open_session` on a config other than the one the open sessions use. Credentials and secrets resolve process-wide, so open on that config or close those sessions first | configuration | 2 |
+| `SESSION_OPEN` | `e2e mcp`: `open_session` while every session slot (`--max-sessions`, default 4) holds an open, opening, or closing session, or while the server shuts down. Close one first | configuration | 2 |
+| `UNKNOWN_TOPIC` | `e2e mcp`: an `e2e://guide/{topic}` resource named a topic the skill does not have. The message lists them | configuration | 2 |
+| `UNKNOWN_TOOL` | `e2e mcp`: `call` or `tools` named a tool the session's catalog does not have. The message lists the catalog. An action verb the engine does not declare is `UNSUPPORTED_CAPABILITY` instead | configuration | 2 |
+| `UNSUPPORTED_ARTIFACT` | A [`video` mode](/reference/config#recording) set on a target, or on a test that will run there, records on some attempt, and the target's engine cannot record video. The message names the target, and the test for a test's mode. A mode from the config root or `--video` never raises it: it applies to the targets that can record, with a notice naming the rest | configuration | 2 |
 | `UNSUPPORTED_CAPABILITY` | A fixture, action, or tier the target's engine does not declare: a contributed fixture it does not contribute, `screen` without `locate`, state restore without `state`, screenshots without `artifacts`, `app.clearState()` on a persistent browser context (the message names the mode), or `agent.assert` `vision` with an executor that does not declare `vision: true` | configuration | 2 |
-| `NO_TESTS` | Zero runnable ordinary test-target pairs. The message names the cause: no file matched the `tests` globs (with look-alike files), a positional matched nothing (naming the `!` entry that excludes a file it names), a file registered no tests, or filters excluded everything. Suppress with `--pass-with-no-tests`. | configuration | 2 |
-| `RUNNER_BUSY` | A suite overlapped another suite, an opening or open host, or a live attempt; or a host opened during a suite. Await the suite and close existing hosts before retrying. | configuration | 2 |
+| `NO_TESTS` | Zero runnable ordinary test-target pairs. The message names the cause: no file matched the `tests` globs (it lists look-alike files), a positional matched nothing (it names the `!` entry that excludes a file the positional names), a file registered no tests, or filters excluded everything. Suppress with `--pass-with-no-tests`. | configuration | 2 |
+| `RUNNER_BUSY` | A suite overlapped another suite, an opening or open host, or a live attempt; or a host opened during a suite. Await the suite and close existing hosts before you retry. | configuration | 2 |
 | `NO_LAST_RUN` | `--last-failed` found no `report.json` where the run writes it, or a file there that is not a `report-1` document. Run once without the flag first. | configuration | 2 |
 | `ONLY_IN_CI` | A focused test reached a CI run | configuration | 2 |
-| `COLLECTION_ERROR` | Registration outside collection, an invalid title, a non-function body, an async `describe` body, nested serial groups, an option forbidden inside a serial group, a setup test inside `describe`, an invalid session name, a malformed test option such as `tags` that is not a list of distinct names, a test or `describe` option key the call does not take (the message names the nearest one), a test file that fails to import (in a run with no positionals, or a file the positionals selected), a session that a selected test consumes or `e2e explore --session` names but no collected setup test produces (the message lists the declared sessions), a consumed session on a target whose engine has no `state` capability to save it, or a positional path outside the project root | configuration | 2 |
-| `POLICY_DENIED` | A denied destination (a navigation scheme other than `http:` or `https:`, `about:blank` aside, a malformed URL), a forbidden fill (an undeclared secret, a non-editable sink, a mismatched purpose, a `Secret` reaching a JSON sink), or tainted pixels (`app.screenshot()` after a secret fill, `vision: 'only'` with pixels withheld, an `act` or `vision: true` step whose semantic capture timed out before the step did); see the [security rules](/reference/security#policy-denials). An unknown tool or an undeclared field goes back to the model as the call's result, never this code | configuration | 2 |
+| `COLLECTION_ERROR` | Registration outside collection; an invalid title; a non-function body; an async `describe` body; nested serial groups; an option forbidden inside a serial group; a setup test inside `describe`; an invalid session name; a malformed test option, such as `tags` that is not a list of distinct names; a test or `describe` option key the call does not take (the message names the nearest one); a test file that fails to import, in a run with no positionals or when the positionals selected it; a session that a selected test consumes, or `e2e explore --session` names, but no collected setup test produces (the message lists the declared sessions); a consumed session on a target whose engine has no `state` capability to save it; or a positional path outside the project root | configuration | 2 |
 | `SESSION_MISMATCH` | A stored session does not match the current run, target, engine, platform, or app identity | configuration | 2 |
 | `SESSION_EXPIRED` | A stored session is past its expiry at restore time (configuration, exit 2), or a setup test saved state whose expiry is not after its creation (test, exit 1) | configuration or test | 2 or 1 |
 | `SESSION_INVALID` | A stored session failed authentication. Tampered or corrupt. | configuration | 2 |
 | `SESSION_UNAVAILABLE` | A test requires a session no selected setup test produced | configuration | 2 |
 | `SESSION_CONTRACT` | A setup test saved a session twice, saved an undeclared name, or finished without saving a declared one | test | 1 |
-| `TEST_TIMEOUT` | The attempt exceeded the resolved test `timeout`. The body is not stopped: the harness operation it is in is cancelled and its fixtures die with the attempt, so it cannot reach the next test. A test that blocks the event loop gets its worker killed once `cleanupTimeout` (at least 5 s) passes after the timeout; see [Hooks](/reference/test#hooks) | test | 1 |
-| `STEP_NOT_AWAITED` | The test body settled while a step it started was still running: a step call without `await`. The step is failed at the line of the call and cancelled; the attempt fails with this code when the body returned, or keeps the body's own error and records this one under `secondaryErrors` when the body threw. An `expect.poll` matcher left running when the body, an `afterEach` hook, or a fixture teardown returned fails that phase the same way at the line of the call, and is cancelled; one left by a `beforeAll` or `afterAll` hook fails the hook with `HOOK_FAILED` | test | 1 |
-| `UNHANDLED_REJECTION` | A promise rejected with no test running. An unhandled rejection is charged to the test running when it surfaces, since Node cannot say which test created the promise; between tests it is this run error, naming the last test that finished in the worker as the likely source. Before any test has finished in the worker it ends the worker instead: the rejection is the run error and the test on its way fails with `WORKER_CRASH`. The body that left it keeps running like one past `TEST_TIMEOUT` | test | 1 |
+| `TEST_TIMEOUT` | The attempt exceeded the resolved test `timeout`. The body is not stopped: the harness operation it is in is cancelled, and its fixtures die with the attempt, so it cannot reach the next test. A test that blocks the event loop gets its worker killed once `cleanupTimeout` (at least 5 s) passes after the timeout. See [Hooks](/reference/test#hooks) | test | 1 |
+| `STEP_NOT_AWAITED` | The test body settled while a step it started was still running: a step call without `await`. The step fails at the line of the call and is cancelled. The attempt fails with this code when the body returned. When the body threw, the attempt keeps the body's error and records this one under `secondaryErrors`. An `expect.poll` matcher still running when the body, an `afterEach` hook, or a fixture teardown returned fails that phase the same way, at the line of the call, and is cancelled. One left by a `beforeAll` or `afterAll` hook fails the hook with `HOOK_FAILED` | test | 1 |
+| `UNHANDLED_REJECTION` | A promise rejected with no test running. Node cannot say which test created a promise, so an unhandled rejection is charged to the test running when it surfaces. Between tests it is this run error, naming the last test that finished in the worker as the likely source. Before any test has finished in the worker, it ends the worker instead: the rejection is the run error, and the test on its way fails with `WORKER_CRASH`. The body that left it keeps running like one past `TEST_TIMEOUT` | test | 1 |
 | `HOOK_FAILED` | `beforeAll` or `afterAll` threw. Skips the suite's tests. | test | 1 |
-| `RUN_TIMEOUT` | The run spent its [`maxDuration`](/reference/config#maxduration) or `--max-duration`: the running tests end as `interrupted`, the rest are skipped with cause `time-limit` | test | 1 |
+| `RUN_TIMEOUT` | The run spent its [`maxDuration`](/reference/config#maxduration) or `--max-duration`. The running tests end as `interrupted`, and the rest are skipped with cause `time-limit` | test | 1 |
 | `LAUNCH_TIMEOUT` | Engine init or attempt start exceeded `config.launchTimeout` | infrastructure | 3 |
 | `CLEANUP_TIMEOUT` | Engine attempt end exceeded `config.cleanupTimeout`, or a route or dialog handler the engine ran for the test was still running when the attempt settled | infrastructure | 3 |
 | `BROWSER_INSTALL_FAILED` | Browser provisioning failed | infrastructure | 3 |
 | `WORKER_INIT_FAILED` | A worker process never became ready | infrastructure | 3 |
-| `WORKER_EXIT` | A worker exited unexpectedly while running a file: the test in flight fails with `WORKER_CRASH` and the file's other tests are skipped. Attempts that finished before the exit stay in the report, a retry's earlier attempts and a serial group's finished members included, and the crash is recorded as the attempt after them, or on the last attempt when it came after it (an `afterAll`). An unhandled rejection is charged to the test running when it surfaces, or recorded as `UNHANDLED_REJECTION` between tests; before any test has finished there it ends the worker, and the rejection itself is the run error, not this one | infrastructure | 3 |
+| `WORKER_EXIT` | A worker exited unexpectedly while running a file. The test in flight fails with `WORKER_CRASH`, and the file's other tests are skipped. Attempts that finished before the exit stay in the report, including a retry's earlier attempts and a serial group's finished members. The crash is recorded as the attempt after them, or on the last attempt when it came after it (an `afterAll`). When an unhandled rejection ends the worker (see `UNHANDLED_REJECTION`), the rejection is the run error, not this one | infrastructure | 3 |
 | `WORKER_CRASH` | A worker exited during a specific test | infrastructure | 3 |
 | `WORKER_ENTRY_MISSING` | The worker entry module could not be resolved. Packaging fault. | infrastructure | 3 |
-| `NODE_UNSUPPORTED` | The runtime cannot run e2e's TypeScript loader: a Node.js outside `^22.22.3 \|\| >=24.8.0`, Bun (no `module.registerHooks`), or Deno. The CLI checks this before it starts and exits 2 with the same sentence; the loader checks it again where it registers its hooks, for a host that runs e2e's runner without the CLI | infrastructure | 3 |
-| `TYPESCRIPT_COMPILER_UNAVAILABLE` | oxc, which compiles e2e's TypeScript, could not be loaded; the message gives the cause. Most often its native bindings for this platform (`@oxc-transform/binding-<platform>-<arch>`, `@oxc-parser/binding-<platform>-<arch>`) are not installed. npm leaves it out with `--omit=optional`, or when `package-lock.json` was written on another platform (npm/cli#4828): install again without `--omit=optional`, or delete `package-lock.json` and `node_modules` and run `npm install` | infrastructure | 3 |
+| `NODE_UNSUPPORTED` | The runtime cannot run e2e's TypeScript loader: a Node.js outside `^22.22.3 \|\| >=24.8.0`, Bun (no `module.registerHooks`), or Deno. The CLI checks this before it starts and exits 2 with the same sentence. The loader checks again where it registers its hooks, for a host that runs e2e's runner without the CLI | infrastructure | 3 |
+| `TYPESCRIPT_COMPILER_UNAVAILABLE` | oxc, which compiles e2e's TypeScript, could not be loaded; the message gives the cause. Most often its native bindings for this platform (`@oxc-transform/binding-<platform>-<arch>`, `@oxc-parser/binding-<platform>-<arch>`) are not installed. npm leaves them out with `--omit=optional`, or when `package-lock.json` was written on another platform (npm/cli#4828). Install again without `--omit=optional`, or delete `package-lock.json` and `node_modules` and run `npm install` | infrastructure | 3 |
 | `ENGINE_FAILURE` | Any engine fault with no more specific mapping | infrastructure | 3 |
-| `WORKER_PROTOCOL` | A worker reported the wrong unit, an attempt overlapped a live attempt, `openHost` tried to acquire an engine instance already owned by another host, or `host.openSession` tried to reuse a closed host or overlap an opening, live, or closing session | infrastructure | 3 |
+| `WORKER_PROTOCOL` | A worker reported the wrong unit, an attempt overlapped a live attempt, `openHost` tried to acquire an engine instance another host owns, or `host.openSession` tried to reuse a closed host or overlap an opening, live, or closing session | infrastructure | 3 |
 | `SESSION_KEY_INVALID` | A worker received a session key that is not 32 bytes. Packaging or IPC fault. | infrastructure | 3 |
-| `REPORT_WRITE_FAILED` | `report.json` or `ai-trace.json` could not be written. Recorded as a run error; the in-memory outcome is still complete. A `junit.xml` that could not be written is the `junit` reporter's stderr line instead | infrastructure | 3 |
+| `REPORT_WRITE_FAILED` | `report.json` or `ai-trace.json` could not be written. Recorded as a run error; the in-memory outcome is still complete. A `junit.xml` that could not be written is a line on the `junit` reporter's stderr instead | infrastructure | 3 |
 | `EVALUATE_FAILED` | `browser.evaluate` threw inside the page (`@e2e-dev/web`) | test | 1 |
-| `INTERRUPTED` | A user or CI signal aborted the run. A test cut short by `--max-failures` or the time limit also ends with this code, its message naming the limit; that run exits 1, not 130 | interrupted | 130 |
-
-Errors raised by argument validation in test code use `INVALID_ARGUMENT` and
-`INVALID_LOCATOR`, both test failures with exit code 1. An unclassified thrown
-value from test code is reported as `ERROR`. The `internal` class and exit
-code 4 are reserved for runner invariant violations; no first-party code raises
-a code in that class today.
+| `INTERRUPTED` | A user or CI signal aborted the run. A test cut short by `--max-failures` or the time limit also ends with this code, and its message names the limit; that run exits 1, not 130 | interrupted | 130 |
 
-## Exit codes
+#### Test code errors
 
-For a mixed run the highest-precedence code present wins, `130 > 4 > 3 > 2 >
-1 > 0`. The report keeps every individual result. A setup assertion failure
-is a test failure and skips its dependents. Missing setup infrastructure keeps
-its configuration or infrastructure class.
+| Code | When it happens | Class | Exit |
+| --- | --- | --- | ---: |
+| `INVALID_ARGUMENT` | Argument validation in test code refused an argument | test | 1 |
+| `INVALID_LOCATOR` | Argument validation in test code refused a locator's arguments | test | 1 |
+| `ERROR` | A value thrown from test code that no other code classifies | test | 1 |