e2e 0.15.1 → 0.16.0

package/docs/reference/cli.mdx

@@ -0,0 +1,819 @@
+---
+title: CLI
+description: Commands, flags, output formats, and exit codes.
+---
+
+<CodeGroup>
+```bash npm
+npx e2e init [directory]
+npx e2e login [provider]
+npx e2e logout [provider]
+npx e2e models [provider]
+npx e2e guide [topic]
+npx e2e mcp [options]
+npx e2e run [files...] [options]
+npx e2e explore [goal] [options]
+npx e2e list [files...] [options]
+npx e2e cache ls|clear|stats [--config <path>]
+npx e2e feedback -m <text> [options]
+npx e2e telemetry [status|enable|disable]
+npx e2e --help
+npx e2e --version
+```
+
+```bash pnpm
+pnpm dlx e2e init [directory]
+pnpm exec e2e login [provider]
+pnpm exec e2e logout [provider]
+pnpm exec e2e models [provider]
+pnpm exec e2e guide [topic]
+pnpm exec e2e mcp [options]
+pnpm exec e2e run [files...] [options]
+pnpm exec e2e explore [goal] [options]
+pnpm exec e2e list [files...] [options]
+pnpm exec e2e cache ls|clear|stats [--config <path>]
+pnpm exec e2e feedback -m <text> [options]
+pnpm exec e2e telemetry [status|enable|disable]
+pnpm exec e2e --help
+pnpm exec e2e --version
+```
+
+```bash bun
+bunx e2e init [directory]
+bunx e2e login [provider]
+bunx e2e logout [provider]
+bunx e2e models [provider]
+bunx e2e guide [topic]
+bunx e2e mcp [options]
+bunx e2e run [files...] [options]
+bunx e2e explore [goal] [options]
+bunx e2e list [files...] [options]
+bunx e2e cache ls|clear|stats [--config <path>]
+bunx e2e feedback -m <text> [options]
+bunx e2e telemetry [status|enable|disable]
+bunx e2e --help
+bunx e2e --version
+```
+</CodeGroup>
+
+The CLI needs Node.js 22.12 or newer.
+
+## Help and version
+
+| Flag | Behavior |
+| --- | --- |
+| `-h`, `--help` | Prints the help of the command it follows and exits 0. `e2e help run` is the same as `e2e run --help`. |
+| `-v`, `--version` | Prints the installed `e2e` version and exits 0. |
+
+```console
+$ npx e2e --version
+0.14.0
+```
+
+Output is colored on a terminal or in CI, plain when piped or when `NO_COLOR`
+is set. `FORCE_COLOR` keeps the colors. `e2e` without a command prints the
+help on stderr and exits 2.
+
+On a terminal at least 55 columns wide, `e2e --help` opens with the e2e
+wordmark, the site's lettering drawn in block characters; piped output and
+narrower terminals get the text alone.
+
+## e2e init
+
+Scaffolds a project and adds `e2e` to `devDependencies`. Existing
+config and test files are skipped. With a `directory` argument the project is
+scaffolded there.
+
+On a terminal, the wizard opens with the e2e wordmark written in over about
+a second, letter by letter, the way a pen writes it. With `--yes`, in CI, on
+a dumb terminal, and on a terminal shorter than 11 rows the wordmark prints at
+once, and piped output never shows it. Ctrl-C or SIGTERM while it is being
+written exits 130.
+
+For a new project, the wizard asks you to choose:
+
+- The engine. **Web** (Playwright, the default) adds `@e2e-dev/web`.
+  **Mobile (iOS/Android)** adds `@e2e-dev/mobile`, defaults to iOS on
+  macOS and Android elsewhere, and opens Settings. **None** adds nothing.
+- A model provider or subscription for agent steps. Choose Vercel AI Gateway,
+  OpenRouter, an OpenAI-compatible endpoint, ChatGPT, GitHub Copilot, or
+  SuperGrok. Choose None for tests that only use locators. The wizard adds
+  the required packages and model config. Subscription users then sign in
+  with `e2e login`. See [Models](/models) and [Subscriptions](/subscriptions).
+- Where to install the agent skill (see [e2e guide](#e2e-guide)):
+  `.agents/skills/` for Codex, Cursor, Copilot, Gemini CLI, OpenCode, and
+  Zed, and `.claude/skills/` for Claude Code. Both are preselected; with
+  both, `.claude/skills/e2e` is a symlink to `.agents/skills/e2e`, the
+  layout `npx skills add` produces.
+- Where to register the [MCP server](/reference/mcp): `.mcp.json` for
+  Claude Code and `.cursor/mcp.json` for Cursor. Both are preselected.
+
+After confirming the file changes, choose whether to install the selected
+dependencies. The installer follows the project's `packageManager` field or
+lockfile, then the invoking package manager, then npm.
+
+| Argument or flag | Type | Default | Behavior |
+| --- | --- | --- | --- |
+| `[directory]` | path | the working directory | Where to scaffold; created when missing |
+| `-y`, `--yes` | boolean | `false` | Uses the Web engine and Vercel AI Gateway, installs the skill and MCP config in both locations, and skips dependency installation |
+
+| Creates | Content |
+| --- | --- |
+| `package.json` | Created as a private ESM package when missing; otherwise only missing dependencies and scripts are added. Sibling packages (`@e2e-dev/web`, `@e2e-dev/mobile`) get the caret range of the version released with the CLI, or that exact version when the CLI is a prerelease. A `test:e2e` script runs `e2e run` unless one exists |
+| `e2e.config.ts` | One target for the selected engine, with the app in its `app`. Playwright reads `APP_URL` (default `http://localhost:3000`) into `app.url` and shows `app.command` in a comment. Mobile sets `app: { bundleId: 'Settings' }` on iOS and `com.android.settings` on Android. With a gateway, `agents.default` is `{ model, system }` |
+| `tests/example.e2e.ts` | An app-open test that passes against any page, plus a commented agent test. Device setup gets a Settings check instead |
+| `.gitignore` | Appends `node_modules/`, `.e2e/artifacts/`, `.e2e/cache/`, `.e2e/sessions/`, `.e2e/report.json`, `.e2e/ai-trace.json`, `.e2e/junit.xml`, `.e2e/summary.md`, `.e2e/failures/`, `.e2e/logs/`, and `.e2e/videos/`. Committing the cache is opt-in; see [commit the replay cache](/cache#commit-the-replay-cache) |
+| `.agents/skills/e2e/`, `.claude/skills/e2e/` | `SKILL.md` and one `references/<topic>.md` per `e2e guide` topic. With both chosen, the Claude Code directory is a relative symlink (`../../.agents/skills/e2e`) to the copy |
+| `.mcp.json`, `.cursor/mcp.json` | Registers `e2e mcp` without changing other servers |
+
+<CodeGroup>
+```bash npm
+npx e2e init --yes
+```
+
+```bash pnpm
+pnpm dlx e2e init --yes
+```
+
+```bash bun
+bunx e2e init --yes
+```
+</CodeGroup>
+
+Re-running `init` keeps an existing config and adds no packages for it. Once
+a skill directory exists, later runs refresh copies that differ from the
+installed version and never add a directory the project did not choose. A
+`.claude/skills/e2e/` copy from an earlier version becomes the symlink when
+it holds nothing but the shipped files; one with other files in it stays a
+copy and is refreshed. A skill directory that is a symlink elsewhere, or sits
+under one, is never written through: `--yes` skips it with a warning naming
+the link and its target, and an interactive run offers to replace a linked
+skill directory with a copy, or with the link where the link goes. A
+copy with a directory where a file goes, or a file where a directory goes, is
+skipped with a warning naming the entry; `init` removes nothing to repair it.
+
+Cancellation leaves files untouched and exits 0. An invalid `package.json`
+exits 2 before any write. Without a terminal on stdin and stdout, `init`
+without `--yes` exits 2 and names the flag.
+
+## e2e login
+
+<CodeGroup>
+```bash npm
+npx e2e login openai              # ChatGPT, browser sign-in
+npx e2e login openai --device     # sign in with a device code
+npx e2e login github-copilot --from-gh  # use your GitHub CLI login
+npx e2e login github-copilot --client-id <id> [--enterprise-url <host>]
+npx e2e login spacexai            # SuperGrok, device code
+npx e2e login                     # choose a provider
+```
+
+```bash pnpm
+pnpm exec e2e login openai              # ChatGPT, browser sign-in
+pnpm exec e2e login openai --device     # sign in with a device code
+pnpm exec e2e login github-copilot --from-gh  # use your GitHub CLI login
+pnpm exec e2e login github-copilot --client-id <id> [--enterprise-url <host>]
+pnpm exec e2e login spacexai            # SuperGrok, device code
+pnpm exec e2e login                     # choose a provider
+```
+
+```bash bun
+bunx e2e login openai              # ChatGPT, browser sign-in
+bunx e2e login openai --device     # sign in with a device code
+bunx e2e login github-copilot --from-gh  # use your GitHub CLI login
+bunx e2e login github-copilot --client-id <id> [--enterprise-url <host>]
+bunx e2e login spacexai            # SuperGrok, device code
+bunx e2e login                     # choose a provider
+```
+</CodeGroup>
+
+Signs in to a subscription and stores credentials in
+`$XDG_CONFIG_HOME/e2e/oauth.json`, by default `~/.config/e2e/oauth.json`. A failed sign-in exits 1, a cancelled one 130; an
+unknown provider exits 2. See [Subscriptions](/subscriptions) for model
+configuration.
+
+| Argument or flag | Type | Default | Behavior |
+| --- | --- | --- | --- |
+| `[provider]` | `openai`, `github-copilot`, or `spacexai` | a picker | The subscription to sign in to |
+| `--device` | boolean | `false` | ChatGPT: show a code to enter on another device instead of opening a browser |
+| `--from-gh` | boolean | `false` | GitHub Copilot: reuse the token of the signed-in GitHub CLI |
+| `--client-id <id>` | string | none | GitHub Copilot: the client id of your GitHub OAuth App with the device flow enabled |
+| `--enterprise-url <host>` | string | none | GitHub Copilot: the GitHub Enterprise host |
+
+## e2e logout
+
+<CodeGroup>
+```bash npm
+npx e2e logout           # pick one of the stored logins
+npx e2e logout openai    # forget one
+```
+
+```bash pnpm
+pnpm exec e2e logout           # pick one of the stored logins
+pnpm exec e2e logout openai    # forget one
+```
+
+```bash bun
+bunx e2e logout           # pick one of the stored logins
+bunx e2e logout openai    # forget one
+```
+</CodeGroup>
+
+## e2e models
+
+<CodeGroup>
+```bash npm
+npx e2e models           # every stored login
+npx e2e models openai    # one provider
+```
+
+```bash pnpm
+pnpm exec e2e models           # every stored login
+pnpm exec e2e models openai    # one provider
+```
+
+```bash bun
+bunx e2e models           # every stored login
+bunx e2e models openai    # one provider
+```
+</CodeGroup>
+
+Asks the vendor which models the stored login serves and prints one line per
+model: the id `chatgpt()`, `copilot()`, or `grok()` takes, the vendor's name
+for it, and its notes (reasoning levels, vision, preview, hidden, and for
+Copilot `not enabled` or `no chat completions` when `copilot()` cannot call
+it). Without a
+stored login, or when the named provider has none, exits 1 with the sign-in
+command; when one of several logins fails to list, the others still print and
+the command exits 1. An unknown provider exits 2. See
+[Pick a model](/subscriptions#pick-a-model).
+
+## e2e guide
+
+<CodeGroup>
+```bash npm
+npx e2e guide            # the overview
+npx e2e guide setup      # one topic
+```
+
+```bash pnpm
+pnpm exec e2e guide            # the overview
+pnpm exec e2e guide setup      # one topic
+```
+
+```bash bun
+bunx e2e guide            # the overview
+bunx e2e guide setup      # one topic
+```
+</CodeGroup>
+
+Prints the skill that `init` installs. `guide` alone prints the overview;
+`guide <topic>` prints one of `setup`, `writing-tests`, `agent`, `running`,
+`explore`, `debugging`, `mcp`, or `bug-bash`. An unknown topic exits 2 and names the
+valid ones. The same skill installs with `npx skills add tester-army/e2e`.
+See [Coding agents](/coding-agents).
+
+## e2e mcp
+
+<CodeGroup>
+```bash npm
+npx e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
+```
+
+```bash pnpm
+pnpm exec e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
+```
+
+```bash bun
+bunx e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
+```
+</CodeGroup>
+
+An MCP server on stdio that gives a coding agent a live session on the app.
+Flags, tools, and session rules are on the [MCP server](/reference/mcp) page.
+
+## e2e run
+
+### Positional arguments
+
+```bash
+e2e run [files...]
+```
+
+Test files, directories, or globs relative to the project root:
+
+- a file path (`tests/signup.e2e.ts`) selects that file;
+- a directory (`tests/agent`, `.`) selects every discovered test file beneath
+  it;
+- a bare name selects every discovered file whose path ends with it at a
+  segment boundary. `signup.e2e.ts` and `agent/signup.e2e.ts` both select
+  `tests/agent/signup.e2e.ts`. Without a `/` or `.` it matches the file name
+  with extensions dropped, so `signup` selects it too;
+- any of these with `:line` appended (`tests/signup.e2e.ts:12`, `signup:12`)
+  selects only the test declared at that line of each file it names: the line
+  the `test(` call opens on, as an editor or a report's `source` shows it. A
+  serial group runs whole when a line names one member. Several lines of one
+  file add up; naming the file without a line selects it whole. When no test
+  is declared at a named line, `NO_TESTS` names the positional and the lines
+  the file declares tests at. A positional that exists on disk as written is
+  a path, even when its name ends in `:12`;
+- a glob (`'tests/**/*.smoke.e2e.ts'`, quoted) uses the grammar of the config
+  `tests` globs: `*`, `?`, and a whole `**` segment. Braces, character
+  classes, and extglobs are `INVALID_GLOB` with a hint rather than a pattern
+  that matches nothing: there is no expansion, so name one glob per
+  alternative. `./`, `..`, a trailing `/`, and an absolute path inside the
+  project resolve as they do for any positional path, and an existing file or
+  directory is selected by its path whatever its name is spelled with.
+
+Positionals narrow the config globs and the tag, platform, and capability
+filters; they never bypass them. Names and globs are case-sensitive on every
+OS. A path outside the project root is `COLLECTION_ERROR`.
+
+The runner imports every discovered test file, even when you select only a
+few. This lets a selected test find its session setup in another file.
+Other ordinary tests remain unselected. A collection error in a selected
+file, or in any file of a run with no positionals, fails the run. A file the
+positionals left out that fails to import or register its tests is skipped
+with a notice, so a half-written file elsewhere in the suite does not stop
+`npx e2e run tests/login.e2e.ts`; if a selected test's session then has no
+setup test, the error names that file as where the setup may be.
+
+A positional starting with `-` that names no file is a usage error, exit 2.
+This is what `pnpm test:e2e -- --headed` produces: the `--` reaches e2e and
+`--headed` would otherwise be taken for a file. The message names the direct
+form, `pnpm exec e2e run --headed`.
+
+When nothing is left to run, `NO_TESTS` names the cause: no file matched the
+globs (with look-alike files such as `tests/login.test.ts`), a positional
+matched nothing (with the nearest path), a matched file registered no tests,
+or every test was filtered out.
+
+### Flags
+
+| Flag | Type | Default | Behavior |
+| --- | --- | --- | --- |
+| `--config <path>` | string | discovered | Explicit config path. A missing file is `CONFIG_NOT_FOUND`. |
+| `--target <ids>` | comma-separated or repeated | every configured target | Selects targets by name. Only the app commands of selected targets start. An unknown ID is `UNKNOWN_TARGET`. |
+| `--tag <tags>` | comma-separated or repeated | none | Only tests carrying the tags: any of them, or every one with `--tag-mode all`. When the filter leaves nothing to run, `NO_TESTS` counts the tests it filtered, marks each tag no test declares, and names the nearest declared one. |
+| `--tag-mode <mode>` | `any` or `all` | `any` | How several tags combine: any of them, or every one. |
+| `--exclude-tag <tags>` | comma-separated or repeated | none | Leaves out every test carrying any of the tags, whatever `--tag` or a positional selected. A serial group still runs whole when any member is selected; tag the serial `describe` to exclude the flow. |
+| `--grep <pattern>` | regular expression, repeatable | none | Only tests whose title matches one of the patterns. The text matched is the describe titles and the test title joined by one space (`checkout pays`); the file and the tags are not part of it. A bare pattern is case-sensitive; `'/pattern/i'` adds flags. A pattern that does not compile is a usage error, exit 2. |
+| `--grep-invert <pattern>` | regular expression, repeatable | none | Leaves out every test whose title matches one of the patterns. Same text and pattern forms as `--grep`; both together keep the tests `--grep` matches minus the ones `--grep-invert` matches. |
+| `--last-failed` | boolean | `false` | Only the tests the previous run did not pass, read from the `report.json` this run overwrites, `<output>/report.json` (`--output`, else the config's [`output`](/reference/config#output)): failed, timed out, interrupted, or skipped because a setup, a serial predecessor, a hook, or the worker failed, and every test it selected in the describe or file, on that target, of a `beforeAll` or `afterAll` that failed (for a setup, the tests that consume its session). A rerun that runs the hook again reports it again if it still fails; another filter that leaves its tests out leaves the failure out of the next report too. A test is named per target and agent. No report is `NO_LAST_RUN`; a previous run with nothing failed selects nothing, `NO_TESTS` unless `--pass-with-no-tests`. |
+| `--shard <index/total>` | `2/3` | none | Runs one contiguous slice of the selected tests, cut once every other filter applied. Slices differ in size by at most one test, a serial group counts as one and is never split, and each shard brings only the setup tests its own tests need. Every shard runs the same command with a different index; the order is the report order, so the same suite shards the same way on every machine. A shard that gets no test is `NO_TESTS` unless `--pass-with-no-tests`. See [Sharding](/ci#sharding). |
+| `--headed` | boolean | `false` | Visible UI when the engine supports it. |
+| `--agent <names>` | comma-separated or repeated | `default` | Runs unpinned tests once per selected agent. For a test with `{ agent }`, selects the shared names when there is overlap; otherwise keeps the test's selection. An unknown name is `INVALID_CONFIG`. |
+| `--retries <n>` | integer 0-10 | config | Replaces `config.retries`. |
+| `--workers <n>` | integer 1-1024 | config | Replaces `config.workers`. |
+| `--repeat-each <n>` | integer, at least 1 | `1` | Runs every selected test this many times. Each run is a result of its own, with `repeat` 0 through `n - 1` and its own `id`; the list reporter titles the later runs `(repeat #1)`, `(repeat #2)`, and their artifacts go under a `repeat-<n>` directory. Setup tests run once. With the replay cache on, the first run records and the later ones replay, so pair it with `--no-cache` to exercise the model each time. The `list` summary adds a `Repeats` row (`1 of 2 tests passed all 5 runs`) and one line per test that did not pass every run, naming each run that failed; the `markdown` page carries the same tally. |
+| `--max-failures <n>` | integer, at least 1 | none | Stops the run once this many tests have failed or timed out. Nothing more is dispatched, the tests running on other workers end as `interrupted`, and the tests not started, the rest of the file whose test reached the limit included, are skipped with cause `failure-limit` and the reason `run stopped after 3 failures (--max-failures 3)`. The exit code is the failures' own, 1, not 130. |
+| `--reporter <ids>` | comma-separated list | config | Replaces the built-in IDs in `config.reporters`: `list`, `json`, `junit`, `markdown`. `json` with `list` is `INVALID_CONFIG`. [Reporter objects](/reference/reporters) in the config keep running. |
+| `--output <dir>` | string | config, else `.e2e` | The results directory for this run, over the config's [`output`](/reference/config#output): the report, the reporter files, the AI trace, `artifacts/` (cleared once the run's tests start), and `sessions/`. Resolved from the project root; one the run cannot own is `INVALID_CONFIG`. The removed `--artifacts <dir>` fails with the `--output` it maps to: `--artifacts out/artifacts` is `--output out`, and a directory whose parent is the project root or outside it maps to `--output .e2e`. |
+| `--no-cache` | boolean | `false` | Replay cache off for this run, whatever `config.cache` says. |
+| `--strict-cache` | boolean | `false` | Turns on `cache.strict` for this run: a step whose recording exists but no longer replays fails with `REPLAY_STALE` instead of handing off to the agent. See [Fail on a stale recording](/cache#fail-on-a-stale-recording). |
+| `--pass-with-no-tests` | boolean | `false` | Exit 0 instead of `NO_TESTS` when nothing is selected. |
+| `--debug` | boolean | `false` | Phase timings and an agent step table on stderr after the run. |
+| `--ai-trace` | boolean | `false` | Records every model call to `<output>/ai-trace.json`, an AI SDK devtools database. Open it with [unbox-ai](https://github.com/tester-army/unbox-ai). |
+| `--trace [mode]` | `off`, `on`, `retain-on-failure`, `on-first-retry`, or `on-all-retries` | config | Which attempts record a trace, over the config's and every target's `trace`; a test's own `trace` still wins. A bare `--trace` is `on`, `--trace off` skips the cost for one run. It applies to the targets whose engine can trace; the run names the others in one notice. The value is optional and greedy, so `--trace tests/a.e2e.ts` reads the file as a mode and fails: write `--trace=<mode>`, or put test files before `--trace`. See [trace](/reference/config#trace). |
+| `--video [mode]` | a recording mode, as `--trace` | config | Which attempts record a video (WebM on a browser engine, MP4 on a device engine), over the config's and every target's `video`; a test's own `video` still wins. A bare `--video` is `on`. It applies to the targets whose engine can record; the run names the others in one notice. Greedy like `--trace`: write `--video=<mode>`, or put test files before `--video`. See [video](/reference/config#video). |
+
+Out-of-range `--retries` and `--workers` fail with `INVALID_CONFIG` before
+anything starts, the same as the config value would. `--target`, `--tag`,
+`--exclude-tag`, and `--agent` take each name once; an empty value (`--tag ''`,
+or `--tag "$TAGS"` with the variable unset) is a usage error, exit 2, never a
+wider selection.
+
+Filters only narrow, and each applies after the one before: positionals, then
+`.only`, `--tag`, `--exclude-tag`, `--grep`, `--grep-invert`, `--last-failed`,
+then the platform and capability rules, then `--shard`. When they leave
+nothing to run, `NO_TESTS` counts the tests each filter removed, names the tags
+and patterns, and marks a tag no test declares.
+
+`--debug` appends two tables to stderr. Phase timings are summed across the
+runner and every worker. The agent step table lists every agent step in
+execution order with its per-phase split, model calls, tokens, and cost when
+the provider reported one.
+
+```console
+$ npx e2e run --debug
+...
+[e2e debug] phase timings (wall 14.2s)
+  phase               count   total     avg     max
+  scheduler               1   11.8s   11.8s   11.8s
+  test.body               6   10.1s   1.7s    3.9s
+  session.launch          6    1.1s   183ms   410ms
+  engine.prepare          1   612ms   612ms   612ms
+  collect                 1   420ms   420ms   420ms
+  config.load             1   118ms   118ms   118ms
+  realm.import            1    96ms    96ms    96ms
+  session.close           6    88ms    15ms    31ms
+  engine.init             1    74ms    74ms    74ms
+
+[e2e debug] agent steps (execution order, model anthropic/claude-sonnet-4.5, total $0.0312)
+  step                                   total   model  observe  action  calls  tokens in/out  cached       cost
+  act "add a todo named groceries"       3.9s    2.8s    410ms   620ms      3  9412/388       61% (5740)   $0.0198
+  assert "the list shows groceries"      1.2s    1.1s     88ms     0ms      1  3120/41        0% (0)       $0.0114
+```
+
+`cached` is the share of input tokens the provider served from its prompt
+cache, `-` when it reports no split. The report records the same per step
+(`model.cacheReadTokens`, `model.cacheWriteTokens`) and per run
+(`usage.modelCachedTokens`).
+
+<CodeGroup>
+```bash npm
+npx e2e run
+npx e2e run tests/signup.e2e.ts --tag smoke
+npx e2e run tests/signup.e2e.ts:12
+npx e2e run tests/agent
+npx e2e run 'tests/**/*.smoke.e2e.ts'
+npx e2e run --target web,webkit --tag smoke,billing --tag-mode all
+npx e2e run --tag smoke --exclude-tag slow
+npx e2e run --grep checkout --grep-invert '/refund/i'
+npx e2e run --last-failed
+npx e2e run --shard 2/3
+npx e2e run --reporter json --workers 4 --retries 2
+npx e2e run --max-failures 3
+npx e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts
+npx e2e run --reporter list,junit
+```
+
+```bash pnpm
+pnpm exec e2e run
+pnpm exec e2e run tests/signup.e2e.ts --tag smoke
+pnpm exec e2e run tests/signup.e2e.ts:12
+pnpm exec e2e run tests/agent
+pnpm exec e2e run 'tests/**/*.smoke.e2e.ts'
+pnpm exec e2e run --target web,webkit --tag smoke,billing --tag-mode all
+pnpm exec e2e run --tag smoke --exclude-tag slow
+pnpm exec e2e run --grep checkout --grep-invert '/refund/i'
+pnpm exec e2e run --last-failed
+pnpm exec e2e run --shard 2/3
+pnpm exec e2e run --reporter json --workers 4 --retries 2
+pnpm exec e2e run --max-failures 3
+pnpm exec e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts
+pnpm exec e2e run --reporter list,junit
+```
+
+```bash bun
+bunx e2e run
+bunx e2e run tests/signup.e2e.ts --tag smoke
+bunx e2e run tests/signup.e2e.ts:12
+bunx e2e run tests/agent
+bunx e2e run 'tests/**/*.smoke.e2e.ts'
+bunx e2e run --target web,webkit --tag smoke,billing --tag-mode all
+bunx e2e run --tag smoke --exclude-tag slow
+bunx e2e run --grep checkout --grep-invert '/refund/i'
+bunx e2e run --last-failed
+bunx e2e run --shard 2/3
+bunx e2e run --reporter json --workers 4 --retries 2
+bunx e2e run --max-failures 3
+bunx e2e run --repeat-each 5 --no-cache tests/checkout.e2e.ts
+bunx e2e run --reporter list,junit
+```
+</CodeGroup>
+
+### Output
+
+| Reporter | Writes |
+| --- | --- |
+| `list` | Terminal progress, failures with code frames, and a run summary. On a TTY, live progress includes the current agent step, model turns, tool calls, and reasoning excerpts. Untrusted fields are capped at 8 KiB and stripped of control characters. |
+| `json` | The report document to stdout and nothing else there, so `e2e run --reporter json \| jq` works. Same document as `<output>/report.json`, described by `schema/report-v1.schema.json` in the package. The shape changes only with the `schemaVersion` field (`report-1` today). Cannot be combined with `list`. |
+| `junit` | `junit.xml` beside the report. One `<testsuite>` per file, one `<testcase>` per test-target pair named `title > path [target]`. A test-class error is a `<failure>`, any other an `<error>`, a skip a `<skipped>` with its reason. Run-level errors form a `run` suite. |
+| `markdown` | `summary.md` beside the report with counts, cost, errors, and failure details. Failed and flaky tests also get a page under `failures/` there, with steps, model turns, and the screen at failure. Exploration runs show findings. [`@e2e-dev/github`](/github) uses this summary for pull request comments. |
+
+Every run that gets to its tests writes `report.json` under the [output directory](/reference/config#output)
+(`.e2e` by default) whatever the reporters are, and the reporters' files
+beside it. A run that stops before (a config or collection error,
+`NO_TESTS`, `NO_LAST_RUN`, `UNSUPPORTED_ARTIFACT`, an app command that
+fails to start, an interrupt) writes none of them and leaves the previous
+run's files in place; its errors are on the terminal and in the exit code. A `junit.xml` or `summary.md` that could not be written is a
+line on stderr, never a run error.
+
+With `--ai-trace`, the run also writes `<output>/ai-trace.json`: one entry per
+model round trip with the exact prompt, tool definitions, response, usage,
+and provider metadata. Observations in it are already redacted. Inline image
+and file bytes are replaced with byte counts. Open it with
+`npx unbox-ai .e2e/ai-trace.json` for the default output; see [Debugging a run](/debugging#agent-steps).
+
+### Signals
+
+Signals escalate.
+
+1. The first `SIGINT` or `SIGTERM` interrupts. The running test ends with
+   status `interrupted`, its `afterEach` hooks and session close run within
+   the cleanup budget, and every worker disposes its engine. The tests not
+   started are skipped with cause `infrastructure-unavailable` and the reason
+   `run interrupted before execution`, so the report lists every planned
+   test and `--last-failed` runs the interrupted and the unstarted ones
+   again. The runner writes the report and exits 130. An interrupt before
+   any test starts (during collection, an engine's `prepare`, or app
+   startup) writes no report and leaves the previous one in place.
+2. A second signal forces. Workers dispose their engines at once and are
+   killed when the cleanup budget is spent. The report is still written.
+3. A third signal exits on the spot, after killing the process groups of every
+   app command the run started. A process attached through
+   `reuseExisting` is left alone.
+
+A worker whose runner disappears disposes its engine and exits within the
+cleanup budget.
+
+## e2e explore
+
+```bash
+e2e explore [goal]
+```
+
+A run with no test file. The agent gets a goal, plans one exploration step
+at a time, reports defects through its `report_finding` tool, and ends with an
+assessment. The default goal is `Explore the app and find bugs`. See
+[Exploring without a test](/explore).
+
+| Flag | Type | Default | Behavior |
+| --- | --- | --- | --- |
+| `--config <path>` | string | discovered | Explicit config path, with the same errors as `run`. |
+| `--target <id>` | string | the first configured target | The one target to explore. Several targets and no flag explores the first and says so on stderr. An unknown name is `UNKNOWN_TARGET`. |
+| `--agent <name>` | string | `default` | The configured agent the exploration runs as. Its model, tools, and context are reused; a hand-rolled `StepExecutor` is replaced by the built-in agent, with a notice. An unknown name is `INVALID_CONFIG`. |
+| `--session <name>` | string | none | Starts signed in. The setup test in the config's test files that declares this session runs first, and the exploration restores what it saved, as a test with `{ session }` does. No other test runs. A name no setup test declares is `COLLECTION_ERROR`, naming the declared ones, before any app process starts. See [Start signed in](/explore#start-signed-in). |
+| `--max-steps <n>` | integer 1-12 | `8` | Exploration steps at most. Out of range exits 2. |
+| `--timeout <ms>` | integer 180000-900000 | `600000` | The run's wall clock. The last minute is kept for the closing assessment. Out of range exits 2. |
+| `--headed` | boolean | `false` | Requests visible UI when the engine supports it. |
+| `--reporter <ids>` | comma-separated list | resolved config value | As for `run`. The exploration summary rows print under the `list` summary; `json` carries the record in the report. |
+| `--output <dir>` | string | config, else `.e2e` | As for `run`; a finding's evidence screenshot is one of the attempt's artifacts under `<output>/artifacts`. |
+| `--debug` | boolean | `false` | As for `run`: phase timings and the agent step table. |
+| `--ai-trace` | boolean | `false` | As for `run`. |
+| `--trace [mode]` | a recording mode | config | As for `run`: one trace of the whole exploration. A bare `--trace` is `on`; `retain-on-failure` keeps it only when the exploration did not pass. The exploration is one attempt, so `on-first-retry` and `on-all-retries` record nothing, and neither does the CI default: `CI=1 e2e explore` needs `--trace on` for a trace, and its plan-time notice says so. Put the goal before `--trace`, or write `--trace=<mode>`. |
+| `--video [mode]` | a recording mode | config | As for `run`: one video of the whole exploration (WebM on a browser engine, MP4 on a device engine). A bare `--video` is `on`; `retain-on-failure` keeps it only when the exploration did not pass. The exploration is one attempt, so a retry mode records nothing. Put the goal before `--video`, or write `--video=<mode>`. |
+
+The run has one result, under the virtual file `explore`, titled by the goal.
+With `--session`, the report also holds the setup test's result and lists the
+project's other tests as skipped with cause `filtered`, as `run` lists the
+tests of a file no positional named.
+The exploration runs with the replay cache off and zero retries whatever the
+config says. A `--session` setup test runs as it would for `run`, with the
+config's agents, cache, and retries.
+`<output>/report.json` gains `run.explore`: the goal, the budgets, why the run
+ended (`finished`, `step-limit`, `time`, `stuck`, or `aborted`), the
+assessment, the steps, and the findings.
+
+Exit codes: `0` when steps ran and no finding of kind `issue` was reported,
+`1` when one was or when no step ran at all, `2` and `3` as for `run`, `130`
+on Ctrl-C.
+
+<CodeGroup>
+```bash npm
+npx e2e explore
+npx e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'
+npx e2e explore --target web --max-steps 4 --headed
+npx e2e explore 'Hunt for broken forms and dead links' --agent thorough --video
+```
+
+```bash pnpm
+pnpm exec e2e explore
+pnpm exec e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'
+pnpm exec e2e explore --target web --max-steps 4 --headed
+pnpm exec e2e explore 'Hunt for broken forms and dead links' --agent thorough --video
+```
+
+```bash bun
+bunx e2e explore
+bunx e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'
+bunx e2e explore --target web --max-steps 4 --headed
+bunx e2e explore 'Hunt for broken forms and dead links' --agent thorough --video
+```
+</CodeGroup>
+
+## e2e list
+
+```bash
+e2e list [files...]
+```
+
+A dry run. `list` selects tests exactly as `run` does, prints one line per
+test-target pair, and exits. No app process, engine, or worker starts, and
+nothing is written.
+
+Positionals and the selection flags (`--config`, `--target`, `--tag`,
+`--tag-mode`, `--exclude-tag`, `--grep`, `--grep-invert`, `--last-failed`,
+`--shard`, `--pass-with-no-tests`) are those of `run`; `--last-failed` reads
+`report.json` under the config's [`output`](/reference/config#output). `--reporter` accepts `list` (the default) or `json`.
+
+| Reporter | Writes |
+| --- | --- |
+| `list` | One line per pair, `file › title [target]`, in selection order, with the describe path joined by ` › ` (`tests/billing.e2e.ts › checkout › pays [web]`), then each tag the test declares as `#tag` (`tests/billing.e2e.ts › checkout › pays [web] #smoke #billing`). A skipped test keeps its line with `(skipped: <reason>)` appended. Setup tests are listed like any other. |
+| `json` | `{ "pairs": [...] }` to stdout. Each pair carries `file`, `title`, `titlePath`, `kind` (`test` or `setup`), `tags` (the tags the test declares, `[]` when none), `target`, `disposition` (`run` or `skip`), and `skipReason` when skipped. |
+
+Pairs a filter removed are not listed.
+
+<CodeGroup>
+```bash npm
+npx e2e list
+npx e2e list tests/signup.e2e.ts
+npx e2e list --tag smoke --target web
+npx e2e list --grep checkout --exclude-tag slow
+npx e2e list --reporter json
+```
+
+```bash pnpm
+pnpm exec e2e list
+pnpm exec e2e list tests/signup.e2e.ts
+pnpm exec e2e list --tag smoke --target web
+pnpm exec e2e list --grep checkout --exclude-tag slow
+pnpm exec e2e list --reporter json
+```
+
+```bash bun
+bunx e2e list
+bunx e2e list tests/signup.e2e.ts
+bunx e2e list --tag smoke --target web
+bunx e2e list --grep checkout --exclude-tag slow
+bunx e2e list --reporter json
+```
+</CodeGroup>
+
+## e2e cache
+
+Inspect or clear the replay cache. These commands load the discovered config
+or `--config <path>`, honor `cache.dir`, and exit 0 on an empty store. The
+default directory is `.e2e/cache/`.
+
+| Command | Prints |
+| --- | --- |
+| `e2e cache ls` | One row per entry: the test and target it was recorded for, the first 12 characters of its instruction digest, its age, and how many actions a replay would run (`(truncated)` when recording hit the action cap). Rows are grouped by test, then target, oldest first. |
+| `e2e cache stats` | The store directory, the number of readable entries, their total size, and the number of unreadable files when there are any. |
+| `e2e cache clear` | Deletes every cache file, then the directory when nothing else is left in it, and reports how many files went. |
+
+<CodeGroup>
+```bash npm
+npx e2e cache ls
+npx e2e cache stats
+npx e2e cache clear
+npx e2e cache ls --config packages/web/e2e.config.ts
+```
+
+```bash pnpm
+pnpm exec e2e cache ls
+pnpm exec e2e cache stats
+pnpm exec e2e cache clear
+pnpm exec e2e cache ls --config packages/web/e2e.config.ts
+```
+
+```bash bun
+bunx e2e cache ls
+bunx e2e cache stats
+bunx e2e cache clear
+bunx e2e cache ls --config packages/web/e2e.config.ts
+```
+</CodeGroup>
+
+```text
+TEST                             TARGET  INSTRUCTION   AGE  ACTIONS
+tests/billing.e2e.ts::upgrades   web     4f1c0a9d3b62  2h   4
+tests/signup.e2e.ts::signs%20up  web     9ab7715ce004  3d   7 (truncated)
+```
+
+An entry recorded before entries carried provenance prints `-` in those
+columns and still replays. A file the store cannot read (truncated, over the
+1 MiB ceiling, or from a newer schema) is counted separately on stderr, treated
+as a miss by a run, and removed by `clear`. Files the runner never wrote are
+left alone.
+
+<Note>
+These commands read the file store only. With a custom `cache.store` they
+exit 2 and say so.
+</Note>
+
+## e2e feedback
+
+<CodeGroup>
+```bash npm
+npx e2e feedback -m <text> [--type bug|docs|feature|other] [options]
+```
+
+```bash pnpm
+pnpm exec e2e feedback -m <text> [--type bug|docs|feature|other] [options]
+```
+
+```bash bun
+bunx e2e feedback -m <text> [--type bug|docs|feature|other] [options]
+```
+</CodeGroup>
+
+Sends one report about e2e itself to the e2e team: something broke, the docs
+or the skill misled, or a capability is missing. Coding agents are welcome to
+send one when they hit a problem with e2e, not with the app under test.
+
+| Flag | Meaning |
+| --- | --- |
+| `-m, --message <text>` | Required. One or two sentences on the problem, at most 2000 characters |
+| `--type <type>` | `bug`, `docs`, `feature`, or `other` (default) |
+| `--task <text>` | What you were trying to do |
+| `--expected <text>` | What you expected to happen |
+| `--actual <text>` | What happened instead, with the error code and message |
+| `--approach <text>` | What you tried, workarounds included |
+| `--command <text>` | The e2e command or API involved, at most 200 characters |
+| `--agent <text>` | The coding agent and model sending it, at most 200 characters |
+| `--dry-run` | Print the event that would be sent and send nothing |
+
+`--task`, `--expected`, `--actual`, and `--approach` take up to 4000
+characters each. A blank or longer value exits 2.
+
+<CodeGroup>
+```bash npm
+npx e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout"
+npx e2e feedback --type docs -m "The mcp topic never says how to close a session"
+npx e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run
+```
+
+```bash pnpm
+pnpm exec e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout"
+pnpm exec e2e feedback --type docs -m "The mcp topic never says how to close a session"
+pnpm exec e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run
+```
+
+```bash bun
+bunx e2e feedback --type bug -m "list ignores --grep" --command "e2e list --grep checkout"
+bunx e2e feedback --type docs -m "The mcp topic never says how to close a session"
+bunx e2e feedback --type feature -m "Need a way to set the viewport per test" --dry-run
+```
+</CodeGroup>
+
+The report goes to the same PostHog project as [telemetry](/telemetry#feedback),
+as one `e2e_feedback` event, together with the anonymous machine facts every
+telemetry event carries. Before sending, the value of every environment
+variable whose name holds `KEY`, `TOKEN`, `SECRET`, `PASS`, `_PWD`,
+`CREDENTIAL`, `AUTH`, `COOKIE`, or `PRIVATE` becomes `<secret:NAME>` when it
+is at least six characters long, the minimum for a config secret, since a
+shorter one would rewrite ordinary words. Bearer tokens,
+URL credentials, private keys, JWTs, and common API key shapes become
+`<redacted>`. Do not paste app data, page content, or test files.
+
+Feedback is sent only when you run the command, so a saved
+`e2e telemetry disable` does not stop it. `E2E_TELEMETRY_DISABLED=1` and
+`DO_NOT_TRACK=1` do: with either set the command exits 2 and sends nothing,
+so a CI job or a fleet that opted out sends nothing even when an agent runs
+it. `E2E_TELEMETRY_DEBUG=1` prints it like `--dry-run`. It prints a reference
+id and exits 0 once PostHog accepts the event, or exits 3 when PostHog did
+not confirm it; the report may still have arrived, so the message names its
+reference id.
+
+## e2e telemetry
+
+Shows whether anonymous usage telemetry is on, and switches it.
+[Telemetry](/telemetry) lists every property sent.
+
+| Command | Effect |
+| --- | --- |
+| `e2e telemetry`, `e2e telemetry status` | Prints `Status: enabled`, or `Status: disabled` with the reason: the variable that is set, the saved choice, or a preferences directory that cannot be written. |
+| `e2e telemetry disable` | Saves the opt-out to `~/.config/e2e/telemetry.json` (`$XDG_CONFIG_HOME/e2e/` when set, `%APPDATA%\e2e\` on Windows) and prints the status. |
+| `e2e telemetry enable` | Saves the opt-in and prints the status. A variable set in the environment still wins, and the status says so. |
+
+<CodeGroup>
+```bash npm
+npx e2e telemetry
+npx e2e telemetry disable
+E2E_TELEMETRY_DEBUG=1 npx e2e run
+```
+
+```bash pnpm
+pnpm exec e2e telemetry
+pnpm exec e2e telemetry disable
+E2E_TELEMETRY_DEBUG=1 pnpm exec e2e run
+```
+
+```bash bun
+bunx e2e telemetry
+bunx e2e telemetry disable
+E2E_TELEMETRY_DEBUG=1 bunx e2e run
+```
+</CodeGroup>
+
+`E2E_TELEMETRY_DISABLED=1` and `DO_NOT_TRACK=1` turn telemetry off without
+touching the file. `E2E_TELEMETRY_DEBUG=1` prints every event to stderr and
+sends nothing. The first command on a machine other than `e2e init` and
+`e2e telemetry` prints a notice once; in CI nothing is printed and no file is written.
+
+## Exit codes
+
+| Code | Meaning |
+| ---: | --- |
+| 0 | All selected tests passed, were flaky, or were explicitly skipped |
+| 1 | Final test or setup failure, or test timeout |
+| 2 | CLI, config, collection, dependency, credential, model-config, or policy error |
+| 3 | Engine, app process, model provider, artifact, or cleanup infrastructure failure; `e2e feedback` that could not be delivered |
+| 4 | Internal runner invariant or unhandled runner error |
+| 130 | Interrupted by a user or CI signal |
+
+Precedence for a mixed run is `130 > 4 > 3 > 2 > 1 > 0`. The report keeps every
+individual result; precedence affects only the process exit code.
+
+Argument parsing failures exit 2 with a one-line error on stderr.
+
+## Diagnostics
+
+A message that rejects a name suggests the nearest valid one. Keys from other
+runners' configs (`testDir`, `baseURL`, `webServer`, `projects`) say where
+that fact lives here. A failing `import` in the config or a test file ends
+with the install or add command for the project's package manager, or the
+subpaths the package exports. A model credential the provider rejects names
+the variable the provider reads.