e2e 0.16.0 → 0.18.0

package/docs/reference/agent.mdx

@@ -1,6 +1,7 @@
 ---
 title: agent
 description: Run agent actions, check conditions, and extract structured data.
+icon: "robot"
 ---
 
 Use `act` to interact with the app, `assert` to check the current screen,
@@ -191,7 +192,9 @@ Every `act` observation with unavailable semantics includes that image and
 warning automatically. Earlier node ids are invalid, and the next successful
 semantic capture presents the whole tree again. The screenshot shows the
 current viewport; an unavailable tree does not prove that a control or
-message is absent.
+message is absent. After a secret fill there is no fallback: `act`, and a
+judgment with `vision: true`, fails with `POLICY_DENIED` when semantic
+capture times out before the step does.
 
 If semantic capture fails, trace replay stops and the step cannot record a
 new trace entry, even if the tree later recovers.
@@ -238,7 +241,12 @@ assert(assertion: string, options?: AssertOptions): Promise<void>;
 
 Checks one fresh observation without polling. The judge sees only the
 assertion and current observation, never prior steps or `act` summaries.
-It answers `holds`, `fails`, or `inconclusive`.
+It answers `holds`, `fails`, or `inconclusive`. When what the assertion is
+about appears more than once, such as a total in a summary and on a pay
+button, every instance must agree: one that contradicts it fails. Name
+the place (`the order summary total is $42.00`) to judge only that one. A
+claim about some item, such as `a todo is marked done`, still holds when one
+item matches.
 
 Throws `ASSERTION_FAILED` on a false judgment and `ASSERTION_INCONCLUSIVE`
 when the screen did not show enough to decide. Both carry the explanation
@@ -271,7 +279,9 @@ Observes and judges at once. After a false or inconclusive judgment it
 waits out `interval`, then observes every 500 ms (or every `interval`, when
 shorter) without a model call, and judges again only once the screen
 differs from the one last judged, pixels included under `vision`. Succeeds
-on the first judgment that holds.
+on the first judgment that holds. Judgments follow the `agent.assert`
+rules, so a value shown in more than one place holds only once every
+instance agrees.
 
 Throws `STEP_TIMEOUT` on deadline expiry and `STEP_BUDGET_EXHAUSTED` on
 budget exhaustion, each carrying the last judgment (with the same vision

package/docs/reference/app.mdx

@@ -1,6 +1,7 @@
 ---
 title: app
 description: The portable app lifecycle fixture, available on every platform.
+icon: "cube"
 ---
 
 `app` controls the app's lifecycle and captures screenshots. Each method
@@ -43,7 +44,7 @@ timeout. They do not accept per-call timeout options.
 
 | Method | Timeout | Throws |
 | --- | --- | --- |
-| `open(path?)` | `config.timeout`, 120000 ms | `APP_URL_REQUIRED` for a path when the target declares no `app.url`, `POLICY_DENIED` for a `file:`, `data:`, or `javascript:` URL, `APP_UNREACHABLE`, `ACTION_FAILED`, `UNSUPPORTED_CAPABILITY` without `session.open` on a target with a URL or without `session.restart` on one without (a device launches its pinned app) |
+| `open(path?)` | `config.timeout`, 120000 ms | `APP_URL_REQUIRED` for a path when the target declares no `app.url`, `POLICY_DENIED` for a URL whose scheme is not `http:` or `https:` (`about:blank` aside), `APP_UNREACHABLE`, `ACTION_FAILED`, `UNSUPPORTED_CAPABILITY` without `session.open` on a target with a URL or without `session.restart` on one without (a device launches its pinned app) |
 | `restart()` | `config.timeout`, 120000 ms | `ACTION_FAILED`, `ENGINE_FAILURE`, `UNSUPPORTED_CAPABILITY` without `session.restart` |
 | `clearState()` | `config.timeout`, 120000 ms | `ACTION_FAILED`, `ENGINE_FAILURE`, `UNSUPPORTED_CAPABILITY` without `session.reset` |
 | `back()` | `config.actionTimeout`, 30000 ms | `APP_NOT_OPEN`, `ACTION_FAILED`, `UNSUPPORTED_CAPABILITY` without `session.back` |

package/docs/reference/cli.mdx

@@ -1,6 +1,7 @@
 ---
 title: CLI
 description: Commands, flags, output formats, and exit codes.
+icon: "terminal"
 ---
 
 <CodeGroup>
@@ -56,7 +57,7 @@ bunx e2e --version
 ```
 </CodeGroup>
 
-The CLI needs Node.js 22.12 or newer.
+The CLI needs Node.js 24.8 or newer, or 22.22.3 or newer on Node.js 22. Any package manager installs it (npm, pnpm, Yarn with or without Plug'n'Play, Bun); the CLI itself runs on Node.js, not on the Bun or Deno runtimes (`bunx e2e` runs it on Node.js). Under Yarn Plug'n'Play on Node.js 24.8 through 24.11.0, a `.cts` file fails to load (Node.js rejects the source Yarn's loader hands on); 24.11.1 and later load it.
 
 ## Help and version
 
@@ -96,9 +97,9 @@ For a new project, the wizard asks you to choose:
   **Mobile (iOS/Android)** adds `@e2e-dev/mobile`, defaults to iOS on
   macOS and Android elsewhere, and opens Settings. **None** adds nothing.
 - A model provider or subscription for agent steps. Choose Vercel AI Gateway,
-  OpenRouter, an OpenAI-compatible endpoint, ChatGPT, GitHub Copilot, or
-  SuperGrok. Choose None for tests that only use locators. The wizard adds
-  the required packages and model config. Subscription users then sign in
+  OpenRouter, an OpenAI-compatible endpoint, ChatGPT, GitHub Copilot,
+  OpenCode Console, or SuperGrok. Choose None for tests that only use
+  locators. The wizard adds the required packages and model config. Subscription users then sign in
   with `e2e login`. See [Models](/models) and [Subscriptions](/subscriptions).
 - Where to install the agent skill (see [e2e guide](#e2e-guide)):
   `.agents/skills/` for Codex, Cursor, Copilot, Gemini CLI, OpenCode, and
@@ -140,7 +141,10 @@ bunx e2e init --yes
 ```
 </CodeGroup>
 
-Re-running `init` keeps an existing config and adds no packages for it. Once
+Re-running `init` keeps an existing config and adds no packages for it. MCP
+registration keeps extra settings on the e2e server, such as environment
+variables and client timeouts, even when repairing its launch command. A
+current registration is left byte-for-byte unchanged. Once
 a skill directory exists, later runs refresh copies that differ from the
 installed version and never add a directory the project did not choose. A
 `.claude/skills/e2e/` copy from an earlier version becomes the symlink when
@@ -164,6 +168,7 @@ npx e2e login openai              # ChatGPT, browser sign-in
 npx e2e login openai --device     # sign in with a device code
 npx e2e login github-copilot --from-gh  # use your GitHub CLI login
 npx e2e login github-copilot --client-id <id> [--enterprise-url <host>]
+npx e2e login opencode-console    # OpenCode Console, device code
 npx e2e login spacexai            # SuperGrok, device code
 npx e2e login                     # choose a provider
 ```
@@ -173,6 +178,7 @@ pnpm exec e2e login openai              # ChatGPT, browser sign-in
 pnpm exec e2e login openai --device     # sign in with a device code
 pnpm exec e2e login github-copilot --from-gh  # use your GitHub CLI login
 pnpm exec e2e login github-copilot --client-id <id> [--enterprise-url <host>]
+pnpm exec e2e login opencode-console    # OpenCode Console, device code
 pnpm exec e2e login spacexai            # SuperGrok, device code
 pnpm exec e2e login                     # choose a provider
 ```
@@ -182,6 +188,7 @@ bunx e2e login openai              # ChatGPT, browser sign-in
 bunx e2e login openai --device     # sign in with a device code
 bunx e2e login github-copilot --from-gh  # use your GitHub CLI login
 bunx e2e login github-copilot --client-id <id> [--enterprise-url <host>]
+bunx e2e login opencode-console    # OpenCode Console, device code
 bunx e2e login spacexai            # SuperGrok, device code
 bunx e2e login                     # choose a provider
 ```
@@ -194,7 +201,7 @@ configuration.
 
 | Argument or flag | Type | Default | Behavior |
 | --- | --- | --- | --- |
-| `[provider]` | `openai`, `github-copilot`, or `spacexai` | a picker | The subscription to sign in to |
+| `[provider]` | `openai`, `github-copilot`, `opencode-console`, or `spacexai` | a picker | The subscription to sign in to |
 | `--device` | boolean | `false` | ChatGPT: show a code to enter on another device instead of opening a browser |
 | `--from-gh` | boolean | `false` | GitHub Copilot: reuse the token of the signed-in GitHub CLI |
 | `--client-id <id>` | string | none | GitHub Copilot: the client id of your GitHub OAuth App with the device flow enabled |
@@ -239,10 +246,11 @@ bunx e2e models openai    # one provider
 </CodeGroup>
 
 Asks the vendor which models the stored login serves and prints one line per
-model: the id `chatgpt()`, `copilot()`, or `grok()` takes, the vendor's name
-for it, and its notes (reasoning levels, vision, preview, hidden, and for
-Copilot `not enabled` or `no chat completions` when `copilot()` cannot call
-it). Without a
+model: the id `chatgpt()`, `copilot()`, `opencodeConsole()`, or `grok()`
+takes, the vendor's name for it, and its notes (reasoning levels, vision,
+preview, hidden, for Copilot `not enabled` or `no chat or responses` when
+`copilot()` cannot call it, and for OpenCode Console the plan, Zen or Go,
+that serves it). Without a
 stored login, or when the named provider has none, exits 1 with the sign-in
 command; when one of several logins fails to list, the others still print and
 the command exits 1. An unknown provider exits 2. See
@@ -277,15 +285,15 @@ See [Coding agents](/coding-agents).
 
 <CodeGroup>
 ```bash npm
-npx e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
+npx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
 ```
 
 ```bash pnpm
-pnpm exec e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
+pnpm exec e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
 ```
 
 ```bash bun
-bunx e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
+bunx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
 ```
 </CodeGroup>
 
@@ -342,6 +350,8 @@ A positional starting with `-` that names no file is a usage error, exit 2.
 This is what `pnpm test:e2e -- --headed` produces: the `--` reaches e2e and
 `--headed` would otherwise be taken for a file. The message names the direct
 form, `pnpm exec e2e run --headed`.
+For npm, the direct form is `npm exec -- e2e run --headed`; npm needs the
+separator before the binary to pass its flags through.
 
 When nothing is left to run, `NO_TESTS` names the cause: no file matched the
 globs (with look-alike files such as `tests/login.test.ts`), a positional
@@ -359,7 +369,7 @@ or every test was filtered out.
 | `--exclude-tag <tags>` | comma-separated or repeated | none | Leaves out every test carrying any of the tags, whatever `--tag` or a positional selected. A serial group still runs whole when any member is selected; tag the serial `describe` to exclude the flow. |
 | `--grep <pattern>` | regular expression, repeatable | none | Only tests whose title matches one of the patterns. The text matched is the describe titles and the test title joined by one space (`checkout pays`); the file and the tags are not part of it. A bare pattern is case-sensitive; `'/pattern/i'` adds flags. A pattern that does not compile is a usage error, exit 2. |
 | `--grep-invert <pattern>` | regular expression, repeatable | none | Leaves out every test whose title matches one of the patterns. Same text and pattern forms as `--grep`; both together keep the tests `--grep` matches minus the ones `--grep-invert` matches. |
-| `--last-failed` | boolean | `false` | Only the tests the previous run did not pass, read from the `report.json` this run overwrites, `<output>/report.json` (`--output`, else the config's [`output`](/reference/config#output)): failed, timed out, interrupted, or skipped because a setup, a serial predecessor, a hook, or the worker failed, and every test it selected in the describe or file, on that target, of a `beforeAll` or `afterAll` that failed (for a setup, the tests that consume its session). A rerun that runs the hook again reports it again if it still fails; another filter that leaves its tests out leaves the failure out of the next report too. A test is named per target and agent. No report is `NO_LAST_RUN`; a previous run with nothing failed selects nothing, `NO_TESTS` unless `--pass-with-no-tests`. |
+| `--last-failed` | boolean | `false` | Only the tests the previous run did not pass, read from the `report.json` this run overwrites, `<output>/report.json` (`--output`, else the config's [`output`](/reference/config#output)): failed, timed out, interrupted, or skipped because a setup, a serial predecessor, a hook, or the worker failed or `--max-failures` stopped the run first, and every test it selected in the describe or file, on that target, of a `beforeAll` or `afterAll` that failed (for a setup, the tests that consume its session). A rerun that runs the hook again reports it again if it still fails, and a rerun that runs any test of its scope without it failing resolves it. A test another filter (`--grep`, `--tag`, `--target`, `--agent`, a positional) leaves out of the rerun stays owed: the rerun's report carries its last result, and the hook failure whose scope it is in, under `run.carried`, the next `--last-failed` selects it again, and the [`@e2e-dev/github`](/github) page stays red until it passes (⏹️ for a test the run before interrupted). The rerun's own exit code and summary count only what it ran. It keeps the artifacts the previous report names and writes its own under `artifacts/rerun-<n>/`, so no attempt overwrites an earlier one's files. A test is named per target and agent. No report is `NO_LAST_RUN`; a previous run with nothing failed selects nothing, `NO_TESTS` unless `--pass-with-no-tests`. |
 | `--shard <index/total>` | `2/3` | none | Runs one contiguous slice of the selected tests, cut once every other filter applied. Slices differ in size by at most one test, a serial group counts as one and is never split, and each shard brings only the setup tests its own tests need. Every shard runs the same command with a different index; the order is the report order, so the same suite shards the same way on every machine. A shard that gets no test is `NO_TESTS` unless `--pass-with-no-tests`. See [Sharding](/ci#sharding). |
 | `--headed` | boolean | `false` | Visible UI when the engine supports it. |
 | `--agent <names>` | comma-separated or repeated | `default` | Runs unpinned tests once per selected agent. For a test with `{ agent }`, selects the shared names when there is overlap; otherwise keeps the test's selection. An unknown name is `INVALID_CONFIG`. |
@@ -368,7 +378,7 @@ or every test was filtered out.
 | `--repeat-each <n>` | integer, at least 1 | `1` | Runs every selected test this many times. Each run is a result of its own, with `repeat` 0 through `n - 1` and its own `id`; the list reporter titles the later runs `(repeat #1)`, `(repeat #2)`, and their artifacts go under a `repeat-<n>` directory. Setup tests run once. With the replay cache on, the first run records and the later ones replay, so pair it with `--no-cache` to exercise the model each time. The `list` summary adds a `Repeats` row (`1 of 2 tests passed all 5 runs`) and one line per test that did not pass every run, naming each run that failed; the `markdown` page carries the same tally. |
 | `--max-failures <n>` | integer, at least 1 | none | Stops the run once this many tests have failed or timed out. Nothing more is dispatched, the tests running on other workers end as `interrupted`, and the tests not started, the rest of the file whose test reached the limit included, are skipped with cause `failure-limit` and the reason `run stopped after 3 failures (--max-failures 3)`. The exit code is the failures' own, 1, not 130. |
 | `--reporter <ids>` | comma-separated list | config | Replaces the built-in IDs in `config.reporters`: `list`, `json`, `junit`, `markdown`. `json` with `list` is `INVALID_CONFIG`. [Reporter objects](/reference/reporters) in the config keep running. |
-| `--output <dir>` | string | config, else `.e2e` | The results directory for this run, over the config's [`output`](/reference/config#output): the report, the reporter files, the AI trace, `artifacts/` (cleared once the run's tests start), and `sessions/`. Resolved from the project root; one the run cannot own is `INVALID_CONFIG`. The removed `--artifacts <dir>` fails with the `--output` it maps to: `--artifacts out/artifacts` is `--output out`, and a directory whose parent is the project root or outside it maps to `--output .e2e`. |
+| `--output <dir>` | string | config, else `.e2e` | The results directory for this run, over the config's [`output`](/reference/config#output): the report, the reporter files, the AI trace, `artifacts/` (cleared once the run's tests start, down to what the previous report names on a `--last-failed` rerun), and `sessions/`. Resolved from the project root; one the run cannot own is `INVALID_CONFIG`. The removed `--artifacts <dir>` fails with the `--output` it maps to: `--artifacts out/artifacts` is `--output out`, and a directory whose parent is the project root or outside it maps to `--output .e2e`. |
 | `--no-cache` | boolean | `false` | Replay cache off for this run, whatever `config.cache` says. |
 | `--strict-cache` | boolean | `false` | Turns on `cache.strict` for this run: a step whose recording exists but no longer replays fails with `REPLAY_STALE` instead of handing off to the agent. See [Fail on a stale recording](/cache#fail-on-a-stale-recording). |
 | `--pass-with-no-tests` | boolean | `false` | Exit 0 instead of `NO_TESTS` when nothing is selected. |
@@ -479,8 +489,8 @@ bunx e2e run --reporter list,junit
 | --- | --- |
 | `list` | Terminal progress, failures with code frames, and a run summary. On a TTY, live progress includes the current agent step, model turns, tool calls, and reasoning excerpts. Untrusted fields are capped at 8 KiB and stripped of control characters. |
 | `json` | The report document to stdout and nothing else there, so `e2e run --reporter json \| jq` works. Same document as `<output>/report.json`, described by `schema/report-v1.schema.json` in the package. The shape changes only with the `schemaVersion` field (`report-1` today). Cannot be combined with `list`. |
-| `junit` | `junit.xml` beside the report. One `<testsuite>` per file, one `<testcase>` per test-target pair named `title > path [target]`. A test-class error is a `<failure>`, any other an `<error>`, a skip a `<skipped>` with its reason. Run-level errors form a `run` suite. |
-| `markdown` | `summary.md` beside the report with counts, cost, errors, and failure details. Failed and flaky tests also get a page under `failures/` there, with steps, model turns, and the screen at failure. Exploration runs show findings. [`@e2e-dev/github`](/github) uses this summary for pull request comments. |
+| `junit` | `junit.xml` beside the report. One `<testsuite>` per file, one `<testcase>` per test-target pair named `title > path [target]`. A test-class error is a `<failure>`, any other an `<error>`, a skip a `<skipped>` with its reason. An interrupted test reached no verdict, so it is a `<skipped>` whose message starts `interrupted:`. Run-level errors form a `run` suite. |
+| `markdown` | `summary.md` beside the report with counts, cost, errors, and failure details. Failed and flaky tests also get a page under `failures/` there, with steps, model turns, and the screen at failure. Interrupted tests are counted on their own and get no page. Exploration runs show findings. [`@e2e-dev/github`](/github) uses this summary for pull request comments. |
 
 Every run that gets to its tests writes `report.json` under the [output directory](/reference/config#output)
 (`.e2e` by default) whatever the reporters are, and the reporters' files
@@ -490,6 +500,16 @@ fails to start, an interrupt) writes none of them and leaves the previous
 run's files in place; its errors are on the terminal and in the exit code. A `junit.xml` or `summary.md` that could not be written is a
 line on stderr, never a run error.
 
+`run.summary` in the report counts the results. `discovered` is every
+test-target pair the runner collected and `selected` the ones this run
+chose; the selection's left-out pairs are in `results` as `selected: false`
+but in no other count. `passed`, `failed` (timed out included),
+`interrupted`, `flaky`, and `skipped` count the selected results and add up
+to `selected`. `interrupted` is a test the run stopped before its verdict
+(a signal, `--max-failures`, or a run-level error such as
+`MODEL_UNAVAILABLE`), never a failure. `executed` is the selected
+results that ran at least one attempt.
+
 With `--ai-trace`, the run also writes `<output>/ai-trace.json`: one entry per
 model round trip with the exact prompt, tool definitions, response, usage,
 and provider metadata. Observations in it are already redacted. Inline image
@@ -510,7 +530,10 @@ Signals escalate.
    any test starts (during collection, an engine's `prepare`, or app
    startup) writes no report and leaves the previous one in place.
 2. A second signal forces. Workers dispose their engines at once and are
-   killed when the cleanup budget is spent. The report is still written.
+   killed when the cleanup budget is spent. The report is still written,
+   and so are `junit.xml` and `summary.md` when the `junit` and
+   `markdown` reporters are on; a custom reporter's
+   `onRunFinished` is abandoned.
 3. A third signal exits on the spot, after killing the process groups of every
    app command the run started. A process attached through
    `reuseExisting` is left alone.

package/docs/reference/config.mdx

@@ -1,6 +1,7 @@
 ---
 title: E2EConfig
 description: Every configuration key, its default, its valid range, and the environment variables.
+icon: "sliders"
 ---
 
 Export an object from `e2e.config.ts` and use `satisfies E2EConfig` to check
@@ -69,10 +70,45 @@ It throws when the file is missing; guard it with `existsSync` from
 `node:fs` when only some machines have one.
 
 Config and test files run as ESM whatever the nearest `package.json` `type`
-says, and so does the TypeScript they import: a helper by path, or a
-workspace package that exports its `.ts` source by name. An installed package
-imported by name keeps its own format. A config or test written with `require`
-or `module.exports` fails to load.
+says, and so does every `.ts`, `.mts`, `.tsx`, and `.jsx` file they reach: a
+helper by path, a package that exports its `.ts` source by name, or a `.ts`
+file a CommonJS file requires. A `.cts` file is CommonJS, written with
+`require`, `import x = require()`, and `export =`; its `import` statements are
+not turned into `require` calls. A `.ts` file written with `require` or
+`module.exports` fails to load: name it `.cts`. JavaScript keeps the format
+its package declares.
+
+e2e compiles TypeScript and JSX itself, and an import written in such a file
+resolves the way TypeScript resolves it: `./helper.js` and `./helper` load
+`helper.ts`, `./view.jsx` loads `view.tsx`, `./lib` loads the file
+`lib/package.json` names as `main`, else `lib/index.ts`, a bare import that
+`tsconfig.json` maps through `paths` or `baseUrl` loads the mapped file, and
+a `#` import or package export that names a missing `./x.js`, `./x.jsx`, or
+`./x` loads its `x.ts` or `x.tsx` source when imported (a `require()` of it fails as Node.js reports
+it). An import written in a JavaScript file resolves
+as Node.js resolves it.
+
+The `tsconfig.json` is the nearest one above the file, or, when that one has
+`references` and does not include the file itself (as Vite writes it), the
+directly referenced one that does. e2e reads its `paths`, `baseUrl`, `jsx`, `jsxFactory`,
+`jsxFragmentFactory`, `jsxImportSource`, `experimentalDecorators`,
+`emitDecoratorMetadata`, `useDefineForClassFields` (and `target`, which sets
+its default), and `verbatimModuleSyntax`, once per process and again for each
+session `e2e mcp` opens. A `tsconfig.json` e2e cannot read, such as one that
+`extends` a package that is not installed, is skipped with a warning.
+TypeScript inside an installed package compiles without the project's
+`tsconfig.json`.
+
+e2e refuses, naming the file and line, two things it cannot compile:
+auto-accessors (`accessor x = 1`) and decorators without
+`experimentalDecorators`. Each file compiles on its own, without type
+information, so a type imported without `import type` fails to load when the
+import stays in the output (a decorated member's type under
+`emitDecoratorMetadata`, a re-exported type); the error names the import to
+change. A JSON file imported without `with { type: 'json' }` loads with its
+parsed contents as the default export. Stack traces and failure locations
+point at the TypeScript source, through another loader's source map too
+(`NODE_OPTIONS=--import tsx`).
 
 ## Top level
 
@@ -96,6 +132,8 @@ export interface E2EConfig {
   cleanupTimeout?: number;
   /** Retries per test, 0 through 10; default 1 in CI, else 0. */
   retries?: number;
+  /** Fail the run when a test skips itself after a soft failure or an earlier failed attempt; default false. */
+  failOnSkippedFailure?: boolean;
   /** Parallel workers, 1 through 1024; default 1 in CI, else half the cores. An engine may cap it lower. */
   workers?: number;
   /** `{ store }` hands every artifact to a host store as it is produced. */
@@ -158,6 +196,17 @@ Every numeric value must be a safe integer. `projectId` falls back to
 `package.json`. `workers` never exceeds what a target's engine declares. A
 targeted agent action is bounded by the smaller of `actionTimeout` and 15 s.
 
+## failOnSkippedFailure
+
+Set `failOnSkippedFailure: true` to fail the run with exit code 1 when a test
+skips after a soft assertion failure or skips a retry after an earlier failed
+attempt. The default is `false`.
+
+The test keeps its `skipped` status and reason. Its failures remain in the
+report, and the CLI shows them under `Skipped After Failure` with either
+setting. Clean skips, successful retries, and engine cleanup diagnostics do
+not trigger this policy. A skip still stops the body and does not retry.
+
 ## targets
 
 ```ts
@@ -267,8 +316,9 @@ it (no `app.command`) is `INVALID_CONFIG`. An `app.command` with neither
 
 ### The app's site
 
-A test or the agent may open any http(s) URL; `file:`, `data:`, and
-`javascript:` are `POLICY_DENIED`. The site of `url`, its registrable domain
+A test or the agent may open any http(s) URL, and `about:blank`; every
+other scheme (`file:`, `data:`, `javascript:`, `view-source:`, `blob:`) is
+`POLICY_DENIED`. The site of `url`, its registrable domain
 (`app.example.com` and `auth.example.com` are one site), scopes two things:
 the browser engine's [`headers`](/reference/web#options) reach the
 site and no other host, and child frames off the site stay out of
@@ -369,7 +419,7 @@ run.
 | `report.json` | The canonical report; `--last-failed` reads the one here |
 | `junit.xml`, `summary.md`, `failures/` | The `junit` and `markdown` reporters' files |
 | `ai-trace.json` | `--ai-trace` |
-| `artifacts/` | Screenshots, traces, videos, downloads, and logs, one directory per attempt. Cleared once a run's tests start, so it holds the latest run's evidence and nothing its report no longer names |
+| `artifacts/` | Screenshots, traces, videos, downloads, and logs, one directory per attempt. Cleared once a run's tests start, so it holds the latest run's evidence and nothing its report no longer names. A `--last-failed` rerun keeps the files the report it reruns names and writes its own attempts under `artifacts/rerun-<n>/` |
 | `sessions/` | The run's encrypted `test.setup` sessions, deleted when it ends |
 | `videos/<session>/` | [`e2e mcp`](/reference/mcp#recording) session recordings |
 
@@ -754,8 +804,9 @@ export interface ModelInstance {
 }
 ```
 
-Construct the model in your config with an AI SDK provider. For subscription
-sign-in, use the [`e2e/oauth/*` models](/subscriptions).
+Construct the model in your config with an AI SDK provider. You can use all
+providers that support tool calls, see [AI SDK providers](/models#ai-sdk-providers).
+For subscription sign-in, use the [`e2e/oauth/*` models](/subscriptions).
 
 | Gateway | Constructor | Package | Reads |
 | --- | --- | --- | --- |
@@ -798,7 +849,9 @@ Credentials and secrets are separate namespaces: `secrets.get(name)` returns a
 reports know that handle as `<name>.password` (`<secret:admin.password>`), so
 `credentials.admin` and `secrets.admin` can both exist; a `secrets` entry
 named exactly `admin.password` beside an `admin` credential is
-`INVALID_CONFIG`.
+`INVALID_CONFIG`. So are two credentials whose names map to one
+`E2E_USER_<NAME>_*` override, like `svc-a` and `svc_a` (see
+[Environment variables](/reference/environment)).
 
 ```ts
 /** Named test identity. The password remains an opaque Secret. */
@@ -824,7 +877,8 @@ export type SecretConfig = string | SecretProvider;
 
 An entry is a static value, or a provider called on every authorized fill (a
 vault lookup, a freshly computed TOTP). It is recorded in the config digest by
-name only, and an `E2E_SECRET_<NAME>` override wins over both. Every entry
+name only, and an `E2E_SECRET_<NAME>` override wins over both. Two entries whose
+names map to one variable, like `api-key` and `api_key`, are `INVALID_CONFIG`. Every entry
 must resolve to a string of at least 6 characters (code points) or a
 function; anything else is `INVALID_CONFIG`. The minimum exists for redaction: every occurrence
 of a value becomes `<secret:name>`, and a shorter value would take ordinary
@@ -877,7 +931,7 @@ See [Secrets that are not passwords](/authentication#secrets-that-are-not-passwo
 In `read-write` mode the replay cache records an `agent.act()` only after a
 later verification passes, such as a locator assertion or `agent.assert()`.
 `read-only` replays existing entries and writes none; `off` disables both.
-See [Caching agent steps](/cache).
+See [Caching](/cache).
 
 ```ts
 /**
@@ -932,7 +986,7 @@ instead of a hand-off to the agent; see
 `e2e init` adds `.e2e/cache/` to `.gitignore`, so committing entries is
 opt-in. Remove that line to share replays with CI and teammates; CI stays
 `read-only` unless `cache: 'read-write'` is set. See
-[Caching agent steps](/cache#commit-the-replay-cache).
+[Caching](/cache#commit-the-replay-cache).
 
 ## Run limits
 

package/docs/reference/engine.mdx

@@ -1,6 +1,7 @@
 ---
 title: e2e/engine
 description: Build an engine with observations, actions, lifecycle hooks, and fixtures.
+icon: "gear"
 ---
 
 Import engine types and helpers from `e2e/engine`. `defineEngine` validates
@@ -709,7 +710,11 @@ export type PointerAction =
       readonly direction: ScrollDirection;
       readonly momentum?: Momentum;
     }
-  | { readonly kind: 'swipeTo'; readonly target: ViewportPoint };
+  | {
+      readonly kind: 'swipeTo';
+      readonly target: ViewportPoint;
+      readonly durationMs?: number;
+    };
 
 /** Every pointer action kind; what an engine declares in `pointerActions`. */
 export type PointerActionKind = PointerAction['kind'];
@@ -923,6 +928,13 @@ export const RETRYABLE_ENGINE_ERROR_CODES: ReadonlySet<EngineErrorCode> = new Se
 | `INVALID_STATE` | `APP_NOT_OPEN` | no |
 | `ENGINE_FAILURE`, any non-`EngineError` | infrastructure `ENGINE_FAILURE` | no |
 
+An `OPERATION_TIMEOUT` on a locate or read that started with under 100 ms of
+a waiting call's deadline left, after one that completed, ends the wait
+instead: an action's or `waitFor`'s `LOCATOR_NOT_FOUND`, or an assertion's
+`ASSERTION_FAILED`, with the timeout as its cause. A negated assertion passes
+there if its negation had held long enough by the time the read was cut off, as
+`pollCondition` below describes.
+
 An `EngineError` a contributed fixture method throws is mapped by the table
 above, so a fixture over a session that ended reports `APP_NOT_OPEN` the way
 an observation would. A method can also fail a step with the runner's own
@@ -960,8 +972,8 @@ Use these helpers to keep behavior consistent across engines:
 | --- | --- |
 | `matchesText(actual, pattern)`, `toTextPattern(match, { exact? })`, `describePattern(pattern)` | `TextPattern` matching for `locate` and text matchers; `toTextPattern` throws `INVALID_ARGUMENT` for a `match` that is neither a string nor a `RegExp` |
 | `resolveExpression(expression, nodes, { selector? })` | The reference `locate`. Resolves one `LocatorExpression` over a semantic tree in document order, with `text` and `label` answering the innermost match when a container echoes a descendant's text. An engine whose tree is the whole truth (a device) calls it; a browser engine reproduces it. A `selector` needs the hook, a `frame` is `FRAME_NOT_FOUND` |
-| `urlMatches(current, expected, baseHref)` | `toHaveURL`-style comparison; an `expected` that is neither a string nor a `RegExp` throws `INVALID_ARGUMENT`, as in `toTextPattern` |
-| `pollCondition({ deadline, signal, negated, evaluate, onTimeout })`, `Deadline` | Assertion polling and the negation grace window |
+| `urlMatches(current, expected, baseHref, ignoreCase?)` | `toHaveURL`-style comparison, `ignoreCase` as on the locator matchers; an `expected` that is neither a string nor a `RegExp` throws `INVALID_ARGUMENT`, as in `toTextPattern` |
+| `pollCondition({ deadline, signal, negated, evaluate, onTimeout })`, `Deadline` | Assertion polling and the negation grace window. An `OPERATION_TIMEOUT` from an `evaluate` that started with under 100 ms of the deadline left, after one that completed, ends the poll: `onTimeout(cause)` builds the failure with it, and a negation passes if it had held long enough by the time that read was cut off. Bound each read `evaluate` makes by `deadline` (`context.operation(deadline.remaining())`): one on the action timeout can outlast the assertion's own |
 | `validateJsonValue(value, label, rules?)` | Checks a value is plain JSON: a top-level `undefined` passes; a non-finite number, a cycle, an object that is not a plain object or array, or a function, symbol, bigint, or `undefined` inside is `INVALID_ARGUMENT` naming `label`. A `Secret` is `POLICY_DENIED` unless `rules.onSecret` throws its own; `rules.maxDepth` caps the nesting |
 | `rejectUnknownKeys(label, value, keys, code?)` | Throws `INVALID_CONFIG` for the first own key of `value` outside `keys`, naming the nearest known key when one is a plausible typo (`web() has unknown key "viewprt"; did you mean "viewport"?`) and the known keys otherwise: the check the runner runs on its own config objects, for an engine factory's options. Pass `'INVALID_ARGUMENT'` as `code` for an object a test passes to a fixture method |
 | `rejectUnknownOptions(api, options, known, code?)` | `INVALID_ARGUMENT` (or `code`, `INVALID_LOCATOR` for a locator option) for an option key a fixture method or matcher does not take, so a JavaScript caller's unsupported option fails instead of running on the default. `options` must be a plain object or `undefined`; a class instance or array is refused, and non-enumerable own keys count as keys |
@@ -1151,16 +1163,16 @@ startedAt }` for `stopVideo` to hand back. A `stop` that throws or returns
 anything else is `ENGINE_FAILURE`, naming the provider and the lease. The
 target argument's type is exported as `ProviderRecordingTarget`.
 
-After a secret fill, the runner redacts every text entry in each trace
-archive and drops its screencast frames: every `screencast/` entry, each
-`screencast-frame` record in a `.trace` entry, and any image a record names.
-After `resolveSecret` returns, it redacts the trace the same way and keeps
-the frames: a trace records the options the engine opened the attempt with,
-and an engine-held secret is protected as text only.
+When the session knows a secret value, the runner redacts every text entry in
+each trace archive, so a value a test spelled into a URL or the options the
+engine opened the attempt with (`resolveSecret`) never stay in it. After a
+secret fill it also drops the screencast frames: every `screencast/` entry,
+each `screencast-frame` record in a `.trace` entry, and any image a record
+names.
 If redaction fails, it deletes the archives and records
 `TRACE_WITHHELD`. Videos are unmasked and carry `redaction: 'incomplete'`.
-A `download` an engine attaches is the app's bytes: after a secret fill or
-once `resolveSecret` returns, the runner rewrites a text one (`.txt`,
+A `download` an engine attaches is the app's bytes: when the session knows
+a secret value, the runner rewrites a text one (`.txt`,
 `.csv`, `.json`, `.html`) through the ledger and labels it `complete`; every
 other download keeps its bytes and carries `redaction: 'incomplete'`.
 
@@ -1185,8 +1197,8 @@ export interface EngineFixtureContext {
     /**
      * Resolves a navigation target against the base URL. Throws
      * `APP_URL_REQUIRED` when the target declares no URL and `POLICY_DENIED`
-     * for a `file:`, `data:`, or `javascript:` scheme, so a fixture never
-     * re-implements the rule the harness owns.
+     * for any scheme but `http:` and `https:` (`about:blank` aside), so a
+     * fixture never re-implements the rule the harness owns.
      */
     resolveUrl(url: string): string;
   };

package/docs/reference/environment.mdx

@@ -1,6 +1,7 @@
 ---
 title: Environment variables
 description: Every variable the runner and the first-party engines read, and what each one does.
+icon: "dollar-sign"
 ---
 
 These variables configure the runner and first-party packages. In the tables,
@@ -21,6 +22,7 @@ See [Models](/models) and [Subscriptions](/subscriptions).
 | Variable | Reading | Effect |
 | --- | --- | --- |
 | `E2E_OAUTH_CREDENTIALS` | value | The stored subscription logins as JSON, the contents of `oauth.json`, for a machine nobody can sign in on. The `e2e/oauth/*` models and `e2e models` read it in place of `$XDG_CONFIG_HOME/e2e/oauth.json`; `e2e login` and `e2e logout` refuse to change it, and refreshed tokens are not written back |
+| `OPENCODE_API_KEY` | value | An OpenCode Console service account key; `opencodeConsole()` uses it in place of the stored `opencode-console` login; `e2e models` still reads the stored login. OpenCode reads the same variable |
 | `XDG_CONFIG_HOME` | value | Directory of the stored logins, `$XDG_CONFIG_HOME/e2e/oauth.json`; default `~/.config/e2e/oauth.json` |
 
 ## Credentials and secrets
@@ -32,7 +34,13 @@ See [Models](/models) and [Subscriptions](/subscriptions).
 | `E2E_SECRET_<NAME>` | set | Replaces the value of `secrets.<name>` |
 
 `<NAME>` is the name uppercased with every character outside `A-Z0-9`
-replaced by `_`, so `power-user` becomes `E2E_USER_POWER_USER_USERNAME`. A
+replaced by `_`, so `power-user` becomes `E2E_USER_POWER_USER_USERNAME`. Two
+credentials, or two secrets, whose names map to the same `<NAME>` (`api-key`
+and `api_key` both read `E2E_SECRET_API_KEY`; `svc-a` and `svc_a` both read
+`E2E_USER_SVC_A_USERNAME` and `_PASSWORD`) are `INVALID_CONFIG` at config
+load, whether or not a variable is set, so one rotation never replaces two
+entries; the message names the entries and the variables. A credential and a
+secret may share a name, since their prefixes differ. A
 variable for a name absent from the config is ignored. An empty value is
 `INVALID_CONFIG`, and so is a password or secret shorter than 6 characters
 (code points): redaction rewrites every occurrence of a value, and one that
@@ -50,7 +58,6 @@ short would take ordinary text with it.
 | --- | --- | --- |
 | `NO_COLOR` | set | Plain CLI output. Colors are otherwise on for a TTY or in CI and off when piped |
 | `FORCE_COLOR` | set | Colored CLI output even when piped |
-| `NODE_OPTIONS` | value | Inspected for a TypeScript `--import` loader, which decides how the runner registers its own module hooks for `.ts` config and test files |
 | `TERM` | value | `dumb` draws the `e2e init` wordmark at rest, without animation, as in CI |
 | `GITHUB_SHA`, `GITHUB_HEAD_REF`, `GITHUB_REF_NAME` | value | The commit and branch the report records under `run.vcs` when `git` cannot read the project root. `GITHUB_HEAD_REF` wins over `GITHUB_REF_NAME` |
 
@@ -100,6 +107,7 @@ What is sent and how to read it is on the [Telemetry](/telemetry) page.
 | Variable | Reading | Effect |
 | --- | --- | --- |
 | `PLAYWRIGHT_BROWSERS_PATH` | value | Playwright's browser cache, honored by the engine's `prepare` hook |
+| `PLAYWRIGHT_SKIP_BROWSER_GC` | value | Defaults to `1` for the engine's browser install, so it keeps other browsers in the cache. Set `0` to let the install remove them |
 | `KERNEL_API_KEY` | value | The API key `kernel()` from `@e2e-dev/kernel` creates browsers with; unset, the browser lease fails. See [Kernel](/integrations/kernel) |
 | `EXPO_TOKEN` | value | The Expo access token `easSimulators()` from `@e2e-dev/eas` starts and stops simulator sessions with; unset, it uses the eas-cli login in `.expo/state.json` under `HOME` (`USERPROFILE` on Windows), and with neither the device lease fails. See [EAS Simulators](/integrations/eas) |
 | `HOME`, `USERPROFILE` | value | The home `easSimulators()` finds the eas-cli login under when `EXPO_TOKEN` is unset: `USERPROFILE` on Windows, `HOME` elsewhere, the OS user's home when unset |