e2e 0.21.1 → 0.21.2

package/docs/reference/test.mdx

@@ -6,9 +6,620 @@ icon: "flask-vial"
 "twitter:image": "https://e2e.tester.army/docs/images/og/reference/test.png?v=d1ad3b01c3"
 ---
 
-Declare tests, groups, and hooks synchronously while the test file is
-imported. Use `test.extend` to add fixtures and `test.skip` to skip a test at
-registration or during its body.
+`test` declares tests, setup tests, groups, and hooks. Declarations run
+synchronously while the test file is imported. Use `test.extend` to add
+fixtures and `test.skip` to skip a test at registration or during its body.
+
+```ts title="tests/checkout.e2e.ts"
+import { beforeEach, describe, expect, test } from 'e2e';
+
+describe('checkout', { tags: ['billing'] }, () => {
+  beforeEach(async ({ app }) => {
+    await app.open('/checkout');
+  });
+
+  test('pays with a saved card', { retries: 2 }, async ({ screen }) => {
+    await screen.getByRole('button', 'Pay').tap();
+    await expect(screen.getByText('Payment received')).toBeVisible();
+  });
+});
+```
+
+## Reference
+
+### Members
+
+| Member | Returns |
+| --- | --- |
+| [`test`](#test) | `TestCase` |
+| [`test.skip`](#test-skip) | `TestCase \| void` |
+| [`test.only`](#test-only) | `TestCase` |
+| [`test.setup`](#test-setup) | `TestCase` |
+| [`test.extend`](#test-extend) | `TestAPI` |
+| [`describe`](#describe) | `void` |
+| [Hooks](#hooks) | `void` |
+
+`describe` and the four hooks are also top-level exports, the same functions
+as `test.describe`, `test.beforeEach`, and the rest:
+`import { describe, beforeEach, test } from 'e2e'`.
+
+- A per-test hook imported from `e2e` is typed with the core fixtures.
+- `@e2e-dev/web` and `@e2e-dev/mobile` export `test`, `describe`, and all
+  four hooks typed with their `browser` or `device` fixture, so a test file
+  registers from one import.
+- A `test` you extended with your own fixtures registers hooks that see them
+  through `test.beforeEach`.
+
+Registering a test, group, or hook outside module evaluation throws
+`COLLECTION_ERROR`. The full declarations are under [Types](#types).
+
+#### `test`
+
+Registers one test, with or without [options](#options).
+
+```ts
+import { describe, test } from '@e2e-dev/web';
+import { expect } from 'e2e';
+
+test('app opens', async ({ app, browser }) => {
+  await app.open();
+  await expect(browser).toHaveURL('/');
+});
+
+test('checks out', { tags: ['billing'], retries: 2, timeout: 60_000 }, async ({ app }) => {
+  await app.open('/checkout');
+});
+```
+
+A test or group title is 1 through 512 UTF-8 bytes after NFC normalization,
+with no NUL character.
+
+| Throws | When |
+| --- | --- |
+| `COLLECTION_ERROR` | Two tests in one file share a title path: the group titles, then the test's own. |
+
+#### `test.skip`
+
+Registers a skipped test, or skips the running test.
+
+```ts
+test.skip('exports a PDF', async ({ app }) => {}); // at registration
+test.skip(organizations.length < 2, 'a single organization'); // inside a body
+test.skip('not on this tenant'); // inside a body, always
+```
+
+- A skipped test does not affect the exit code by default.
+- To skip at collection by option, use the [`skip`](#skip) option.
+- At registration it returns a `TestCase`; inside a body, `void`.
+
+Call `test.skip(condition, reason)` inside a test for a condition you learn
+at runtime, such as a disabled feature flag:
+
+```ts
+test('switches organizations', { session: 'admin' }, async ({ app, agent }) => {
+  const organizations = await listOrganizations();
+  test.skip(organizations.length < 2, 'the demo tenant has a single organization');
+  await app.open('/backend');
+  await agent.act(`switch to the organization "${organizations[1].name}"`);
+});
+```
+
+- When the condition is true, the call throws, as Playwright's does. Nothing
+  after the call runs.
+- With no failure before it, the test is reported `skipped` with `reason`,
+  with no retry.
+- Steps before it stay in the report as they ran.
+- A failed `expect.soft` before the call still fails the attempt, as in
+  Playwright. The attempt is `failed`, with the soft failures as its error and
+  the skip reason in its `skip`. It retries like any failure.
+- Hooks and fixtures still run their teardown.
+- When the condition is a precondition, call it first, before any step.
+- A failed attempt followed by a skip on retry leaves the test `skipped`. The
+  CLI lists it under `Skipped After Failure` with its original error. Set
+  [`failOnSkippedFailure: true`](/reference/config) to fail the run with exit
+  code 1 in that case. The default is `false`.
+
+| Throws | When |
+| --- | --- |
+| `INVALID_ARGUMENT` | Called in a setup test. A setup test cannot skip because other tests depend on its sessions. |
+| `COLLECTION_ERROR` | The conditional form, outside a test body. |
+
+#### `test.only`
+
+Registers one focused local test.
+
+```ts
+test.only('pays', async ({ app }) => {
+  await app.open('/checkout');
+});
+```
+
+Other tests are reported as `filtered`. A CI run with a focused test fails
+with `ONLY_IN_CI`.
+
+#### `test.setup`
+
+Registers a setup test that declares the sessions it saves, and must save
+each exactly once.
+
+```ts
+import { credentials } from 'e2e';
+
+test.setup('authenticate', { sessions: ['member'] }, async ({ app, screen, session }) => {
+  await app.open('/login');
+  await screen.getByLabel('Email').fill(credentials.user('member').username);
+  await screen.getByLabel('Password').fill(credentials.user('member').password);
+  await screen.getByRole('button', 'Sign in').tap();
+  await session.save('member');
+});
+```
+
+- `test.setup` must be top-level, not inside `describe`.
+- The [`sessions`](#sessions) option declares the names.
+- A test consumes a saved session with the [`session`](#session) option.
+- The setup body gets one extra fixture, `session`:
+
+```ts
+export interface SetupSession {
+  /** Saves state under a setup-declared name. */
+  save(name: string): Promise<void>;
+}
+```
+
+| Throws | When |
+| --- | --- |
+| `SESSION_CONTRACT` | At runtime, a declared session was not saved, or was saved twice. |
+
+#### `test.extend`
+
+Returns a new `test` typed with more fixtures. The one it was called on is
+unchanged.
+
+```ts
+import { test as base } from 'e2e';
+import type { Device } from '@e2e-dev/mobile';
+
+export const test = base.extend<{ device: Device }>(); // types only
+```
+
+- With no argument, `extend` refines only the types: it returns the same
+  runtime `test`, typed with an engine's contributed fixtures. They still
+  resolve from the target's engine at runtime. This types a device, desktop,
+  or web surface without a global `declare module` augmentation.
+- With definitions, it adds fixtures of your own. See
+  [Your own fixtures](#your-own-fixtures).
+
+#### `describe`
+
+Declares a group of tests, with or without options.
+
+```ts
+import { describe, test } from 'e2e';
+
+describe('checkout', { tags: ['smoke'], serial: true }, () => {
+  test('adds an item', async ({ screen }) => {});
+  test('pays', async ({ screen }) => {});
+});
+```
+
+- A group takes the [options](#options) a test takes, except `only`, plus
+  [`serial`](#serial).
+- The body is synchronous.
+
+| Throws | When |
+| --- | --- |
+| `COLLECTION_ERROR` | An `async` body. It is also a type error. |
+| `COLLECTION_ERROR` | `test.setup` inside the body. |
+
+### Hooks
+
+| Hook | Runs |
+| --- | --- |
+| [`beforeEach`](#beforeeach) | Before each test attempt |
+| [`afterEach`](#aftereach) | After each test attempt |
+| [`beforeAll`](#beforeall) | Once per suite instance |
+| [`afterAll`](#afterall) | After the scope's last test |
+
+Hook order follows scope nesting, not position in the file, so a hook
+declared after a test still applies to it.
+
+#### `beforeEach`
+
+Registers a test-attempt setup hook.
+
+```ts
+test.beforeEach(async ({ app }) => {
+  await app.open('/');
+});
+```
+
+- Runs outermost group first, then in declaration order within a group.
+- `beforeEach` and the body share the test timeout.
+- Receives the test fixtures.
+
+#### `afterEach`
+
+Registers a test-attempt teardown hook.
+
+```ts
+test.afterEach(async ({ app }) => {
+  await app.screenshot('after');
+});
+```
+
+- Runs innermost group first, in reverse declaration order, after a failure
+  or timeout too.
+- Each `afterEach` hook gets its own `cleanupTimeout` budget with working
+  fixtures, so teardown can still drive the app after the body timed out.
+- Receives the test fixtures.
+
+#### `beforeAll`
+
+Registers a suite-instance setup hook. It runs once per suite instance, on
+first entry into the scope.
+
+```ts
+beforeAll(async ({ platform }) => {
+  await seedDatabase(platform);
+});
+```
+
+The module is re-imported for every retry, serial group, and setup test, so
+`beforeAll` runs again in each. Its only fixture is `platform`.
+
+| Throws | When |
+| --- | --- |
+| `HOOK_FAILED` | The hook throws. The suite's tests are skipped with cause `hook-failed`. Inside a serial group it skips every remaining member and fails the group without a retry. |
+
+#### `afterAll`
+
+Registers a suite-instance teardown hook. It runs as soon as the scope's last
+test in the realm is done.
+
+```ts
+afterAll(async () => {
+  await resetDatabase();
+});
+```
+
+A realm that a failure discards runs `afterAll` for every scope whose
+`beforeAll` started. Its only fixture is `platform`. A throw is `HOOK_FAILED`,
+as for [`beforeAll`](#beforeall).
+
+#### Blocked event loop
+
+A test that blocks its worker's event loop (`while (true) {}` in a fixture,
+`beforeEach`, the body, or `afterEach`) stops every timer in that worker, so
+the worker cannot time it out.
+
+- When the worker has not responded `cleanupTimeout` (at least 5 s) after the
+  test timeout, the runner kills it.
+- That attempt is reported `timed-out` with `TEST_TIMEOUT` and no artifacts,
+  after the attempts before it, and is not retried.
+- The rest of its file runs on a new worker. In a serial group, the members
+  after it are skipped.
+- A `beforeAll` or `afterAll` that blocks still hangs the run.
+
+<Note>
+A worker with a debugger attached when the test starts (an editor's
+auto-attach, or `NODE_OPTIONS=--inspect=0`) is never killed this way, so a
+breakpoint can stay paused. A programmatic run with an inline config or
+in-memory tests runs in the runner's own process and has no worker to kill: a
+blocked event loop hangs it.
+</Note>
+
+### Options
+
+| Option | Type | Default |
+| --- | --- | --- |
+| [`timeout`](#timeout) | `number` | `config.timeout` |
+| [`retries`](#retries) | `number` | `config.retries` |
+| [`tags`](#tags) | `readonly string[]` | - |
+| [`skip`](#skip) | `boolean \| string` | - |
+| [`only`](#only) | `boolean` | - |
+| [`platforms`](#platforms) | `readonly string[]` | - |
+| [`requires`](#requires) | `readonly Capability[]` | - |
+| [`session`](#session) | `string` | - |
+| [`agentContext`](#agentcontext) | `string` | - |
+| [`agent`](#agent) | `string \| readonly string[]` | the run's agents |
+| [`trace`](#trace) | `RecordingMode` | the run's |
+| [`video`](#video) | `RecordingMode` | the run's |
+| [`serial`](#serial) | `boolean` | - |
+| [`sessions`](#sessions) | `readonly string[]` | required |
+
+`test` takes every option but `serial` and `sessions`. `describe` takes
+every option but `only` and `sessions`. `test.setup` requires `sessions` and
+takes no `session`, `only`, or `skip`.
+
+Resolution order: test, nearest group, outer groups, config, built-in
+default. Inside a [serial group](#serial), the group's `retries`, `trace`, and
+`video` apply.
+
+| Throws | When |
+| --- | --- |
+| `COLLECTION_ERROR` | An option key the call does not take (`{ timout: 1000 }`). The error names the nearest known key when it is a likely typo, and every key the call takes otherwise. |
+
+#### `timeout`
+
+The attempt deadline in milliseconds. Innermost wins.
+
+```ts
+test('imports a large file', { timeout: 300_000 }, async ({ app }) => {}); // default: config.timeout
+```
+
+`timeout` must be a positive safe integer.
+
+#### `retries`
+
+How many times a failed test is retried.
+
+```ts
+test('checks out', { retries: 2 }, async ({ app }) => {}); // default: config.retries
+```
+
+An integer from 0 through 10.
+
+#### `tags`
+
+Tags that `--tag` selects on. Every layer's tags are unioned.
+
+```ts
+describe('checkout', { tags: ['billing', 'smoke'] }, () => {});
+```
+
+Tags are distinct names, none blank, with no comma and no leading or
+trailing whitespace.
+
+#### `skip`
+
+Skips the test at collection. A string is the reason. Any truthy layer skips.
+
+```ts
+test('exports a PDF', { skip: 'PDF export is off on staging' }, async ({ app }) => {});
+```
+
+#### `only`
+
+Focuses the test locally. CI rejects it with `ONLY_IN_CI`.
+
+```ts
+test('pays', { only: true }, async ({ app }) => {});
+```
+
+#### `platforms`
+
+Platforms the test runs on. Other targets skip it. Innermost wins.
+
+```ts
+test('uses Face ID', { platforms: ['ios'] }, async ({ app }) => {});
+```
+
+#### `requires`
+
+Capabilities the target must have. A target that lacks one skips the test at
+selection. Innermost wins.
+
+```ts
+test('onboarding asks for the camera', { requires: ['native-app'] }, async ({ app, agent }) => {
+  await app.open();
+  await agent.assert('The app asks for camera access');
+});
+```
+
+Accepted values:
+
+- The harness capabilities of the target's engine: `observation`, `actions`,
+  `location`, `pointer`, `keyboard`, `state`, `artifacts`.
+- One per contributed fixture: `browser`, `device`. See
+  [the engine contract](/reference/engine#engine).
+- The kind of app the target declares: `browser` for an `app.url`,
+  `native-app` for an `app.bundleId` or `app.appPath`.
+
+#### `session`
+
+The session a setup test saved, restored before the body runs.
+
+```ts
+test('sees the dashboard', { session: 'member' }, async ({ app }) => {});
+```
+
+See [`test.setup`](#test-setup).
+
+#### `agentContext`
+
+Extra context for agent steps. Layers are concatenated outermost first.
+
+```ts
+describe('pricing', { agentContext: 'Prices are shown in EUR.' }, () => {});
+```
+
+#### `agent`
+
+The configured agent (`agents.<name>`) this test or group runs with, in
+place of the run's agents.
+
+```ts
+test('checks out', { agent: ['fast', 'smart'] }, async ({ agent }) => {}); // one result per agent
+```
+
+- A list runs the test once per agent named, as one result each.
+- Innermost wins. A call's own `agent` option wins over it.
+- `--agent` narrows a list to the names both name, and never overrides a pin
+  the flag does not name.
+- A setup test runs once per target, so it pins at most one agent.
+
+| Throws | When |
+| --- | --- |
+| `COLLECTION_ERROR` | At selection, before any test runs: a name that `agents` does not define. |
+
+#### `trace`
+
+Whether the test keeps a trace page, in place of the run's.
+
+```ts
+test('checkout', { trace: 'retain-on-failure' }, async ({ agent }) => {});
+```
+
+- Takes a [recording mode](/reference/config#recording), the same as the
+  config's `trace`: `off`, `on`, `retain-on-failure`, `on-first-retry`, or
+  `on-all-retries`.
+- A test's or group's own value wins over `--trace`, the target's, and the
+  config's. One test can record on a run that records nothing, or opt out of
+  one that records everything.
+
+#### `video`
+
+Which of the test's attempts record a video, in place of the run's.
+
+```ts
+test('checkout', { video: 'on' }, async ({ agent }) => {
+  await agent.act('Buy the first product in the catalog');
+});
+```
+
+- Takes the same [recording modes](/reference/config#recording) as `trace`
+  and the config's `video`.
+- A test's or group's own value wins over `--video`, the target's, and the
+  config's.
+
+| Throws | When |
+| --- | --- |
+| `UNSUPPORTED_ARTIFACT` | Before any test starts, naming the test: the mode records on some attempt and the target's engine cannot record video. |
+
+#### `serial`
+
+`describe` only. Runs the group as one ordered retry unit with shared app
+state.
+
+```ts
+describe('checkout', { serial: true }, () => {
+  test('adds an item', async ({ screen }) => {});
+  test('pays', async ({ screen }) => {});
+});
+```
+
+- The group also shares secret state: a secret fill withholds pixels for the
+  rest of the group's attempt. A retry starts fresh.
+- Members take the `trace` and `video` on the serial `describe` or a group
+  around it. The group records one video per group attempt.
+- A group attempt may not start because its file fails to load or its session
+  fails to launch. Then the first member fails with that error, and the rest
+  skip with cause `serial-predecessor-failed`. When the run is interrupted
+  during the launch, every member skips instead.
+- On a retry, the members keep the verdicts of the attempt before it.
+- The exit code follows the error's category, as for any other test.
+
+| Throws | When |
+| --- | --- |
+| `COLLECTION_ERROR` | A member sets `retries`, `trace`, `video`, `session`, `platforms`, `requires`, `skip`, `only`, or a nested `serial`. |
+
+#### `sessions`
+
+`test.setup` only. The session names the setup test must save.
+
+```ts
+test.setup('authenticate', { sessions: ['member', 'admin'] }, async ({ session }) => {});
+```
+
+A static, non-empty array of unique names. Each name matches
+`/^[A-Za-z0-9_.-]{1,128}$/`. See [`test.setup`](#test-setup).
+
+### Fixtures
+
+| Fixture | Type |
+| --- | --- |
+| [`agent`](/reference/agent) | `Agent` |
+| [`app`](/reference/app) | `App` |
+| [`screen`](/reference/screen) | `Screen` |
+| `platform` | `string` |
+| [`session`](#test-setup) | `SetupSession` |
+
+`agent`, `app`, and `screen` reach test bodies, `beforeEach`, and
+`afterEach`. `platform` reaches every hook too, and `session` only
+`test.setup` bodies. [`TestFixtures`](#testfixtures) describes each one.
+
+Everything else is a fixture the target's engine contributes, or one of
+[your own](#your-own-fixtures). `@e2e-dev/web` contributes
+[`browser`](/reference/web); a device engine contributes `device`.
+
+| Throws | When |
+| --- | --- |
+| `UNSUPPORTED_CAPABILITY` | A test reaches for a fixture the engine does not contribute. Declare `requires: ['browser']` to skip such tests at selection instead. |
+
+<Note>
+The fixtures above are lazy. Destructuring in the parameter list acquires
+them before the body's first statement. [Your own
+fixtures](#your-own-fixtures) are not lazy.
+</Note>
+
+#### Your own fixtures
+
+`test.extend(fixtures)` defines resources set up before each test and cleaned
+up afterward, including after a failure. In each fixture function, code before
+`await use(value)` sets up the resource. The test receives `value`, and code
+after `use` tears it down.
+
+```ts
+import { test as base } from '@e2e-dev/web';
+
+export const test = base.extend<{ workspace: Workspace }>({
+  workspace: async ({ browser }, use) => {
+    const workspace = await createWorkspace();
+    await signIn(browser, workspace);
+    await use(workspace); // the hooks and the body run here
+    await workspace.cleanup(); // teardown, after the body even when it failed
+  },
+});
+
+test('renames the workspace', async ({ workspace, screen }) => {
+  await screen.getByLabel('Name').fill(`${workspace.name} (renamed)`);
+});
+```
+
+A definition is typed against the fixtures of the `test` it extends, not its siblings in
+the same call. A fixture that needs another goes in a chained second
+`extend`: `base.extend({ a }).extend({ b })` gives `b` a fixture object that
+already has `a`.
+
+Per attempt:
+
+1. Each definition in the `extend` chain the test was registered through
+   runs in declaration order, after the engine's fixtures, until it calls
+   `use`. It runs whether or not the test names it. A fixture that opens a
+   page or changes app state does so for every test of that `test`. So a
+   file whose tests need different setups registers them through different
+   `extend` chains. Setup counts against the test `timeout`. A failure is
+   reported in phase `beforeEach`, before any `beforeEach` hook has run.
+2. `beforeEach` hooks, the body, then `afterEach` hooks run with the same
+   fixture object. The fixture set is the test's own chain: a hook that reads
+   `workspace` around a test registered through `base` fails with
+   `UNSUPPORTED_CAPABILITY`.
+3. Teardowns run in reverse order after the last `afterEach`, whether or not
+   the body passed, each with its own `cleanupTimeout` budget. An error there
+   fails a passing attempt in phase `afterEach` and joins `secondaryErrors`
+   after a body failure.
+
+| Mistake | Fails |
+| --- | --- |
+| A value that is not a plain object of functions, an empty name, a core name (`agent`, `app`, `screen`, `platform`, `session`), or a name an earlier `extend` in the chain defined | At import, `COLLECTION_ERROR` naming the fixture |
+| A name the target's engine contributes (`browser` on a Playwright target) | The attempt, in `beforeEach`, `TEST_SETUP_FAILED` |
+| A definition that returns without calling `use` | The attempt, in `beforeEach`, `TEST_SETUP_FAILED` |
+| A definition that calls `use` twice | The attempt, at teardown, `TEST_SETUP_FAILED` |
+| A definition that throws before `use` | The attempt, in `beforeEach`, with that error |
+| A definition that throws after `use` | The attempt, in `afterEach`, with that error |
+
+<Note>
+A body or hook that hits the test `timeout` is abandoned, not cancelled, so
+teardown starts while it may still be running. A setup abandoned this way
+that reaches `use` later gets an immediate resolve. Its teardown runs
+detached from the run, so what it allocated is still released. Later
+fixtures, hooks, and the test body do not start after an abandoned setup or
+hook finishes.
+</Note>
+
+## Types
+
+### `TestAPI`
 
 ```ts
 export interface TestAPI<Fixtures = TestFixtures> {
@@ -20,9 +631,10 @@ export interface TestAPI<Fixtures = TestFixtures> {
   skip(title: string, fn: TestFn<Fixtures>): TestCase;
   /**
    * Inside a running test: skips it when `condition` is true (or always,
-   * with no condition), ending the body there. The attempt is reported
-   * `skipped` with `reason`; steps that already ran stay in the report, and
-   * teardown still runs. A setup test cannot skip: its sessions are owed.
+   * with no condition) by throwing, so nothing after the call runs. The
+   * attempt is reported `skipped` with `reason`; steps that already ran stay
+   * in the report without deciding the status, and teardown still runs. A
+   * setup test cannot skip: its sessions are owed.
    * Outside a test body this form is `COLLECTION_ERROR`; use the `skip`
    * option to skip at collection.
    */
@@ -82,138 +694,7 @@ export const beforeAll: TestAPI['beforeAll'];
 export const afterAll: TestAPI['afterAll'];
 ```
 
-`describe` and the four hooks are also top-level exports, the same functions
-as `test.describe` and `test.beforeEach` and the rest:
-`import { describe, beforeEach, test } from 'e2e'`. A per-test hook imported
-from `e2e` is typed with the core fixtures. `@e2e-dev/web` and
-`@e2e-dev/mobile` export `test`, `describe`, and all four hooks typed with
-their `browser` or `device` fixture, so a test file registers from one
-import, and a `test` you extended with your own fixtures registers hooks
-that see them through `test.beforeEach`.
-
-Registering a test, group, or hook outside module evaluation throws
-`COLLECTION_ERROR`.
-
-A test or group title is 1 through 512 UTF-8 bytes after NFC normalization,
-with no NUL character. Two tests in one file with the same title path, the
-group titles and then the test's own, are a `COLLECTION_ERROR`.
-
-## Members
-
-A skipped test does not affect the exit code by default. With `test.only`, other tests
-are reported as `filtered`, and a CI run fails with `ONLY_IN_CI`.
-
-Call `test.skip(condition, reason)` inside a test for conditions you learn at
-runtime, such as a disabled feature flag. When the condition is true, the call
-throws: the body stops at that line, as Playwright's does. Nothing after the
-call runs. With no failure before it, the test is reported `skipped` with
-`reason`, with no retry.
-Steps before it stay in the report as they ran. A failed `expect.soft` before
-the call still fails the attempt, as in Playwright: the attempt is `failed`
-with the soft failures as its error and the skip reason in its `skip`, and it
-retries like any failure. Hooks and fixtures still run their teardown. When
-the condition is a precondition, call it first, before any step.
-
-A failed attempt followed by a skip on retry leaves the test `skipped`, and
-the CLI lists it under `Skipped After Failure` with its original error. Set
-[`failOnSkippedFailure: true`](/reference/config) to fail the run with exit
-code 1 in that case. The default is `false`.
-
-A setup test cannot skip because other tests depend on its sessions; the
-call fails with `INVALID_ARGUMENT`. Outside a test body, the conditional
-form is `COLLECTION_ERROR`. Use the `skip` option when declaring the test.
-
-```ts
-test('switches organizations', { session: 'admin' }, async ({ app, agent }) => {
-  const organizations = await listOrganizations();
-  test.skip(organizations.length < 2, 'the demo tenant has a single organization');
-  await app.open('/backend');
-  await agent.act(`switch to the organization "${organizations[1].name}"`);
-});
-```
-`test.setup` must be top-level, not inside `describe`. An `async`
-`describe` body is a type error and a `COLLECTION_ERROR`.
-
-```ts
-import { describe, test } from '@e2e-dev/web';
-import { expect } from 'e2e';
-
-test('app opens', async ({ app, browser }) => {
-  await app.open();
-  await expect(browser).toHaveURL('/');
-});
-
-test('checks out', { tags: ['billing'], retries: 2, timeout: 60_000 }, async ({ app }) => {
-  await app.open('/checkout');
-});
-
-describe('checkout', { tags: ['smoke'], serial: true }, () => {
-  test('adds an item', async ({ screen }) => {});
-  test('pays', async ({ screen }) => {});
-});
-```
-
-`test.setup` declares the sessions it saves, and must save each exactly once:
-
-```ts
-import { credentials } from 'e2e';
-
-test.setup('authenticate', { sessions: ['member'] }, async ({ app, screen, session }) => {
-  await app.open('/login');
-  await screen.getByLabel('Email').fill(credentials.user('member').username);
-  await screen.getByLabel('Password').fill(credentials.user('member').password);
-  await screen.getByRole('button', 'Sign in').tap();
-  await session.save('member');
-});
-```
-
-Session names must match `/^[A-Za-z0-9_.-]{1,128}$/` and be unique.
-Not saving a declared session, or saving one twice, throws `SESSION_CONTRACT`
-at runtime.
-
-```ts
-export interface SetupSession {
-  /** Saves state under a setup-declared name. */
-  save(name: string): Promise<void>;
-}
-```
-
-## Hooks
-
-`beforeEach` hooks run outermost group first, then in declaration order
-within a group. `afterEach` hooks run innermost first, in reverse declaration
-order, after a failure or timeout too. `beforeAll` runs once per suite
-instance on first entry into the scope; `afterAll` when the scope's last test
-in the realm finishes. Hook order follows scope nesting, not position in the
-file, so a hook declared after a test still applies to it.
-
-The module is re-imported for every retry, serial group, and setup test, so
-`beforeAll` runs again in each. A scope's `afterAll` runs as soon as its last
-test in that realm is done. A realm that a failure discards runs `afterAll`
-for every scope whose `beforeAll` started.
-
-`beforeEach` and the body share the test timeout. Each `afterEach` hook gets
-its own `cleanupTimeout` budget with working fixtures, so teardown can still
-drive the app after the body timed out.
-
-A test that blocks its worker's event loop (`while (true) {}` in a fixture,
-`beforeEach`, the body, or `afterEach`) stops every timer in that worker, so
-the worker cannot time it out. A `beforeAll` or `afterAll` that blocks still
-hangs the run. When the worker has not responded `cleanupTimeout` (at least 5 s) after
-the test timeout, the runner kills it. That attempt is reported `timed-out`
-with `TEST_TIMEOUT` and no artifacts, after the attempts before it, and is
-not retried. The rest of its file runs on a new worker. In a serial group,
-the members after it are skipped. A worker with a debugger attached when the
-test starts (an editor's auto-attach, or `NODE_OPTIONS=--inspect=0`) is
-never killed this way, so a breakpoint can stay paused. A programmatic run with an inline
-config or in-memory tests runs in the runner's own process and has no worker
-to kill: a blocked event loop hangs it.
-
-A throw inside `beforeAll` or `afterAll` is `HOOK_FAILED` and skips the
-suite's tests with cause `hook-failed`. Inside a serial group it skips every
-remaining member and fails the group without a retry.
-
-## Options
+### `TestOptions`
 
 ```ts
 export interface TestOptions {
@@ -244,9 +725,10 @@ export interface TestOptions {
    */
   agent?: string | readonly string[];
   /**
-   * Whether the test keeps a trace page, in place of the run's: the same
-   * modes as the config's `trace`. Innermost wins, over `--trace` too;
-   * inside a serial group the group's value applies.
+   * Which of the test's attempts keep a trace, in place of the run's: the
+   * same modes as the config's `trace`. Innermost wins, over `--trace` too;
+   * inside a serial group the group's value applies, since the group runs as
+   * one unit.
    */
   trace?: RecordingMode;
   /**
@@ -271,62 +753,7 @@ export interface SetupOptions extends Omit<TestOptions, 'session' | 'only' | 'sk
 }
 ```
 
-`timeout` must be a positive safe integer; `retries` must be an integer from
-0 through 10. Groups do not accept `only`. An option key the call does not
-take (`{ timout: 1000 }`) is `COLLECTION_ERROR`, naming the nearest known key
-when it is a likely typo and every key the call takes otherwise. `test.setup` requires `sessions`
-to be a static, non-empty array of unique names.
-
-Resolution order: test, nearest group, outer groups, config, built-in default.
-
-`requires` names capabilities of the target's engine, the harness ones
-(`observation`, `actions`, `location`, `pointer`, `keyboard`, `state`,
-`artifacts`) and one per contributed fixture (`browser`, `device`; see
-[the engine contract](/reference/engine#engine)), or the kind of app the
-target declares: `browser` for an `app.url`, `native-app` for an
-`app.bundleId` or `app.appPath`. A target that lacks one skips the test at
-selection:
-
-```ts
-test('onboarding asks for the camera', { requires: ['native-app'] }, async ({ app, agent }) => {
-  await app.open();
-  await agent.assert('The app asks for camera access');
-});
-```
-
-An `agent` name that `agents` does not define is a `COLLECTION_ERROR` at
-selection, before any test runs.
-
-`trace` and `video` take a [recording mode](/reference/config#recording):
-`off`, `on`, `retain-on-failure`, `on-first-retry`, or `on-all-retries`. A
-test's or group's own value wins over `--trace` / `--video`, the target's,
-and the config's, so one test can record on a run that records nothing, or
-opt out of one that records everything. A `video` mode set here is required:
-when it records on some attempt and the target's engine cannot record video,
-the run fails with `UNSUPPORTED_ARTIFACT` before any test starts, naming the
-test:
-
-```ts
-test('checkout', { video: 'on' }, async ({ agent }) => {
-  await agent.act('Buy the first product in the catalog');
-});
-```
-
-`serial: true` makes the group one ordered retry unit with shared app state
-and shared secret state: a secret fill withholds pixels for the rest of the
-group's attempt. A retry starts fresh. Inside it, `retries`, `trace`,
-`video`, `session`, `platforms`, `requires`, `skip`, `only`, and a nested
-`serial` on a member are collection errors. Set them on the group. Members
-take the `trace` and `video` on the serial `describe` or a group around it,
-and the group records one video per group attempt. A group attempt that
-cannot start, because its file fails to load or its session fails to launch,
-fails the first member with that error and skips the rest with cause
-`serial-predecessor-failed`. When the run is interrupted during the launch,
-every member skips instead. On a retry, the members keep the verdicts of
-the attempt before it. The exit code follows the error's category, as for any
-other test.
-
-## Fixtures
+### `TestFixtures`
 
 ```ts
 export interface TestFixtures {
@@ -351,41 +778,7 @@ export interface SuiteFixtures {
 }
 ```
 
-Everything else is a fixture the target's engine contributes, or one of
-[your own](#your-own-fixtures). `@e2e-dev/web` contributes
-[`browser`](/reference/web) and exports a `test` typed with it; a device
-engine contributes `device`. Reaching for a fixture the engine does not
-contribute fails with `UNSUPPORTED_CAPABILITY`; declare `requires: ['browser']`
-to skip such tests at selection instead.
-
-The fixtures above are lazy. Destructuring in the parameter list acquires
-them before the body's first statement. Your own fixtures are not: every
-fixture in the `extend` chain a test was registered through sets up for that
-test, even when its parameters omit it.
-
-### Your own fixtures
-
-`test.extend(fixtures)` defines resources set up before each test and cleaned
-up afterward, including after a failure. In each fixture function, code before
-`await use(value)` sets up the resource. The test receives `value`, and code
-after `use` tears it down.
-
-```ts
-import { test as base } from '@e2e-dev/web';
-
-export const test = base.extend<{ workspace: Workspace }>({
-  workspace: async ({ browser }, use) => {
-    const workspace = await createWorkspace();
-    await signIn(browser, workspace);
-    await use(workspace); // the hooks and the body run here
-    await workspace.cleanup(); // teardown, after the body even when it failed
-  },
-});
-
-test('renames the workspace', async ({ workspace, screen }) => {
-  await screen.getByLabel('Name').fill(`${workspace.name} (renamed)`);
-});
-```
+### `FixtureDefinitions`
 
 ```ts
 /**
@@ -410,47 +803,7 @@ export type FixtureDefinitions<Base, Extra> = {
 };
 ```
 
-`extend` returns a new `test`; the one it was called on is unchanged. A
-definition is typed against the fixtures of the `test` it extends, not its
-siblings in the same call. A fixture that needs another goes in a chained
-second `extend`: `base.extend({ a }).extend({ b })` gives `b` a fixture
-object that already has `a`.
-
-Per attempt:
-
-1. Each definition runs in declaration order until it calls `use`, whether
-   or not the test names it: a fixture that opens a page or changes app
-   state does so for every test of that `test`, so a file whose tests need
-   different setups registers them through different `extend` chains. Setup
-   counts against the test `timeout`; a failure is reported in phase
-   `beforeEach`, before any `beforeEach` hook has run.
-2. `beforeEach` hooks, the body, then `afterEach` hooks run with the same
-   fixture object. The fixture set is the test's own chain: a hook that reads
-   `workspace` around a test registered through `base` fails with
-   `UNSUPPORTED_CAPABILITY`.
-3. Teardowns run in reverse order after the last `afterEach`, whether or not
-   the body passed, each with its own `cleanupTimeout` budget. An error there
-   fails a passing attempt in phase `afterEach` and joins `secondaryErrors`
-   after a body failure.
-
-What fails, and where:
-
-| Mistake | Fails |
-| --- | --- |
-| A value that is not a plain object of functions, an empty name, a core name (`agent`, `app`, `screen`, `platform`, `session`), or a name an earlier `extend` in the chain defined | At import, `COLLECTION_ERROR` naming the fixture |
-| A name the target's engine contributes (`browser` on a Playwright target) | The attempt, in `beforeEach`, `TEST_SETUP_FAILED` |
-| A definition that returns without calling `use` | The attempt, in `beforeEach`, `TEST_SETUP_FAILED` |
-| A definition that calls `use` twice | The attempt, at teardown, `TEST_SETUP_FAILED` |
-| A definition that throws before `use` | The attempt, in `beforeEach`, with that error |
-| A definition that throws after `use` | The attempt, in `afterEach`, with that error |
-
-A body or hook that hits the test `timeout` is abandoned, not cancelled, so
-teardown starts while it may still be running. A setup abandoned this way
-that reaches `use` later gets an immediate resolve, and its teardown runs
-detached from the run, so what it allocated is still released.
-Later fixtures, hooks, and the test body do not start after an abandoned setup or hook finishes.
-
-## Function types
+### Function types
 
 ```ts
 /** A test body. */