e2e 0.21.1 → 0.21.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (133) hide show
  1. package/README.md +2 -0
  2. package/dist/.build.json +1 -1
  3. package/dist/agent/act-validation.js +1 -0
  4. package/dist/agent/act.js +7 -3
  5. package/dist/agent/action-dispatcher.js +30 -8
  6. package/dist/agent/actions.d.ts +2 -0
  7. package/dist/agent/actions.js +5 -2
  8. package/dist/agent/error.js +1 -0
  9. package/dist/agent/executor.d.ts +14 -6
  10. package/dist/agent/index.js +5 -1
  11. package/dist/agent/invocation.d.ts +3 -0
  12. package/dist/agent/model/sdk.d.ts +5 -0
  13. package/dist/agent/model/sdk.js +18 -1
  14. package/dist/agent/observation.js +1 -0
  15. package/dist/agent/primitives.js +14 -3
  16. package/dist/agent/replay.js +9 -3
  17. package/dist/agent/step-cache.d.ts +3 -2
  18. package/dist/agent/step-cache.js +7 -4
  19. package/dist/agent/tool-loop.js +4 -1
  20. package/dist/cache/context.js +1 -1
  21. package/dist/cache/recorder.d.ts +2 -1
  22. package/dist/cache/recorder.js +11 -2
  23. package/dist/cache/trace-fields.d.ts +10 -0
  24. package/dist/cache/trace-fields.js +17 -0
  25. package/dist/cache/trace.d.ts +9 -1
  26. package/dist/cache/trace.js +26 -7
  27. package/dist/cli/feedback.js +2 -1
  28. package/dist/cli/init/sibling-versions.json +1 -1
  29. package/dist/collect/select.js +12 -2
  30. package/dist/config/agent.d.ts +1 -1
  31. package/dist/config/diagnose.js +9 -3
  32. package/dist/config/esm-hooks.js +3 -2
  33. package/dist/config/resolve.js +3 -2
  34. package/dist/config/typescript.js +52 -2
  35. package/dist/engine/contract.d.ts +3 -2
  36. package/dist/engine/manifest.js +2 -2
  37. package/dist/internal/errors.js +5 -2
  38. package/dist/internal/source.js +2 -0
  39. package/dist/internal/unserved-model.d.ts +5 -0
  40. package/dist/internal/unserved-model.js +22 -0
  41. package/dist/locator/screen.js +1 -2
  42. package/dist/oauth/copilot.js +23 -5
  43. package/dist/oauth/device-code.js +35 -19
  44. package/dist/oauth/errors.d.ts +3 -0
  45. package/dist/oauth/errors.js +26 -11
  46. package/dist/oauth/providers/github-copilot.d.ts +4 -0
  47. package/dist/oauth/providers/github-copilot.js +64 -16
  48. package/dist/oauth/providers/openai.d.ts +2 -2
  49. package/dist/oauth/providers/openai.js +47 -19
  50. package/dist/oauth/providers/opencode-console.js +36 -15
  51. package/dist/oauth/providers/xai.js +18 -9
  52. package/dist/oauth/token-endpoint.d.ts +15 -6
  53. package/dist/oauth/token-endpoint.js +18 -6
  54. package/dist/report/code-frame.js +9 -3
  55. package/dist/report/failure-text.d.ts +10 -2
  56. package/dist/report/failure-text.js +24 -8
  57. package/dist/report/list-model.d.ts +13 -0
  58. package/dist/report/list-steps.d.ts +6 -1
  59. package/dist/report/list-steps.js +29 -0
  60. package/dist/report/list.d.ts +1 -0
  61. package/dist/report/list.js +16 -7
  62. package/dist/report/live-window.js +2 -1
  63. package/dist/report/markdown.js +1 -1
  64. package/dist/run/fixtures.js +2 -0
  65. package/dist/run/host-trace.d.ts +3 -2
  66. package/dist/run/host-trace.js +2 -1
  67. package/dist/run/last-run.d.ts +9 -0
  68. package/dist/run/last-run.js +28 -0
  69. package/dist/run/open-session.d.ts +3 -1
  70. package/dist/run/open-session.js +1 -1
  71. package/dist/run/runner.js +6 -2
  72. package/dist/run/sessions.js +3 -3
  73. package/dist/run/standalone-attempt.d.ts +2 -1
  74. package/dist/run/standalone-attempt.js +3 -1
  75. package/dist/run/steps.d.ts +3 -1
  76. package/dist/run/units.d.ts +8 -0
  77. package/dist/run/units.js +14 -0
  78. package/dist/run/worker-models.d.ts +2 -0
  79. package/dist/run/worker-models.js +7 -0
  80. package/dist/types.d.ts +2 -2
  81. package/docs/authentication.mdx +26 -23
  82. package/docs/browser.mdx +7 -4
  83. package/docs/bug-bash.mdx +2 -2
  84. package/docs/cache.mdx +6 -1
  85. package/docs/ci/bitrise.mdx +1 -1
  86. package/docs/ci/codemagic.mdx +2 -2
  87. package/docs/ci/eas-workflows.mdx +1 -1
  88. package/docs/coding-agents.mdx +7 -35
  89. package/docs/debugging.mdx +24 -36
  90. package/docs/executors.mdx +1 -1
  91. package/docs/github.mdx +2 -1
  92. package/docs/integrations/eas.mdx +1 -1
  93. package/docs/integrations/index.mdx +6 -0
  94. package/docs/integrations/lightpanda.mdx +192 -0
  95. package/docs/integrations/testmu.mdx +152 -0
  96. package/docs/mcp.mdx +123 -0
  97. package/docs/migrate/detox.mdx +1 -1
  98. package/docs/migrate/maestro.mdx +101 -105
  99. package/docs/migrate/playwright.mdx +65 -70
  100. package/docs/mobile.mdx +18 -3
  101. package/docs/models.mdx +4 -2
  102. package/docs/quickstart.mdx +7 -0
  103. package/docs/reference/agent.mdx +369 -277
  104. package/docs/reference/app.mdx +141 -32
  105. package/docs/reference/cli.mdx +674 -558
  106. package/docs/reference/config.mdx +1406 -752
  107. package/docs/reference/engine.mdx +1062 -256
  108. package/docs/reference/environment.mdx +156 -84
  109. package/docs/reference/errors.mdx +106 -72
  110. package/docs/reference/expect.mdx +752 -372
  111. package/docs/reference/mcp.mdx +366 -141
  112. package/docs/reference/mobile.mdx +844 -403
  113. package/docs/reference/reporters.mdx +312 -106
  114. package/docs/reference/runner.mdx +373 -174
  115. package/docs/reference/screen.mdx +931 -398
  116. package/docs/reference/security.mdx +106 -40
  117. package/docs/reference/test.mdx +626 -273
  118. package/docs/reference/web.mdx +956 -533
  119. package/docs/security.mdx +6 -5
  120. package/docs/snippets/model-provider.jsx +763 -0
  121. package/docs/snippets/setup-prompt-card.jsx +22 -0
  122. package/docs/snippets/show-more.jsx +22 -0
  123. package/docs/snippets/video-card.jsx +34 -0
  124. package/docs/snippets/video.jsx +25 -0
  125. package/docs/subscriptions.mdx +10 -3
  126. package/docs/telemetry.mdx +39 -39
  127. package/package.json +3 -2
  128. package/schema/report-v1.schema.json +6 -0
  129. package/skills/e2e/references/agent.md +8 -3
  130. package/skills/e2e/references/debugging.md +1 -1
  131. package/skills/e2e/references/mcp.md +1 -1
  132. package/skills/e2e/references/setup.md +12 -3
  133. package/skills/e2e/references/writing-tests.md +1 -1
@@ -6,13 +6,12 @@ icon: "file-lines"
6
6
  "twitter:image": "https://e2e.tester.army/docs/images/og/reference/runner.png?v=98c298d3ed"
7
7
  ---
8
8
 
9
- `e2e/runner` runs a suite from code without invoking the CLI. `run` executes tests and returns their report; `list` returns a selection. `openHost` prepares a target for your test runner, and `host.openSession` opens a fresh attempt for each test. Importing `e2e` from a test file does not load the runner.
9
+ `e2e/runner` runs a suite from code without invoking the CLI. `run` executes
10
+ tests and returns their report, `list` returns a selection, and `openHost`
11
+ hosts e2e sessions in another test runner. Importing `e2e` from a test file
12
+ does not load the runner.
10
13
 
11
- ## run
12
-
13
- `run(options?: RunOptions)` executes a complete suite and returns `Promise<RunResult>`. It finishes cleanup before resolving and never exits the caller's process.
14
-
15
- ```ts
14
+ ```ts title="scripts/run-e2e.ts"
16
15
  import type { E2EConfig } from 'e2e';
17
16
  import { run } from 'e2e/runner';
18
17
  import { web } from '@e2e-dev/web';
@@ -33,23 +32,47 @@ const result = await run({
33
32
  console.log(result.status, result.report.run.summary);
34
33
  ```
35
34
 
36
- ### Execution options
35
+ ## Reference
37
36
 
38
- `run` accepts these options. Put execution settings, such as retries and workers, in the config.
37
+ ### Functions
39
38
 
40
- | Option | Meaning |
39
+ | Function | Returns |
41
40
  | --- | --- |
42
- | `cwd` | Project directory. Defaults to the current working directory. |
43
- | `config` | Config file path or an `E2EConfig` value. Omit it to discover `e2e.config.ts`. |
44
- | `files` | File selection: paths, globs, bare names, or `file:line`. Omit it to run the configured suite. |
45
- | `interruptSignal` | Abort to interrupt tests and execute teardown. |
46
- | `onEvent` | Receives typed progress events. Must not block; a throwing or rejecting callback is disabled without failing the run. |
41
+ | [`run`](#run) | `Promise<RunResult>` |
42
+ | [`list`](#list) | `Promise<ListResult>` |
43
+ | [`openHost`](#openhost) | `Promise<E2EHost<Fixtures>>` |
44
+ | [`isE2EError`](#ise2eerror) | `boolean` |
45
+
46
+ #### `run`
47
+
48
+ `run(options?: RunOptions)` executes a complete suite. It finishes cleanup
49
+ before resolving and never exits the caller's process.
50
+
51
+ ```ts
52
+ const result = await run({ files: ['tests/signup.e2e.ts'] });
53
+ process.exitCode = result.exitCode;
54
+ ```
55
+
56
+ ##### Options
47
57
 
48
- Runtime config runs with one worker in the caller's process. A file config supports the configured worker processes. Terminal progress is disabled; use `onEvent` for progress.
58
+ | Option | Example | Type | Default |
59
+ | --- | --- | --- | --- |
60
+ | `cwd` | `'apps/web'` | `string` | the current working directory |
61
+ | `config` | `'e2e.ci.config.ts'` | `string \| E2EConfig` | discovered `e2e.config.ts` |
62
+ | `files` | `['tests/signup.e2e.ts:12']` | `readonly string[]` | the configured suite |
63
+ | `interruptSignal` | `controller.signal` | `AbortSignal` | - |
64
+ | `onEvent` | `(event) => {}` | `RunEventSink` | - |
49
65
 
50
- Helper modules your test files import stay cached between `run` calls in the same process, so edits to them need a new process.
66
+ - `files` takes paths, globs, bare names, or `file:line`.
67
+ - Aborting `interruptSignal` interrupts tests and executes teardown.
68
+ - `onEvent` receives typed progress events. Terminal progress is off. It
69
+ must not block. A throwing or rejecting callback is disabled without
70
+ failing the run.
71
+ - Put execution settings, such as retries and workers, in the config.
72
+ - A runtime config (an `E2EConfig` value) runs with one worker in the
73
+ caller's process. A file config supports the configured worker processes.
51
74
 
52
- ### Completed result
75
+ ##### Result
53
76
 
54
77
  | Field | Meaning |
55
78
  | --- | --- |
@@ -59,15 +82,27 @@ Helper modules your test files import stay cached between `run` calls in the sam
59
82
  | `reportPath` | Absolute path to `report.json`, or `undefined` if it could not be written. |
60
83
  | `aiTracePath` | AI trace path, or `undefined` when no AI trace was written. |
61
84
 
62
- Test failures, config failures, and interruption resolve with their status and errors in the report. Overlapping suites reject with `RUNNER_BUSY`. Close every host before running a suite, and await the suite before opening a host.
85
+ Test failures, config failures, and interruption resolve with their status
86
+ and errors in the report.
63
87
 
64
- Each run that returns a report sends one anonymous [`e2e_run_completed`](/telemetry) event, with the same opt-outs as the CLI. The first call can print the telemetry notice on stderr.
88
+ | Throws | When |
89
+ | --- | --- |
90
+ | `RUNNER_BUSY` | Another suite is running, or a host is open. Close every host before running a suite, and await the suite before opening a host. |
65
91
 
66
- Suite output uses UTF-8 text redaction and does not preserve binary data. Your application's logs and progress callback output stay under your control.
92
+ - Helper modules your test files import stay cached between `run` calls in
93
+ one process. An edit to them needs a new process.
94
+ - Each run that returns a report sends one anonymous
95
+ [`e2e_run_completed`](/telemetry) event, with the same opt-outs as the CLI.
96
+ The first call can print the telemetry notice on stderr.
97
+ - Suite output uses UTF-8 text redaction and does not preserve binary data.
98
+ Your application's logs and progress callback output stay under your
99
+ control.
67
100
 
68
- ## list
101
+ #### `list`
69
102
 
70
- `list` collects and selects the way [`e2e list`](/reference/cli#e2e-list) does, and stops there. No app process, engine, or worker starts, and nothing is written.
103
+ `list(options?: ListOptions)` collects and selects the way
104
+ [`e2e list`](/reference/cli#e2e-list) does, and stops there. No app process,
105
+ engine, or worker starts, and nothing is written.
71
106
 
72
107
  ```ts
73
108
  import { isE2EError, list } from 'e2e/runner';
@@ -82,29 +117,49 @@ for (const pair of pairs) {
82
117
  }
83
118
  ```
84
119
 
85
- ### Options
86
-
87
- `list` takes one object. Omit `config` to discover `e2e.config.ts` from `cwd`, the same walk `e2e list` does.
88
-
89
- | Option | Meaning |
90
- | --- | --- |
91
- | `config` | A config file path, or an `E2EConfig` value in place of a file. |
92
- | `targets` | Target names (`--target`). An unknown name is `UNKNOWN_TARGET`. |
93
- | `cwd` | Project directory. Defaults to the current working directory. |
94
- | `files` | Positional file selection: paths, globs, bare names, and `file:line`. |
95
- | `tags`, `tagMode`, `excludeTags` | The `--tag`, `--tag-mode`, and `--exclude-tag` filters. |
96
- | `grep`, `grepInvert` | `RegExp` arrays. A test's describe titles and title, joined by spaces, must match one `grep` pattern and no `grepInvert` pattern. |
97
- | `lastFailed` | Only tests the previous run did not pass, read from `report.json` under `output`. |
98
- | `shard` | `{ index, total }`, the `--shard index/total` slice. |
99
- | `passWithNoTests` | Resolve with no runnable test instead of throwing `NO_TESTS`. |
100
- | `output` | Results directory, over the config's `output`. `lastFailed` reads `report.json` there. |
101
- | `env` | Environment the config resolves against. Defaults to `process.env`. |
102
-
103
- ### Result
120
+ Each call imports the test files again, so a later `list()` in the same
121
+ process sees an edit.
122
+
123
+ ##### Options
124
+
125
+ | Option | Example | Type | Default |
126
+ | --- | --- | --- | --- |
127
+ | `config` | `'e2e.ci.config.ts'` | `string \| E2EConfig` | discovered from `cwd`, the same walk `e2e list` does |
128
+ | `targets` | `['web']` | `readonly string[]` | - |
129
+ | `cwd` | `'apps/web'` | `string` | the current working directory |
130
+ | `files` | `['tests/', 'signup.e2e.ts:12']` | `readonly string[]` | the configured suite |
131
+ | `tags`, `tagMode`, `excludeTags` | `{ tags: ['smoke'], tagMode: 'all' }` | `readonly string[]`, `'any' \| 'all'` | - |
132
+ | `grep`, `grepInvert` | `[/checkout/]` | `readonly RegExp[]` | - |
133
+ | `lastFailed` | `true` | `boolean` | - |
134
+ | `shard` | `{ index: 1, total: 4 }` | `{ index, total }` | - |
135
+ | `passWithNoTests` | `true` | `boolean` | - |
136
+ | `output` | `'.e2e'` | `string` | the config's `output` |
137
+ | `env` | `{ ...process.env, CI: '1' }` | `NodeJS.ProcessEnv` | `process.env` |
138
+
139
+ - `targets` names targets, as `--target` does.
140
+ - `files` is the positional file selection: paths, globs, bare names, and
141
+ `file:line`.
142
+ - `tags`, `tagMode`, and `excludeTags` are the `--tag`, `--tag-mode`, and
143
+ `--exclude-tag` filters.
144
+ - `grep` and `grepInvert`: a test's describe titles and title, joined by
145
+ spaces, must match one `grep` pattern and no `grepInvert` pattern.
146
+ - `lastFailed` keeps only tests the previous run did not pass, read from
147
+ `report.json` under `output`.
148
+ - `shard` is the `--shard index/total` slice.
149
+ - `passWithNoTests` resolves with no runnable test instead of throwing
150
+ `NO_TESTS`.
151
+ - `output` is the results directory.
152
+ - `env` is the environment the config resolves against.
153
+
154
+ ##### Result
104
155
 
105
156
  `ListResult` is `{ pairs, unmatched, targets }`.
106
157
 
107
- `pairs` is one `ListedPair` per test, target, and agent, in selection order, including pairs a filter removed. Repeats are not expanded. `unmatched` is the positional arguments, as written, that selected no discovered file. `targets` is the selected target names, in config order.
158
+ - `pairs` is one `ListedPair` per test, target, and agent, in selection
159
+ order, including pairs a filter removed. Repeats are not expanded.
160
+ - `unmatched` is the positional arguments, as written, that selected no
161
+ discovered file.
162
+ - `targets` is the selected target names, in config order.
108
163
 
109
164
  | Field | Meaning |
110
165
  | --- | --- |
@@ -116,15 +171,15 @@ for (const pair of pairs) {
116
171
  | `serialId` | Set when the test belongs to a serial group, otherwise `undefined`. |
117
172
  | `target`, `agent` | The target name and the configured agent this pair runs as. |
118
173
  | `disposition` | `run`, `skip`, or `filtered`. |
119
- | `reason` | Why the pair is skipped or filtered. Omitted when it runs. |
120
-
121
- `e2e list` prints only `run` and `skip`. `list` returns `filtered` pairs too, each with `reason` (for example `title does not match --grep`).
174
+ | `reason` | Why the pair is skipped or filtered, such as `title does not match --grep`. Omitted when it runs. |
122
175
 
123
- ### Errors
176
+ `e2e list` prints only `run` and `skip` pairs. `list` returns `filtered`
177
+ pairs too.
124
178
 
125
- A failure is the same classified error `e2e list` exits with. `instanceof` can miss it across copies of the package. Recognize one with `isE2EError` and branch on `error.code`. `ConfigurationError` and `isE2EError` are both exported from `e2e/runner`.
179
+ ##### Errors
126
180
 
127
- An empty selection throws `ConfigurationError` `NO_TESTS` unless `passWithNoTests`. An unknown target is `UNKNOWN_TARGET`. A missing config, a config the loader refused, and a collection failure throw the same codes `run` records.
181
+ A failure is the same classified error `e2e list` exits with. Recognize one
182
+ with [`isE2EError`](#ise2eerror) and branch on `error.code`.
128
183
 
129
184
  ```ts
130
185
  import { isE2EError, list } from 'e2e/runner';
@@ -138,19 +193,281 @@ try {
138
193
  }
139
194
  ```
140
195
 
141
- Each call imports the test files again, so a later `list()` in the same process sees an edit.
196
+ | Throws | When |
197
+ | --- | --- |
198
+ | `NO_TESTS` | The selection is empty and `passWithNoTests` is not set. A `ConfigurationError`. |
199
+ | `UNKNOWN_TARGET` | A name in `targets` is not a configured target. |
200
+ | The codes `run` records | The config is missing, the loader refused it, or collection failed. |
201
+
202
+ #### `openHost`
203
+
204
+ `openHost<Fixtures>(options?: OpenHostOptions)` prepares e2e for Playwright
205
+ Test, Vitest, Jest, or your own harness, and returns a [host](#host).
206
+
207
+ ```ts
208
+ import { openHost } from 'e2e/runner';
209
+
210
+ await using host = await openHost({ target: 'web' });
211
+ await using session = await host.openSession({ name: 'checkout' });
212
+ await session.fixtures.app.open('/checkout');
213
+ ```
214
+
215
+ - Your runner owns collection, retries, and reporting.
216
+ - The host prepares the target's engine, starts app processes, and
217
+ initializes the engine once per worker lifetime. Use a worker-scoped host
218
+ and a test-scoped session; see the
219
+ [Playwright Test example](#playwright-test-fixtures).
220
+ - Each [`host.openSession`](#host-opensession) creates a test-lifetime attempt
221
+ with fresh fixtures, a deadline, steps, and artifacts.
222
+ - If opening fails, e2e tears down what started before throwing the error.
223
+
224
+ ##### Options
225
+
226
+ | Option | Example | Type | Default |
227
+ | --- | --- | --- | --- |
228
+ | `config` | `'e2e.config.ts'` | `string \| E2EConfig` | nearest `e2e.config.ts` or `e2e.config.mts` |
229
+ | `cwd` | `'apps/web'` | `string` | `process.cwd()` |
230
+ | `target` | `'android'` | `string` | the only target |
231
+ | `env` | `process.env` | `NodeJS.ProcessEnv` | `process.env` |
232
+ | `signal` | `controller.signal` | `AbortSignal` | - |
233
+ | `headed` | `true` | `boolean` | `false` |
234
+ | `onNotice` | `(target, message) => {}` | `(target: string, message: string) => void` | - |
235
+
236
+ - `config` is a config file path relative to `cwd`, or an `E2EConfig` value.
237
+ Omit it to discover the config from `cwd`, the same walk `list` does.
238
+ - `cwd` is the directory for config discovery, and the project root of a
239
+ config value.
240
+ - `target` is required when the config declares several targets.
241
+ - `env` is the environment the config, engines, and providers read.
242
+ - `signal` cancels host provisioning, a pending session open, or live
243
+ fixture calls.
244
+ - `headed` runs a browser engine headed.
245
+ - `onNotice` receives progress outside a step, such as provisioning or an
246
+ app process. It must not throw.
247
+
248
+ A host given a config path loads the file and its project-file imports fresh. Package
249
+ imports stay shared, so their singleton state is not recreated per host. A
250
+ config value uses its own engine instances. Each instance belongs to one
251
+ host until that host closes, so a second host must use another instance.
252
+
253
+ | Throws | When |
254
+ | --- | --- |
255
+ | `UNKNOWN_TARGET` | `target` names no configured target. |
256
+ | `TARGET_REQUIRED` | The config declares several targets and `target` names none. |
257
+ | `RUNNER_BUSY` | A suite is running. Await it before opening a host. |
258
+
259
+ Opening failures are classified, redacted errors; raw engine causes are not
260
+ exposed. Match them with [`isE2EError`](#ise2eerror) and `error.code`.
261
+
262
+ #### `isE2EError`
263
+
264
+ A type guard that recognizes a classified e2e error. Branch on its
265
+ `error.code`.
266
+
267
+ ```ts
268
+ if (isE2EError(error) && error.code === 'NO_TESTS') process.exitCode = 0;
269
+ ```
270
+
271
+ `instanceof` can miss an e2e error across copies of the package, so use
272
+ `isE2EError`. `ConfigurationError` and `isE2EError` are both exported
273
+ from `e2e/runner`.
274
+
275
+ ### Host
276
+
277
+ `E2EHost<Fixtures>` is what [`openHost`](#openhost) returns.
278
+
279
+ | Member | Returns |
280
+ | --- | --- |
281
+ | `runId`, `target` | `string` |
282
+ | [`host.openSession`](#host-opensession) | `Promise<E2ESession<Fixtures>>` |
283
+ | [`host.close`](#host-close) | `Promise<readonly SerializedError[]>` |
284
+
285
+ `runId` and `target` are the identity the host's sessions share.
286
+
287
+ Only one hosted attempt may be opening, live, or closing per process, even
288
+ across different hosts and whether the config is a file or a value.
289
+
290
+ - An overlapping open is rejected before engine attempt state starts. Await
291
+ `session.close()` before opening the next session on the same host.
292
+ - A launch that fails or is cancelled ends what it opened and leaves the host
293
+ usable, as a run keeps its worker for the retry.
294
+ - A session whose engine cleanup failed closes the host permanently, since
295
+ its isolation is uncertain; open a new host.
296
+
297
+ | Throws | When |
298
+ | --- | --- |
299
+ | `WORKER_PROTOCOL` | An overlapping hosted attempt, or a call on a closed host. |
300
+
301
+ #### `host.openSession`
302
+
303
+ `host.openSession(options: OpenSessionOptions)` opens a fresh
304
+ [session](#session) on this host's target. A session is one test attempt with
305
+ no retry loop. A retry opens a new session on the same worker host.
306
+
307
+ ```ts
308
+ const session = await host.openSession({
309
+ name: 'checkout',
310
+ timeout: 60_000, // default: the config's timeout
311
+ onStep: (step) => {
312
+ if (step.phase === 'end') console.log(`${step.api} ${step.label}: ${step.status}`);
313
+ },
314
+ });
315
+ ```
316
+
317
+ ##### Options
318
+
319
+ | Option | Example | Type | Default |
320
+ | --- | --- | --- | --- |
321
+ | `name` | `'checkout'` | `string` | required |
322
+ | `timeout` | `60_000` | `number` | the config's `timeout` |
323
+ | `signal` | `controller.signal` | `AbortSignal` | - |
324
+ | `agent` | `'smart'` | `string` | the first configured agent |
325
+ | `onStep` | `(step) => {}` | `(progress: StepProgress) => void` | - |
326
+
327
+ - `name` is the test name for step and trace reporting.
328
+ - `timeout` is the deadline for the session's operations in milliseconds,
329
+ positive and finite, counted from when the session opens. It does not bound
330
+ provisioning or opening, and it does not close the session.
331
+ - `signal` cancels opening or live fixture calls. The host's `signal` also
332
+ applies. Provisioning happens in `openHost`, so only the host's `signal`
333
+ cancels it; either signal cancels a launch.
334
+ - `agent` is the agent the `agent` fixture uses when a call names none.
335
+ - `onStep` receives live `StepProgress`, the same feed reporters receive. It
336
+ must not block or throw. Its `phase` is `start`, `event`, `activity`,
337
+ `replay`, or `end`. An `end` event includes `status`, `durationMs`,
338
+ `modelCalls`, and a redacted `error` when the step failed.
339
+ - `timeout`, `agent`, and `onStep` belong to the session, not `openHost`.
340
+
341
+ | Throws | When |
342
+ | --- | --- |
343
+ | `INVALID_CONFIG` | `name` is not a string, or `timeout` is not positive and finite. |
344
+ | `CANCELLED` | The session's or the host's `signal` aborted before or during opening. |
345
+ | `WORKER_PROTOCOL` | Another hosted attempt is opening, live, or closing, or the host is closed. |
346
+
347
+ #### `host.close`
348
+
349
+ `host.close()` cancels any active or pending session, waits for it to close,
350
+ then disposes the engine, stops app processes, and releases provisioning.
351
+
352
+ ```ts
353
+ const failures = await host.close();
354
+ if (failures.length > 0) throw new AggregateError(failures, 'e2e host close failed');
355
+ ```
356
+
357
+ - Idempotent: it disposes and releases resources only once.
358
+ - A host is `AsyncDisposable`: `await using host = await openHost()` closes
359
+ it at scope exit and throws an `AggregateError` when closing reports
360
+ failures.
142
361
 
143
- ## openHost
362
+ ### Session
144
363
 
145
- `openHost` prepares e2e for Playwright Test, Vitest, Jest, or your own harness.
146
- Your runner owns collection, retries, and reporting. The host prepares the
147
- target's engine, starts app processes, and initializes the engine once per
148
- worker lifetime. Each `host.openSession` creates a test-lifetime attempt with
149
- fresh fixtures, a deadline, steps, and artifacts.
364
+ `E2ESession<Fixtures>` is what [`host.openSession`](#host-opensession) returns.
150
365
 
151
- Use a worker-scoped host and a test-scoped session:
366
+ | Member | Returns |
367
+ | --- | --- |
368
+ | `fixtures` | `TestFixtures & Fixtures` |
369
+ | `runId`, `attemptId`, `target` | `string` |
370
+ | `steps()` | `readonly StepRecord[]` |
371
+ | `report()` | `Report \| undefined` |
372
+ | `artifactsDir` | `string` |
373
+ | `tracePath` | `string` |
374
+ | [`session.close`](#session-close) | `Promise<readonly SerializedError[]>` |
375
+
376
+ - `fixtures` holds `agent`, `app`, `screen`, `platform`, and the engine's
377
+ fixtures.
378
+ - `runId` and `target` are the host's run and target; `attemptId` is unique
379
+ to this attempt.
380
+ - `steps()` returns read-only `StepRecord`s for steps run so far, as the
381
+ [report](/reference/reporters) records them.
382
+ - `report()` returns the session `Report` the trace is rendered from, after
383
+ close and a successful trace write; otherwise `undefined`. Its status
384
+ describes the SDK session, not your runner's test-body verdict.
385
+ - `artifactsDir` is the absolute directory for this attempt's screenshots,
386
+ recordings, and trace.
387
+ - `tracePath` is the absolute path to `trace.md`; attach or read it after
388
+ `close()` settles.
389
+
390
+ #### `session.close`
391
+
392
+ `session.close()` cancels remaining operations, ends the attempt, and writes
393
+ its trace. It returns soft expectation, abandoned-operation (an unsettled
394
+ poll or step), and cleanup failures instead of throwing them.
152
395
 
153
396
  ```ts
397
+ const failures = await session.close();
398
+ if (failures.length > 0) throw new AggregateError(failures, 'e2e session close failed');
399
+ ```
400
+
401
+ - Idempotent: later calls return the same failures.
402
+ - The host stays open for the next test.
403
+ - A session is `AsyncDisposable`:
404
+ `await using session = await host.openSession({ name: 'checkout' })`
405
+ closes it at scope exit and throws an `AggregateError` of the returned
406
+ failures.
407
+
408
+ #### Deadline and soft failures
409
+
410
+ `expect.soft` collects failures for the session's whole lifetime. Check
411
+ `close()`'s result to fail the host runner's test.
412
+
413
+ - `expect.poll` uses the session's remaining budget, even when its own
414
+ timeout is longer.
415
+ - Contributed fixture operations also honor that budget, including
416
+ operations that disable their own timeout.
417
+ - When the deadline passes, a poll or operation that was watching it reports
418
+ its own cut. Everything else still running, including an operation an
419
+ engine or a model never answered, fails with `TEST_TIMEOUT`.
420
+ - Either way the session's outcome is timed out and `close()` returns the
421
+ timeout. Later operations fail with `TEST_TIMEOUT`; close the session.
422
+
423
+ #### Artifacts
424
+
425
+ Artifacts live under `<output>/results/host/<resultSegment>/`. At close,
426
+ e2e writes `trace.md` there through the same redacted formatting reports
427
+ use.
428
+
429
+ - e2e generates a safe `resultSegment` from the name and the unique attempt
430
+ identity. The name never becomes a raw path component.
431
+ - Repeated names and retries get separate directories.
432
+ - With provider-backed secrets, the directory uses an identity-only segment,
433
+ since unresolved values cannot be redacted from a title slug. The trace
434
+ still shows the redacted name.
435
+ - Artifact paths are relative to `<output>/results`, not
436
+ `session.artifactsDir`.
437
+ - Video follows the target's mode for a first attempt. Failure-only video is
438
+ kept for soft failures and for failed, blocked, timed-out, or cancelled
439
+ steps.
440
+
441
+ <Note>
442
+ A hosted session writes its trace, not `report.json`. For your runner's
443
+ reporting, use `onStep`, `steps()`, `report()`, and `close()`'s failures.
444
+ Your runner's test-body failures are not passed to e2e.
445
+
446
+ The one-shot attempt `e2e mcp` uses stays internal. There is no top-level
447
+ `openSession` export.
448
+ </Note>
449
+
450
+ ## Types
451
+
452
+ `e2e/runner` exports these types beside the functions.
453
+
454
+ | Type | Describes |
455
+ | --- | --- |
456
+ | `RunOptions`, `RunResult` | [`run`](#run)'s options and result |
457
+ | `RunEventSink` | The `onEvent` callback |
458
+ | `ListOptions`, `ListResult`, `ListedPair` | [`list`](#list)'s options, result, and one pair |
459
+ | `OpenHostOptions`, `E2EHost` | [`openHost`](#openhost)'s options and the [host](#host) |
460
+ | `OpenSessionOptions`, `E2ESession` | [`openSession`](#host-opensession)'s options and the [session](#session) |
461
+ | `StepProgress`, `StepRecord` | One `onStep` progress event, and one step `steps()` returns |
462
+ | `SerializedError` | One failure `close()` returns |
463
+
464
+ ## Examples
465
+
466
+ ### Playwright Test fixtures
467
+
468
+ A worker-scoped host and a test-scoped session:
469
+
470
+ ```ts title="tests/fixtures.ts"
154
471
  import { test as base } from '@playwright/test';
155
472
  import { openHost, type E2EHost, type E2ESession } from 'e2e/runner';
156
473
  import type { Device } from '@e2e-dev/mobile';
@@ -195,121 +512,3 @@ test('turns on dark mode', async ({ e2e }) => {
195
512
  await app.screenshot();
196
513
  });
197
514
  ```
198
-
199
- ### Host options
200
-
201
- `openHost<Fixtures>(options?: OpenHostOptions)` returns
202
- `Promise<E2EHost<Fixtures>>`. Omit `config` to discover the config from `cwd`,
203
- the same walk `list` does. If opening fails, e2e tears down what started
204
- before throwing the error.
205
-
206
- | Option | Default | Meaning |
207
- | --- | --- | --- |
208
- | `config` | nearest `e2e.config.ts` or `e2e.config.mts` | Config file path relative to `cwd`, or an `E2EConfig` value |
209
- | `cwd` | `process.cwd()` | Directory for config discovery, and the project root of a config value |
210
- | `target` | the only target | Target name; required when the config declares several |
211
- | `env` | `process.env` | Environment the config, engines, and providers read |
212
- | `signal` | none | Cancels host provisioning, a pending session open, or live fixture calls |
213
- | `headed` | `false` | Runs a browser engine headed |
214
- | `onNotice` | none | Progress outside a step, such as provisioning or an app process |
215
-
216
- A config file and its project-file imports are loaded fresh for each host.
217
- Package imports remain shared; their singleton state is not recreated per host.
218
- A config value uses the engine instances in that value. Neither form allows
219
- overlapping hosted attempts in one process. A config value's engine instance
220
- belongs to one host until it closes; a second host must use another instance.
221
-
222
- ### Host
223
-
224
- | Member | Meaning |
225
- | --- | --- |
226
- | `runId`, `target` | Identity shared by the host's sessions |
227
- | `openSession(options)` | Opens a fresh `E2ESession<Fixtures>` on this host's target |
228
- | `close()` | Cancels any active or pending session, waits for it to close, then disposes the engine, stops app processes, and releases provisioning. Returns `Promise<readonly SerializedError[]>` |
229
-
230
- Only one hosted attempt may be opening, live, or closing per process, even
231
- across different hosts. An overlapping open is rejected before engine
232
- attempt state starts. Await `session.close()` before opening the next
233
- session on the same host. A launch that fails or is cancelled ends what it
234
- opened and leaves the host usable, as a run keeps its worker for the retry.
235
- A session whose engine cleanup failed closes the host permanently, since
236
- its isolation is uncertain; open a new host. Overlap and closed-host errors
237
- use `WORKER_PROTOCOL`.
238
-
239
- `host.close()` is idempotent and disposes and releases resources only once.
240
- A host is `AsyncDisposable`; `await using host = await openHost()` closes it
241
- at scope exit and throws an `AggregateError` when closing reports failures.
242
-
243
- ## host.openSession
244
-
245
- `host.openSession(options: OpenSessionOptions)` returns
246
- `Promise<E2ESession<Fixtures>>`. A session is one test attempt with no retry
247
- loop. A retry opens a new session on the same worker host.
248
-
249
- ### Session options
250
-
251
- | Option | Default | Meaning |
252
- | --- | --- | --- |
253
- | `name` | required | Test name for step and trace reporting; a string |
254
- | `timeout` | config `timeout` | Deadline for the session's operations in milliseconds, positive and finite, counted from when it opens. Does not bound provisioning or opening, and does not close the session |
255
- | `signal` | none | Cancels opening or live fixture calls; the host signal also applies |
256
- | `agent` | the first configured agent | Agent the `agent` fixture uses when a call names none |
257
- | `onStep` | none | Live `StepProgress`, the same feed reporters receive |
258
-
259
- `timeout`, `agent`, and `onStep` belong to the session, not `openHost`.
260
- Use `signal` to cancel provisioning or launch. `onStep` must not block or
261
- throw. Its `phase` is `start`, `event`, `activity`, `replay`, or `end`.
262
- An `end` event includes `status`, `durationMs`, `modelCalls`, and a redacted
263
- `error` when the step failed.
264
-
265
- ### Session
266
-
267
- | Member | Meaning |
268
- | --- | --- |
269
- | `fixtures` | `agent`, `app`, `screen`, `platform`, and the engine's fixtures, typed as `TestFixtures & Fixtures` |
270
- | `runId`, `attemptId`, `target` | Host run and target, plus a unique attempt id |
271
- | `steps()` | Read-only `StepRecord`s for steps run so far, as the [report](/reference/reporters) records them |
272
- | `artifactsDir` | Absolute directory for this attempt's screenshots, recordings, and trace |
273
- | `tracePath` | Absolute string path to `trace.md`; attach or read it after `close()` settles |
274
- | `close()` | Cancels remaining operations, ends the attempt, and writes its trace. Returns soft expectation and cleanup failures as `Promise<readonly SerializedError[]>` instead of throwing them |
275
-
276
- `session.close()` is idempotent and returns the same failures on later calls.
277
- The host stays open for the next test. A session is `AsyncDisposable`:
278
- `await using session = await host.openSession({ name: 'checkout' })` closes
279
- it at scope exit and throws an `AggregateError` of the returned failures.
280
-
281
- `expect.soft` collects failures for the session's whole lifetime. Check
282
- `close()`'s result to fail the host runner's test. `expect.poll` uses the
283
- session's remaining budget, even when its own timeout is longer. Contributed
284
- fixture operations also honor that budget, including operations that disable
285
- their own timeout. When the deadline passes, a poll or operation that was
286
- watching it reports its own cut; everything else still running, including an
287
- operation an engine or a model never answered, fails with `TEST_TIMEOUT`.
288
- Either way the session's outcome is timed out and `close()` returns the
289
- timeout. Later operations fail with `TEST_TIMEOUT`; close the session.
290
-
291
- Artifacts live under `<output>/results/host/<resultSegment>/`, with
292
- `trace.md` written at close through the existing redacted report-formatting
293
- pipeline. e2e generates the safe `resultSegment` from the name and unique
294
- attempt identity; the name never becomes a raw path component. Repeated
295
- names and retries get separate directories. With provider-backed secrets,
296
- the directory uses an identity-only segment because unresolved values cannot
297
- be redacted from a title slug. The trace still displays the redacted name.
298
-
299
- Opening failures are classified, redacted errors; raw engine causes are not
300
- exposed. Match them with `isE2EError` and
301
- `error.code`, as for `list`. `openHost` reports `UNKNOWN_TARGET` for an
302
- unknown target and `TARGET_REQUIRED` when several targets exist and none
303
- was named.
304
-
305
- <Note>
306
- A hosted session writes its trace, not a run report. Your runner owns retries
307
- and reporting; use `onStep`, `steps()`, and `close()`'s failures. Video follows
308
- the target's mode for a first attempt, including retaining failure-only video
309
- for soft failures and failed, blocked, timed-out, or cancelled steps. Your
310
- runner's test-body failures are not
311
- passed to e2e.
312
-
313
- The one-shot attempt used by `e2e mcp` remains internal. There is no
314
- top-level `openSession` export.
315
- </Note>