e2e 0.21.1 → 0.21.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -0
- package/dist/.build.json +1 -1
- package/dist/agent/act-validation.js +1 -0
- package/dist/agent/act.js +7 -3
- package/dist/agent/action-dispatcher.js +30 -8
- package/dist/agent/actions.d.ts +2 -0
- package/dist/agent/actions.js +5 -2
- package/dist/agent/error.js +1 -0
- package/dist/agent/executor.d.ts +14 -6
- package/dist/agent/index.js +5 -1
- package/dist/agent/invocation.d.ts +3 -0
- package/dist/agent/model/sdk.d.ts +5 -0
- package/dist/agent/model/sdk.js +18 -1
- package/dist/agent/observation.js +1 -0
- package/dist/agent/primitives.js +14 -3
- package/dist/agent/replay.js +9 -3
- package/dist/agent/step-cache.d.ts +3 -2
- package/dist/agent/step-cache.js +7 -4
- package/dist/agent/tool-loop.js +4 -1
- package/dist/cache/context.js +1 -1
- package/dist/cache/recorder.d.ts +2 -1
- package/dist/cache/recorder.js +11 -2
- package/dist/cache/trace-fields.d.ts +10 -0
- package/dist/cache/trace-fields.js +17 -0
- package/dist/cache/trace.d.ts +9 -1
- package/dist/cache/trace.js +26 -7
- package/dist/cli/feedback.js +2 -1
- package/dist/cli/init/sibling-versions.json +1 -1
- package/dist/collect/select.js +12 -2
- package/dist/config/agent.d.ts +1 -1
- package/dist/config/diagnose.js +9 -3
- package/dist/config/esm-hooks.js +3 -2
- package/dist/config/resolve.js +3 -2
- package/dist/config/typescript.js +52 -2
- package/dist/engine/contract.d.ts +3 -2
- package/dist/engine/manifest.js +2 -2
- package/dist/internal/errors.js +5 -2
- package/dist/internal/source.js +2 -0
- package/dist/internal/unserved-model.d.ts +5 -0
- package/dist/internal/unserved-model.js +22 -0
- package/dist/locator/screen.js +1 -2
- package/dist/oauth/copilot.js +23 -5
- package/dist/oauth/device-code.js +35 -19
- package/dist/oauth/errors.d.ts +3 -0
- package/dist/oauth/errors.js +26 -11
- package/dist/oauth/providers/github-copilot.d.ts +4 -0
- package/dist/oauth/providers/github-copilot.js +64 -16
- package/dist/oauth/providers/openai.d.ts +2 -2
- package/dist/oauth/providers/openai.js +47 -19
- package/dist/oauth/providers/opencode-console.js +36 -15
- package/dist/oauth/providers/xai.js +18 -9
- package/dist/oauth/token-endpoint.d.ts +15 -6
- package/dist/oauth/token-endpoint.js +18 -6
- package/dist/report/code-frame.js +9 -3
- package/dist/report/failure-text.d.ts +10 -2
- package/dist/report/failure-text.js +24 -8
- package/dist/report/list-model.d.ts +13 -0
- package/dist/report/list-steps.d.ts +6 -1
- package/dist/report/list-steps.js +29 -0
- package/dist/report/list.d.ts +1 -0
- package/dist/report/list.js +16 -7
- package/dist/report/live-window.js +2 -1
- package/dist/report/markdown.js +1 -1
- package/dist/run/fixtures.js +2 -0
- package/dist/run/host-trace.d.ts +3 -2
- package/dist/run/host-trace.js +2 -1
- package/dist/run/last-run.d.ts +9 -0
- package/dist/run/last-run.js +28 -0
- package/dist/run/open-session.d.ts +3 -1
- package/dist/run/open-session.js +1 -1
- package/dist/run/runner.js +6 -2
- package/dist/run/sessions.js +3 -3
- package/dist/run/standalone-attempt.d.ts +2 -1
- package/dist/run/standalone-attempt.js +3 -1
- package/dist/run/steps.d.ts +3 -1
- package/dist/run/units.d.ts +8 -0
- package/dist/run/units.js +14 -0
- package/dist/run/worker-models.d.ts +2 -0
- package/dist/run/worker-models.js +7 -0
- package/dist/types.d.ts +2 -2
- package/docs/authentication.mdx +26 -23
- package/docs/browser.mdx +7 -4
- package/docs/bug-bash.mdx +2 -2
- package/docs/cache.mdx +6 -1
- package/docs/ci/bitrise.mdx +1 -1
- package/docs/ci/codemagic.mdx +2 -2
- package/docs/ci/eas-workflows.mdx +1 -1
- package/docs/coding-agents.mdx +7 -35
- package/docs/debugging.mdx +24 -36
- package/docs/executors.mdx +1 -1
- package/docs/github.mdx +2 -1
- package/docs/integrations/eas.mdx +1 -1
- package/docs/integrations/index.mdx +6 -0
- package/docs/integrations/lightpanda.mdx +192 -0
- package/docs/integrations/testmu.mdx +152 -0
- package/docs/mcp.mdx +123 -0
- package/docs/migrate/detox.mdx +1 -1
- package/docs/migrate/maestro.mdx +101 -105
- package/docs/migrate/playwright.mdx +65 -70
- package/docs/mobile.mdx +18 -3
- package/docs/models.mdx +4 -2
- package/docs/quickstart.mdx +7 -0
- package/docs/reference/agent.mdx +369 -277
- package/docs/reference/app.mdx +141 -32
- package/docs/reference/cli.mdx +674 -558
- package/docs/reference/config.mdx +1406 -752
- package/docs/reference/engine.mdx +1062 -256
- package/docs/reference/environment.mdx +156 -84
- package/docs/reference/errors.mdx +106 -72
- package/docs/reference/expect.mdx +752 -372
- package/docs/reference/mcp.mdx +366 -141
- package/docs/reference/mobile.mdx +844 -403
- package/docs/reference/reporters.mdx +312 -106
- package/docs/reference/runner.mdx +373 -174
- package/docs/reference/screen.mdx +931 -398
- package/docs/reference/security.mdx +106 -40
- package/docs/reference/test.mdx +626 -273
- package/docs/reference/web.mdx +956 -533
- package/docs/security.mdx +6 -5
- package/docs/snippets/model-provider.jsx +763 -0
- package/docs/snippets/setup-prompt-card.jsx +22 -0
- package/docs/snippets/show-more.jsx +22 -0
- package/docs/snippets/video-card.jsx +34 -0
- package/docs/snippets/video.jsx +25 -0
- package/docs/subscriptions.mdx +10 -3
- package/docs/telemetry.mdx +39 -39
- package/package.json +3 -2
- package/schema/report-v1.schema.json +6 -0
- package/skills/e2e/references/agent.md +8 -3
- package/skills/e2e/references/debugging.md +1 -1
- package/skills/e2e/references/mcp.md +1 -1
- package/skills/e2e/references/setup.md +12 -3
- package/skills/e2e/references/writing-tests.md +1 -1
|
@@ -6,13 +6,12 @@ icon: "file-lines"
|
|
|
6
6
|
"twitter:image": "https://e2e.tester.army/docs/images/og/reference/runner.png?v=98c298d3ed"
|
|
7
7
|
---
|
|
8
8
|
|
|
9
|
-
`e2e/runner` runs a suite from code without invoking the CLI. `run` executes
|
|
9
|
+
`e2e/runner` runs a suite from code without invoking the CLI. `run` executes
|
|
10
|
+
tests and returns their report, `list` returns a selection, and `openHost`
|
|
11
|
+
hosts e2e sessions in another test runner. Importing `e2e` from a test file
|
|
12
|
+
does not load the runner.
|
|
10
13
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
`run(options?: RunOptions)` executes a complete suite and returns `Promise<RunResult>`. It finishes cleanup before resolving and never exits the caller's process.
|
|
14
|
-
|
|
15
|
-
```ts
|
|
14
|
+
```ts title="scripts/run-e2e.ts"
|
|
16
15
|
import type { E2EConfig } from 'e2e';
|
|
17
16
|
import { run } from 'e2e/runner';
|
|
18
17
|
import { web } from '@e2e-dev/web';
|
|
@@ -33,23 +32,47 @@ const result = await run({
|
|
|
33
32
|
console.log(result.status, result.report.run.summary);
|
|
34
33
|
```
|
|
35
34
|
|
|
36
|
-
|
|
35
|
+
## Reference
|
|
37
36
|
|
|
38
|
-
|
|
37
|
+
### Functions
|
|
39
38
|
|
|
40
|
-
|
|
|
39
|
+
| Function | Returns |
|
|
41
40
|
| --- | --- |
|
|
42
|
-
| `
|
|
43
|
-
| `
|
|
44
|
-
| `
|
|
45
|
-
| `
|
|
46
|
-
|
|
41
|
+
| [`run`](#run) | `Promise<RunResult>` |
|
|
42
|
+
| [`list`](#list) | `Promise<ListResult>` |
|
|
43
|
+
| [`openHost`](#openhost) | `Promise<E2EHost<Fixtures>>` |
|
|
44
|
+
| [`isE2EError`](#ise2eerror) | `boolean` |
|
|
45
|
+
|
|
46
|
+
#### `run`
|
|
47
|
+
|
|
48
|
+
`run(options?: RunOptions)` executes a complete suite. It finishes cleanup
|
|
49
|
+
before resolving and never exits the caller's process.
|
|
50
|
+
|
|
51
|
+
```ts
|
|
52
|
+
const result = await run({ files: ['tests/signup.e2e.ts'] });
|
|
53
|
+
process.exitCode = result.exitCode;
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
##### Options
|
|
47
57
|
|
|
48
|
-
|
|
58
|
+
| Option | Example | Type | Default |
|
|
59
|
+
| --- | --- | --- | --- |
|
|
60
|
+
| `cwd` | `'apps/web'` | `string` | the current working directory |
|
|
61
|
+
| `config` | `'e2e.ci.config.ts'` | `string \| E2EConfig` | discovered `e2e.config.ts` |
|
|
62
|
+
| `files` | `['tests/signup.e2e.ts:12']` | `readonly string[]` | the configured suite |
|
|
63
|
+
| `interruptSignal` | `controller.signal` | `AbortSignal` | - |
|
|
64
|
+
| `onEvent` | `(event) => {}` | `RunEventSink` | - |
|
|
49
65
|
|
|
50
|
-
|
|
66
|
+
- `files` takes paths, globs, bare names, or `file:line`.
|
|
67
|
+
- Aborting `interruptSignal` interrupts tests and executes teardown.
|
|
68
|
+
- `onEvent` receives typed progress events. Terminal progress is off. It
|
|
69
|
+
must not block. A throwing or rejecting callback is disabled without
|
|
70
|
+
failing the run.
|
|
71
|
+
- Put execution settings, such as retries and workers, in the config.
|
|
72
|
+
- A runtime config (an `E2EConfig` value) runs with one worker in the
|
|
73
|
+
caller's process. A file config supports the configured worker processes.
|
|
51
74
|
|
|
52
|
-
|
|
75
|
+
##### Result
|
|
53
76
|
|
|
54
77
|
| Field | Meaning |
|
|
55
78
|
| --- | --- |
|
|
@@ -59,15 +82,27 @@ Helper modules your test files import stay cached between `run` calls in the sam
|
|
|
59
82
|
| `reportPath` | Absolute path to `report.json`, or `undefined` if it could not be written. |
|
|
60
83
|
| `aiTracePath` | AI trace path, or `undefined` when no AI trace was written. |
|
|
61
84
|
|
|
62
|
-
Test failures, config failures, and interruption resolve with their status
|
|
85
|
+
Test failures, config failures, and interruption resolve with their status
|
|
86
|
+
and errors in the report.
|
|
63
87
|
|
|
64
|
-
|
|
88
|
+
| Throws | When |
|
|
89
|
+
| --- | --- |
|
|
90
|
+
| `RUNNER_BUSY` | Another suite is running, or a host is open. Close every host before running a suite, and await the suite before opening a host. |
|
|
65
91
|
|
|
66
|
-
|
|
92
|
+
- Helper modules your test files import stay cached between `run` calls in
|
|
93
|
+
one process. An edit to them needs a new process.
|
|
94
|
+
- Each run that returns a report sends one anonymous
|
|
95
|
+
[`e2e_run_completed`](/telemetry) event, with the same opt-outs as the CLI.
|
|
96
|
+
The first call can print the telemetry notice on stderr.
|
|
97
|
+
- Suite output uses UTF-8 text redaction and does not preserve binary data.
|
|
98
|
+
Your application's logs and progress callback output stay under your
|
|
99
|
+
control.
|
|
67
100
|
|
|
68
|
-
|
|
101
|
+
#### `list`
|
|
69
102
|
|
|
70
|
-
`list` collects and selects the way
|
|
103
|
+
`list(options?: ListOptions)` collects and selects the way
|
|
104
|
+
[`e2e list`](/reference/cli#e2e-list) does, and stops there. No app process,
|
|
105
|
+
engine, or worker starts, and nothing is written.
|
|
71
106
|
|
|
72
107
|
```ts
|
|
73
108
|
import { isE2EError, list } from 'e2e/runner';
|
|
@@ -82,29 +117,49 @@ for (const pair of pairs) {
|
|
|
82
117
|
}
|
|
83
118
|
```
|
|
84
119
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
|
91
|
-
|
|
|
92
|
-
| `
|
|
93
|
-
| `
|
|
94
|
-
| `
|
|
95
|
-
| `
|
|
96
|
-
| `
|
|
97
|
-
| `
|
|
98
|
-
| `
|
|
99
|
-
| `
|
|
100
|
-
| `
|
|
101
|
-
| `
|
|
102
|
-
|
|
103
|
-
|
|
120
|
+
Each call imports the test files again, so a later `list()` in the same
|
|
121
|
+
process sees an edit.
|
|
122
|
+
|
|
123
|
+
##### Options
|
|
124
|
+
|
|
125
|
+
| Option | Example | Type | Default |
|
|
126
|
+
| --- | --- | --- | --- |
|
|
127
|
+
| `config` | `'e2e.ci.config.ts'` | `string \| E2EConfig` | discovered from `cwd`, the same walk `e2e list` does |
|
|
128
|
+
| `targets` | `['web']` | `readonly string[]` | - |
|
|
129
|
+
| `cwd` | `'apps/web'` | `string` | the current working directory |
|
|
130
|
+
| `files` | `['tests/', 'signup.e2e.ts:12']` | `readonly string[]` | the configured suite |
|
|
131
|
+
| `tags`, `tagMode`, `excludeTags` | `{ tags: ['smoke'], tagMode: 'all' }` | `readonly string[]`, `'any' \| 'all'` | - |
|
|
132
|
+
| `grep`, `grepInvert` | `[/checkout/]` | `readonly RegExp[]` | - |
|
|
133
|
+
| `lastFailed` | `true` | `boolean` | - |
|
|
134
|
+
| `shard` | `{ index: 1, total: 4 }` | `{ index, total }` | - |
|
|
135
|
+
| `passWithNoTests` | `true` | `boolean` | - |
|
|
136
|
+
| `output` | `'.e2e'` | `string` | the config's `output` |
|
|
137
|
+
| `env` | `{ ...process.env, CI: '1' }` | `NodeJS.ProcessEnv` | `process.env` |
|
|
138
|
+
|
|
139
|
+
- `targets` names targets, as `--target` does.
|
|
140
|
+
- `files` is the positional file selection: paths, globs, bare names, and
|
|
141
|
+
`file:line`.
|
|
142
|
+
- `tags`, `tagMode`, and `excludeTags` are the `--tag`, `--tag-mode`, and
|
|
143
|
+
`--exclude-tag` filters.
|
|
144
|
+
- `grep` and `grepInvert`: a test's describe titles and title, joined by
|
|
145
|
+
spaces, must match one `grep` pattern and no `grepInvert` pattern.
|
|
146
|
+
- `lastFailed` keeps only tests the previous run did not pass, read from
|
|
147
|
+
`report.json` under `output`.
|
|
148
|
+
- `shard` is the `--shard index/total` slice.
|
|
149
|
+
- `passWithNoTests` resolves with no runnable test instead of throwing
|
|
150
|
+
`NO_TESTS`.
|
|
151
|
+
- `output` is the results directory.
|
|
152
|
+
- `env` is the environment the config resolves against.
|
|
153
|
+
|
|
154
|
+
##### Result
|
|
104
155
|
|
|
105
156
|
`ListResult` is `{ pairs, unmatched, targets }`.
|
|
106
157
|
|
|
107
|
-
`pairs` is one `ListedPair` per test, target, and agent, in selection
|
|
158
|
+
- `pairs` is one `ListedPair` per test, target, and agent, in selection
|
|
159
|
+
order, including pairs a filter removed. Repeats are not expanded.
|
|
160
|
+
- `unmatched` is the positional arguments, as written, that selected no
|
|
161
|
+
discovered file.
|
|
162
|
+
- `targets` is the selected target names, in config order.
|
|
108
163
|
|
|
109
164
|
| Field | Meaning |
|
|
110
165
|
| --- | --- |
|
|
@@ -116,15 +171,15 @@ for (const pair of pairs) {
|
|
|
116
171
|
| `serialId` | Set when the test belongs to a serial group, otherwise `undefined`. |
|
|
117
172
|
| `target`, `agent` | The target name and the configured agent this pair runs as. |
|
|
118
173
|
| `disposition` | `run`, `skip`, or `filtered`. |
|
|
119
|
-
| `reason` | Why the pair is skipped or filtered
|
|
120
|
-
|
|
121
|
-
`e2e list` prints only `run` and `skip`. `list` returns `filtered` pairs too, each with `reason` (for example `title does not match --grep`).
|
|
174
|
+
| `reason` | Why the pair is skipped or filtered, such as `title does not match --grep`. Omitted when it runs. |
|
|
122
175
|
|
|
123
|
-
|
|
176
|
+
`e2e list` prints only `run` and `skip` pairs. `list` returns `filtered`
|
|
177
|
+
pairs too.
|
|
124
178
|
|
|
125
|
-
|
|
179
|
+
##### Errors
|
|
126
180
|
|
|
127
|
-
|
|
181
|
+
A failure is the same classified error `e2e list` exits with. Recognize one
|
|
182
|
+
with [`isE2EError`](#ise2eerror) and branch on `error.code`.
|
|
128
183
|
|
|
129
184
|
```ts
|
|
130
185
|
import { isE2EError, list } from 'e2e/runner';
|
|
@@ -138,19 +193,281 @@ try {
|
|
|
138
193
|
}
|
|
139
194
|
```
|
|
140
195
|
|
|
141
|
-
|
|
196
|
+
| Throws | When |
|
|
197
|
+
| --- | --- |
|
|
198
|
+
| `NO_TESTS` | The selection is empty and `passWithNoTests` is not set. A `ConfigurationError`. |
|
|
199
|
+
| `UNKNOWN_TARGET` | A name in `targets` is not a configured target. |
|
|
200
|
+
| The codes `run` records | The config is missing, the loader refused it, or collection failed. |
|
|
201
|
+
|
|
202
|
+
#### `openHost`
|
|
203
|
+
|
|
204
|
+
`openHost<Fixtures>(options?: OpenHostOptions)` prepares e2e for Playwright
|
|
205
|
+
Test, Vitest, Jest, or your own harness, and returns a [host](#host).
|
|
206
|
+
|
|
207
|
+
```ts
|
|
208
|
+
import { openHost } from 'e2e/runner';
|
|
209
|
+
|
|
210
|
+
await using host = await openHost({ target: 'web' });
|
|
211
|
+
await using session = await host.openSession({ name: 'checkout' });
|
|
212
|
+
await session.fixtures.app.open('/checkout');
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
- Your runner owns collection, retries, and reporting.
|
|
216
|
+
- The host prepares the target's engine, starts app processes, and
|
|
217
|
+
initializes the engine once per worker lifetime. Use a worker-scoped host
|
|
218
|
+
and a test-scoped session; see the
|
|
219
|
+
[Playwright Test example](#playwright-test-fixtures).
|
|
220
|
+
- Each [`host.openSession`](#host-opensession) creates a test-lifetime attempt
|
|
221
|
+
with fresh fixtures, a deadline, steps, and artifacts.
|
|
222
|
+
- If opening fails, e2e tears down what started before throwing the error.
|
|
223
|
+
|
|
224
|
+
##### Options
|
|
225
|
+
|
|
226
|
+
| Option | Example | Type | Default |
|
|
227
|
+
| --- | --- | --- | --- |
|
|
228
|
+
| `config` | `'e2e.config.ts'` | `string \| E2EConfig` | nearest `e2e.config.ts` or `e2e.config.mts` |
|
|
229
|
+
| `cwd` | `'apps/web'` | `string` | `process.cwd()` |
|
|
230
|
+
| `target` | `'android'` | `string` | the only target |
|
|
231
|
+
| `env` | `process.env` | `NodeJS.ProcessEnv` | `process.env` |
|
|
232
|
+
| `signal` | `controller.signal` | `AbortSignal` | - |
|
|
233
|
+
| `headed` | `true` | `boolean` | `false` |
|
|
234
|
+
| `onNotice` | `(target, message) => {}` | `(target: string, message: string) => void` | - |
|
|
235
|
+
|
|
236
|
+
- `config` is a config file path relative to `cwd`, or an `E2EConfig` value.
|
|
237
|
+
Omit it to discover the config from `cwd`, the same walk `list` does.
|
|
238
|
+
- `cwd` is the directory for config discovery, and the project root of a
|
|
239
|
+
config value.
|
|
240
|
+
- `target` is required when the config declares several targets.
|
|
241
|
+
- `env` is the environment the config, engines, and providers read.
|
|
242
|
+
- `signal` cancels host provisioning, a pending session open, or live
|
|
243
|
+
fixture calls.
|
|
244
|
+
- `headed` runs a browser engine headed.
|
|
245
|
+
- `onNotice` receives progress outside a step, such as provisioning or an
|
|
246
|
+
app process. It must not throw.
|
|
247
|
+
|
|
248
|
+
A host given a config path loads the file and its project-file imports fresh. Package
|
|
249
|
+
imports stay shared, so their singleton state is not recreated per host. A
|
|
250
|
+
config value uses its own engine instances. Each instance belongs to one
|
|
251
|
+
host until that host closes, so a second host must use another instance.
|
|
252
|
+
|
|
253
|
+
| Throws | When |
|
|
254
|
+
| --- | --- |
|
|
255
|
+
| `UNKNOWN_TARGET` | `target` names no configured target. |
|
|
256
|
+
| `TARGET_REQUIRED` | The config declares several targets and `target` names none. |
|
|
257
|
+
| `RUNNER_BUSY` | A suite is running. Await it before opening a host. |
|
|
258
|
+
|
|
259
|
+
Opening failures are classified, redacted errors; raw engine causes are not
|
|
260
|
+
exposed. Match them with [`isE2EError`](#ise2eerror) and `error.code`.
|
|
261
|
+
|
|
262
|
+
#### `isE2EError`
|
|
263
|
+
|
|
264
|
+
A type guard that recognizes a classified e2e error. Branch on its
|
|
265
|
+
`error.code`.
|
|
266
|
+
|
|
267
|
+
```ts
|
|
268
|
+
if (isE2EError(error) && error.code === 'NO_TESTS') process.exitCode = 0;
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
`instanceof` can miss an e2e error across copies of the package, so use
|
|
272
|
+
`isE2EError`. `ConfigurationError` and `isE2EError` are both exported
|
|
273
|
+
from `e2e/runner`.
|
|
274
|
+
|
|
275
|
+
### Host
|
|
276
|
+
|
|
277
|
+
`E2EHost<Fixtures>` is what [`openHost`](#openhost) returns.
|
|
278
|
+
|
|
279
|
+
| Member | Returns |
|
|
280
|
+
| --- | --- |
|
|
281
|
+
| `runId`, `target` | `string` |
|
|
282
|
+
| [`host.openSession`](#host-opensession) | `Promise<E2ESession<Fixtures>>` |
|
|
283
|
+
| [`host.close`](#host-close) | `Promise<readonly SerializedError[]>` |
|
|
284
|
+
|
|
285
|
+
`runId` and `target` are the identity the host's sessions share.
|
|
286
|
+
|
|
287
|
+
Only one hosted attempt may be opening, live, or closing per process, even
|
|
288
|
+
across different hosts and whether the config is a file or a value.
|
|
289
|
+
|
|
290
|
+
- An overlapping open is rejected before engine attempt state starts. Await
|
|
291
|
+
`session.close()` before opening the next session on the same host.
|
|
292
|
+
- A launch that fails or is cancelled ends what it opened and leaves the host
|
|
293
|
+
usable, as a run keeps its worker for the retry.
|
|
294
|
+
- A session whose engine cleanup failed closes the host permanently, since
|
|
295
|
+
its isolation is uncertain; open a new host.
|
|
296
|
+
|
|
297
|
+
| Throws | When |
|
|
298
|
+
| --- | --- |
|
|
299
|
+
| `WORKER_PROTOCOL` | An overlapping hosted attempt, or a call on a closed host. |
|
|
300
|
+
|
|
301
|
+
#### `host.openSession`
|
|
302
|
+
|
|
303
|
+
`host.openSession(options: OpenSessionOptions)` opens a fresh
|
|
304
|
+
[session](#session) on this host's target. A session is one test attempt with
|
|
305
|
+
no retry loop. A retry opens a new session on the same worker host.
|
|
306
|
+
|
|
307
|
+
```ts
|
|
308
|
+
const session = await host.openSession({
|
|
309
|
+
name: 'checkout',
|
|
310
|
+
timeout: 60_000, // default: the config's timeout
|
|
311
|
+
onStep: (step) => {
|
|
312
|
+
if (step.phase === 'end') console.log(`${step.api} ${step.label}: ${step.status}`);
|
|
313
|
+
},
|
|
314
|
+
});
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
##### Options
|
|
318
|
+
|
|
319
|
+
| Option | Example | Type | Default |
|
|
320
|
+
| --- | --- | --- | --- |
|
|
321
|
+
| `name` | `'checkout'` | `string` | required |
|
|
322
|
+
| `timeout` | `60_000` | `number` | the config's `timeout` |
|
|
323
|
+
| `signal` | `controller.signal` | `AbortSignal` | - |
|
|
324
|
+
| `agent` | `'smart'` | `string` | the first configured agent |
|
|
325
|
+
| `onStep` | `(step) => {}` | `(progress: StepProgress) => void` | - |
|
|
326
|
+
|
|
327
|
+
- `name` is the test name for step and trace reporting.
|
|
328
|
+
- `timeout` is the deadline for the session's operations in milliseconds,
|
|
329
|
+
positive and finite, counted from when the session opens. It does not bound
|
|
330
|
+
provisioning or opening, and it does not close the session.
|
|
331
|
+
- `signal` cancels opening or live fixture calls. The host's `signal` also
|
|
332
|
+
applies. Provisioning happens in `openHost`, so only the host's `signal`
|
|
333
|
+
cancels it; either signal cancels a launch.
|
|
334
|
+
- `agent` is the agent the `agent` fixture uses when a call names none.
|
|
335
|
+
- `onStep` receives live `StepProgress`, the same feed reporters receive. It
|
|
336
|
+
must not block or throw. Its `phase` is `start`, `event`, `activity`,
|
|
337
|
+
`replay`, or `end`. An `end` event includes `status`, `durationMs`,
|
|
338
|
+
`modelCalls`, and a redacted `error` when the step failed.
|
|
339
|
+
- `timeout`, `agent`, and `onStep` belong to the session, not `openHost`.
|
|
340
|
+
|
|
341
|
+
| Throws | When |
|
|
342
|
+
| --- | --- |
|
|
343
|
+
| `INVALID_CONFIG` | `name` is not a string, or `timeout` is not positive and finite. |
|
|
344
|
+
| `CANCELLED` | The session's or the host's `signal` aborted before or during opening. |
|
|
345
|
+
| `WORKER_PROTOCOL` | Another hosted attempt is opening, live, or closing, or the host is closed. |
|
|
346
|
+
|
|
347
|
+
#### `host.close`
|
|
348
|
+
|
|
349
|
+
`host.close()` cancels any active or pending session, waits for it to close,
|
|
350
|
+
then disposes the engine, stops app processes, and releases provisioning.
|
|
351
|
+
|
|
352
|
+
```ts
|
|
353
|
+
const failures = await host.close();
|
|
354
|
+
if (failures.length > 0) throw new AggregateError(failures, 'e2e host close failed');
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
- Idempotent: it disposes and releases resources only once.
|
|
358
|
+
- A host is `AsyncDisposable`: `await using host = await openHost()` closes
|
|
359
|
+
it at scope exit and throws an `AggregateError` when closing reports
|
|
360
|
+
failures.
|
|
142
361
|
|
|
143
|
-
|
|
362
|
+
### Session
|
|
144
363
|
|
|
145
|
-
`
|
|
146
|
-
Your runner owns collection, retries, and reporting. The host prepares the
|
|
147
|
-
target's engine, starts app processes, and initializes the engine once per
|
|
148
|
-
worker lifetime. Each `host.openSession` creates a test-lifetime attempt with
|
|
149
|
-
fresh fixtures, a deadline, steps, and artifacts.
|
|
364
|
+
`E2ESession<Fixtures>` is what [`host.openSession`](#host-opensession) returns.
|
|
150
365
|
|
|
151
|
-
|
|
366
|
+
| Member | Returns |
|
|
367
|
+
| --- | --- |
|
|
368
|
+
| `fixtures` | `TestFixtures & Fixtures` |
|
|
369
|
+
| `runId`, `attemptId`, `target` | `string` |
|
|
370
|
+
| `steps()` | `readonly StepRecord[]` |
|
|
371
|
+
| `report()` | `Report \| undefined` |
|
|
372
|
+
| `artifactsDir` | `string` |
|
|
373
|
+
| `tracePath` | `string` |
|
|
374
|
+
| [`session.close`](#session-close) | `Promise<readonly SerializedError[]>` |
|
|
375
|
+
|
|
376
|
+
- `fixtures` holds `agent`, `app`, `screen`, `platform`, and the engine's
|
|
377
|
+
fixtures.
|
|
378
|
+
- `runId` and `target` are the host's run and target; `attemptId` is unique
|
|
379
|
+
to this attempt.
|
|
380
|
+
- `steps()` returns read-only `StepRecord`s for steps run so far, as the
|
|
381
|
+
[report](/reference/reporters) records them.
|
|
382
|
+
- `report()` returns the session `Report` the trace is rendered from, after
|
|
383
|
+
close and a successful trace write; otherwise `undefined`. Its status
|
|
384
|
+
describes the SDK session, not your runner's test-body verdict.
|
|
385
|
+
- `artifactsDir` is the absolute directory for this attempt's screenshots,
|
|
386
|
+
recordings, and trace.
|
|
387
|
+
- `tracePath` is the absolute path to `trace.md`; attach or read it after
|
|
388
|
+
`close()` settles.
|
|
389
|
+
|
|
390
|
+
#### `session.close`
|
|
391
|
+
|
|
392
|
+
`session.close()` cancels remaining operations, ends the attempt, and writes
|
|
393
|
+
its trace. It returns soft expectation, abandoned-operation (an unsettled
|
|
394
|
+
poll or step), and cleanup failures instead of throwing them.
|
|
152
395
|
|
|
153
396
|
```ts
|
|
397
|
+
const failures = await session.close();
|
|
398
|
+
if (failures.length > 0) throw new AggregateError(failures, 'e2e session close failed');
|
|
399
|
+
```
|
|
400
|
+
|
|
401
|
+
- Idempotent: later calls return the same failures.
|
|
402
|
+
- The host stays open for the next test.
|
|
403
|
+
- A session is `AsyncDisposable`:
|
|
404
|
+
`await using session = await host.openSession({ name: 'checkout' })`
|
|
405
|
+
closes it at scope exit and throws an `AggregateError` of the returned
|
|
406
|
+
failures.
|
|
407
|
+
|
|
408
|
+
#### Deadline and soft failures
|
|
409
|
+
|
|
410
|
+
`expect.soft` collects failures for the session's whole lifetime. Check
|
|
411
|
+
`close()`'s result to fail the host runner's test.
|
|
412
|
+
|
|
413
|
+
- `expect.poll` uses the session's remaining budget, even when its own
|
|
414
|
+
timeout is longer.
|
|
415
|
+
- Contributed fixture operations also honor that budget, including
|
|
416
|
+
operations that disable their own timeout.
|
|
417
|
+
- When the deadline passes, a poll or operation that was watching it reports
|
|
418
|
+
its own cut. Everything else still running, including an operation an
|
|
419
|
+
engine or a model never answered, fails with `TEST_TIMEOUT`.
|
|
420
|
+
- Either way the session's outcome is timed out and `close()` returns the
|
|
421
|
+
timeout. Later operations fail with `TEST_TIMEOUT`; close the session.
|
|
422
|
+
|
|
423
|
+
#### Artifacts
|
|
424
|
+
|
|
425
|
+
Artifacts live under `<output>/results/host/<resultSegment>/`. At close,
|
|
426
|
+
e2e writes `trace.md` there through the same redacted formatting reports
|
|
427
|
+
use.
|
|
428
|
+
|
|
429
|
+
- e2e generates a safe `resultSegment` from the name and the unique attempt
|
|
430
|
+
identity. The name never becomes a raw path component.
|
|
431
|
+
- Repeated names and retries get separate directories.
|
|
432
|
+
- With provider-backed secrets, the directory uses an identity-only segment,
|
|
433
|
+
since unresolved values cannot be redacted from a title slug. The trace
|
|
434
|
+
still shows the redacted name.
|
|
435
|
+
- Artifact paths are relative to `<output>/results`, not
|
|
436
|
+
`session.artifactsDir`.
|
|
437
|
+
- Video follows the target's mode for a first attempt. Failure-only video is
|
|
438
|
+
kept for soft failures and for failed, blocked, timed-out, or cancelled
|
|
439
|
+
steps.
|
|
440
|
+
|
|
441
|
+
<Note>
|
|
442
|
+
A hosted session writes its trace, not `report.json`. For your runner's
|
|
443
|
+
reporting, use `onStep`, `steps()`, `report()`, and `close()`'s failures.
|
|
444
|
+
Your runner's test-body failures are not passed to e2e.
|
|
445
|
+
|
|
446
|
+
The one-shot attempt `e2e mcp` uses stays internal. There is no top-level
|
|
447
|
+
`openSession` export.
|
|
448
|
+
</Note>
|
|
449
|
+
|
|
450
|
+
## Types
|
|
451
|
+
|
|
452
|
+
`e2e/runner` exports these types beside the functions.
|
|
453
|
+
|
|
454
|
+
| Type | Describes |
|
|
455
|
+
| --- | --- |
|
|
456
|
+
| `RunOptions`, `RunResult` | [`run`](#run)'s options and result |
|
|
457
|
+
| `RunEventSink` | The `onEvent` callback |
|
|
458
|
+
| `ListOptions`, `ListResult`, `ListedPair` | [`list`](#list)'s options, result, and one pair |
|
|
459
|
+
| `OpenHostOptions`, `E2EHost` | [`openHost`](#openhost)'s options and the [host](#host) |
|
|
460
|
+
| `OpenSessionOptions`, `E2ESession` | [`openSession`](#host-opensession)'s options and the [session](#session) |
|
|
461
|
+
| `StepProgress`, `StepRecord` | One `onStep` progress event, and one step `steps()` returns |
|
|
462
|
+
| `SerializedError` | One failure `close()` returns |
|
|
463
|
+
|
|
464
|
+
## Examples
|
|
465
|
+
|
|
466
|
+
### Playwright Test fixtures
|
|
467
|
+
|
|
468
|
+
A worker-scoped host and a test-scoped session:
|
|
469
|
+
|
|
470
|
+
```ts title="tests/fixtures.ts"
|
|
154
471
|
import { test as base } from '@playwright/test';
|
|
155
472
|
import { openHost, type E2EHost, type E2ESession } from 'e2e/runner';
|
|
156
473
|
import type { Device } from '@e2e-dev/mobile';
|
|
@@ -195,121 +512,3 @@ test('turns on dark mode', async ({ e2e }) => {
|
|
|
195
512
|
await app.screenshot();
|
|
196
513
|
});
|
|
197
514
|
```
|
|
198
|
-
|
|
199
|
-
### Host options
|
|
200
|
-
|
|
201
|
-
`openHost<Fixtures>(options?: OpenHostOptions)` returns
|
|
202
|
-
`Promise<E2EHost<Fixtures>>`. Omit `config` to discover the config from `cwd`,
|
|
203
|
-
the same walk `list` does. If opening fails, e2e tears down what started
|
|
204
|
-
before throwing the error.
|
|
205
|
-
|
|
206
|
-
| Option | Default | Meaning |
|
|
207
|
-
| --- | --- | --- |
|
|
208
|
-
| `config` | nearest `e2e.config.ts` or `e2e.config.mts` | Config file path relative to `cwd`, or an `E2EConfig` value |
|
|
209
|
-
| `cwd` | `process.cwd()` | Directory for config discovery, and the project root of a config value |
|
|
210
|
-
| `target` | the only target | Target name; required when the config declares several |
|
|
211
|
-
| `env` | `process.env` | Environment the config, engines, and providers read |
|
|
212
|
-
| `signal` | none | Cancels host provisioning, a pending session open, or live fixture calls |
|
|
213
|
-
| `headed` | `false` | Runs a browser engine headed |
|
|
214
|
-
| `onNotice` | none | Progress outside a step, such as provisioning or an app process |
|
|
215
|
-
|
|
216
|
-
A config file and its project-file imports are loaded fresh for each host.
|
|
217
|
-
Package imports remain shared; their singleton state is not recreated per host.
|
|
218
|
-
A config value uses the engine instances in that value. Neither form allows
|
|
219
|
-
overlapping hosted attempts in one process. A config value's engine instance
|
|
220
|
-
belongs to one host until it closes; a second host must use another instance.
|
|
221
|
-
|
|
222
|
-
### Host
|
|
223
|
-
|
|
224
|
-
| Member | Meaning |
|
|
225
|
-
| --- | --- |
|
|
226
|
-
| `runId`, `target` | Identity shared by the host's sessions |
|
|
227
|
-
| `openSession(options)` | Opens a fresh `E2ESession<Fixtures>` on this host's target |
|
|
228
|
-
| `close()` | Cancels any active or pending session, waits for it to close, then disposes the engine, stops app processes, and releases provisioning. Returns `Promise<readonly SerializedError[]>` |
|
|
229
|
-
|
|
230
|
-
Only one hosted attempt may be opening, live, or closing per process, even
|
|
231
|
-
across different hosts. An overlapping open is rejected before engine
|
|
232
|
-
attempt state starts. Await `session.close()` before opening the next
|
|
233
|
-
session on the same host. A launch that fails or is cancelled ends what it
|
|
234
|
-
opened and leaves the host usable, as a run keeps its worker for the retry.
|
|
235
|
-
A session whose engine cleanup failed closes the host permanently, since
|
|
236
|
-
its isolation is uncertain; open a new host. Overlap and closed-host errors
|
|
237
|
-
use `WORKER_PROTOCOL`.
|
|
238
|
-
|
|
239
|
-
`host.close()` is idempotent and disposes and releases resources only once.
|
|
240
|
-
A host is `AsyncDisposable`; `await using host = await openHost()` closes it
|
|
241
|
-
at scope exit and throws an `AggregateError` when closing reports failures.
|
|
242
|
-
|
|
243
|
-
## host.openSession
|
|
244
|
-
|
|
245
|
-
`host.openSession(options: OpenSessionOptions)` returns
|
|
246
|
-
`Promise<E2ESession<Fixtures>>`. A session is one test attempt with no retry
|
|
247
|
-
loop. A retry opens a new session on the same worker host.
|
|
248
|
-
|
|
249
|
-
### Session options
|
|
250
|
-
|
|
251
|
-
| Option | Default | Meaning |
|
|
252
|
-
| --- | --- | --- |
|
|
253
|
-
| `name` | required | Test name for step and trace reporting; a string |
|
|
254
|
-
| `timeout` | config `timeout` | Deadline for the session's operations in milliseconds, positive and finite, counted from when it opens. Does not bound provisioning or opening, and does not close the session |
|
|
255
|
-
| `signal` | none | Cancels opening or live fixture calls; the host signal also applies |
|
|
256
|
-
| `agent` | the first configured agent | Agent the `agent` fixture uses when a call names none |
|
|
257
|
-
| `onStep` | none | Live `StepProgress`, the same feed reporters receive |
|
|
258
|
-
|
|
259
|
-
`timeout`, `agent`, and `onStep` belong to the session, not `openHost`.
|
|
260
|
-
Use `signal` to cancel provisioning or launch. `onStep` must not block or
|
|
261
|
-
throw. Its `phase` is `start`, `event`, `activity`, `replay`, or `end`.
|
|
262
|
-
An `end` event includes `status`, `durationMs`, `modelCalls`, and a redacted
|
|
263
|
-
`error` when the step failed.
|
|
264
|
-
|
|
265
|
-
### Session
|
|
266
|
-
|
|
267
|
-
| Member | Meaning |
|
|
268
|
-
| --- | --- |
|
|
269
|
-
| `fixtures` | `agent`, `app`, `screen`, `platform`, and the engine's fixtures, typed as `TestFixtures & Fixtures` |
|
|
270
|
-
| `runId`, `attemptId`, `target` | Host run and target, plus a unique attempt id |
|
|
271
|
-
| `steps()` | Read-only `StepRecord`s for steps run so far, as the [report](/reference/reporters) records them |
|
|
272
|
-
| `artifactsDir` | Absolute directory for this attempt's screenshots, recordings, and trace |
|
|
273
|
-
| `tracePath` | Absolute string path to `trace.md`; attach or read it after `close()` settles |
|
|
274
|
-
| `close()` | Cancels remaining operations, ends the attempt, and writes its trace. Returns soft expectation and cleanup failures as `Promise<readonly SerializedError[]>` instead of throwing them |
|
|
275
|
-
|
|
276
|
-
`session.close()` is idempotent and returns the same failures on later calls.
|
|
277
|
-
The host stays open for the next test. A session is `AsyncDisposable`:
|
|
278
|
-
`await using session = await host.openSession({ name: 'checkout' })` closes
|
|
279
|
-
it at scope exit and throws an `AggregateError` of the returned failures.
|
|
280
|
-
|
|
281
|
-
`expect.soft` collects failures for the session's whole lifetime. Check
|
|
282
|
-
`close()`'s result to fail the host runner's test. `expect.poll` uses the
|
|
283
|
-
session's remaining budget, even when its own timeout is longer. Contributed
|
|
284
|
-
fixture operations also honor that budget, including operations that disable
|
|
285
|
-
their own timeout. When the deadline passes, a poll or operation that was
|
|
286
|
-
watching it reports its own cut; everything else still running, including an
|
|
287
|
-
operation an engine or a model never answered, fails with `TEST_TIMEOUT`.
|
|
288
|
-
Either way the session's outcome is timed out and `close()` returns the
|
|
289
|
-
timeout. Later operations fail with `TEST_TIMEOUT`; close the session.
|
|
290
|
-
|
|
291
|
-
Artifacts live under `<output>/results/host/<resultSegment>/`, with
|
|
292
|
-
`trace.md` written at close through the existing redacted report-formatting
|
|
293
|
-
pipeline. e2e generates the safe `resultSegment` from the name and unique
|
|
294
|
-
attempt identity; the name never becomes a raw path component. Repeated
|
|
295
|
-
names and retries get separate directories. With provider-backed secrets,
|
|
296
|
-
the directory uses an identity-only segment because unresolved values cannot
|
|
297
|
-
be redacted from a title slug. The trace still displays the redacted name.
|
|
298
|
-
|
|
299
|
-
Opening failures are classified, redacted errors; raw engine causes are not
|
|
300
|
-
exposed. Match them with `isE2EError` and
|
|
301
|
-
`error.code`, as for `list`. `openHost` reports `UNKNOWN_TARGET` for an
|
|
302
|
-
unknown target and `TARGET_REQUIRED` when several targets exist and none
|
|
303
|
-
was named.
|
|
304
|
-
|
|
305
|
-
<Note>
|
|
306
|
-
A hosted session writes its trace, not a run report. Your runner owns retries
|
|
307
|
-
and reporting; use `onStep`, `steps()`, and `close()`'s failures. Video follows
|
|
308
|
-
the target's mode for a first attempt, including retaining failure-only video
|
|
309
|
-
for soft failures and failed, blocked, timed-out, or cancelled steps. Your
|
|
310
|
-
runner's test-body failures are not
|
|
311
|
-
passed to e2e.
|
|
312
|
-
|
|
313
|
-
The one-shot attempt used by `e2e mcp` remains internal. There is no
|
|
314
|
-
top-level `openSession` export.
|
|
315
|
-
</Note>
|