e2e 0.16.0 → 0.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (256) hide show
  1. package/README.md +10 -0
  2. package/dist/.build.json +1 -1
  3. package/dist/agent/act.js +7 -4
  4. package/dist/agent/action-dispatcher.js +4 -5
  5. package/dist/agent/actions.d.ts +32 -23
  6. package/dist/agent/actions.js +38 -24
  7. package/dist/agent/ai-sdk.js +5 -4
  8. package/dist/agent/error.js +3 -3
  9. package/dist/agent/invocation.js +3 -4
  10. package/dist/agent/model/sdk.js +2 -0
  11. package/dist/agent/model/tool-choice.d.ts +9 -0
  12. package/dist/agent/model/tool-choice.js +14 -0
  13. package/dist/agent/observation-feed.d.ts +5 -6
  14. package/dist/agent/observation-feed.js +8 -11
  15. package/dist/agent/observation.d.ts +40 -11
  16. package/dist/agent/observation.js +109 -59
  17. package/dist/agent/phases.d.ts +7 -1
  18. package/dist/agent/phases.js +12 -3
  19. package/dist/agent/point-tap.d.ts +8 -8
  20. package/dist/agent/point-tap.js +1 -1
  21. package/dist/agent/prompts.js +3 -0
  22. package/dist/agent/replay.d.ts +19 -8
  23. package/dist/agent/replay.js +29 -29
  24. package/dist/agent/scroll-target.d.ts +2 -3
  25. package/dist/agent/step-cache.d.ts +43 -15
  26. package/dist/agent/step-cache.js +156 -67
  27. package/dist/agent/tool-loop.js +94 -5
  28. package/dist/cache/anchors.d.ts +72 -23
  29. package/dist/cache/anchors.js +233 -60
  30. package/dist/cache/context.d.ts +25 -14
  31. package/dist/cache/context.js +50 -8
  32. package/dist/cache/decide.d.ts +2 -2
  33. package/dist/cache/decide.js +4 -6
  34. package/dist/cache/identity.d.ts +18 -4
  35. package/dist/cache/identity.js +9 -4
  36. package/dist/cache/recorder.d.ts +15 -1
  37. package/dist/cache/recorder.js +36 -23
  38. package/dist/cache/rekeyed.d.ts +46 -0
  39. package/dist/cache/rekeyed.js +89 -0
  40. package/dist/cache/relocate.d.ts +14 -14
  41. package/dist/cache/relocate.js +33 -25
  42. package/dist/cache/route.d.ts +32 -33
  43. package/dist/cache/route.js +94 -50
  44. package/dist/cache/store.d.ts +6 -0
  45. package/dist/cache/store.js +21 -1
  46. package/dist/cache/trace.d.ts +53 -14
  47. package/dist/cache/trace.js +76 -30
  48. package/dist/cli/bin.js +2 -2
  49. package/dist/cli/index.js +43 -32
  50. package/dist/cli/init/engines.d.ts +1 -3
  51. package/dist/cli/init/engines.js +2 -2
  52. package/dist/cli/init/gateways.d.ts +1 -1
  53. package/dist/cli/init/gateways.js +11 -1
  54. package/dist/cli/init/mcp-config.d.ts +2 -1
  55. package/dist/cli/init/mcp-config.js +8 -3
  56. package/dist/cli/init/sibling-versions.json +1 -1
  57. package/dist/cli/init/versions.d.ts +0 -8
  58. package/dist/cli/init/versions.js +1 -10
  59. package/dist/cli/mcp.d.ts +3 -2
  60. package/dist/cli/mcp.js +52 -16
  61. package/dist/cli/run-output.d.ts +17 -0
  62. package/dist/cli/run-output.js +45 -0
  63. package/dist/collect/collect.js +8 -4
  64. package/dist/collect/registry.js +27 -16
  65. package/dist/config/compiled-files.d.ts +36 -0
  66. package/dist/config/compiled-files.js +50 -0
  67. package/dist/config/diagnose.js +21 -19
  68. package/dist/config/esm-hooks.d.ts +39 -44
  69. package/dist/config/esm-hooks.js +238 -63
  70. package/dist/config/load.d.ts +1 -7
  71. package/dist/config/load.js +23 -66
  72. package/dist/config/register.d.ts +8 -0
  73. package/dist/config/register.js +9 -0
  74. package/dist/config/resolve.d.ts +1 -0
  75. package/dist/config/resolve.js +33 -1
  76. package/dist/config/specifiers.d.ts +19 -0
  77. package/dist/config/specifiers.js +33 -0
  78. package/dist/config/tsconfig.d.ts +56 -0
  79. package/dist/config/tsconfig.js +115 -0
  80. package/dist/config/typescript.d.ts +19 -0
  81. package/dist/config/typescript.js +253 -0
  82. package/dist/engine/contract.d.ts +1 -0
  83. package/dist/engine/index.d.ts +4 -2
  84. package/dist/engine/session.js +2 -2
  85. package/dist/expect/async.js +4 -2
  86. package/dist/expect/poll-scope.d.ts +52 -0
  87. package/dist/expect/poll-scope.js +113 -0
  88. package/dist/expect/poll.js +33 -6
  89. package/dist/expect/values.js +11 -4
  90. package/dist/internal/abandoned.d.ts +11 -0
  91. package/dist/internal/abandoned.js +23 -0
  92. package/dist/internal/brands.d.ts +2 -0
  93. package/dist/internal/brands.js +2 -0
  94. package/dist/internal/client-identity.d.ts +9 -0
  95. package/dist/internal/client-identity.js +14 -0
  96. package/dist/internal/errors.d.ts +10 -3
  97. package/dist/internal/errors.js +23 -8
  98. package/dist/internal/node-version.d.ts +16 -7
  99. package/dist/internal/node-version.js +50 -15
  100. package/dist/internal/package-manager.js +1 -1
  101. package/dist/internal/realm-slot.d.ts +5 -4
  102. package/dist/internal/redact.d.ts +9 -4
  103. package/dist/internal/redact.js +120 -23
  104. package/dist/internal/time.d.ts +26 -4
  105. package/dist/internal/time.js +58 -14
  106. package/dist/internal/urls.d.ts +9 -5
  107. package/dist/internal/urls.js +19 -10
  108. package/dist/locator/engine.d.ts +12 -2
  109. package/dist/locator/engine.js +55 -14
  110. package/dist/locator/screen.js +59 -9
  111. package/dist/mcp/catalog.d.ts +3 -2
  112. package/dist/mcp/catalog.js +6 -6
  113. package/dist/mcp/output.d.ts +59 -0
  114. package/dist/mcp/output.js +129 -0
  115. package/dist/mcp/server.d.ts +4 -0
  116. package/dist/mcp/server.js +9 -12
  117. package/dist/mcp/session.d.ts +8 -1
  118. package/dist/mcp/session.js +56 -15
  119. package/dist/mcp/tools.js +7 -5
  120. package/dist/oauth/chatgpt.js +3 -23
  121. package/dist/oauth/cli.js +1 -0
  122. package/dist/oauth/copilot.d.ts +5 -4
  123. package/dist/oauth/copilot.js +80 -7
  124. package/dist/oauth/grok.js +2 -1
  125. package/dist/oauth/models.js +2 -1
  126. package/dist/oauth/opencode-console.d.ts +10 -0
  127. package/dist/oauth/opencode-console.js +133 -0
  128. package/dist/oauth/providers/github-copilot.d.ts +12 -0
  129. package/dist/oauth/providers/github-copilot.js +77 -23
  130. package/dist/oauth/providers/opencode-console.d.ts +31 -0
  131. package/dist/oauth/providers/opencode-console.js +148 -0
  132. package/dist/oauth/providers.d.ts +2 -6
  133. package/dist/oauth/providers.js +3 -7
  134. package/dist/oauth/responses.d.ts +9 -0
  135. package/dist/oauth/responses.js +23 -0
  136. package/dist/report/build.d.ts +26 -0
  137. package/dist/report/build.js +16 -8
  138. package/dist/report/failure-text.d.ts +6 -1
  139. package/dist/report/failure-text.js +14 -4
  140. package/dist/report/format.d.ts +10 -5
  141. package/dist/report/format.js +12 -5
  142. package/dist/report/junit.js +11 -12
  143. package/dist/report/list.d.ts +12 -0
  144. package/dist/report/list.js +61 -24
  145. package/dist/report/markdown.d.ts +1 -1
  146. package/dist/report/markdown.js +37 -14
  147. package/dist/report/repeats.d.ts +6 -1
  148. package/dist/report/repeats.js +12 -3
  149. package/dist/run/artifacts.d.ts +22 -4
  150. package/dist/run/artifacts.js +71 -3
  151. package/dist/run/execute.d.ts +34 -5
  152. package/dist/run/execute.js +93 -44
  153. package/dist/run/extended-fixtures.js +6 -2
  154. package/dist/run/failure-evidence.js +10 -4
  155. package/dist/run/fixtures.js +6 -8
  156. package/dist/run/in-process.d.ts +2 -0
  157. package/dist/run/in-process.js +52 -10
  158. package/dist/run/last-run.d.ts +42 -8
  159. package/dist/run/last-run.js +121 -14
  160. package/dist/run/process-output.d.ts +73 -0
  161. package/dist/run/process-output.js +130 -0
  162. package/dist/run/provision.d.ts +2 -0
  163. package/dist/run/provision.js +1 -0
  164. package/dist/run/realm.d.ts +4 -1
  165. package/dist/run/realm.js +20 -6
  166. package/dist/run/records.d.ts +20 -4
  167. package/dist/run/records.js +26 -0
  168. package/dist/run/retry.d.ts +10 -1
  169. package/dist/run/retry.js +30 -12
  170. package/dist/run/runner.d.ts +9 -1
  171. package/dist/run/runner.js +101 -39
  172. package/dist/run/scheduler.js +255 -19
  173. package/dist/run/secrecy.d.ts +22 -19
  174. package/dist/run/secrecy.js +22 -18
  175. package/dist/run/serial.d.ts +75 -2
  176. package/dist/run/serial.js +243 -59
  177. package/dist/run/standalone.js +1 -1
  178. package/dist/run/steps.d.ts +0 -6
  179. package/dist/run/steps.js +3 -20
  180. package/dist/run/trace-redaction.d.ts +5 -6
  181. package/dist/run/trace-redaction.js +5 -6
  182. package/dist/run/worker/entry.js +40 -18
  183. package/dist/run/worker/handle.js +10 -5
  184. package/dist/run/worker/protocol.d.ts +85 -3
  185. package/dist/run/worker/session.d.ts +2 -0
  186. package/dist/run/worker/session.js +29 -0
  187. package/dist/telemetry/environment.d.ts +2 -1
  188. package/dist/telemetry/environment.js +12 -2
  189. package/dist/telemetry/events.js +1 -0
  190. package/dist/telemetry/project.d.ts +5 -6
  191. package/dist/telemetry/project.js +6 -11
  192. package/dist/telemetry/store.d.ts +5 -5
  193. package/dist/telemetry/store.js +5 -7
  194. package/dist/telemetry/telemetry.d.ts +2 -3
  195. package/dist/telemetry/telemetry.js +2 -4
  196. package/dist/types.d.ts +15 -6
  197. package/docs/agent-steps.mdx +1 -0
  198. package/docs/agents.mdx +7 -3
  199. package/docs/api-testing.mdx +2 -1
  200. package/docs/authentication.mdx +6 -3
  201. package/docs/browser.mdx +49 -12
  202. package/docs/bug-bash.mdx +2 -1
  203. package/docs/cache.mdx +66 -19
  204. package/docs/ci.mdx +14 -9
  205. package/docs/coding-agents.mdx +2 -1
  206. package/docs/core-concepts.mdx +208 -0
  207. package/docs/debugging.mdx +25 -12
  208. package/docs/decision-models.mdx +142 -0
  209. package/docs/example-projects.mdx +32 -0
  210. package/docs/executors.mdx +7 -2
  211. package/docs/explore.mdx +1 -0
  212. package/docs/github.mdx +14 -5
  213. package/docs/index.mdx +9 -5
  214. package/docs/integrations/eas.mdx +1 -0
  215. package/docs/integrations/index.mdx +1 -0
  216. package/docs/integrations/kernel.mdx +3 -2
  217. package/docs/migrate/cypress.mdx +26 -3
  218. package/docs/migrate/detox.mdx +24 -1
  219. package/docs/migrate/maestro.mdx +28 -4
  220. package/docs/migrate/playwright.mdx +44 -18
  221. package/docs/migrate/selenium.mdx +27 -3
  222. package/docs/mobile-ci.mdx +6 -5
  223. package/docs/mobile.mdx +96 -8
  224. package/docs/models.mdx +60 -9
  225. package/docs/quickstart.mdx +28 -16
  226. package/docs/reference/agent.mdx +13 -3
  227. package/docs/reference/app.mdx +2 -1
  228. package/docs/reference/cli.mdx +41 -18
  229. package/docs/reference/config.mdx +67 -13
  230. package/docs/reference/engine.mdx +25 -13
  231. package/docs/reference/environment.mdx +10 -2
  232. package/docs/reference/errors.mdx +13 -10
  233. package/docs/reference/expect.mdx +36 -13
  234. package/docs/reference/mcp.mdx +21 -8
  235. package/docs/reference/mobile.mdx +24 -13
  236. package/docs/reference/reporters.mdx +9 -2
  237. package/docs/reference/screen.mdx +49 -16
  238. package/docs/reference/test.mdx +32 -5
  239. package/docs/reference/web.mdx +133 -23
  240. package/docs/security.mdx +58 -42
  241. package/docs/subscriptions.mdx +92 -12
  242. package/docs/telemetry.mdx +9 -6
  243. package/docs/tools.mdx +1 -0
  244. package/docs/{starting-your-app.mdx → web.mdx} +157 -11
  245. package/docs/writing-an-engine.mdx +2 -1
  246. package/package.json +25 -8
  247. package/schema/report-v1.schema.json +14 -1
  248. package/skills/e2e/SKILL.md +5 -3
  249. package/skills/e2e/references/agent.md +33 -11
  250. package/skills/e2e/references/debugging.md +4 -4
  251. package/skills/e2e/references/mcp.md +8 -3
  252. package/skills/e2e/references/running.md +15 -9
  253. package/skills/e2e/references/setup.md +45 -16
  254. package/skills/e2e/references/writing-tests.md +35 -18
  255. package/docs/viewports.mdx +0 -146
  256. package/docs/writing-tests.mdx +0 -256
@@ -1,6 +1,7 @@
1
1
  ---
2
2
  title: Errors
3
3
  description: Every code you can see, its class, and its exit code.
4
+ icon: "triangle-exclamation"
4
5
  ---
5
6
 
6
7
  Every error carries a class. The class determines the exit code; the code
@@ -56,8 +57,8 @@ try {
56
57
  | `TEST_SETUP_FAILED` | Test prerequisites outside the product failed to arrange (blocked: test_setup). Also a `test.extend()` fixture that broke its contract: returned without calling `use()`, called it twice, or redefined a fixture the target's engine contributes | configuration | 2 |
57
58
  | `AUTOMATION_UNSUPPORTED` | The step needs an interaction the available toolset cannot perform, e.g. drag-and-drop (blocked: automation) | test | 1 |
58
59
  | `MODEL_UNAVAILABLE` | No model configured: the built-in agent's `agents.<name>.model` holds no AI SDK model instance; the message names the agent that was selected. Checked once per run when the `agent` fixture is first acquired; recorded as one run-level error that stops the run. `e2e explore` checks before anything starts, a custom executor's entry included, since explore runs the built-in agent. A custom executor without a model is not checked: its `act` and `assert` steps run, and `waitFor` or `extract` fails with this code on its first call. Evicts no [recording](/cache). | configuration | 2 |
59
- | `POLICY_DENIED` | A denied destination (`file:`, `data:`, `javascript:`, a malformed URL), a forbidden fill (an undeclared secret, a non-editable sink, a password into a field that is not a password field), or pixel evidence after a secret fill (`vision: 'only'` with pixels withheld). Also reading a value, text, or attribute from a secure field, the getters and the `toHaveValue`, `toHaveText`, `toContainText`, and `toHaveAttribute` matchers alike. An unknown tool or an undeclared field is refused to the model as the call's result and is not this code | configuration | 2 |
60
- | `APP_UNREACHABLE` | The app URL could not be reached, or an app command exited early or exceeded `startupTimeout`. The startup error includes recent log output with `command.env` values redacted, and suggests adding a `log` if none is configured. Under the agent, also any engine fault with no more specific code: on a device, an iOS automation runner that is busy or wedged reports here with a message that names the runner, the session and device, and the recovery; see [Testing iOS and Android](/mobile#troubleshooting) | infrastructure | 3 |
60
+ | `POLICY_DENIED` | A denied destination (a navigation scheme other than `http:` or `https:`, `about:blank` aside, a malformed URL), a forbidden fill (an undeclared secret, a non-editable sink, a password into a field that is not a password field), or pixel evidence after a secret fill (`vision: 'only'` with pixels withheld, or an `act` or `vision: true` step whose semantic capture timed out before the step did). Also reading a value, text, or attribute from a secure field, the getters and the `toHaveValue`, `toHaveText`, `toContainText`, and `toHaveAttribute` matchers alike. An unknown tool or an undeclared field is refused to the model as the call's result and is not this code | configuration | 2 |
61
+ | `APP_UNREACHABLE` | The app URL could not be reached, or an app command exited early or exceeded `startupTimeout`. The startup error includes recent log output with `command.env` values redacted, and suggests adding a `log` if none is configured. Under the agent, also any engine fault with no more specific code: on a device, an iOS automation runner that is busy or wedged reports here with a message that names the runner, the session and device, and the recovery; see [Mobile](/mobile#troubleshooting) | infrastructure | 3 |
61
62
  | `APP_ALREADY_RUNNING` | `readyUrl` already answered before the target's `app.command` was spawned and `reuseExisting` was not in effect; stop that process or set `reuseExisting: true` locally | infrastructure | 3 |
62
63
  | `MODEL_PROVIDER_FAILED` | Provider transport failure (network error, 5xx, rate limit, or a request with no response within 120 s) after the adapter's transport retries (up to 5 per model call, honoring `retry-after`, bounded by the remaining step timeout). A request that stalls is aborted at 120 s and sent again, and an `agent.act` step's `--debug` transcript notes it. The bound covers the whole request, so a model that needs longer than 120 s for one answer fails here too. Evicts no [recording](/cache) | infrastructure | 3 |
63
64
  | `CANCELLED` | The runner cancelled the operation, without a process signal: an `e2e mcp` `open_session` whose request the client cancelled or whose connection closed, or that the server's shutdown cut short, after stopping what it had started | infrastructure | 3 |
@@ -73,7 +74,7 @@ try {
73
74
  | `ASSERTION_INCONCLUSIVE` | `agent.assert` could not judge: the screen did not show enough to decide either way (the relevant part is not on screen, still loading, or unreadable). `agent.extract` raises it when the model answers that the screen does not show the requested data, instead of returning a placeholder. A failure, never a pass; the explanation says what was missing. Judged from the tree alone with `vision` off, the message adds `pass vision: true when the answer is in pixels`, as a condition (`if the engine captures pixels, ...`) until a step of the attempt has received pixels; no hint when the engine declares no screenshot capture, a secret was filled, or an earlier step's pixel request was degraded | test | 1 |
74
75
  | `AUTHENTICATION_FAILED` | The executor concluded that signing in failed inside the app: a product verdict. `AUTH_CREDENTIAL_INVALID` is the blocked counterpart for a configured credential the app rejects | test | 1 |
75
76
  | `STEP_NO_CONCLUSION` | The executor used every turn without calling `complete_step`; the step fails with the agent's last summary | test | 1 |
76
- | `REPLAY_STALE` | Under `cache.strict` or `--strict-cache`, the step's recording exists but no longer replays (`step.cache.reason` is one of `invalid-entry`, `wrong-context`, `target-not-found`, `target-ambiguous`, `viewport-changed`, `action-failed`, `action-uncertain`, `end-mismatch`), so the step fails with no model call instead of handing off to the agent. Not retried: a retry never replays and would pass live. Re-record with a read-write run without the flag, and with `cache.strict` off when the config set it, then commit the changed entry; the message names the knob that turned strict on and the configured cache directory (or `cache.store`) | configuration | 2 |
77
+ | `REPLAY_STALE` | Under `cache.strict` or `--strict-cache`, the step's recording exists but no longer replays (`step.cache.reason` is one of `invalid-entry`, `wrong-context`, `target-not-found`, `target-ambiguous`, `viewport-changed`, `action-failed`, `action-uncertain`, `end-mismatch`, or `no-entry` while the cache directory, not a custom `cache.store`, holds a recording of the step under another key, which the message names), so the step fails with no model call instead of handing off to the agent. Not retried: a retry never replays and would pass live. Re-record with a read-write run without the flag, and with `cache.strict` off when the config set it, then commit the changed entry; the message names the knob that turned strict on and the configured cache directory (or `cache.store`) | configuration | 2 |
77
78
 
78
79
  ## Runner codes
79
80
 
@@ -83,8 +84,8 @@ These appear in terminal output and in the report as plain string codes.
83
84
  | --- | --- | --- | ---: |
84
85
  | `CONFIG_NOT_FOUND` | `--config <path>` names a file that does not exist, or no `e2e.config.ts` or `e2e.config.mts` exists from the working directory up to the repository root | configuration | 2 |
85
86
  | `CONFIG_AMBIGUOUS` | Both `e2e.config.ts` and `e2e.config.mts` in one directory | configuration | 2 |
86
- | `CONFIG_LOAD_FAILED` | The config module failed to import: a package that is declared but not installed, an undeclared package, a wrong subpath, a removed export, or any other throw while it evaluates. The message names the fix. A config error raised while it evaluates keeps its own code: an option a factory the config calls (`web()`, `mobile()`, `defineEngine()`) refuses, or a `secrets.get()` handle turned into a string, is `INVALID_CONFIG` | configuration | 2 |
87
- | `INVALID_CONFIG` | A factory the config module calls (`web()`, `mobile()`, `defineEngine()`) refused its options, with the factory's own message; or an unknown key, out-of-range value, invalid enum, missing `targets`, a target key outside `{ name, platform, engine, app, trace, video }`, a target `app` key that is unknown or malformed, `app` on a target without an engine, a `web()` target without `app.url` or with a device field, a `mobile()` target without `app.bundleId` or `app.appPath`, with `app.url` (not supported yet), a link in `bundleId`, or an unknown permission name or state (the message names the nearest name), an unknown key in a `web()` or `mobile()` option, including `web({ connect })` and `web({ basicAuth })` (the message names the nearest one), a `trace` or `video` that is not a recording mode, the removed artifact kinds list (`artifacts: [...]` or `artifacts.kinds`) and `artifacts.trace` block (the message names the `trace` mode they meant), the removed `artifacts.video` block or `video` artifact kind (the message names the `video` option), the removed `specVersion` key, a `secrets.get()` in an engine option that names no `secrets` entry (a credential's password is not one), a `secrets.get()` reference turned into a string while the config evaluates (a template literal, `String()`) or given where a string is needed (an `app.command` `args` entry or `env` value; only an engine option that declares secrets takes one), an `app.command` `args` entry or `env` value that is not a string (an `undefined` `env` value is dropped instead), an unknown key in `app.command` (the message names the nearest one), an `engine` that is not a `defineEngine` handle, `json` reporter combined with `list`, duplicate or malformed target name, a secret or credential password shorter than 6 characters (code points), oversized `agents.<name>.context`, an `expect.poll` `timeout` or `interval` that is not a finite number of milliseconds, a `tests` value that is not a glob or a list of globs, a `tests` list of only `!` exclusions, the removed top-level `limits` key, an agents entry that is a bare `StepExecutor` (the message names `{ executor }`) or a model instance (the message names `{ model }`), an unknown agent key (the message names the nearest one), the removed agent keys `timeout` and `maxTurns` (the message names `judgmentTimeout` and `maxModelCalls`), `system` or `tools` beside `executor`, a `tools` entry `defineTool` did not build or that takes a built-in tool name (a grammar verb, `complete_step`, `locate`, `start_recording`, `stop_recording`, or `report_finding`), `createToolLoopExecutor({ maxTurns })`, `reuseExisting` on an app whose `app.url` asks for a free port, or a port-0 `app.url` with no `app.command` to serve it | configuration | 2 |
87
+ | `CONFIG_LOAD_FAILED` | The config module failed to import: a package that is declared but not installed, an undeclared package, a wrong subpath, a removed export, or any other throw while it evaluates. The message names the fix. A config error raised while it evaluates keeps its own code: an option a factory the config calls (`web()`, `mobile()`, `defineEngine()`, `decisionExecutor()`) refuses, or a `secrets.get()` handle turned into a string, is `INVALID_CONFIG` | configuration | 2 |
88
+ | `INVALID_CONFIG` | A factory the config module calls (`web()`, `mobile()`, `defineEngine()`, `decisionExecutor()`) refused its options, with the factory's own message; or an unknown key, out-of-range value, invalid enum, missing `targets`, a target key outside `{ name, platform, engine, app, trace, video }`, a target `app` key that is unknown or malformed, `app` on a target without an engine, a `web()` target without `app.url` or with a device field, a `mobile()` target without `app.bundleId` or `app.appPath`, with `app.url` (not supported yet), a link in `bundleId`, or an unknown permission name or state (the message names the nearest name), an unknown key in a `web()` or `mobile()` option, including `web({ connect })` and `web({ basicAuth })` (the message names the nearest one), a `trace` or `video` that is not a recording mode, the removed artifact kinds list (`artifacts: [...]` or `artifacts.kinds`) and `artifacts.trace` block (the message names the `trace` mode they meant), the removed `artifacts.video` block or `video` artifact kind (the message names the `video` option), the removed `specVersion` key, a `secrets.get()` in an engine option that names no `secrets` entry (a credential's password is not one), a `secrets.get()` reference turned into a string while the config evaluates (a template literal, `String()`) or given where a string is needed (an `app.command` `args` entry or `env` value; only an engine option that declares secrets takes one), an `app.command` `args` entry or `env` value that is not a string (an `undefined` `env` value is dropped instead), an unknown key in `app.command` (the message names the nearest one), an `engine` that is not a `defineEngine` handle, `json` reporter combined with `list`, duplicate or malformed target name, a secret or credential password shorter than 6 characters (code points), two credentials or two secrets whose names map to one `E2E_USER_*` or `E2E_SECRET_*` override variable (`api-key` and `api_key`), oversized `agents.<name>.context`, an `expect.poll` `timeout` or `interval` that is not a finite number of milliseconds, a `tests` value that is not a glob or a list of globs, a `tests` list of only `!` exclusions, the removed top-level `limits` key, an agents entry that is a bare `StepExecutor` (the message names `{ executor }`) or a model instance (the message names `{ model }`), an unknown agent key (the message names the nearest one), the removed agent keys `timeout` and `maxTurns` (the message names `judgmentTimeout` and `maxModelCalls`), `system` or `tools` beside `executor`, a `tools` entry `defineTool` did not build or that takes a built-in tool name (a grammar verb, `complete_step`, `locate`, `start_recording`, `stop_recording`, or `report_finding`), `createToolLoopExecutor({ maxTurns })`, `reuseExisting` on an app whose `app.url` asks for a free port, or a port-0 `app.url` with no `app.command` to serve it | configuration | 2 |
88
89
  | `CONFIG_NOT_DETERMINISTIC` | A worker re-resolved the config module and got a different digest than the runner | configuration | 2 |
89
90
  | `APP_URL_REQUIRED` | A navigation or relative URL on a target that declares no `app.url`, or an `app.command` with no `app.url` or `app.readyUrl` to poll | configuration | 2 |
90
91
  | `INVALID_APP_URL` | Base URL with userinfo, a query, a fragment, a non-HTTP scheme, plain HTTP on a non-loopback host, port 0 on a host other than a literal loopback address (`127.0.0.1`, `[::1]`), a loopback address the run could not bind a free port on, or unparseable | configuration | 2 |
@@ -104,24 +105,26 @@ These appear in terminal output and in the report as plain string codes.
104
105
  | `NO_LAST_RUN` | `--last-failed` found no `report.json` where the run writes it, or a file there that is not a `report-1` document. Run once without the flag first. | configuration | 2 |
105
106
  | `ONLY_IN_CI` | A focused test reached a CI run | configuration | 2 |
106
107
  | `COLLECTION_ERROR` | Registration outside collection, an invalid title, a non-function body, an async `describe` body, nested serial groups, an option forbidden inside a serial group, a setup test inside `describe`, an invalid session name, a malformed test option such as `tags` that is not a list of distinct names, a test or `describe` option key the call does not take (the message names the nearest one), a test file that fails to import (in a run with no positionals, or a file the positionals selected), a session that a selected test consumes or `e2e explore --session` names but no collected setup test produces (the message lists the declared sessions), a consumed session on a target whose engine has no `state` capability to save it, or a positional path outside the project root | configuration | 2 |
107
- | `POLICY_DENIED` | A denied destination (`file:`, `data:`, `javascript:`, a malformed URL), a forbidden fill (an undeclared secret, a non-editable sink, a mismatched purpose, a `Secret` reaching a JSON sink), or tainted pixels (`app.screenshot()` after a secret fill, `vision: 'only'` with pixels withheld); see the [security model](/security). An unknown tool or an undeclared field goes back to the model as the call's result, never this code | configuration | 2 |
108
+ | `POLICY_DENIED` | A denied destination (a navigation scheme other than `http:` or `https:`, `about:blank` aside, a malformed URL), a forbidden fill (an undeclared secret, a non-editable sink, a mismatched purpose, a `Secret` reaching a JSON sink), or tainted pixels (`app.screenshot()` after a secret fill, `vision: 'only'` with pixels withheld, an `act` or `vision: true` step whose semantic capture timed out before the step did); see the [security model](/security). An unknown tool or an undeclared field goes back to the model as the call's result, never this code | configuration | 2 |
108
109
  | `SESSION_MISMATCH` | A stored session does not match the current run, target, engine, platform, or app identity | configuration | 2 |
109
110
  | `SESSION_EXPIRED` | A stored session is past its expiry at restore time (configuration, exit 2), or a setup test saved state whose expiry is not after its creation (test, exit 1) | configuration or test | 2 or 1 |
110
111
  | `SESSION_INVALID` | A stored session failed authentication. Tampered or corrupt. | configuration | 2 |
111
112
  | `SESSION_UNAVAILABLE` | A test requires a session no selected setup test produced | configuration | 2 |
112
113
  | `SESSION_CONTRACT` | A setup test saved a session twice, saved an undeclared name, or finished without saving a declared one | test | 1 |
113
- | `TEST_TIMEOUT` | The attempt exceeded the resolved test `timeout`. The body is not stopped: the harness operation it is in is cancelled and its fixtures die with the attempt, so it cannot reach the next test | test | 1 |
114
- | `STEP_NOT_AWAITED` | The test body settled while a step it started was still running: a step call without `await`. The step is failed at the line of the call and cancelled; the attempt fails with this code when the body returned, or keeps the body's own error and records this one under `secondaryErrors` when the body threw | test | 1 |
114
+ | `TEST_TIMEOUT` | The attempt exceeded the resolved test `timeout`. The body is not stopped: the harness operation it is in is cancelled and its fixtures die with the attempt, so it cannot reach the next test. A test that blocks the event loop gets its worker killed once `cleanupTimeout` (at least 5 s) passes after the timeout; see [Hooks](/reference/test#hooks) | test | 1 |
115
+ | `STEP_NOT_AWAITED` | The test body settled while a step it started was still running: a step call without `await`. The step is failed at the line of the call and cancelled; the attempt fails with this code when the body returned, or keeps the body's own error and records this one under `secondaryErrors` when the body threw. An `expect.poll` matcher left running when the body, an `afterEach` hook, or a fixture teardown returned fails that phase the same way at the line of the call, and is cancelled; one left by a `beforeAll` or `afterAll` hook fails the hook with `HOOK_FAILED` | test | 1 |
115
116
  | `UNHANDLED_REJECTION` | A promise rejected with no test running. An unhandled rejection is charged to the test running when it surfaces, since Node cannot say which test created the promise; between tests it is this run error, naming the last test that finished in the worker as the likely source. Before any test has finished in the worker it ends the worker instead: the rejection is the run error and the test on its way fails with `WORKER_CRASH`. The body that left it keeps running like one past `TEST_TIMEOUT` | test | 1 |
116
117
  | `HOOK_FAILED` | `beforeAll` or `afterAll` threw. Skips the suite's tests. | test | 1 |
117
118
  | `LAUNCH_TIMEOUT` | Engine init or attempt start exceeded `config.launchTimeout` | infrastructure | 3 |
118
119
  | `CLEANUP_TIMEOUT` | Engine attempt end exceeded `config.cleanupTimeout`, or a route or dialog handler the engine ran for the test was still running when the attempt settled | infrastructure | 3 |
119
- | `TRACE_WITHHELD` | A trace from an attempt a secret reached (filled, or held by the engine) could not be rewritten, so it was deleted. Recorded in the attempt's `secondaryErrors`; never fails the attempt. | infrastructure | 3 |
120
+ | `TRACE_WITHHELD` | A trace from a session that knows a secret value could not be rewritten, so it was deleted, or the engine returned an archive path outside the attempt's artifact directory, so nothing was registered. Recorded in the attempt's `secondaryErrors`; never fails the attempt. | infrastructure | 3 |
120
121
  | `BROWSER_INSTALL_FAILED` | Browser provisioning failed | infrastructure | 3 |
121
122
  | `WORKER_INIT_FAILED` | A worker process never became ready | infrastructure | 3 |
122
- | `WORKER_EXIT` | A worker exited unexpectedly while running a file: the test in flight fails with `WORKER_CRASH` and the file's other tests are skipped. An unhandled rejection is charged to the test running when it surfaces, or recorded as `UNHANDLED_REJECTION` between tests; before any test has finished there it ends the worker, and the rejection itself is the run error, not this one | infrastructure | 3 |
123
+ | `WORKER_EXIT` | A worker exited unexpectedly while running a file: the test in flight fails with `WORKER_CRASH` and the file's other tests are skipped. Attempts that finished before the exit stay in the report, a retry's earlier attempts and a serial group's finished members included, and the crash is recorded as the attempt after them, or on the last attempt when it came after it (an `afterAll`). An unhandled rejection is charged to the test running when it surfaces, or recorded as `UNHANDLED_REJECTION` between tests; before any test has finished there it ends the worker, and the rejection itself is the run error, not this one | infrastructure | 3 |
123
124
  | `WORKER_CRASH` | A worker exited during a specific test | infrastructure | 3 |
124
125
  | `WORKER_ENTRY_MISSING` | The worker entry module could not be resolved. Packaging fault. | infrastructure | 3 |
126
+ | `NODE_UNSUPPORTED` | The runtime cannot run e2e's TypeScript loader: a Node.js outside `^22.22.3 \|\| >=24.8.0`, Bun (no `module.registerHooks`), or Deno. The CLI checks this before it starts and exits 2 with the same sentence; the loader checks it again where it registers its hooks, for a host that runs e2e's runner without the CLI | infrastructure | 3 |
127
+ | `TYPESCRIPT_COMPILER_UNAVAILABLE` | oxc, which compiles e2e's TypeScript, could not be loaded; the message gives the cause. Most often its native bindings for this platform (`@oxc-transform/binding-<platform>-<arch>`, `@oxc-parser/binding-<platform>-<arch>`) are not installed. npm leaves it out with `--omit=optional`, or when `package-lock.json` was written on another platform (npm/cli#4828): install again without `--omit=optional`, or delete `package-lock.json` and `node_modules` and run `npm install` | infrastructure | 3 |
125
128
  | `ENGINE_FAILURE` | Any engine fault with no more specific mapping | infrastructure | 3 |
126
129
  | `WORKER_PROTOCOL` | A worker reported a unit other than the one it was running, or started an attempt while another was still running | infrastructure | 3 |
127
130
  | `SESSION_KEY_INVALID` | A worker received a session key that is not 32 bytes. Packaging or IPC fault. | infrastructure | 3 |
@@ -1,6 +1,7 @@
1
1
  ---
2
2
  title: expect
3
3
  description: "Locator assertions, fixture assertions, and plain value matchers."
4
+ icon: "circle-check"
4
5
  ---
5
6
 
6
7
  Use `expect(locator)` to wait for an app condition and `expect(value)` to
@@ -52,14 +53,17 @@ before passing its result to `expect`.
52
53
  | Default timeout | `config.assertionTimeout`, 5000 ms |
53
54
  | Override | `{ timeout }` in milliseconds, on every locator and `browser` matcher |
54
55
  | Unknown option | `INVALID_ARGUMENT` before the first read, from JavaScript too |
55
- | Negation grace window | A negated matcher passes after 1000 ms of continuous truth, or after the whole budget when `{ timeout }` is shorter than that |
56
+ | Negation grace window | A negated matcher passes after 1000 ms of continuous truth, or at the deadline when `{ timeout }` is shorter than that and the negation held for the whole budget. The window counts from the moment the first read that saw the negation started, so a slow read counts toward it |
56
57
  | Failure code | `ASSERTION_FAILED`, exit code 1 |
57
58
 
58
59
  Locator strictness: except for `toHaveCount` and the list form of
59
60
  `toHaveText` and `toContainText`, a positive matcher requires exactly one
60
61
  match; zero matches keep polling until the timeout, and multiple matches fail
61
- immediately with `LOCATOR_AMBIGUOUS`. `toBeHidden`, negated visibility, and
62
- negated `toBeAttached` accept zero matches.
62
+ immediately with `LOCATOR_AMBIGUOUS`. `toBeHidden`, `toBeVisible({ visible:
63
+ false })`, `toBeAttached({ attached: false })`, negated visibility, and
64
+ negated `toBeAttached` accept zero matches; for them, and for
65
+ `waitFor({ state: 'detached' })` and `'hidden'`, a locator under a frame that
66
+ is not in the document is zero matches rather than a wait for the frame.
63
67
 
64
68
  Text comparison is normalized: leading and trailing whitespace is trimmed and
65
69
  each run of whitespace collapses to one space. `toHaveText` and
@@ -201,8 +205,8 @@ Every matcher call is a recorded `expect.<matcher>` assertion step.
201
205
  export interface BrowserExpectation {
202
206
  /** Inverts the matcher. */
203
207
  readonly not: BrowserExpectation;
204
- /** Waits for the current URL to match. */
205
- toHaveURL(expected: string | RegExp, options?: { timeout?: number }): Promise<void>;
208
+ /** Waits for the current URL to match; `ignoreCase` compares a string case-insensitively and adds the `i` flag to a RegExp, `false` removes it. */
209
+ toHaveURL(expected: string | RegExp, options?: { ignoreCase?: boolean; timeout?: number }): Promise<void>;
206
210
  /** Waits for the current title to match. */
207
211
  toHaveTitle(expected: TextMatch, options?: { timeout?: number }): Promise<void>;
208
212
  /** Waits for the target's `class` attribute to match: a string is the whole normalized class list, a RegExp is tested against it. */
@@ -212,9 +216,10 @@ export interface BrowserExpectation {
212
216
 
213
217
  A relative `toHaveURL` string resolves against the base URL and matches
214
218
  exactly after normalization; a `RegExp` tests the full URL. Anything else,
215
- such as a predicate function, is `INVALID_ARGUMENT`. Each `browser` matcher takes
216
- `timeout` only; any other key, such as Playwright's `ignoreCase` on
217
- `toHaveURL`, is `INVALID_ARGUMENT`.
219
+ such as a predicate function, is `INVALID_ARGUMENT`. `toHaveURL` also takes
220
+ Playwright's `ignoreCase`, with the locator matchers' semantics. Otherwise each
221
+ `browser` matcher takes `timeout` only; any other key, such as `ignoreCase` on
222
+ `toHaveTitle`, which Playwright does not take either, is `INVALID_ARGUMENT`.
218
223
 
219
224
  `toHaveClass` polls for the node too: a target that matches nothing yet is one
220
225
  more poll, not a failure, until the assertion timeout runs out. An ambiguous
@@ -326,8 +331,11 @@ side is refused.
326
331
  `toHaveProperty` walks a dotted path or an array of keys. Use the array form
327
332
  when a key contains a dot, or to index an array by number
328
333
  (`['lines', 0, 'sku']`). Inherited properties count; a property holding
329
- `undefined` exists. With a second argument the value at the path must equal it
330
- by `toEqual` rules.
334
+ `undefined` exists. A string, number, or boolean on the path is read through
335
+ its wrapper object, as in Jest, so `toHaveProperty('label.length', 3)` and
336
+ `toHaveProperty(['label', 0], 'a')` reach into a string; `null` and
337
+ `undefined` have no properties. With a second argument the value at the path
338
+ must equal it by `toEqual` rules.
331
339
 
332
340
  ### Matching a schema
333
341
 
@@ -363,7 +371,7 @@ The unit-runner matchers (`toThrow`, `toBeInstanceOf`, `toBeNaN`,
363
371
  `toStrictEqual`, `resolves`, `rejects`, `expect.extend`) are not part of
364
372
  `expect`. An end-to-end test checks what the app shows and what its API
365
373
  returns; the value matchers cover that, see
366
- [Testing APIs](/api-testing). Code-level
374
+ [API](/api-testing). Code-level
367
375
  checks belong in the unit runner.
368
376
 
369
377
  Pass a second argument to describe the check in its failure message:
@@ -455,11 +463,15 @@ ASSERTION_FAILED: 2 soft assertions failed
455
463
  A soft locator or `browser` matcher is still a recorded `expect.<matcher>` step,
456
464
  marked failed, so the report shows where in the flow it landed. A soft value
457
465
  matcher is not a step, the same as a hard one; its failure appears in the
458
- final error only.
466
+ attempt's error records.
459
467
 
460
468
  When the body throws on its own, or times out, that error stays the
461
469
  attempt's verdict and the soft failures are recorded beside it under
462
- `secondaryErrors`. A body that skips itself with `test.skip(...)` drops them.
470
+ `secondaryErrors`. A body that skips itself with `test.skip(...)` keeps them
471
+ there too. The CLI shows `Skipped After Failure`; set
472
+ [`failOnSkippedFailure: true`](/reference/config) to make that run fail with
473
+ exit code 1. Without that option, the test stays skipped and does not affect
474
+ the exit code.
463
475
 
464
476
  A soft `toMatchSchema` returns the schema's output when the value matches
465
477
  and `undefined` after a kept failure, and its type says so.
@@ -544,3 +556,14 @@ await expect.poll(() => readStatus(), { message: 'the batch never finished' }).n
544
556
 
545
557
  `expect.poll` is not recorded as a report step; a timeout surfaces only as
546
558
  the test's failure.
559
+
560
+ Await every poll. A poll belongs to the phase that started it: the test body
561
+ (with its fixtures and `beforeEach` hooks), one `afterEach` hook or fixture
562
+ teardown, or one `beforeAll` or `afterAll` hook. When that phase returns while
563
+ the poll still runs, the poll is cancelled and the phase fails with
564
+ `STEP_NOT_AWAITED` at the line of the call, whether or not the poll would
565
+ have passed. It never fails a later test: a poll started by work its phase
566
+ left running, such as a body past its timeout, is cancelled before its first
567
+ read. A body that threw keeps its own
568
+ error and records this one under `secondaryErrors`, and a suite hook reports
569
+ it as `HOOK_FAILED`.
@@ -1,19 +1,20 @@
1
1
  ---
2
2
  title: MCP server (e2e mcp)
3
3
  description: Let a coding agent inspect and interact with your app over MCP.
4
+ icon: "plug"
4
5
  ---
5
6
 
6
7
  <CodeGroup>
7
8
  ```bash npm
8
- npx e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
9
+ npx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
9
10
  ```
10
11
 
11
12
  ```bash pnpm
12
- pnpm exec e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
13
+ pnpm exec e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
13
14
  ```
14
15
 
15
16
  ```bash bun
16
- bunx e2e mcp [--config <path>] [--target <name>] [--headless] [--max-sessions <n>]
17
+ bunx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
17
18
  ```
18
19
  </CodeGroup>
19
20
 
@@ -64,21 +65,32 @@ Start the server in the project directory. Config discovery works like
64
65
  | --- | --- |
65
66
  | `--config <path>` | The config every session loads unless `open_session` names one. Configs are reloaded on every `open_session`, with every file they import by path. |
66
67
  | `--target <name>` | The target every session opens on. Otherwise `open_session` names one, or the only target is used; several targets and no name is `TARGET_REQUIRED`. |
67
- | `--headless` | Hide the browser or simulator. Sessions are headed by default outside CI. |
68
+ | `--headed` | Show the browser or simulator, when the engine supports it, in every session whose `open_session` does not set `headed`. Sessions are headless by default, as in `e2e run`. |
68
69
  | `--max-sessions <n>` | How many sessions may be open at once, 1 through 16; default 4. Each session is its own browser or device. |
69
70
 
70
71
  Operator output goes to stderr, prefixed `e2e mcp:`. Stdout carries only the
71
- protocol; anything a config module writes to stdout is redirected.
72
+ protocol. What your code writes through `console`, `process.stdout`, or
73
+ `process.stderr` (the config's top-level code, project tools) goes to stderr
74
+ with secret values redacted. Output printed while a config loads appears once
75
+ it has loaded; if the load fails, that output is withheld, since the config's
76
+ secrets are unknown. An error nobody caught is logged redacted and shuts the
77
+ server down. A child process that inherits the server's stdio writes past
78
+ this.
72
79
 
73
80
  ## Tools
74
81
 
75
82
  | Tool | Arguments | Result |
76
83
  | --- | --- | --- |
77
- | `open_session` | `target` (optional), `config` (optional path) | Loads the config, starts the target's app command, boots the engine, starts one attempt, opens the app URL when the engine can navigate. Returns the session id, the catalog, and the first observation. |
84
+ | `open_session` | `target` (optional), `config` (optional path), `headed` (optional boolean) | Loads the config, starts the target's app command, boots the engine, starts one attempt, opens the app URL when the engine can navigate. Returns the session id, the catalog, and the first observation. `headed: true` shows this session's browser or simulator, `false` hides it; without it the session follows `--headed`. |
78
85
  | `tools` | `tool` (optional), `session` (required while several are open) | The catalog, one line per tool. With `tool`, its full description and argument schema. |
79
86
  | `call` | `tool`, `args` (optional), `session` (required while several are open) | Runs one catalog tool. Arguments are validated against the tool's closed schema first: a wrong type or an argument the tool does not declare is `INVALID_ARGUMENT`, and nothing runs. An action that fails is an error result that still carries the screen it re-observed, and leads with the tool, its target, and the code: `navigate file:///etc/passwd failed: POLICY_DENIED: forbidden URL scheme: file:`. |
80
87
  | `close_session` | `session` (required while several are open) | Saves a recording still running, ends the attempt, disposes the engine, stops the app processes the session started once no other session uses them. |
81
88
 
89
+ Project tools honor their AI SDK `toModelOutput` renderer, including when
90
+ `execute` returns a string. Rendered `error-text` and `error-json` outputs
91
+ are MCP error results (`isError: true`) so the coding agent can distinguish
92
+ a failed tool call from a successful one.
93
+
82
94
  ## Several sessions
83
95
 
84
96
  Several sessions can be open at once, up to `--max-sessions`, each on its
@@ -183,8 +195,9 @@ Sessions enforce what tests enforce. Node ids are valid only against the
183
195
  newest observation. A secret fills only through `type_secret`; once one has
184
196
  been filled, `screenshot` and the point tools (`tap_at`, `hover_at`,
185
197
  `type_at`, `press_at`, `select_at`) stay in the catalog but answer
186
- `PIXEL_TAINTED` for the rest of the session. Every text a `call` returns, a node `locate` found,
187
- an action's result, or an error, passes the attempt's secret ledger. Nothing a
198
+ `PIXEL_TAINTED` for the rest of the session. Every text a tool returns, a node `locate` found,
199
+ an action's result, an `open_session` or `close_session` failure, or a cleanup line, has the
200
+ config's secret values redacted once the config has loaded, as do the server's own log lines. Nothing a
188
201
  session does is recorded as a test result or into the replay cache; its
189
202
  artifacts land under `<output>/artifacts/<target>/sessions/`, its videos under
190
203
  `<output>/videos/`. A later `e2e run` clears `<output>/artifacts`; the
@@ -1,9 +1,11 @@
1
1
  ---
2
2
  title: Mobile
3
3
  description: Mobile engine options and the device fixture for iOS and Android.
4
+ icon: "mobile-screen"
4
5
  ---
5
6
 
6
- `@e2e-dev/mobile` drives iOS simulators and Android emulators through
7
+ `@e2e-dev/mobile` drives iOS simulators, Android emulators, and connected
8
+ phones through
7
9
  [agent-device](https://github.com/callstack/agent-device). `agent`, `app`,
8
10
  `screen`, and `expect` work as they do everywhere; this page covers what is
9
11
  specific to a device target. Import `test` from the package to have `device`
@@ -79,8 +81,9 @@ export interface MobileOptions {
79
81
  /** Platform the target's device runs. */
80
82
  readonly platform: MobilePlatform;
81
83
  /**
82
- * Simulator or emulator to use, by name, simulator UDID, or emulator
83
- * serial (`emulator-5554`). A list is a pool: the
84
+ * Device to use: a simulator or emulator by name, simulator UDID, or
85
+ * emulator serial (`emulator-5554`), or a connected phone by name. A list
86
+ * is a pool: the
84
87
  * engine declares one worker per entry and worker slot `n` drives the
85
88
  * `n`th, so `workers` at or above the pool size runs the target's files
86
89
  * across every device at once. Omitted, the pool is every booted device of
@@ -278,8 +281,8 @@ export interface Device {
278
281
  * `relaunch` restarts it fresh. `launchArguments` and `permissions` apply
279
282
  * to this launch alone; the target's configured ones apply to a `relaunch`
280
283
  * of the pinned app, and to no foreground-only open. A link is
281
- * `INVALID_ARGUMENT` (`file:`, `data:`, and `javascript:` are
282
- * `POLICY_DENIED`): `openLink` opens one.
284
+ * `INVALID_ARGUMENT` (`file:`, `data:`, `javascript:`, `view-source:`,
285
+ * `blob:`, and `filesystem:` are `POLICY_DENIED`): `openLink` opens one.
283
286
  */
284
287
  openApp(app: string, options?: OpenAppOptions): Promise<void>;
285
288
  /**
@@ -289,8 +292,8 @@ export interface Device {
289
292
  * a web link the app does not claim goes to Safari, and the next
290
293
  * observation brings the app back. Android starts the link on that
291
294
  * package. Without an app, Android lets the OS route the link; iOS needs
292
- * one (`INVALID_ARGUMENT` without). `file:`, `data:`, and `javascript:`
293
- * links are `POLICY_DENIED`.
295
+ * one (`INVALID_ARGUMENT` without). `file:`, `data:`, `javascript:`,
296
+ * `view-source:`, `blob:`, and `filesystem:` links are `POLICY_DENIED`.
294
297
  */
295
298
  openLink(url: string, options?: { app?: string }): Promise<void>;
296
299
  /**
@@ -426,7 +429,10 @@ session opened. Android reads the device's foreground activity.
426
429
  `installApp` installs a build from inside a test or a fixture: the run's build once per device (no path installs `app.appPath`), or another build for an upgrade path. A plain
427
430
  install keeps the app's data; `reinstall: true` removes the app named by
428
431
  `app` (default: the pinned app) first. It resolves to the id to `openApp`
429
- the build by.
432
+ the build by: the bundle id or package agent-device read from the build, else
433
+ the `app` passed in. With no reported id and no `app`, the install is
434
+ `ENGINE_FAILURE` instead of a file path posing as an id; see
435
+ [Troubleshooting](/mobile#troubleshooting).
430
436
 
431
437
  ```ts
432
438
  const { app } = await device.installApp('./build/MyApp.app', { reinstall: true });
@@ -444,9 +450,9 @@ name (`Settings`); `relaunch: true` restarts it fresh, and `launchArguments`
444
450
  and `permissions` apply to that launch alone (the target's own apply to a
445
451
  `relaunch` of the pinned app). It takes no link:
446
452
  agent-device opens any `scheme:rest` string as a URL, so a link is refused
447
- before the device sees it. A `file:`, `data:`, or `javascript:` link is
448
- `POLICY_DENIED`, the rule every navigation follows; any other link is
449
- `INVALID_ARGUMENT` pointing at `openLink`.
453
+ before the device sees it. A `file:`, `data:`, `javascript:`,
454
+ `view-source:`, `blob:`, or `filesystem:` link is `POLICY_DENIED`; any other
455
+ link is `INVALID_ARGUMENT` pointing at `openLink`.
450
456
 
451
457
  `openLink` opens a deep link or a web link into `app` (default: the pinned
452
458
  app), for magic-link sign-in and deep-link routes, and the session observes
@@ -456,8 +462,13 @@ observation brings the app back. Android starts the link on that package.
456
462
  Without an app, Android lets the OS route the link, to a verified app link
457
463
  owner or the browser, and the session follows the package that took it. iOS
458
464
  needs an app: an open bound to none leaves no app session to observe, so it
459
- is `INVALID_ARGUMENT`. `file:`, `data:`, and `javascript:` links are
460
- `POLICY_DENIED`, as on the web. The report shows the link cut before its
465
+ is `INVALID_ARGUMENT`. `file:`, `data:`, `javascript:`, `view-source:`,
466
+ `blob:`, and `filesystem:` links are `POLICY_DENIED`. A device link may use
467
+ any app's custom scheme, so the rule refuses these schemes rather than
468
+ admitting http(s) alone as the web does. A string that does not parse as an
469
+ absolute URL is `INVALID_ARGUMENT` too, and its message names no part of the
470
+ input, which may carry a magic-link token in its query, path, or userinfo.
471
+ The report shows the link cut before its
461
472
  query and fragment, where a magic link carries its token. The replay cache
462
473
  anchors on the location the link lands on: a step recorded after `openLink`
463
474
  replays when the link lands on the same screen again.
@@ -1,6 +1,7 @@
1
1
  ---
2
2
  title: Reporters
3
3
  description: Receive run events and publish results with a custom reporter.
4
+ icon: "file-lines"
4
5
  ---
5
6
 
6
7
  Add reporter objects to `reporters` in `e2e.config.ts`, alongside the
@@ -171,7 +172,7 @@ prints. All reporters run concurrently, each with a one-minute deadline.
171
172
  A timeout, exception, or invalid return value is reported on stderr without
172
173
  changing the run's status, exit code, or report.
173
174
 
174
- One Ctrl-C still lets reporters finish. A second Ctrl-C stops waiting for them.
175
+ One Ctrl-C still lets reporters finish. A second Ctrl-C stops waiting for them, except the built-in `junit` and `markdown`, which still write their files so none of the previous run's is left beside the new report.
175
176
 
176
177
  The second argument is an `AbortSignal` that aborts when the budget runs out
177
178
  or the run is forced to stop. Hand it to every request and timer, or an
@@ -180,7 +181,13 @@ abandoned reporter keeps the CLI alive after the run has ended.
180
181
  With `--last-failed`, `lastRun` is the report the rerun selected from. The
181
182
  rerun's own report lists the tests it left out as unselected, so a reporter
182
183
  that keeps one place current, the way `@e2e-dev/github` keeps a pull request
183
- comment, folds the rerun into `lastRun` to show both runs as one.
184
+ comment, folds the rerun into `lastRun` to show both runs as one. A test the
185
+ rerun owed but another filter left out is in the rerun's `run.carried`: its
186
+ last result, the serial group its attempts live on, and the `beforeAll` or
187
+ `afterAll` failure whose scope holds it, from whichever earlier run reported
188
+ them. A fold shows the carried result in place of the unselected one and
189
+ counts a carried failure or hook error against the outcome. The artifacts
190
+ both reports name are on disk under `artifactsRoot`.
184
191
 
185
192
  The rows a reporter resolves with print under the `list` summary in the same
186
193
  layout as `Report`; the `json` reporter does not carry them:
@@ -1,6 +1,7 @@
1
1
  ---
2
2
  title: screen and Locator
3
3
  description: Queries, refinement, actions, and reads.
4
+ icon: "display"
4
5
  ---
5
6
 
6
7
  A `Locator` stores a query and resolves it when an action, read, or assertion
@@ -107,7 +108,14 @@ await screen.getByTestId('memory-list', { visible: true }).getByRole('listitem')
107
108
  ```
108
109
 
109
110
  `visible` uses the same visibility check as `toBeVisible()`, `isVisible()`,
110
- and `waitFor()`. It works with scopes, `filter`, `first`, `last`, and `nth`:
111
+ and `waitFor()`: what renders, as Playwright reads it. `display: none`,
112
+ `visibility: hidden`, a box with no size, a closed `<details>` body, and
113
+ content under `content-visibility: hidden` are hidden; `aria-hidden` is not,
114
+ since a decorative spinner marked with it still paints, so
115
+ `toBeHidden()` on one waits until it leaves. A child that sets
116
+ `visibility: visible` under a hidden parent is visible. On the web, text
117
+ directly under a `display: contents` element with `visibility: hidden` is
118
+ hidden, where Playwright calls it visible. It works with scopes, `filter`, `first`, `last`, and `nth`:
111
119
  `getByText('Save', { visible: true }).first()` is the first visible match.
112
120
  Prefer it over `browser.locator('p:visible')`, which ties the test to CSS.
113
121
 
@@ -171,14 +179,15 @@ not the finger: `'down'` reveals what is below. `momentum` is its fling
171
179
  strength, `'none'` by default: `'none'` a plain drag of half the surface,
172
180
  `'slow'` a short fling, `'fast'` a long one. How far each carries is the
173
181
  engine's.
174
- `{ from, to }` is a swipe along a path between two viewport points: a touch
175
- swipe on a device, a pointer drag on a document platform, which is what
176
- reveals a row action, dismisses a sheet, or drags a card between columns.
177
- Uses `config.actionTimeout`. Throws
178
- `ACTION_FAILED` when the gesture is rejected, `INVALID_ARGUMENT` for a mix
179
- of the two shapes or a coordinate off the plane, and `UNSUPPORTED_CAPABILITY`
180
- when the engine does not list `swipe` in its actions (a direction) or
181
- `swipeTo` in its pointer actions (a path).
182
+ `{ from, to, duration? }` is a swipe along a path between two viewport points:
183
+ a touch swipe on a device, a pointer drag on a document platform, which is
184
+ what reveals a row action, dismisses a sheet, or drags a card between
185
+ columns. `duration` sets how long the gesture takes in milliseconds; omitted
186
+ leaves the duration to the engine's default. Uses `config.actionTimeout`.
187
+ Throws `ACTION_FAILED` when the gesture is rejected, `INVALID_ARGUMENT` for
188
+ a mix of the two shapes, an out-of-range duration, or a coordinate off the
189
+ plane, and `UNSUPPORTED_CAPABILITY` when the engine does not list `swipe` in
190
+ its actions (a direction) or `swipeTo` in its pointer actions (a path).
182
191
 
183
192
  ```ts
184
193
  await screen.swipe({ direction: 'up', momentum: 'fast' });
@@ -187,6 +196,7 @@ const row = await screen.getByText('Design review').boundingBox();
187
196
  await screen.swipe({
188
197
  from: { x: row.x + row.width - 10, y: row.y + row.height / 2 },
189
198
  to: { x: row.x + 10, y: row.y + row.height / 2 },
199
+ duration: 300,
190
200
  });
191
201
  ```
192
202
 
@@ -292,8 +302,12 @@ export interface Locator extends Screen {
292
302
  all(): Promise<Locator[]>;
293
303
  /** Reads the normalized text of every current match, without auto-waiting; empty when nothing matches. */
294
304
  allTextContents(): Promise<string[]>;
295
- /** Waits for the requested locator state. */
296
- waitFor(options?: { state?: 'visible' | 'hidden'; timeout?: number }): Promise<void>;
305
+ /**
306
+ * Waits for the requested locator state, `visible` by default: `attached`
307
+ * for one match, visible or not; `detached` for none; `hidden` for none or
308
+ * a hidden one.
309
+ */
310
+ waitFor(options?: { state?: 'attached' | 'detached' | 'visible' | 'hidden'; timeout?: number }): Promise<void>;
297
311
  /** Adds deterministic locator filters. */
298
312
  filter(options: { hasText?: TextMatch; has?: Locator }): Locator;
299
313
  /** Selects the first current match. */
@@ -382,7 +396,11 @@ await screen.getByLabel('City').pressSequentially('War', { delay: 50 });
382
396
  ## Locator reads
383
397
 
384
398
  `waitFor` defaults `state` to `'visible'` and `timeout` to
385
- `config.actionTimeout`, and throws `LOCATOR_NOT_FOUND` on timeout.
399
+ `config.actionTimeout`, and throws `LOCATOR_NOT_FOUND` on timeout. `'attached'`
400
+ waits for one match, hidden or shown, `'detached'` for none, and `'hidden'` for
401
+ a hidden match or none. Several matches are `LOCATOR_AMBIGUOUS` in every
402
+ state. Any other state, or an option key besides `state` and `timeout`, is
403
+ `INVALID_ARGUMENT` before anything is resolved.
386
404
 
387
405
  `textContent` and `allTextContents` return the node's rendered text with
388
406
  whitespace collapsed to single spaces. On the web that is what `innerText`
@@ -399,6 +417,12 @@ a heading; it does not throw.
399
417
  `toHaveText`, `toContainText`, and `toHaveAttribute`.
400
418
  `isEnabled`, `isDisabled`, `isChecked`, and `boundingBox` are allowed there.
401
419
 
420
+ `boundingBox()` is `null` for a node that lays out no box (`display: none`
421
+ on it or an ancestor), and a box for one `visibility: hidden` keeps.
422
+ `inputValue()` and `toHaveValue` on a checkbox or radio read its value
423
+ attribute, `on` when it has none, whatever its checked state; `isChecked()`
424
+ reads the state.
425
+
402
426
  `isHidden` is `!isVisible()` and `isDisabled` is `!isEnabled()`, with the
403
427
  same strictness: `isHidden` is `true` for zero matches, `isDisabled` is
404
428
  `LOCATOR_NOT_FOUND` there, and both are `LOCATOR_AMBIGUOUS` on several.
@@ -521,7 +545,7 @@ export interface TextMatchOptions {
521
545
  /**
522
546
  * Role query options. A role query never matches a node hidden from the
523
547
  * accessibility tree, on every engine; `visible` (inherited) is the one knob
524
- * that narrows the other query kinds the same way.
548
+ * that narrows the other query kinds, to the nodes `toBeVisible()` accepts.
525
549
  */
526
550
  export interface RoleOptions extends TextMatchOptions {
527
551
  /** Accessible name filter. */
@@ -564,6 +588,7 @@ export interface SwipeOptions {
564
588
  momentum?: Momentum;
565
589
  from?: never;
566
590
  to?: never;
591
+ duration?: never;
567
592
  }
568
593
 
569
594
  /** A swipe along a path between two viewport points: a touch swipe on a device, a pointer drag on a document platform. */
@@ -572,6 +597,8 @@ export interface SwipePathOptions {
572
597
  from: Point;
573
598
  /** Where it lifts. */
574
599
  to: Point;
600
+ /** Gesture duration in milliseconds; omitted = the engine's default. */
601
+ duration?: number;
575
602
  direction?: never;
576
603
  momentum?: never;
577
604
  }
@@ -623,8 +650,10 @@ export type ScrollDirection = 'up' | 'down' | 'left' | 'right';
623
650
  export type Momentum = 'none' | 'slow' | 'fast';
624
651
  ```
625
652
 
626
- A role query never matches a node hidden from the accessibility tree.
627
- `visible` applies the same rule to the other query kinds. See
653
+ A role query never matches a node hidden from the accessibility tree: one
654
+ that does not render, or one under `aria-hidden`. `visible` narrows the
655
+ other query kinds to the nodes that render, the check `toBeVisible()` makes,
656
+ so an `aria-hidden` node that still paints counts as visible there. See
628
657
  [Hidden duplicates](#hidden-duplicates).
629
658
 
630
659
  `img` is an alias, not a role: `getByRole('img', { name })` builds the same
@@ -640,7 +669,11 @@ and `<details>` are `group`, `<input type="number">` is `spinbutton`,
640
669
  `<select size="3">` is a `listbox` like a `multiple` one, an `<input list>`
641
670
  naming a `<datalist>` is a `combobox`, a `<td>` of a `role="grid"` table is a
642
671
  `gridcell`,
643
- `<thead>` and `<tbody>` are `rowgroup`, `<th scope="row">` is `rowheader`,
672
+ `<thead>` and `<tbody>` are `rowgroup`, `<th scope="row">` is `rowheader`.
673
+ A scope-less `<th>` beside a data cell with text or children is also a
674
+ `rowheader`; header-only rows keep `columnheader`. A lone scope-less header in a one-row
675
+ table has no header role, while `scope="col"` and `scope="colgroup"` explicitly
676
+ keep `columnheader`.
644
677
  `<figure>` is `figure`, `<article>` is `article`, `<header>` and `<footer>` are
645
678
  `banner` and `contentinfo` only outside sectioning content or an element with
646
679
  a sectioning role (`article`, `complementary`, `main`, `navigation`, `region`),
@@ -1,6 +1,7 @@
1
1
  ---
2
2
  title: test
3
3
  description: Tests, setup tests, groups, hooks, options, and fixtures.
4
+ icon: "flask-vial"
4
5
  ---
5
6
 
6
7
  Declare tests, groups, and hooks synchronously while the test file is
@@ -97,16 +98,23 @@ group titles and then the test's own, are a `COLLECTION_ERROR`.
97
98
 
98
99
  ## Members
99
100
 
100
- A skipped test never affects the exit code. With `test.only`, other tests
101
+ A skipped test does not affect the exit code by default. With `test.only`, other tests
101
102
  are reported as `filtered`, and a CI run fails with `ONLY_IN_CI`.
102
103
 
103
104
  Call `test.skip(condition, reason)` inside a test for conditions you learn at
104
105
  runtime, such as a disabled feature flag. When the condition is true, the call
105
106
  throws: the body stops at that line, as Playwright's does, and the test is
106
107
  reported `skipped` with `reason`, with no retry. Nothing after the call runs.
107
- Steps before it stay in the report as they ran, but they do not decide the
108
- status: a failed `expect.soft` before the call is dropped. Hooks and fixtures still run their teardown. When the condition is
109
- a precondition, call it first, before any step.
108
+ Steps before it stay in the report as they ran. Soft failures before the call
109
+ remain in the attempt's `secondaryErrors`, and the CLI lists the test under
110
+ `Skipped After Failure`. A failed attempt followed by a skip on retry appears
111
+ there too, with its original error. Hooks and fixtures still run their teardown.
112
+ When the condition is a precondition, call it first, before any step.
113
+
114
+ Set [`failOnSkippedFailure: true`](/reference/config) to fail the run with exit
115
+ code 1 when a runtime skip follows a failure in that test. The test remains
116
+ `skipped` with its reason, and the skip does not trigger another retry. Clean
117
+ skips and retries that pass are unaffected. The default is `false`.
110
118
 
111
119
  A setup test cannot skip because other tests depend on its sessions; the
112
120
  call fails with `INVALID_ARGUMENT`. Outside a test body, the conditional
@@ -185,6 +193,19 @@ for every scope whose `beforeAll` started.
185
193
  its own `cleanupTimeout` budget with working fixtures, so teardown can still
186
194
  drive the app after the body timed out.
187
195
 
196
+ A test that blocks its worker's event loop (`while (true) {}` in a fixture,
197
+ `beforeEach`, the body, or `afterEach`) stops every timer in that worker, so
198
+ the worker cannot time it out. A `beforeAll` or `afterAll` that blocks still
199
+ hangs the run. When the worker has not responded `cleanupTimeout` (at least 5 s) after
200
+ the test timeout, the runner kills it. That attempt is reported `timed-out`
201
+ with `TEST_TIMEOUT` and no artifacts, after the attempts before it, and is
202
+ not retried. The rest of its file runs on a new worker. In a serial group,
203
+ the members after it are skipped. A worker with a debugger attached when the
204
+ test starts (an editor's auto-attach, or `NODE_OPTIONS=--inspect=0`) is
205
+ never killed this way, so a breakpoint can stay paused. A programmatic run with an inline
206
+ config or in-memory tests runs in the runner's own process and has no worker
207
+ to kill: a blocked event loop hangs it.
208
+
188
209
  A throw inside `beforeAll` or `afterAll` is `HOOK_FAILED` and skips the
189
210
  suite's tests with cause `hook-failed`. Inside a serial group it skips every
190
211
  remaining member and fails the group without a retry.
@@ -295,7 +316,13 @@ group's attempt. A retry starts fresh. Inside it, `retries`, `trace`,
295
316
  `video`, `session`, `platforms`, `requires`, `skip`, `only`, and a nested
296
317
  `serial` on a member are collection errors. Set them on the group. The group
297
318
  records one trace and one video per group attempt, from the `trace` and
298
- `video` on the serial `describe` or a group around it.
319
+ `video` on the serial `describe` or a group around it. A group attempt that
320
+ cannot start, because its file fails to load or its session fails to launch,
321
+ fails the first member with that error and skips the rest with cause
322
+ `serial-predecessor-failed`. When the run is interrupted during the launch,
323
+ every member skips instead. On a retry, the members keep the verdicts of
324
+ the attempt before it. The exit code follows the error's category, as for any
325
+ other test.
299
326
 
300
327
  ## Fixtures
301
328