e2e 0.21.1 → 0.21.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (133) hide show
  1. package/README.md +2 -0
  2. package/dist/.build.json +1 -1
  3. package/dist/agent/act-validation.js +1 -0
  4. package/dist/agent/act.js +7 -3
  5. package/dist/agent/action-dispatcher.js +30 -8
  6. package/dist/agent/actions.d.ts +2 -0
  7. package/dist/agent/actions.js +5 -2
  8. package/dist/agent/error.js +1 -0
  9. package/dist/agent/executor.d.ts +14 -6
  10. package/dist/agent/index.js +5 -1
  11. package/dist/agent/invocation.d.ts +3 -0
  12. package/dist/agent/model/sdk.d.ts +5 -0
  13. package/dist/agent/model/sdk.js +18 -1
  14. package/dist/agent/observation.js +1 -0
  15. package/dist/agent/primitives.js +14 -3
  16. package/dist/agent/replay.js +9 -3
  17. package/dist/agent/step-cache.d.ts +3 -2
  18. package/dist/agent/step-cache.js +7 -4
  19. package/dist/agent/tool-loop.js +4 -1
  20. package/dist/cache/context.js +1 -1
  21. package/dist/cache/recorder.d.ts +2 -1
  22. package/dist/cache/recorder.js +11 -2
  23. package/dist/cache/trace-fields.d.ts +10 -0
  24. package/dist/cache/trace-fields.js +17 -0
  25. package/dist/cache/trace.d.ts +9 -1
  26. package/dist/cache/trace.js +26 -7
  27. package/dist/cli/feedback.js +2 -1
  28. package/dist/cli/init/sibling-versions.json +1 -1
  29. package/dist/collect/select.js +12 -2
  30. package/dist/config/agent.d.ts +1 -1
  31. package/dist/config/diagnose.js +9 -3
  32. package/dist/config/esm-hooks.js +3 -2
  33. package/dist/config/resolve.js +3 -2
  34. package/dist/config/typescript.js +52 -2
  35. package/dist/engine/contract.d.ts +3 -2
  36. package/dist/engine/manifest.js +2 -2
  37. package/dist/internal/errors.js +5 -2
  38. package/dist/internal/source.js +2 -0
  39. package/dist/internal/unserved-model.d.ts +5 -0
  40. package/dist/internal/unserved-model.js +22 -0
  41. package/dist/locator/screen.js +1 -2
  42. package/dist/oauth/copilot.js +23 -5
  43. package/dist/oauth/device-code.js +35 -19
  44. package/dist/oauth/errors.d.ts +3 -0
  45. package/dist/oauth/errors.js +26 -11
  46. package/dist/oauth/providers/github-copilot.d.ts +4 -0
  47. package/dist/oauth/providers/github-copilot.js +64 -16
  48. package/dist/oauth/providers/openai.d.ts +2 -2
  49. package/dist/oauth/providers/openai.js +47 -19
  50. package/dist/oauth/providers/opencode-console.js +36 -15
  51. package/dist/oauth/providers/xai.js +18 -9
  52. package/dist/oauth/token-endpoint.d.ts +15 -6
  53. package/dist/oauth/token-endpoint.js +18 -6
  54. package/dist/report/code-frame.js +9 -3
  55. package/dist/report/failure-text.d.ts +10 -2
  56. package/dist/report/failure-text.js +24 -8
  57. package/dist/report/list-model.d.ts +13 -0
  58. package/dist/report/list-steps.d.ts +6 -1
  59. package/dist/report/list-steps.js +29 -0
  60. package/dist/report/list.d.ts +1 -0
  61. package/dist/report/list.js +16 -7
  62. package/dist/report/live-window.js +2 -1
  63. package/dist/report/markdown.js +1 -1
  64. package/dist/run/fixtures.js +2 -0
  65. package/dist/run/host-trace.d.ts +3 -2
  66. package/dist/run/host-trace.js +2 -1
  67. package/dist/run/last-run.d.ts +9 -0
  68. package/dist/run/last-run.js +28 -0
  69. package/dist/run/open-session.d.ts +3 -1
  70. package/dist/run/open-session.js +1 -1
  71. package/dist/run/runner.js +6 -2
  72. package/dist/run/sessions.js +3 -3
  73. package/dist/run/standalone-attempt.d.ts +2 -1
  74. package/dist/run/standalone-attempt.js +3 -1
  75. package/dist/run/steps.d.ts +3 -1
  76. package/dist/run/units.d.ts +8 -0
  77. package/dist/run/units.js +14 -0
  78. package/dist/run/worker-models.d.ts +2 -0
  79. package/dist/run/worker-models.js +7 -0
  80. package/dist/types.d.ts +2 -2
  81. package/docs/authentication.mdx +26 -23
  82. package/docs/browser.mdx +7 -4
  83. package/docs/bug-bash.mdx +2 -2
  84. package/docs/cache.mdx +6 -1
  85. package/docs/ci/bitrise.mdx +1 -1
  86. package/docs/ci/codemagic.mdx +2 -2
  87. package/docs/ci/eas-workflows.mdx +1 -1
  88. package/docs/coding-agents.mdx +7 -35
  89. package/docs/debugging.mdx +24 -36
  90. package/docs/executors.mdx +1 -1
  91. package/docs/github.mdx +2 -1
  92. package/docs/integrations/eas.mdx +1 -1
  93. package/docs/integrations/index.mdx +6 -0
  94. package/docs/integrations/lightpanda.mdx +192 -0
  95. package/docs/integrations/testmu.mdx +152 -0
  96. package/docs/mcp.mdx +123 -0
  97. package/docs/migrate/detox.mdx +1 -1
  98. package/docs/migrate/maestro.mdx +101 -105
  99. package/docs/migrate/playwright.mdx +65 -70
  100. package/docs/mobile.mdx +18 -3
  101. package/docs/models.mdx +4 -2
  102. package/docs/quickstart.mdx +7 -0
  103. package/docs/reference/agent.mdx +369 -277
  104. package/docs/reference/app.mdx +141 -32
  105. package/docs/reference/cli.mdx +674 -558
  106. package/docs/reference/config.mdx +1406 -752
  107. package/docs/reference/engine.mdx +1062 -256
  108. package/docs/reference/environment.mdx +156 -84
  109. package/docs/reference/errors.mdx +106 -72
  110. package/docs/reference/expect.mdx +752 -372
  111. package/docs/reference/mcp.mdx +366 -141
  112. package/docs/reference/mobile.mdx +844 -403
  113. package/docs/reference/reporters.mdx +312 -106
  114. package/docs/reference/runner.mdx +373 -174
  115. package/docs/reference/screen.mdx +931 -398
  116. package/docs/reference/security.mdx +106 -40
  117. package/docs/reference/test.mdx +626 -273
  118. package/docs/reference/web.mdx +956 -533
  119. package/docs/security.mdx +6 -5
  120. package/docs/snippets/model-provider.jsx +763 -0
  121. package/docs/snippets/setup-prompt-card.jsx +22 -0
  122. package/docs/snippets/show-more.jsx +22 -0
  123. package/docs/snippets/video-card.jsx +34 -0
  124. package/docs/snippets/video.jsx +25 -0
  125. package/docs/subscriptions.mdx +10 -3
  126. package/docs/telemetry.mdx +39 -39
  127. package/package.json +3 -2
  128. package/schema/report-v1.schema.json +6 -0
  129. package/skills/e2e/references/agent.md +8 -3
  130. package/skills/e2e/references/debugging.md +1 -1
  131. package/skills/e2e/references/mcp.md +1 -1
  132. package/skills/e2e/references/setup.md +12 -3
  133. package/skills/e2e/references/writing-tests.md +1 -1
@@ -0,0 +1,152 @@
1
+ ---
2
+ title: TestMu AI
3
+ sidebarTitle: TestMu AI
4
+ description: Run mobile targets on TestMu AI (formerly LambdaTest) Android emulators, iOS simulators, and real devices with the testmu() device provider from @e2e-dev/testmu.
5
+ icon: "/images/icons/testmu.svg"
6
+ "og:image": "https://e2e.tester.army/docs/images/og/integrations/testmu.png?v=6ef69e0285"
7
+ "twitter:image": "https://e2e.tester.army/docs/images/og/integrations/testmu.png?v=6ef69e0285"
8
+ ---
9
+
10
+ `testmu()` from `@e2e-dev/testmu` is a
11
+ [device provider](/mobile#hosted-devices) for
12
+ [TestMu AI](https://www.lambdatest.com) (formerly LambdaTest) Android
13
+ emulators, iOS simulators, and real devices. Each worker slot leases one
14
+ device when the run starts and releases it when the run ends.
15
+
16
+ <Tip>
17
+ No Mac, Xcode, or Android SDK on the machine that runs e2e.
18
+ </Tip>
19
+
20
+ ## Setup
21
+
22
+ Install the integration alongside the mobile engine:
23
+
24
+ <CodeGroup>
25
+ ```bash npm
26
+ npm install --save-dev e2e @e2e-dev/mobile @e2e-dev/testmu
27
+ ```
28
+
29
+ ```bash pnpm
30
+ pnpm add -D e2e @e2e-dev/mobile @e2e-dev/testmu
31
+ ```
32
+
33
+ ```bash bun
34
+ bun add -d e2e @e2e-dev/mobile @e2e-dev/testmu
35
+ ```
36
+ </CodeGroup>
37
+
38
+ ```ts title="e2e.config.ts"
39
+ import type { E2EConfig } from 'e2e';
40
+ import { mobile } from '@e2e-dev/mobile';
41
+ import { testmu } from '@e2e-dev/testmu';
42
+
43
+ const apk = 'https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_android.apk';
44
+ const ipa = 'https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa';
45
+
46
+ export default {
47
+ targets: [
48
+ {
49
+ name: 'android-emulator',
50
+ engine: mobile({
51
+ platform: 'android',
52
+ device: testmu({ device: 'Galaxy S22 Ultra 5G', osVersion: '14', app: apk }),
53
+ }),
54
+ app: { bundleId: 'com.lambdatest.proverbial' },
55
+ },
56
+ {
57
+ name: 'ios-real',
58
+ engine: mobile({
59
+ platform: 'ios',
60
+ device: testmu({ device: 'iPhone 16', osVersion: '18', app: ipa, deviceType: 'real' }),
61
+ }),
62
+ app: { bundleId: 'proverbial' },
63
+ },
64
+ {
65
+ name: 'android-real',
66
+ engine: mobile({
67
+ platform: 'android',
68
+ device: testmu({ device: 'Pixel 6', osVersion: '14', app: apk, deviceType: 'real' }),
69
+ }),
70
+ app: { bundleId: 'com.lambdatest.proverbial' },
71
+ },
72
+ ],
73
+ workers: 1,
74
+ } satisfies E2EConfig;
75
+ ```
76
+
77
+ `device` and `osVersion` must match TestMu AI's catalog exactly, as the
78
+ [capabilities generator](https://www.lambdatest.com/capabilities-generator)
79
+ lists them: `18.0` for a virtual iOS device, `18` for a real one, `14` for
80
+ Android. Real and virtual devices have separate catalogs.
81
+
82
+ `app` is the build TestMu AI installs on every session: an `lt://` app id,
83
+ an `https` URL, or a local path, resolved against the directory of
84
+ `e2e.config.ts` and uploaded. Use an `.apk` or `.aab` on Android, a zipped
85
+ `.app` on an iOS simulator, and a signed `.ipa` on a real iOS device. Leave
86
+ the target's `app.appPath` out, and keep `app.bundleId` for `app.open()`.
87
+
88
+ Tests are the same as on a local device:
89
+
90
+ ```ts title="tests/home.e2e.ts"
91
+ import { test } from '@e2e-dev/mobile';
92
+ import { expect } from 'e2e';
93
+
94
+ test('Proverbial home screen', async ({ app, screen }) => {
95
+ await app.open();
96
+ await expect(screen.getByText('Proverbial')).toBeVisible();
97
+ // Android labels the button GEOLOCATION, iOS GeoLocation.
98
+ await expect(screen.getByRole('button', /^geolocation$/i)).toBeVisible();
99
+ });
100
+ ```
101
+
102
+ ## Authenticate
103
+
104
+ Set `LT_USERNAME` and `LT_ACCESS_KEY` in the environment `e2e run` starts
105
+ in, such as CI secrets. Copy them from **Account Settings → Password &
106
+ Security → Username and Access Key**
107
+ ([accounts.lambdatest.com/security/username-accesskey](https://accounts.lambdatest.com/security/username-accesskey)).
108
+ Without either, the lease fails before anything starts.
109
+
110
+ ## Options
111
+
112
+ | Option | Default | Meaning |
113
+ | ------------- | --------------- | ------- |
114
+ | `device` | required | Device name as TestMu AI's catalog lists it (`'Galaxy S22 Ultra 5G'`, `'iPhone 16'`). |
115
+ | `osVersion` | required | OS version as the catalog lists it for that device. |
116
+ | `app` | required | The build TestMu AI installs: an `lt://` app id, an `https` URL, or a local path. |
117
+ | `deviceType` | `'virtual'` | `'virtual'` for an emulator or simulator, `'real'` for a real device. |
118
+ | `project` | `'e2e'` | Dashboard project the sessions are grouped under. |
119
+ | `build` | the run id | Dashboard build the sessions are grouped under. |
120
+ | `sessionName` | `e2e-<run id>-<target>-<slot>` | Session name on the dashboard. With more than one worker slot, `-<slot>` is appended. |
121
+ | `orientation` | the device's | `'portrait'` or `'landscape'`. |
122
+ | `geoLocation` | TestMu AI's | Country code the device's IP geolocates to (`'US'`). |
123
+ | `timezone` | the device's | Time zone (`'UTC+05:30'`). |
124
+ | `language` | the device's | Language code (`'fr'`). |
125
+ | `locale` | the device's | Locale (`'fr_FR'`). |
126
+ | `appiumVersion` | TestMu AI's | Appium version for the session (`'2.16.2'`). |
127
+
128
+ Device features are set when the session starts and hold for the whole
129
+ session. An unknown option or an invalid value fails the config load with
130
+ `INVALID_CONFIG`.
131
+
132
+ An attempt that records [video](/reference/config#video) links TestMu AI's
133
+ recording of the session, found by its `build` and `sessionName`. The
134
+ recording covers the whole session, so every attempt on a worker slot links
135
+ the same video. Runs that overlap and share a fixed `build` and
136
+ `sessionName` can link each other's videos; the defaults never do.
137
+
138
+ ## Limits
139
+
140
+ - Each session starts when its slot is leased, which takes about 30 to 75
141
+ seconds, and is billed until the run releases it, even when no test runs
142
+ on it. Keep `workers` times the number of TestMu AI targets within your
143
+ plan's parallel sessions.
144
+ - If the runner is killed before it releases a device, nothing sends the
145
+ session another command, so TestMu AI's
146
+ [idle timeout](https://www.lambdatest.com/support/docs/desired-capabilities-in-appium/)
147
+ (120 seconds by default) ends it, which takes about three minutes after its
148
+ last command.
149
+ - Device settings (permissions, location, network, appearance), system
150
+ alerts, recording and device logs through agent-device, and port reverse
151
+ are not available. Read logs on the TestMu AI dashboard, and use TestMu AI
152
+ Tunnel to reach a server on your network.
package/docs/mcp.mdx ADDED
@@ -0,0 +1,123 @@
1
+ ---
2
+ title: MCP server
3
+ description: Let your coding agent use your app and check its own work.
4
+ icon: "plug"
5
+ "og:image": "https://e2e.tester.army/docs/images/og/mcp.png?v=17998906c0"
6
+ "twitter:image": "https://e2e.tester.army/docs/images/og/mcp.png?v=17998906c0"
7
+ ---
8
+
9
+ `e2e mcp` gives a coding agent such as Claude Code, Codex, or Cursor a live
10
+ session on your app, on web, iOS, or Android. The agent sees the screen,
11
+ acts on it, and checks its work.
12
+
13
+ ## Why it matters
14
+
15
+ | Task | Without `e2e mcp` | With `e2e mcp` |
16
+ | --- | --- | --- |
17
+ | Write a test | Guesses locators, runs, fails, guesses again | Checks each locator before it writes it |
18
+ | Change the UI | Done when it compiles | Opens the page and checks the result |
19
+ | Reproduce a bug | Guesses from the code | Follows the steps in the app |
20
+ | Show a change in a PR | Describes it | Records a video |
21
+
22
+ The server needs no model.
23
+
24
+ ## Set it up
25
+
26
+ `e2e init` offers to register the server in `.mcp.json` (Claude Code) and
27
+ `.cursor/mcp.json` (Cursor). By hand:
28
+
29
+ <Tabs>
30
+ <Tab title="Claude Code">
31
+ <CodeGroup>
32
+ ```bash npm
33
+ claude mcp add --scope project e2e -- npx e2e mcp
34
+ ```
35
+
36
+ ```bash pnpm
37
+ claude mcp add --scope project e2e -- pnpm exec e2e mcp
38
+ ```
39
+
40
+ ```bash bun
41
+ claude mcp add --scope project e2e -- bunx e2e mcp
42
+ ```
43
+ </CodeGroup>
44
+ </Tab>
45
+ <Tab title="Codex">
46
+ Add the server to `.codex/config.toml`, trust the project, and start a
47
+ new session:
48
+
49
+ ```toml title=".codex/config.toml"
50
+ [mcp_servers.e2e]
51
+ command = "npx"
52
+ args = ["e2e", "mcp"]
53
+ ```
54
+ </Tab>
55
+ <Tab title="Cursor">
56
+ Add the server to `.cursor/mcp.json`. Other clients that read an
57
+ `mcpServers` map take the same block.
58
+
59
+ ```json title=".cursor/mcp.json"
60
+ {
61
+ "mcpServers": {
62
+ "e2e": { "command": "npx", "args": ["e2e", "mcp"] }
63
+ }
64
+ }
65
+ ```
66
+ </Tab>
67
+ </Tabs>
68
+
69
+ A session starts the target's app command, if it has one.
70
+
71
+ ## Ask the agent
72
+
73
+ - "Add a Remember me checkbox to the login form. Check it in the app with e2e."
74
+ - "Write a test for checkout. Check every locator in the app first."
75
+ - "Reproduce issue #123 in the app and record a video."
76
+
77
+ Add "headed" to watch a local browser or simulator.
78
+
79
+ ## How it checks a locator
80
+
81
+ A locator guessed from the source fails in the session, not in a test run:
82
+
83
+ ```text wrap
84
+ 0 nodes match getByLabel("Email").
85
+ A test using this locator would fail with LOCATOR_NOT_FOUND. ...
86
+
87
+ 1 node matches getByRole("button", name: "Sign in").
88
+ Use: screen.getByRole("button", "Sign in")
89
+ ```
90
+
91
+ ## Run tests next to a session
92
+
93
+ A session doesn't run tests and writes no test results or cache
94
+ recordings. The agent runs tests with `e2e run`. On web, set
95
+ [`reuseExisting: true`](/web#starting-your-app) to share the app a session
96
+ started, or close the session first. On mobile, run on a device no session
97
+ uses.
98
+
99
+ ## Run several agents
100
+
101
+ Each agent, subagents included, opens its own session with its own browser
102
+ or device, up to four by default. On mobile, give each session its own
103
+ target.
104
+
105
+ ## Secrets
106
+
107
+ `type_secret` fills a secret without showing it to the agent. After a
108
+ secret is filled, the session stops returning screenshots.
109
+
110
+ <Warning>
111
+ Videos from `start_recording` are not masked.
112
+ </Warning>
113
+
114
+ ## Next
115
+
116
+ <CardGroup cols={2}>
117
+ <Card title="MCP server reference" icon="book" href="/reference/mcp">
118
+ Every tool, flag, session rule, and error code.
119
+ </Card>
120
+ <Card title="Coding agents" icon="robot" href="/coding-agents">
121
+ The skill, running tests, and reading a failure.
122
+ </Card>
123
+ </CardGroup>
@@ -11,7 +11,7 @@ import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
11
11
  export const migrationPrompt = `Migrate this project's Detox tests to e2e, one spec at a time. Docs: \`https://e2e.tester.army/docs/migrate/detox.md\` and \`https://e2e.tester.army/docs/mobile.md\`
12
12
 
13
13
  1. Read the migration guide above. It maps \`.detoxrc.js\` and the Detox API to e2e. The mobile guide covers devices and app builds.
14
- 2. If the project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent). It writes a web config and an example test, and installs nothing yet. Replace @e2e-dev/web with @e2e-dev/mobile, and rewrite \`tests/example.e2e.ts\` for the mobile engine or delete it.
14
+ 2. If the project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes --engine mobile\` (or the pnpm or bun equivalent). It writes a mobile config with one target for this machine's platform and an example test against the Settings app, and installs nothing yet. Rewrite \`tests/example.e2e.ts\` for your app or delete it.
15
15
  3. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
16
16
  4. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
17
17
  5. Port \`.detoxrc.js\` into the e2e config: one target per platform with \`mobile({ platform })\`, the app's \`bundleId\` and \`appPath\` on the target, and the device type as \`mobile({ device })\`. e2e uses the normal debug or release build, with no Detox framework.
@@ -12,13 +12,13 @@ export const migrationPrompt = `Migrate this project's Maestro flows to e2e, one
12
12
 
13
13
  1. Read the migration guide above. It maps every Maestro config option and command to e2e and marks what e2e does not have yet. The mobile guide covers devices and app builds.
14
14
  2. e2e tests are TypeScript. If the project has no \`package.json\`, as in a native iOS or Android project, create the test project in an \`e2e/\` folder beside the app.
15
- 3. If the test project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent) there. It writes a web config and an example test, and installs nothing yet. Replace @e2e-dev/web with @e2e-dev/mobile, and rewrite \`tests/example.e2e.ts\` for the mobile engine or delete it.
15
+ 3. If the test project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes --engine mobile\` (or the pnpm or bun equivalent) there. It writes a mobile config with one target for this machine's platform and an example test against the Settings app, and installs nothing yet. Rewrite \`tests/example.e2e.ts\` for your app or delete it.
16
16
  4. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
17
17
  5. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
18
18
  6. Port the flows' \`appId\` and \`.maestro/config.yaml\` into the e2e config: one target per platform with \`mobile({ platform })\`, \`app: { bundleId }\` on each target, and tags as \`{ tags }\` on tests. Move passwords and tokens from \`env\` into \`credentials\` or \`secrets\`.
19
19
  7. Install dependencies, then run \`npx agent-device doctor\` and fix what it reports.
20
20
  8. List the flows and ask me which one to move first. Suggest one whose \`optional\` steps and \`runFlow: { when }\` branches often need updating.
21
- 9. Rewrite that flow as a \`.e2e.ts\` test under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default), with one awaited call per command. Start it with \`app.open()\`, since nothing launches on its own. Follow the guide's "Rewrite selectors" section: e2e matches the whole string and fails on several matches. Where the guide marks a command missing, use the workaround it gives, or leave that flow in Maestro and tell me why.
21
+ 9. Rewrite that flow as a \`.e2e.ts\` test under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default), with one awaited call per command. Start it with \`app.open()\`, since nothing launches on its own. Follow the guide's "Rewrite selectors" section: e2e matches the whole string and fails on several matches. Where the guide says "Not yet" or "No equivalent", use the workaround it gives, or leave that flow in Maestro and tell me why.
22
22
  10. Keep the launch and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted taps breaks often, propose an \`agent.act\` goal for it instead.
23
23
  11. Build and install the app, run the new file with \`npx e2e run <file>\` on each platform target, and fix it until it passes. Keep the Maestro flow until I confirm it can go.
24
24
  12. Summarize which flows moved, which stayed in Maestro, and why.`;
@@ -43,26 +43,26 @@ The reference pages hold the details: [screen](/reference/screen),
43
43
  [expect](/reference/expect), [test](/reference/test), and the
44
44
  [mobile engine](/reference/mobile).
45
45
 
46
- | Maestro | e2e | Status |
47
- | --- | --- | --- |
48
- | `appId` at the top of every flow | `app: { bundleId: 'com.example.app' }` on the target, the app `app.open()` launches | renamed |
49
- | a different `appId` per flow | `device.openApp(id)` inside the test, or separate app-specific configs and runs; every test runs on every selected target | renamed |
50
- | `--platform ios` | `mobile({ platform: 'ios' })`, one engine per platform | renamed |
51
- | `--device` | `mobile({ device: 'iPhone 17' })`; a list is a pool, one worker per device | renamed |
52
- | `.maestro/config.yaml` `flows` | `tests`, default `tests/**/*.e2e.ts`; Maestro runs one folder level, e2e recurses | renamed |
53
- | `flows: ['!helper.yaml']` | a `tests` entry with a leading `!`: `tests: ['tests/**/*.e2e.ts', '!tests/helpers/**']` | renamed |
54
- | `tags` and `--include-tags` | `{ tags: [...] }` on a test or group, `--tag` on the CLI | renamed |
55
- | `--exclude-tags` | `--exclude-tag` | renamed |
56
- | `env` and `${VAR}` | module constants, and `params` in a goal: `agent.act('sign in as {email}', { params: { email } })` | renamed |
57
- | `MAESTRO_*` variables | `process.env` in the config; a password or token goes under `credentials` or `secrets`, which the model never sees, see [Signing in](/authentication) | renamed |
58
- | `--format junit`, `--output` | `--reporter list,junit`, written to `.e2e/junit.xml` | renamed |
59
- | `--format html` | no equivalent; `markdown` writes `.e2e/summary.md`, and each failed test gets a `trace.md` page under `.e2e/results/` | missing |
60
- | `properties` (JUnit `<property>`) | not yet | missing |
61
- | `--shards`, `--shard-split` | `device: [...]` plus `workers` on one machine; `--shard <index/total>` across machines, see [Sharding](/ci#sharding) | renamed |
62
- | `maestro record` | `npx e2e run --video`, one MP4 of the device per attempt, or `video` in the config | renamed |
63
- | `maestro test -c`, Maestro Studio | not yet; `npx e2e mcp` gives a coding agent a live session on the device, see [Coding agents](/coding-agents) | missing |
64
- | `maestro hierarchy` | not yet; `device.locator` takes agent-device selectors for nodes `screen` cannot name | missing |
65
- | Maestro Cloud | your CI, see [Continuous integration](/ci) | by design |
46
+ | Maestro | e2e |
47
+ | --- | --- |
48
+ | `appId` at the top of every flow | `app: { bundleId: 'com.example.app' }` on the target, the app `app.open()` launches |
49
+ | a different `appId` per flow, or a second one mid-flow | `device.openApp(id)` inside the test, or separate app-specific configs and runs; every test runs on every selected target. With `mobileTools`, the agent's `open_app` tool switches apps inside a goal, see [Agent tools](/mobile#agent-tools) |
50
+ | `--platform ios` | `mobile({ platform: 'ios' })`, one engine per platform |
51
+ | `--device` | `mobile({ device: 'iPhone 17' })`; a list is a pool, one worker per device |
52
+ | `.maestro/config.yaml` `flows` | `tests`, default `tests/**/*.e2e.ts`; Maestro runs one folder level, e2e recurses |
53
+ | `flows: ['!helper.yaml']` | a `tests` entry with a leading `!`: `tests: ['tests/**/*.e2e.ts', '!tests/helpers/**']` |
54
+ | `tags` and `--include-tags` | `{ tags: [...] }` on a test or group, `--tag` on the CLI |
55
+ | `--exclude-tags` | `--exclude-tag` |
56
+ | `env` and `${VAR}` | module constants, and `params` in a goal: `agent.act('sign in as {email}', { params: { email } })` |
57
+ | `MAESTRO_*` variables | `process.env` in the config; a password or token goes under `credentials` or `secrets`, which the model never sees, see [Signing in](/authentication) |
58
+ | `--format junit`, `--output` | `--reporter list,junit`, written to `.e2e/junit.xml` |
59
+ | `--format html` | Not yet. `markdown` writes `.e2e/summary.md`, and each failed test gets a `trace.md` page under `.e2e/results/` |
60
+ | `properties` (JUnit `<property>`) | Not yet |
61
+ | `--shards`, `--shard-split` | `device: [...]` plus `workers` on one machine; `--shard <index/total>` across machines, see [Sharding](/ci#sharding) |
62
+ | `maestro record` | `npx e2e run --video`, one MP4 of the device per attempt, or `video` in the config |
63
+ | `maestro test -c`, Maestro Studio | Not yet. `npx e2e mcp` gives a coding agent a live session on the device, see [MCP server](/mcp) |
64
+ | `maestro hierarchy` | Not yet. `device.locator` takes agent-device selectors for nodes `screen` cannot name |
65
+ | Maestro Cloud | your CI, see [Continuous integration](/ci) |
66
66
 
67
67
  ## Rewrite one flow
68
68
 
@@ -99,111 +99,108 @@ test('asks for the email', async ({ app, agent, screen }) => {
99
99
  });
100
100
  ```
101
101
 
102
- A flow can also stay fully deterministic, with a `screen.getBy*(...).tap()`
103
- line for every `tapOn`.
104
-
105
102
  ## Map each command
106
103
 
107
- `Status` is `renamed`, `device only`, `by design`, or `missing` with what to
108
- do today. Calls on `device` run only on a device, so a test that uses it
109
- declares `requires: ['device']` and other targets skip it. Calls on `screen`,
110
- `app`, `agent`, and `expect(locator)` run on every target.
104
+ Calls on `device` run only on a device, so a test that uses it declares
105
+ `requires: ['device']` and other targets skip it. Calls on `screen`, `app`,
106
+ `agent`, and `expect(locator)` run on every target. "Not yet" marks a
107
+ command e2e doesn't have today, and "No equivalent" one it leaves out on
108
+ purpose.
111
109
 
112
110
  <AccordionGroup>
113
111
  <Accordion title="App lifecycle">
114
112
 
115
- | Maestro | e2e | Status |
116
- | --- | --- | --- |
117
- | `launchApp` | `app.open()`, the fresh launch of the target's `app.bundleId`; nothing launches on its own | renamed |
118
- | `launchApp: { clearState: true }`, `clearState` | `app.clearState()` | renamed |
119
- | `launchApp: { stopApp: false }` | `device.openApp(id)` without `relaunch` | device only |
120
- | `launchApp: { arguments }` | `app.launchArguments` on the target, or `device.openApp(app, { relaunch: true, launchArguments })` for one launch; a list of strings, not a map | renamed |
121
- | `launchApp: { permissions }` | `app.permissions` on the target, set before every fresh launch, or `device.openApp(app, { relaunch: true, permissions })` for one; `grant`, `deny`, `reset` for `allow`, `deny`, `unset`; no `all`, nothing is granted unless named | renamed |
122
- | `stopApp`, `killApp` | `device.closeApp()` | device only |
123
- | `stopApp`, then `launchApp` | `app.restart()` | renamed |
124
- | `clearKeychain` | `device.clearKeychain()`, the whole simulator's keychain; iOS only | renamed |
125
- | `openLink` | `device.openLink(url, { app })`; on iOS the link opens into `app`, default the pinned app | renamed |
126
- | `addMedia` | not yet | missing |
127
- | `back` | `app.back()`, or `device.back()` | renamed |
113
+ | Maestro | e2e |
114
+ | --- | --- |
115
+ | `launchApp` | `app.open()`, the fresh launch of the target's `app.bundleId`; nothing launches on its own |
116
+ | `launchApp: { clearState: true }`, `clearState` | `app.clearState()` |
117
+ | `launchApp: { stopApp: false }` | `device.openApp(id)` without `relaunch` |
118
+ | `launchApp: { arguments }` | `app.launchArguments` on the target, or `device.openApp(app, { relaunch: true, launchArguments })` for one launch; a list of strings, not a map |
119
+ | `launchApp: { permissions }` | `app.permissions` on the target, set before every fresh launch, or `device.openApp(app, { relaunch: true, permissions })` for one; `grant`, `deny`, `reset` for `allow`, `deny`, `unset`; no `all`, nothing is granted unless named |
120
+ | `stopApp`, `killApp` | `device.closeApp()` |
121
+ | `stopApp`, then `launchApp` | `app.restart()` |
122
+ | `clearKeychain` | `device.clearKeychain()`, the whole simulator's keychain; iOS only |
123
+ | `openLink` | `device.openLink(url, { app })`; on iOS the link opens into `app`, default the pinned app |
124
+ | `addMedia` | Not yet |
125
+ | `back` | `app.back()`, or `device.back()` |
128
126
 
129
127
  </Accordion>
130
128
  <Accordion title="Taps and input">
131
129
 
132
- | Maestro | e2e | Status |
133
- | --- | --- | --- |
134
- | `tapOn: "Text"` | `screen.getByText('Text').tap()`; exact, see [Rewrite selectors](#rewrite-selectors) | renamed |
135
- | `tapOn: { id: x }` | `screen.getByTestId('x').tap()` | renamed |
136
- | `tapOn: { text, index: 2 }` | `.nth(2)` | renamed |
137
- | `tapOn: { below: "Email" }`, `above`, `leftOf`, `rightOf`, `containsDescendants` | no equivalent, see [Rewrite selectors](#rewrite-selectors) | by design |
138
- | `tapOn: { point: "x,y" }` | `screen.tapAt({ x, y })`; inside a node, `.tap({ position: { x, y } })` | renamed |
139
- | `retryTapIfNoChange`, `waitToSettleTimeoutMs` | `mobile({ settle, transition })`, engine-wide; no per-call knob | renamed |
140
- | `doubleTapOn`, `longPressOn` | `.doubleTap()`, `.longPress({ duration })` | renamed |
141
- | `inputText` | `screen.getByTestId('field').fill('text')` | renamed |
142
- | `inputRandomEmail`, `inputRandomText`, `faker` | `unique()` around a value in `act` params so the cache still replays; any npm faker for the rest | renamed |
143
- | `eraseText` | `.clear()` | renamed |
144
- | `hideKeyboard` | `device.dismissKeyboard()` | renamed |
145
- | `pressKey: Enter` | `.press('Enter')` on the field | renamed |
146
- | `pressKey: Home`, `back` | `device.home()`, `device.back()` | device only |
147
- | `pressKey: lock`, `power`, `volume up`, TV remote keys | not yet | missing |
148
- | `copyTextFrom`, `pasteText` | `await locator.textContent()`, then `.fill(text)` | renamed |
149
- | `setClipboard` | `device.setClipboard(text)`; `device.clipboard()` reads it | device only |
150
- | `label: "..."` on a command | not yet; every `screen`, `device`, and `agent` call is a report step named by the call | missing |
130
+ | Maestro | e2e |
131
+ | --- | --- |
132
+ | `tapOn: "Text"` | `screen.getByText('Text').tap()`; exact, see [Rewrite selectors](#rewrite-selectors) |
133
+ | `tapOn: { id: x }` | `screen.getByTestId('x').tap()` |
134
+ | `tapOn: { text, index: 2 }` | `.nth(2)` |
135
+ | `tapOn: { below: "Email" }`, `above`, `leftOf`, `rightOf`, `containsDescendants` | No equivalent, see [Rewrite selectors](#rewrite-selectors) |
136
+ | `tapOn: { point: "x,y" }` | `screen.tapAt({ x, y })`; inside a node, `.tap({ position: { x, y } })` |
137
+ | `retryTapIfNoChange`, `waitToSettleTimeoutMs` | `mobile({ settle, transition })`, engine-wide; no per-call knob |
138
+ | `doubleTapOn`, `longPressOn` | `.doubleTap()`, `.longPress({ duration })` |
139
+ | `inputText` | `screen.getByTestId('field').fill('text')` |
140
+ | `inputRandomEmail`, `inputRandomText`, `faker` | `unique()` around a value in `act` params so the cache still replays; any npm faker for the rest |
141
+ | `eraseText` | `.clear()` |
142
+ | `hideKeyboard` | `device.dismissKeyboard()` |
143
+ | `pressKey: Enter` | `.press('Enter')` on the field |
144
+ | `pressKey: Home`, `back` | `device.home()`, `device.back()` |
145
+ | `pressKey: lock`, `power`, `volume up`, TV remote keys | Not yet |
146
+ | `copyTextFrom`, `pasteText` | `await locator.textContent()`, then `.fill(text)` |
147
+ | `setClipboard` | `device.setClipboard(text)`; `device.clipboard()` reads it |
148
+ | `label: "..."` on a command | Not yet. Every `screen`, `device`, and `agent` call is a report step named by the call |
151
149
 
152
150
  </Accordion>
153
151
  <Accordion title="Gestures">
154
152
 
155
- | Maestro | e2e | Status |
156
- | --- | --- | --- |
157
- | `swipe: { direction: UP }` | `screen.swipe({ direction: 'down' })`, or `locator.swipe(...)` inside one node; Maestro names the finger's direction, e2e the content's | renamed |
158
- | `swipe: { start, end }` | `screen.swipe({ from: { x, y }, to: { x, y } })` | renamed |
159
- | `scroll` | `screen.swipe({ direction: 'down' })` | renamed |
160
- | `scrollUntilVisible: { element }` | `screen.scrollUntilVisible(locator, { direction })` | renamed |
161
- | `waitForAnimationToEnd` | an assertion on the settled state; `mobile({ settle, transition })` for the engine | renamed |
153
+ | Maestro | e2e |
154
+ | --- | --- |
155
+ | `swipe: { direction: UP }` | `screen.swipe({ direction: 'down' })`, or `locator.swipe(...)` inside one node; Maestro names the finger's direction, e2e the content's |
156
+ | `swipe: { start, end }` | `screen.swipe({ from: { x, y }, to: { x, y } })` |
157
+ | `scroll` | `screen.swipe({ direction: 'down' })` |
158
+ | `scrollUntilVisible: { element }` | `screen.scrollUntilVisible(locator, { direction })` |
159
+ | `waitForAnimationToEnd` | an assertion on the settled state; `mobile({ settle, transition })` for the engine |
162
160
 
163
161
  </Accordion>
164
162
  <Accordion title="Assertions and waits">
165
163
 
166
- | Maestro | e2e | Status |
167
- | --- | --- | --- |
168
- | `assertVisible`, `assertNotVisible` | `await expect(locator).toBeVisible()`, `.toBeHidden()` | renamed |
169
- | `assertVisible: { text, enabled: true }` | `.toBeVisible()` and `.toBeEnabled()` | renamed |
170
- | `assertVisible: { checked, selected, focused }` | `.toBeChecked()`, `.toBeSelected()`, `.toBeFocused()` | renamed |
171
- | `assertTrue: ${...}` | `expect(value).toBe(true)`, or `expect.poll` for a value that is still changing | renamed |
172
- | `extendedWaitUntil: { visible, timeout }` | `.toBeVisible({ timeout })` | renamed |
173
- | `assertScreenshot: { path, thresholdPercentage }` | `await expect(screen).toHaveScreenshot(name, { maxDiffPixelRatio })`, see [Visual comparison](/visual-testing) | renamed |
174
- | `assertDarkMode`, `assertLightMode` | not yet; assert on a themed control | missing |
175
- | `retry: { maxRetries, commands }` | `expect.poll` for a value, `{ retries }` on the test for the flow | renamed |
176
- | `takeScreenshot` | `app.screenshot('label')`, redacted, withheld after a secret fill | renamed |
164
+ | Maestro | e2e |
165
+ | --- | --- |
166
+ | `assertVisible`, `assertNotVisible` | `await expect(locator).toBeVisible()`, `.toBeHidden()` |
167
+ | `assertVisible: { text, enabled: true }` | `.toBeVisible()` and `.toBeEnabled()` |
168
+ | `assertVisible: { checked, selected, focused }` | `.toBeChecked()`, `.toBeSelected()`, `.toBeFocused()` |
169
+ | `assertTrue: ${...}` | `expect(value).toBe(true)`, or `expect.poll` for a value that is still changing |
170
+ | `extendedWaitUntil: { visible, timeout }` | `.toBeVisible({ timeout })` |
171
+ | `assertScreenshot: { path, thresholdPercentage }` | `await expect(screen).toHaveScreenshot(name, { maxDiffPixelRatio })`, see [Visual comparison](/visual-testing) |
172
+ | `assertDarkMode`, `assertLightMode` | Not yet. Assert on a themed control |
173
+ | `retry: { maxRetries, commands }` | `expect.poll` for a value, `{ retries }` on the test for the flow |
174
+ | `takeScreenshot` | `app.screenshot('label')`, redacted, withheld after a secret fill |
177
175
 
178
176
  </Accordion>
179
177
  <Accordion title="Device">
180
178
 
181
- | Maestro | e2e | Status |
182
- | --- | --- | --- |
183
- | `setLocation` | `device.setLocation({ latitude, longitude })` | device only |
184
- | `travel: { points }` | `device.setLocation` in a loop | device only |
185
- | `setAirplaneMode`, `toggleAirplaneMode` | `device.setAirplaneMode(true)` or `false`, the state you want; `true` is Android only | device only |
186
- | `setDarkMode`, `toggleDarkMode` | `device.setAppearance('dark')` or `'light'` | device only |
187
- | `setOrientation` | `device.setOrientation('landscape-left')` | device only |
188
- | `setPermissions` | `device.setPermission(name, 'grant' \| 'deny' \| 'reset')`, one permission per call, on the open app | device only |
189
- | a second `appId` mid-flow | `device.openApp(id)`; the `open_app` tool lets the agent switch inside a goal, see [Agent tools](/mobile#agent-tools) | device only |
190
- | platform-specific selectors | `device.locator('role=text label="..."')`, agent-device's selector terms | device only |
179
+ | Maestro | e2e |
180
+ | --- | --- |
181
+ | `setLocation` | `device.setLocation({ latitude, longitude })` |
182
+ | `travel: { points }` | `device.setLocation` in a loop |
183
+ | `setAirplaneMode`, `toggleAirplaneMode` | `device.setAirplaneMode(true)` or `false`, the state you want; `true` is Android only |
184
+ | `setDarkMode`, `toggleDarkMode` | `device.setAppearance('dark')` or `'light'` |
185
+ | `setOrientation` | `device.setOrientation('landscape-left')` |
186
+ | `setPermissions` | `device.setPermission(name, 'grant' \| 'deny' \| 'reset')`, one permission per call, on the target's app, which need not be running; call it before `app.open()` |
187
+ | platform-specific selectors | `device.locator('role=text label="..."')`, agent-device's selector terms |
191
188
 
192
189
  </Accordion>
193
190
  <Accordion title="AI commands and flow control">
194
191
 
195
- | Maestro | e2e | Status |
196
- | --- | --- | --- |
197
- | `assertWithAI`, `assertNoDefectsWithAI` | `agent.assert('...')`, `agent.assert('...', { vision: true })` | renamed |
198
- | `extractTextWithAI` | `agent.extract('...', { schema })` | renamed |
199
- | `runFlow: file.yaml` | a function, or a fixture from `test.extend` | renamed |
200
- | `runFlow: { when: { visible } }` | `if (await locator.isVisible()) { ... }`, see [Rewrite selectors](#rewrite-selectors) | renamed |
201
- | `optional: true` on a command | the same `isVisible()` check | renamed |
202
- | `repeat: { times: n }`, `repeat: { while }` | a `for` or `while` loop | renamed |
203
- | `runScript`, `evalScript`, `${output.x}` | TypeScript in the test | renamed |
204
- | `http.get`, `http.post`, `json()` | Node `fetch`, see [API](/api-testing) | renamed |
205
- | `onFlowStart`, `onFlowComplete` | `beforeEach`, `afterEach` | renamed |
206
- | `startRecording`, `stopRecording` | `test('x', { video: 'on' }, ...)` records that test's whole attempts; no start and stop inside a test | renamed |
192
+ | Maestro | e2e |
193
+ | --- | --- |
194
+ | `assertWithAI`, `assertNoDefectsWithAI` | `agent.assert('...')`, `agent.assert('...', { vision: true })` |
195
+ | `extractTextWithAI` | `agent.extract('...', { schema })` |
196
+ | `runFlow: file.yaml` | a function, or a fixture from `test.extend` |
197
+ | `runFlow: { when: { visible } }` | `if (await locator.isVisible()) { ... }`, see [Rewrite selectors](#rewrite-selectors) |
198
+ | `optional: true` on a command | the same `isVisible()` check |
199
+ | `repeat: { times: n }`, `repeat: { while }` | a `for` or `while` loop |
200
+ | `runScript`, `evalScript`, `${output.x}` | TypeScript in the test |
201
+ | `http.get`, `http.post`, `json()` | Node `fetch`, see [API](/api-testing) |
202
+ | `onFlowStart`, `onFlowComplete` | `beforeEach`, `afterEach` |
203
+ | `startRecording`, `stopRecording` | `test('x', { video: 'on' }, ...)` records that test's whole attempts; no start and stop inside a test |
207
204
 
208
205
  </Accordion>
209
206
  </AccordionGroup>
@@ -231,10 +228,9 @@ Four differences from Maestro's selectors cause most edits.
231
228
 
232
229
  ## What differs on purpose
233
230
 
234
- - No agent-device client in tests. `device.locator` covers platform-specific
235
- selectors. Everything else goes through `screen`, `app`, `device`, and the
236
- agent, so it is recorded and redacted.
237
- - No `test.setup` or `session:` on a device, see
231
+ - Tests get no agent-device client. Every call goes through `screen`, `app`,
232
+ `device`, or the agent, so e2e records and redacts it.
233
+ - A device has no `test.setup` or `{ session }`, see
238
234
  [what differs from the web](/mobile#what-differs-from-the-web).
239
235
 
240
236
  ## Next