e2e 0.17.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -0
- package/dist/.build.json +1 -1
- package/dist/agent/ai-sdk.js +5 -4
- package/dist/agent/error.js +3 -3
- package/dist/agent/invocation.js +3 -4
- package/dist/agent/model/tool-choice.d.ts +9 -0
- package/dist/agent/model/tool-choice.js +14 -0
- package/dist/agent/observation-feed.js +3 -4
- package/dist/agent/phases.d.ts +7 -1
- package/dist/agent/phases.js +12 -3
- package/dist/agent/prompts.js +3 -0
- package/dist/agent/replay.js +18 -15
- package/dist/agent/step-cache.d.ts +2 -1
- package/dist/agent/step-cache.js +9 -5
- package/dist/agent/tool-loop.js +92 -5
- package/dist/cache/trace.js +6 -2
- package/dist/cli/bin.js +2 -2
- package/dist/cli/index.js +35 -24
- package/dist/cli/init/mcp-config.d.ts +2 -1
- package/dist/cli/init/mcp-config.js +8 -3
- package/dist/cli/init/sibling-versions.json +1 -1
- package/dist/cli/mcp.d.ts +3 -2
- package/dist/cli/mcp.js +52 -16
- package/dist/cli/run-output.d.ts +17 -0
- package/dist/cli/run-output.js +45 -0
- package/dist/collect/collect.js +8 -4
- package/dist/collect/registry.js +27 -16
- package/dist/config/compiled-files.d.ts +36 -0
- package/dist/config/compiled-files.js +50 -0
- package/dist/config/diagnose.js +21 -19
- package/dist/config/esm-hooks.d.ts +39 -44
- package/dist/config/esm-hooks.js +238 -63
- package/dist/config/load.d.ts +1 -7
- package/dist/config/load.js +23 -66
- package/dist/config/register.d.ts +8 -0
- package/dist/config/register.js +9 -0
- package/dist/config/specifiers.d.ts +19 -0
- package/dist/config/specifiers.js +33 -0
- package/dist/config/tsconfig.d.ts +56 -0
- package/dist/config/tsconfig.js +115 -0
- package/dist/config/typescript.d.ts +19 -0
- package/dist/config/typescript.js +253 -0
- package/dist/engine/contract.d.ts +1 -0
- package/dist/engine/index.d.ts +2 -0
- package/dist/engine/session.js +2 -2
- package/dist/expect/async.js +2 -1
- package/dist/expect/poll.js +2 -2
- package/dist/internal/errors.d.ts +10 -3
- package/dist/internal/errors.js +23 -8
- package/dist/internal/node-version.d.ts +16 -7
- package/dist/internal/node-version.js +50 -15
- package/dist/internal/package-manager.js +1 -1
- package/dist/internal/realm-slot.d.ts +5 -4
- package/dist/internal/redact.d.ts +2 -0
- package/dist/internal/redact.js +4 -0
- package/dist/internal/time.d.ts +21 -5
- package/dist/internal/time.js +35 -3
- package/dist/locator/engine.js +47 -11
- package/dist/locator/screen.js +31 -5
- package/dist/mcp/output.d.ts +59 -0
- package/dist/mcp/output.js +129 -0
- package/dist/mcp/server.d.ts +4 -0
- package/dist/mcp/server.js +9 -12
- package/dist/mcp/session.d.ts +8 -1
- package/dist/mcp/session.js +55 -14
- package/dist/mcp/tools.js +7 -5
- package/dist/report/list.d.ts +11 -0
- package/dist/report/list.js +23 -6
- package/dist/run/artifacts.d.ts +4 -4
- package/dist/run/artifacts.js +3 -2
- package/dist/run/execute.d.ts +25 -5
- package/dist/run/execute.js +44 -20
- package/dist/run/failure-evidence.js +6 -2
- package/dist/run/in-process.js +51 -10
- package/dist/run/process-output.d.ts +73 -0
- package/dist/run/process-output.js +130 -0
- package/dist/run/provision.d.ts +2 -0
- package/dist/run/provision.js +1 -0
- package/dist/run/records.d.ts +4 -4
- package/dist/run/retry.d.ts +7 -0
- package/dist/run/retry.js +27 -11
- package/dist/run/runner.d.ts +8 -0
- package/dist/run/runner.js +32 -9
- package/dist/run/scheduler.js +255 -19
- package/dist/run/secrecy.d.ts +22 -19
- package/dist/run/secrecy.js +22 -18
- package/dist/run/serial.d.ts +73 -2
- package/dist/run/serial.js +239 -58
- package/dist/run/standalone.js +1 -1
- package/dist/run/trace-redaction.d.ts +5 -6
- package/dist/run/trace-redaction.js +5 -6
- package/dist/run/worker/entry.js +35 -14
- package/dist/run/worker/handle.js +10 -5
- package/dist/run/worker/protocol.d.ts +83 -3
- package/dist/run/worker/session.js +28 -0
- package/dist/telemetry/environment.d.ts +2 -1
- package/dist/telemetry/environment.js +12 -2
- package/dist/telemetry/project.d.ts +5 -6
- package/dist/telemetry/project.js +6 -11
- package/dist/telemetry/store.d.ts +5 -5
- package/dist/telemetry/store.js +5 -7
- package/dist/telemetry/telemetry.d.ts +2 -3
- package/dist/telemetry/telemetry.js +2 -4
- package/dist/types.d.ts +6 -3
- package/docs/agent-steps.mdx +1 -0
- package/docs/agents.mdx +1 -0
- package/docs/api-testing.mdx +2 -1
- package/docs/authentication.mdx +3 -2
- package/docs/browser.mdx +7 -6
- package/docs/bug-bash.mdx +2 -1
- package/docs/cache.mdx +2 -1
- package/docs/ci.mdx +3 -2
- package/docs/coding-agents.mdx +2 -1
- package/docs/core-concepts.mdx +208 -0
- package/docs/debugging.mdx +19 -9
- package/docs/decision-models.mdx +142 -0
- package/docs/example-projects.mdx +32 -0
- package/docs/executors.mdx +3 -0
- package/docs/explore.mdx +1 -0
- package/docs/github.mdx +2 -1
- package/docs/index.mdx +9 -5
- package/docs/integrations/eas.mdx +1 -0
- package/docs/integrations/index.mdx +1 -0
- package/docs/integrations/kernel.mdx +2 -1
- package/docs/migrate/cypress.mdx +26 -3
- package/docs/migrate/detox.mdx +24 -1
- package/docs/migrate/maestro.mdx +28 -4
- package/docs/migrate/playwright.mdx +27 -4
- package/docs/migrate/selenium.mdx +27 -3
- package/docs/mobile-ci.mdx +6 -5
- package/docs/mobile.mdx +79 -8
- package/docs/models.mdx +7 -2
- package/docs/quickstart.mdx +26 -15
- package/docs/reference/agent.mdx +13 -3
- package/docs/reference/app.mdx +1 -0
- package/docs/reference/cli.mdx +11 -5
- package/docs/reference/config.mdx +42 -6
- package/docs/reference/engine.mdx +22 -10
- package/docs/reference/environment.mdx +2 -1
- package/docs/reference/errors.mdx +11 -8
- package/docs/reference/expect.mdx +2 -1
- package/docs/reference/mcp.mdx +21 -8
- package/docs/reference/mobile.mdx +1 -0
- package/docs/reference/reporters.mdx +1 -0
- package/docs/reference/screen.mdx +19 -9
- package/docs/reference/test.mdx +21 -1
- package/docs/reference/web.mdx +8 -2
- package/docs/security.mdx +33 -27
- package/docs/subscriptions.mdx +1 -0
- package/docs/telemetry.mdx +8 -5
- package/docs/tools.mdx +1 -0
- package/docs/{starting-your-app.mdx → web.mdx} +157 -11
- package/docs/writing-an-engine.mdx +2 -1
- package/package.json +11 -7
- package/skills/e2e/SKILL.md +3 -1
- package/skills/e2e/references/agent.md +11 -0
- package/skills/e2e/references/debugging.md +1 -1
- package/skills/e2e/references/mcp.md +8 -3
- package/skills/e2e/references/running.md +2 -1
- package/skills/e2e/references/setup.md +9 -1
- package/skills/e2e/references/writing-tests.md +1 -1
- package/docs/assertions.mdx +0 -94
- package/docs/goals.mdx +0 -176
- package/docs/locators.mdx +0 -64
- package/docs/physical-devices.mdx +0 -76
- package/docs/viewports.mdx +0 -146
package/docs/migrate/detox.mdx
CHANGED
|
@@ -1,8 +1,31 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: From Detox
|
|
3
3
|
description: Use your existing app build and testIDs with screen, expect, and agent goals.
|
|
4
|
+
icon: "/images/icons/detox.svg"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
8
|
+
|
|
9
|
+
export const migrationPrompt = `Migrate this project's Detox tests to e2e, one spec at a time. Docs: \`https://e2e.tester.army/docs/migrate/detox.md\` and \`https://e2e.tester.army/docs/mobile.md\`
|
|
10
|
+
|
|
11
|
+
1. Read the migration guide above. It maps \`.detoxrc.js\` and the Detox API to e2e. The mobile guide covers devices and app builds.
|
|
12
|
+
2. If the project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent). It writes a web config and an example test, and installs nothing yet. Replace @e2e-dev/web with @e2e-dev/mobile, and rewrite \`tests/example.e2e.ts\` for the mobile engine or delete it.
|
|
13
|
+
3. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
14
|
+
4. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
15
|
+
5. Port \`.detoxrc.js\` into the e2e config: one target per platform with \`mobile({ platform })\`, the app's \`bundleId\` and \`appPath\` on the target, and the device type as \`mobile({ device })\`. e2e uses the normal debug or release build, with no Detox framework.
|
|
16
|
+
6. Install dependencies, then run \`npx agent-device doctor\` and fix what it reports.
|
|
17
|
+
7. List the Detox specs and ask me which one to move first. Suggest a flow that needs frequent maintenance.
|
|
18
|
+
8. Rewrite that spec as a \`.e2e.ts\` file under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default), not in the Detox folder. Start it with \`app.open()\`, since nothing launches on its own. \`testID\` props work with \`screen.getByTestId\`. Replace \`waitFor(...).withTimeout()\` with a locator assertion and its \`timeout\`. e2e fails when a query matches several nodes, so narrow those queries.
|
|
19
|
+
9. Keep the launch and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted interactions breaks often, propose an \`agent.act\` goal for it instead.
|
|
20
|
+
10. Build and install the app, run the new file with \`npx e2e run <file>\` on each platform target, and fix it until it passes. Keep the Detox spec until I confirm it can go.
|
|
21
|
+
11. Summarize which tests moved, which stayed in Detox, and why.`;
|
|
22
|
+
|
|
23
|
+
<SetupPrompt>
|
|
24
|
+
|
|
25
|
+
<Prompt description={migrationPrompt} children={migrationPrompt} actions={["copy", "cursor"]} />
|
|
26
|
+
|
|
27
|
+
</SetupPrompt>
|
|
28
|
+
|
|
6
29
|
`@e2e-dev/mobile` drives a simulator or emulator using your normal debug
|
|
7
30
|
or release build. It does not need the Detox framework. React Native
|
|
8
31
|
`testID` props work with `getByTestId` on both platforms.
|
|
@@ -131,7 +154,7 @@ to settle, configure the engine's
|
|
|
131
154
|
[`settle` and `transition` options](/reference/mobile#engine).
|
|
132
155
|
|
|
133
156
|
<CardGroup cols={2}>
|
|
134
|
-
<Card title="
|
|
157
|
+
<Card title="Mobile" icon="mobile-screen" href="/mobile">
|
|
135
158
|
Configure your app and devices.
|
|
136
159
|
</Card>
|
|
137
160
|
<Card title="Mobile engine reference" icon="book" href="/reference/mobile">
|
package/docs/migrate/maestro.mdx
CHANGED
|
@@ -1,8 +1,32 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: From Maestro
|
|
3
3
|
description: Move Maestro flows to TypeScript with locators, assertions, and agent goals.
|
|
4
|
+
icon: "/images/icons/maestro.svg"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
8
|
+
|
|
9
|
+
export const migrationPrompt = `Migrate this project's Maestro flows to e2e, one flow at a time. Docs: \`https://e2e.tester.army/docs/migrate/maestro.md\` and \`https://e2e.tester.army/docs/mobile.md\`
|
|
10
|
+
|
|
11
|
+
1. Read the migration guide above. It maps every Maestro config option and command to e2e and marks what e2e does not have yet. The mobile guide covers devices and app builds.
|
|
12
|
+
2. e2e tests are TypeScript. If the project has no \`package.json\`, as in a native iOS or Android project, create the test project in an \`e2e/\` folder beside the app.
|
|
13
|
+
3. If the test project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent) there. It writes a web config and an example test, and installs nothing yet. Replace @e2e-dev/web with @e2e-dev/mobile, and rewrite \`tests/example.e2e.ts\` for the mobile engine or delete it.
|
|
14
|
+
4. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
15
|
+
5. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
16
|
+
6. Port the flows' \`appId\` and \`.maestro/config.yaml\` into the e2e config: one target per platform with \`mobile({ platform })\`, \`app: { bundleId }\` on each target, and tags as \`{ tags }\` on tests. Move passwords and tokens from \`env\` into \`credentials\` or \`secrets\`.
|
|
17
|
+
7. Install dependencies, then run \`npx agent-device doctor\` and fix what it reports.
|
|
18
|
+
8. List the flows and ask me which one to move first. Suggest one whose \`optional\` steps and \`runFlow: { when }\` branches often need updating.
|
|
19
|
+
9. Rewrite that flow as a \`.e2e.ts\` test under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default), with one awaited call per command. Start it with \`app.open()\`, since nothing launches on its own. Maestro matches text and ids as regular expressions and taps the first of several matches; e2e matches the whole string and fails on several, so use a RegExp, \`.first()\`, or \`.nth()\` where the flow relied on that. \`optional\` and \`runFlow: { when }\` become \`if (await locator.isVisible())\`. Relative selectors such as \`below\` have no port; use a label, a test id, or a goal. Where the guide marks a command missing, use the workaround it gives, or leave that flow in Maestro and tell me why.
|
|
20
|
+
10. Keep the launch and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted taps breaks often, propose an \`agent.act\` goal for it instead.
|
|
21
|
+
11. Build and install the app, run the new file with \`npx e2e run <file>\` on each platform target, and fix it until it passes. Keep the Maestro flow until I confirm it can go.
|
|
22
|
+
12. Summarize which flows moved, which stayed in Maestro, and why.`;
|
|
23
|
+
|
|
24
|
+
<SetupPrompt>
|
|
25
|
+
|
|
26
|
+
<Prompt description={migrationPrompt} children={migrationPrompt} actions={["copy", "cursor"]} />
|
|
27
|
+
|
|
28
|
+
</SetupPrompt>
|
|
29
|
+
|
|
6
30
|
Translate Maestro commands into awaited TypeScript calls. Keep exact
|
|
7
31
|
interactions as locator calls, or give the agent a goal that covers several
|
|
8
32
|
of them. Both runners can use the same simulator or emulator.
|
|
@@ -167,7 +191,7 @@ declares `requires: ['device']` so other targets skip it. `Status` is `same`,
|
|
|
167
191
|
| `optional: true` on a command | the same `isVisible()` check | renamed |
|
|
168
192
|
| `repeat: { times: n }`, `repeat: { while }` | a `for` or `while` loop | renamed |
|
|
169
193
|
| `runScript`, `evalScript`, `${output.x}` | TypeScript in the test | renamed |
|
|
170
|
-
| `http.get`, `http.post`, `json()` | Node `fetch`, see [
|
|
194
|
+
| `http.get`, `http.post`, `json()` | Node `fetch`, see [API](/api-testing) | renamed |
|
|
171
195
|
| `onFlowStart`, `onFlowComplete` | `beforeEach`, `afterEach` | renamed |
|
|
172
196
|
| `startRecording`, `stopRecording` | `test('x', { video: 'on' }, ...)` records that test's whole attempts, on a device the mobile engine records; no start and stop inside a test | renamed |
|
|
173
197
|
|
|
@@ -220,10 +244,10 @@ Start with a flow whose `optional` steps and `runFlow: { when }` branches
|
|
|
220
244
|
often need updating.
|
|
221
245
|
|
|
222
246
|
<CardGroup cols={2}>
|
|
223
|
-
<Card title="
|
|
247
|
+
<Card title="Mobile" icon="mobile-screen" href="/mobile">
|
|
224
248
|
Configure your app and devices.
|
|
225
249
|
</Card>
|
|
226
|
-
<Card title="
|
|
227
|
-
|
|
250
|
+
<Card title="Core concepts" icon="pen-to-square" href="/core-concepts">
|
|
251
|
+
Goals, assertions, and locators instead of scripted selectors.
|
|
228
252
|
</Card>
|
|
229
253
|
</CardGroup>
|
|
@@ -1,8 +1,31 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: From Playwright
|
|
3
3
|
description: Run e2e alongside Playwright and migrate tests one file at a time.
|
|
4
|
+
icon: "/images/icons/playwright.svg"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
8
|
+
|
|
9
|
+
export const migrationPrompt = `Migrate this project's Playwright tests to e2e, one spec at a time. Docs: \`https://e2e.tester.army/docs/migrate/playwright.md\`
|
|
10
|
+
|
|
11
|
+
1. Read the migration guide above. It maps every Playwright config key, API, and CLI flag to e2e and marks what e2e does not have yet.
|
|
12
|
+
2. If the project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent). It writes a config and an example test, and installs nothing yet.
|
|
13
|
+
3. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
14
|
+
4. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
15
|
+
5. Port \`playwright.config.ts\` into the e2e config with the guide's Config tables. \`baseURL\` and \`webServer\` become the target's \`app\`, \`projects\` become \`targets\`, and browser options under \`use\` (viewport, locale, testIdAttribute) become \`web(...)\` options; \`trace\`, \`video\`, and the timeouts are top-level keys. Keep \`@playwright/test\` installed and the two file patterns apart.
|
|
16
|
+
6. Install dependencies.
|
|
17
|
+
7. List the Playwright specs and ask me which one to move first. Suggest a flow whose steps change often, such as a wizard or a checkout.
|
|
18
|
+
8. Rewrite that spec as a \`.e2e.ts\` file under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default). Map each \`page\` call to \`screen\`, \`app\`, or \`browser\` with the guide. String names in e2e queries match the whole text, case-sensitive, so add \`{ exact: false }\` or a RegExp where the Playwright query matched a fragment. Where the guide marks a feature missing, use the workaround it gives, or leave that test in Playwright and tell me why.
|
|
19
|
+
9. Keep navigation and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted interactions breaks often, propose an \`agent.act\` goal for it instead.
|
|
20
|
+
10. Run the new file with \`npx e2e run <file>\` and fix it until it passes. Keep the Playwright spec until I confirm it can go.
|
|
21
|
+
11. Summarize which tests moved, which stayed in Playwright, and why.`;
|
|
22
|
+
|
|
23
|
+
<SetupPrompt>
|
|
24
|
+
|
|
25
|
+
<Prompt description={migrationPrompt} children={migrationPrompt} actions={["copy", "cursor"]} />
|
|
26
|
+
|
|
27
|
+
</SetupPrompt>
|
|
28
|
+
|
|
6
29
|
`@e2e-dev/web` brings its own pinned `playwright-core`, so your
|
|
7
30
|
`@playwright/test` version is yours to keep. Both runners can stay in one
|
|
8
31
|
project. e2e reads `e2e.config.ts` and finds `tests/**/*.e2e.ts` by default,
|
|
@@ -27,7 +50,7 @@ A Playwright key the runner recognizes (`testDir`, `baseURL`, `webServer`,
|
|
|
27
50
|
| Playwright | e2e | Status |
|
|
28
51
|
| --- | --- | --- |
|
|
29
52
|
| `use.baseURL` | `app: { url }` on the target | renamed |
|
|
30
|
-
| `webServer` | `app: { url, command }` on the target, see [Starting your app](/starting-your-app) | renamed |
|
|
53
|
+
| `webServer` | `app: { url, command }` on the target, see [Starting your app](/web#starting-your-app) | renamed |
|
|
31
54
|
| `webServer` array | `app: { url, command }` on each target; targets declaring the same command share one process. Other dependency processes are not started by this version | missing |
|
|
32
55
|
| `webServer.reuseExistingServer` | `app.command: { reuseExisting: true }`; CI ignores it | renamed |
|
|
33
56
|
| `globalSetup`, `globalTeardown` | not yet; `test.setup` for sign-in, hooks and fixtures for setup code | missing |
|
|
@@ -238,7 +261,7 @@ what to do today.
|
|
|
238
261
|
| `page.exposeFunction()`, `addStyleTag()`, `setExtraHTTPHeaders()`, `emulateMedia()`, `clock` | not yet; `browser.evaluate` appends a `<style>` once the page is open, and returns `null` since its result must be JSON | missing |
|
|
239
262
|
| `context.addCookies()`, `context.cookies()` | `browser.setCookies()`, `browser.cookies()` | web only |
|
|
240
263
|
| `context.clearCookies()` | not yet; `app.clearState()` also clears other persisted state and recreates the context | missing |
|
|
241
|
-
| `request` fixture | Node `fetch`, see [
|
|
264
|
+
| `request` fixture | Node `fetch`, see [API](/api-testing) | by design |
|
|
242
265
|
| `context.storageState({ path })` | not yet; sessions are per run and encrypted, see [By design](#by-design) | by design |
|
|
243
266
|
|
|
244
267
|
### Test structure
|
|
@@ -424,8 +447,8 @@ Start with a flow whose interaction sequence changes often, such as a wizard
|
|
|
424
447
|
or checkout; stable Playwright tests can stay where they are.
|
|
425
448
|
|
|
426
449
|
<CardGroup cols={2}>
|
|
427
|
-
<Card title="
|
|
428
|
-
|
|
450
|
+
<Card title="Core concepts" icon="pen-to-square" href="/core-concepts">
|
|
451
|
+
Goals, assertions, and locators instead of scripted selectors.
|
|
429
452
|
</Card>
|
|
430
453
|
<Card title="Web engine reference" icon="masks-theater" href="/reference/web">
|
|
431
454
|
Every option and every `browser` method.
|
|
@@ -1,8 +1,32 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: From Selenium
|
|
3
3
|
description: Rewrite Selenium flows in TypeScript with locators, retrying assertions, and agent goals.
|
|
4
|
+
icon: "/images/icons/selenium.svg"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
8
|
+
|
|
9
|
+
export const migrationPrompt = `Migrate this project's Selenium tests to e2e, one flow at a time. Docs: \`https://e2e.tester.army/docs/migrate/selenium.md\`
|
|
10
|
+
|
|
11
|
+
1. Read the migration guide above. It maps Selenium config and WebDriver calls to e2e.
|
|
12
|
+
2. e2e tests are TypeScript. If the Selenium suite is Java, Python, or another language, ask me where the new test project should live, for example an \`e2e/\` folder with its own \`package.json\`.
|
|
13
|
+
3. If that project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent) there. It writes a config and an example test, and installs nothing yet.
|
|
14
|
+
4. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
15
|
+
5. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
16
|
+
6. Set the target's \`app\` to the URL every \`driver.get\` used, and the command that starts the app if the suite starts it. Move passwords and tokens into \`credentials\` or \`secrets\`.
|
|
17
|
+
7. Install dependencies.
|
|
18
|
+
8. List the page objects and the flows that use them, and ask me which to move first. Suggest one page object and its flows.
|
|
19
|
+
9. Rewrite those flows as \`.e2e.ts\` files under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default). Prefer role, label, and test id queries on \`screen\` over \`By.id\`, CSS, and XPath. Replace explicit waits, implicit waits, and \`Thread.sleep\` with locator assertions, which retry. \`findElement\` takes the first match and e2e fails on several, so narrow those queries. Turn the page object's setup code into a fixture.
|
|
20
|
+
10. Keep navigation and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted interactions breaks often, propose an \`agent.act\` goal for it instead.
|
|
21
|
+
11. Run the new files with \`npx e2e run <file>\` and fix them until they pass. Keep the Selenium tests until I confirm they can go.
|
|
22
|
+
12. Summarize which flows moved, which stayed in Selenium, and why.`;
|
|
23
|
+
|
|
24
|
+
<SetupPrompt>
|
|
25
|
+
|
|
26
|
+
<Prompt description={migrationPrompt} children={migrationPrompt} actions={["copy", "cursor"]} />
|
|
27
|
+
|
|
28
|
+
</SetupPrompt>
|
|
29
|
+
|
|
6
30
|
e2e tests use TypeScript, locators based on roles and labels, and assertions
|
|
7
31
|
that retry until they pass or time out. A Java or Python suite needs to be
|
|
8
32
|
rewritten. Run both suites while you move one flow at a time.
|
|
@@ -15,7 +39,7 @@ agent calls; the locator-based example below works without one.
|
|
|
15
39
|
| Selenium | e2e |
|
|
16
40
|
| --- | --- |
|
|
17
41
|
| The URL in every `driver.get` | `app: { url }` on the target, then relative paths |
|
|
18
|
-
| Starting the app in a script | `app: { url, command }`, see [Starting your app](/starting-your-app) |
|
|
42
|
+
| Starting the app in a script | `app: { url, command }`, see [Starting your app](/web#starting-your-app) |
|
|
19
43
|
| One driver per browser | One target per browser, `web({ browser: 'firefox' })` |
|
|
20
44
|
| Selenium Grid | `workers` for parallelism; `connect` attaches to a remote Chromium over CDP, and a [browser provider](/browser#hosted-browsers) such as [Kernel](/integrations/kernel) leases hosted browsers |
|
|
21
45
|
| `manage().window().setSize` | `web({ viewport })` |
|
|
@@ -101,8 +125,8 @@ Start with one page object and the flows that use it; its setup code can
|
|
|
101
125
|
become a fixture.
|
|
102
126
|
|
|
103
127
|
<CardGroup cols={2}>
|
|
104
|
-
<Card title="
|
|
105
|
-
|
|
128
|
+
<Card title="Core concepts" icon="pen-to-square" href="/core-concepts">
|
|
129
|
+
Goals, assertions, and locators instead of scripted selectors.
|
|
106
130
|
</Card>
|
|
107
131
|
<Card title="screen reference" icon="crosshairs" href="/reference/screen">
|
|
108
132
|
Every query, refinement, action, and read.
|
package/docs/mobile-ci.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Mobile CI
|
|
3
3
|
description: Run iOS and Android tests on GitHub Actions, EAS Workflows, Bitrise, and Codemagic.
|
|
4
|
+
icon: "mobile-screen-button"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
A mobile CI job does the same four things on every service: build the app
|
|
@@ -16,7 +17,7 @@ credentials, sharding, and exit codes; everything there applies here.
|
|
|
16
17
|
| Machine | macOS with Xcode and a simulator runtime | Linux with KVM (nested virtualization) |
|
|
17
18
|
| Build | Release simulator `.app`, no signing | Release `.apk` |
|
|
18
19
|
| Device | A booted simulator | A booted emulator |
|
|
19
|
-
| Node | 22.
|
|
20
|
+
| Node | 24.8 or later, or 22.22.3 or later on 22 | 24.8 or later, or 22.22.3 or later on 22 |
|
|
20
21
|
|
|
21
22
|
Build Release on both platforms: a React Native or Expo debug build loads
|
|
22
23
|
its JavaScript from Metro, which no CI job here starts, so the app opens on
|
|
@@ -115,7 +116,7 @@ simulator; swap in your own build command as long as it writes a simulator
|
|
|
115
116
|
Release --device generic --no-bundler --output build/ios` builds without
|
|
116
117
|
booting a simulator and writes the `.app` under `build/ios`.
|
|
117
118
|
|
|
118
|
-
```yaml title=".github/workflows/e2e-mobile.yml"
|
|
119
|
+
```yaml title=".github/workflows/e2e-mobile.yml" expandable
|
|
119
120
|
name: e2e mobile
|
|
120
121
|
|
|
121
122
|
on:
|
|
@@ -247,7 +248,7 @@ The iOS job runs on a macOS runner; the Android one needs a
|
|
|
247
248
|
`nested-virtualization` Linux runner, the only EAS machines that can run the
|
|
248
249
|
emulator:
|
|
249
250
|
|
|
250
|
-
```yaml title=".eas/workflows/e2e.yml"
|
|
251
|
+
```yaml title=".eas/workflows/e2e.yml" expandable
|
|
251
252
|
name: e2e
|
|
252
253
|
|
|
253
254
|
on:
|
|
@@ -357,7 +358,7 @@ booting both devices: `xcode-start-simulator` on macOS, and `avd-manager`
|
|
|
357
358
|
with `wait-for-android-emulator` on Linux. Start the emulator before the
|
|
358
359
|
Gradle build so it boots while the APK compiles.
|
|
359
360
|
|
|
360
|
-
```yaml title="bitrise.yml"
|
|
361
|
+
```yaml title="bitrise.yml" expandable
|
|
361
362
|
format_version: '25'
|
|
362
363
|
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
|
|
363
364
|
|
|
@@ -482,7 +483,7 @@ workflows:
|
|
|
482
483
|
Android runs on a Linux instance. The Ubuntu image comes with AVDs named `emulator` (API 34),
|
|
483
484
|
`emulator-35`, and `emulator-36`.
|
|
484
485
|
|
|
485
|
-
```yaml title="codemagic.yaml"
|
|
486
|
+
```yaml title="codemagic.yaml" expandable
|
|
486
487
|
workflows:
|
|
487
488
|
e2e-ios:
|
|
488
489
|
name: e2e iOS
|
package/docs/mobile.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
|
-
title:
|
|
3
|
-
description: Set up a simulator or
|
|
2
|
+
title: Mobile
|
|
3
|
+
description: Set up a simulator, emulator, or connected phone as a target, write a device test, and know what differs from the web.
|
|
4
|
+
icon: "mobile-screen"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
`@e2e-dev/mobile` runs tests on iOS simulators, Android emulators, and
|
|
@@ -180,9 +181,79 @@ If none are booted, it boots one. Two workers never drive the same device.
|
|
|
180
181
|
|
|
181
182
|
## Physical devices
|
|
182
183
|
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
184
|
+
agent-device drives a phone plugged into your machine the same way it
|
|
185
|
+
drives a simulator, so a physical device needs no other engine or option.
|
|
186
|
+
List what agent-device sees, then name the phone in `device` as it is
|
|
187
|
+
listed:
|
|
188
|
+
|
|
189
|
+
<CodeGroup>
|
|
190
|
+
```bash npm
|
|
191
|
+
npx agent-device devices --platform ios
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
```bash pnpm
|
|
195
|
+
pnpm exec agent-device devices --platform ios
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
```bash bun
|
|
199
|
+
bunx agent-device devices --platform ios
|
|
200
|
+
```
|
|
201
|
+
</CodeGroup>
|
|
202
|
+
|
|
203
|
+
```ts
|
|
204
|
+
mobile({ platform: 'ios', device: 'QA iPhone' })
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
Name a phone by its name, not its id. The engine reads a `device` string as
|
|
208
|
+
an id only when it is a simulator UDID or an `emulator-<port>` serial, so a
|
|
209
|
+
phone's UDID or adb serial is looked up as a name and not found. The first
|
|
210
|
+
device with a matching name wins, so give each phone a unique name: two
|
|
211
|
+
phones both called `iPhone` or `Pixel 8` would bind one phone to two
|
|
212
|
+
workers. Without `device`, a connected phone counts as booted and joins the
|
|
213
|
+
pool beside any booted simulators, so name it to be sure a run lands on it.
|
|
214
|
+
|
|
215
|
+
Set the phone up once:
|
|
216
|
+
|
|
217
|
+
- **iOS**: the phone is paired and trusted, unlocked, with Developer Mode
|
|
218
|
+
on. agent-device builds and signs its XCTest runner onto the phone, so set
|
|
219
|
+
`AGENT_DEVICE_IOS_TEAM_ID` to your team id and `AGENT_DEVICE_IOS_BUNDLE_ID`
|
|
220
|
+
to a unique reverse-DNS id (`com.example.agentdevice.runner`); the
|
|
221
|
+
signing profile must allow that id and `<id>.uitests`. Set them before the
|
|
222
|
+
first agent-device command, or run `npx agent-device daemon stop` after:
|
|
223
|
+
a running daemon keeps the environment it started with. The first run
|
|
224
|
+
builds the runner and takes longer. `app.appPath` must be a signed device
|
|
225
|
+
build: a simulator `.app` does not install on a phone.
|
|
226
|
+
- **Android**: USB debugging is on and the phone is authorized, listed as
|
|
227
|
+
`device` in `adb devices`. `app.appPath` is the same `.apk`.
|
|
228
|
+
|
|
229
|
+
`npx agent-device doctor` checks the setup, and
|
|
230
|
+
`npx agent-device help physical-device` covers iOS signing in detail.
|
|
231
|
+
|
|
232
|
+
Some device controls need a simulator or emulator, and fail on a phone with
|
|
233
|
+
agent-device's message:
|
|
234
|
+
|
|
235
|
+
- On an iPhone, every device setting is simulator-only: `app.permissions`
|
|
236
|
+
and `device.setPermission`, `app.clearState()`, `device.setNetwork`,
|
|
237
|
+
`device.setAirplaneMode`, `device.setLocation`, `device.clearLocation`,
|
|
238
|
+
`device.setAppearance`, the biometric methods, and `device.clearKeychain`.
|
|
239
|
+
So are `device.clipboard`, `device.setClipboard`, and `device.fold`. For a
|
|
240
|
+
fresh start, reinstall with `device.installApp(path, { reinstall: true })`,
|
|
241
|
+
with `app.bundleId` on the target or `app` in the options, so agent-device
|
|
242
|
+
knows which app to remove.
|
|
243
|
+
- An older iPhone that `xcrun devicectl list devices` does not list is
|
|
244
|
+
driven without CoreDevice: taps, typing, and the screen work, but
|
|
245
|
+
installs, `--video`, `device.openLink`, and `app.launchArguments` fail.
|
|
246
|
+
Install the build from Xcode and pin `app.bundleId`.
|
|
247
|
+
- On an Android phone, `device.setLocation` is emulator-only, and
|
|
248
|
+
`device.setBiometrics('fingerprint', ...)` works only where the phone
|
|
249
|
+
exposes `cmd fingerprint`.
|
|
250
|
+
- `--video` on an iPhone is built from screenshots at about 15 frames a
|
|
251
|
+
second, and drops frames.
|
|
252
|
+
- A phone is not this machine, so `localhost` in the app is the phone
|
|
253
|
+
itself. Point the app at your machine's network address, or on Android
|
|
254
|
+
forward the port with `adb reverse tcp:<port> tcp:<port>`.
|
|
255
|
+
|
|
256
|
+
To run on phones in a device farm, see [Hosted devices](#hosted-devices).
|
|
186
257
|
|
|
187
258
|
## Hosted devices
|
|
188
259
|
|
|
@@ -270,7 +341,7 @@ record leaves `record` out, and the engine records with agent-device.
|
|
|
270
341
|
A provider against a typical session API has three requests: create a
|
|
271
342
|
session, poll it until the daemon is reachable, stop it.
|
|
272
343
|
|
|
273
|
-
```ts title="device-provider.ts"
|
|
344
|
+
```ts title="device-provider.ts" expandable
|
|
274
345
|
import type { DeviceProvider } from '@e2e-dev/mobile';
|
|
275
346
|
|
|
276
347
|
/**
|
|
@@ -350,7 +421,7 @@ A device cloud agent-device speaks to itself needs no session API of your
|
|
|
350
421
|
own: the provider allocates a lease through a daemon it starts for the run and
|
|
351
422
|
hands the worker the lease scope as `client`.
|
|
352
423
|
|
|
353
|
-
```ts title="device-cloud-provider.ts"
|
|
424
|
+
```ts title="device-cloud-provider.ts" expandable
|
|
354
425
|
import { createAgentDeviceClient } from 'agent-device';
|
|
355
426
|
import type { DeviceProvider } from '@e2e-dev/mobile';
|
|
356
427
|
|
|
@@ -606,7 +677,7 @@ xcrun simctl shutdown <udid> && xcrun simctl boot <udid>
|
|
|
606
677
|
<Card title="Mobile engine reference" icon="mobile-screen" href="/reference/mobile">
|
|
607
678
|
Every option, the `device` fixture, and the tool pack.
|
|
608
679
|
</Card>
|
|
609
|
-
<Card title="Locators" icon="crosshairs" href="/locators">
|
|
680
|
+
<Card title="Locators" icon="crosshairs" href="/core-concepts#locators">
|
|
610
681
|
Exact taps, swipes, and checks with `screen` and `expect`, on any platform.
|
|
611
682
|
</Card>
|
|
612
683
|
</CardGroup>
|
package/docs/models.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Models
|
|
3
3
|
description: Configure a model with your own subscription, an API key, or a local server.
|
|
4
|
+
icon: "brain"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
import { ModelProvider } from './snippets/model-provider.jsx';
|
|
@@ -12,8 +13,9 @@ e2e uses the [AI SDK](https://ai-sdk.dev/providers). You can use
|
|
|
12
13
|
`e2e init` offers these choices and writes the config for you. For the first
|
|
13
14
|
setup, follow the [quickstart](/quickstart#use-a-subscription-or-api-key).
|
|
14
15
|
|
|
15
|
-
Tests without agent steps do not need a model.
|
|
16
|
-
|
|
16
|
+
Tests without agent steps do not need a model. [Decision models](/decision-models), hosted or locally
|
|
17
|
+
hosted, such as Clef or Jev, can drive bounded actions and judge assertions through a custom executor. For the built-in agent,
|
|
18
|
+
choose a model that supports tool calls and images. Steps using the built-in agent need the `ai`
|
|
17
19
|
package installed, whichever provider you use:
|
|
18
20
|
|
|
19
21
|
<CodeGroup>
|
|
@@ -232,6 +234,9 @@ Judgments use the current screen and the question, without the acting
|
|
|
232
234
|
agent's transcript or summaries. Reports include the model used for each
|
|
233
235
|
judgment. See [Custom executors](/executors) if you replace the built-in agent.
|
|
234
236
|
|
|
237
|
+
`judge` takes an AI SDK language model. Clef and Jev use a different API;
|
|
238
|
+
use the [decision executor](/decision-models) for `agent.act` and `agent.assert`.
|
|
239
|
+
|
|
235
240
|
## Data retention and compliance
|
|
236
241
|
|
|
237
242
|
Model input is the redacted screen, the step instruction, and the
|
package/docs/quickstart.mdx
CHANGED
|
@@ -1,15 +1,17 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Quickstart
|
|
3
3
|
description: Set up e2e with one prompt to your coding agent, or step by step.
|
|
4
|
+
icon: "rocket"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { ModelProvider } from './snippets/model-provider.jsx';
|
|
6
8
|
import { SetupPrompt } from './snippets/setup-prompt-card.jsx';
|
|
7
9
|
|
|
8
10
|
export const setupPrompt = `Set up e2e end-to-end tests in this project. Docs: \`https://e2e.tester.army/docs/quickstart.md\`
|
|
9
11
|
|
|
10
12
|
1. Run \`npx e2e init --yes\` (or the pnpm or bun equivalent for this project). It writes a web config using Vercel AI Gateway, an example test, the e2e skill, and MCP config. It installs nothing yet.
|
|
11
13
|
2. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
12
|
-
3. Point the target at this app. For a web app, set its URL and the command that starts its dev server. For an iOS or Android app, replace @e2e-dev/web
|
|
14
|
+
3. Point the target at this app. For a web app, set its URL and the command that starts its dev server. For an iOS or Android app, replace @e2e-dev/web with @e2e-dev/mobile and rewrite the example test for the mobile engine.
|
|
13
15
|
4. Ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
14
16
|
5. Install dependencies, run the example test, and fix setup errors until it passes.
|
|
15
17
|
6. Write one test for the most important user flow in this app, run it, and iterate until it passes.`;
|
|
@@ -29,7 +31,7 @@ The agent installs the [e2e skill](/coding-agents), so later prompts like
|
|
|
29
31
|
"add a test for checkout" or "why did this run fail" work without more setup.
|
|
30
32
|
|
|
31
33
|
<Note>
|
|
32
|
-
Use Node.js 22.
|
|
34
|
+
Use Node.js 24.8 or newer (22.22.3 or newer on Node.js 22). On Windows, run inside WSL. Mobile tests also
|
|
33
35
|
need Xcode with an iOS simulator runtime, or the Android SDK with an emulator.
|
|
34
36
|
</Note>
|
|
35
37
|
|
|
@@ -63,7 +65,7 @@ need Xcode with an iOS simulator runtime, or the Android SDK with an emulator.
|
|
|
63
65
|
<Tabs>
|
|
64
66
|
<Tab title="Web">
|
|
65
67
|
Start your app's dev server at `http://localhost:3000`, or set its address
|
|
66
|
-
in `e2e.config.ts`. You can also [let e2e start it](/starting-your-app).
|
|
68
|
+
in `e2e.config.ts`. You can also [let e2e start it](/web#starting-your-app).
|
|
67
69
|
|
|
68
70
|
<Accordion title="Example web config and test">
|
|
69
71
|
This config uses Vercel AI Gateway. The wizard configures the provider you chose.
|
|
@@ -122,7 +124,7 @@ test('Settings opens', async ({ app, screen }) => {
|
|
|
122
124
|
```
|
|
123
125
|
</Accordion>
|
|
124
126
|
|
|
125
|
-
See [
|
|
127
|
+
See [Mobile](/mobile) to use your own app.
|
|
126
128
|
</Tab>
|
|
127
129
|
</Tabs>
|
|
128
130
|
</Step>
|
|
@@ -166,17 +168,26 @@ Connect the provider you chose in the wizard.
|
|
|
166
168
|
[Subscriptions](/subscriptions) for details.
|
|
167
169
|
</Tab>
|
|
168
170
|
<Tab title="API key">
|
|
169
|
-
|
|
170
|
-
|
|
171
|
+
Select your provider. Install its package and set the variables it reads,
|
|
172
|
+
if any, in the terminal where you run e2e.
|
|
171
173
|
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
174
|
+
<ModelProvider show="select" />
|
|
175
|
+
|
|
176
|
+
<ModelProvider show="install" />
|
|
177
|
+
|
|
178
|
+
<ModelProvider show="env" />
|
|
179
|
+
|
|
180
|
+
Vercel AI Gateway reads `AI_GATEWAY_API_KEY`, or a
|
|
181
|
+
[Vercel OIDC token](/models#vercel-oidc-instead-of-a-key) in a project
|
|
182
|
+
linked with `vercel link`. OpenRouter reads `OPENROUTER_API_KEY`.
|
|
183
|
+
|
|
184
|
+
The wizard writes the config for Vercel AI Gateway, OpenRouter, and
|
|
185
|
+
OpenAI-compatible endpoints. For another provider, replace the model import
|
|
186
|
+
and `model` value in `e2e.config.ts`, and use `agent` as `agents.default`:
|
|
187
|
+
|
|
188
|
+
<ModelProvider show="model" />
|
|
175
189
|
|
|
176
|
-
|
|
177
|
-
[Vercel OIDC token](/models#vercel-oidc-instead-of-a-key) instead.
|
|
178
|
-
OpenRouter uses `OPENROUTER_API_KEY`. See [Models](/models) for other
|
|
179
|
-
providers and local models.
|
|
190
|
+
See [Models](/models) for local models and more options.
|
|
180
191
|
</Tab>
|
|
181
192
|
</Tabs>
|
|
182
193
|
|
|
@@ -246,8 +257,8 @@ to watch a web test, `--no-cache` to skip replaying cached actions.
|
|
|
246
257
|
## Next
|
|
247
258
|
|
|
248
259
|
<CardGroup cols={2}>
|
|
249
|
-
<Card title="
|
|
250
|
-
|
|
260
|
+
<Card title="Core concepts" icon="pen-to-square" href="/core-concepts">
|
|
261
|
+
Goals, assertions, and locators.
|
|
251
262
|
</Card>
|
|
252
263
|
<Card title="Signing in" icon="key" href="/authentication">
|
|
253
264
|
Reuse an app login and keep passwords out of model input.
|
package/docs/reference/agent.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: agent
|
|
3
3
|
description: Run agent actions, check conditions, and extract structured data.
|
|
4
|
+
icon: "robot"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
Use `act` to interact with the app, `assert` to check the current screen,
|
|
@@ -191,7 +192,9 @@ Every `act` observation with unavailable semantics includes that image and
|
|
|
191
192
|
warning automatically. Earlier node ids are invalid, and the next successful
|
|
192
193
|
semantic capture presents the whole tree again. The screenshot shows the
|
|
193
194
|
current viewport; an unavailable tree does not prove that a control or
|
|
194
|
-
message is absent.
|
|
195
|
+
message is absent. After a secret fill there is no fallback: `act`, and a
|
|
196
|
+
judgment with `vision: true`, fails with `POLICY_DENIED` when semantic
|
|
197
|
+
capture times out before the step does.
|
|
195
198
|
|
|
196
199
|
If semantic capture fails, trace replay stops and the step cannot record a
|
|
197
200
|
new trace entry, even if the tree later recovers.
|
|
@@ -238,7 +241,12 @@ assert(assertion: string, options?: AssertOptions): Promise<void>;
|
|
|
238
241
|
|
|
239
242
|
Checks one fresh observation without polling. The judge sees only the
|
|
240
243
|
assertion and current observation, never prior steps or `act` summaries.
|
|
241
|
-
It answers `holds`, `fails`, or `inconclusive`.
|
|
244
|
+
It answers `holds`, `fails`, or `inconclusive`. When what the assertion is
|
|
245
|
+
about appears more than once, such as a total in a summary and on a pay
|
|
246
|
+
button, every instance must agree: one that contradicts it fails. Name
|
|
247
|
+
the place (`the order summary total is $42.00`) to judge only that one. A
|
|
248
|
+
claim about some item, such as `a todo is marked done`, still holds when one
|
|
249
|
+
item matches.
|
|
242
250
|
|
|
243
251
|
Throws `ASSERTION_FAILED` on a false judgment and `ASSERTION_INCONCLUSIVE`
|
|
244
252
|
when the screen did not show enough to decide. Both carry the explanation
|
|
@@ -271,7 +279,9 @@ Observes and judges at once. After a false or inconclusive judgment it
|
|
|
271
279
|
waits out `interval`, then observes every 500 ms (or every `interval`, when
|
|
272
280
|
shorter) without a model call, and judges again only once the screen
|
|
273
281
|
differs from the one last judged, pixels included under `vision`. Succeeds
|
|
274
|
-
on the first judgment that holds.
|
|
282
|
+
on the first judgment that holds. Judgments follow the `agent.assert`
|
|
283
|
+
rules, so a value shown in more than one place holds only once every
|
|
284
|
+
instance agrees.
|
|
275
285
|
|
|
276
286
|
Throws `STEP_TIMEOUT` on deadline expiry and `STEP_BUDGET_EXHAUSTED` on
|
|
277
287
|
budget exhaustion, each carrying the last judgment (with the same vision
|
package/docs/reference/app.mdx
CHANGED
package/docs/reference/cli.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: CLI
|
|
3
3
|
description: Commands, flags, output formats, and exit codes.
|
|
4
|
+
icon: "terminal"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
<CodeGroup>
|
|
@@ -56,7 +57,7 @@ bunx e2e --version
|
|
|
56
57
|
```
|
|
57
58
|
</CodeGroup>
|
|
58
59
|
|
|
59
|
-
The CLI needs Node.js 22.
|
|
60
|
+
The CLI needs Node.js 24.8 or newer, or 22.22.3 or newer on Node.js 22. Any package manager installs it (npm, pnpm, Yarn with or without Plug'n'Play, Bun); the CLI itself runs on Node.js, not on the Bun or Deno runtimes (`bunx e2e` runs it on Node.js). Under Yarn Plug'n'Play on Node.js 24.8 through 24.11.0, a `.cts` file fails to load (Node.js rejects the source Yarn's loader hands on); 24.11.1 and later load it.
|
|
60
61
|
|
|
61
62
|
## Help and version
|
|
62
63
|
|
|
@@ -140,7 +141,10 @@ bunx e2e init --yes
|
|
|
140
141
|
```
|
|
141
142
|
</CodeGroup>
|
|
142
143
|
|
|
143
|
-
Re-running `init` keeps an existing config and adds no packages for it.
|
|
144
|
+
Re-running `init` keeps an existing config and adds no packages for it. MCP
|
|
145
|
+
registration keeps extra settings on the e2e server, such as environment
|
|
146
|
+
variables and client timeouts, even when repairing its launch command. A
|
|
147
|
+
current registration is left byte-for-byte unchanged. Once
|
|
144
148
|
a skill directory exists, later runs refresh copies that differ from the
|
|
145
149
|
installed version and never add a directory the project did not choose. A
|
|
146
150
|
`.claude/skills/e2e/` copy from an earlier version becomes the symlink when
|
|
@@ -281,15 +285,15 @@ See [Coding agents](/coding-agents).
|
|
|
281
285
|
|
|
282
286
|
<CodeGroup>
|
|
283
287
|
```bash npm
|
|
284
|
-
npx e2e mcp [--config <path>] [--target <name>] [--
|
|
288
|
+
npx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
|
|
285
289
|
```
|
|
286
290
|
|
|
287
291
|
```bash pnpm
|
|
288
|
-
pnpm exec e2e mcp [--config <path>] [--target <name>] [--
|
|
292
|
+
pnpm exec e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
|
|
289
293
|
```
|
|
290
294
|
|
|
291
295
|
```bash bun
|
|
292
|
-
bunx e2e mcp [--config <path>] [--target <name>] [--
|
|
296
|
+
bunx e2e mcp [--config <path>] [--target <name>] [--headed] [--max-sessions <n>]
|
|
293
297
|
```
|
|
294
298
|
</CodeGroup>
|
|
295
299
|
|
|
@@ -346,6 +350,8 @@ A positional starting with `-` that names no file is a usage error, exit 2.
|
|
|
346
350
|
This is what `pnpm test:e2e -- --headed` produces: the `--` reaches e2e and
|
|
347
351
|
`--headed` would otherwise be taken for a file. The message names the direct
|
|
348
352
|
form, `pnpm exec e2e run --headed`.
|
|
353
|
+
For npm, the direct form is `npm exec -- e2e run --headed`; npm needs the
|
|
354
|
+
separator before the binary to pass its flags through.
|
|
349
355
|
|
|
350
356
|
When nothing is left to run, `NO_TESTS` names the cause: no file matched the
|
|
351
357
|
globs (with look-alike files such as `tests/login.test.ts`), a positional
|