e2e 0.16.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -0
- package/dist/.build.json +1 -1
- package/dist/agent/act.js +7 -4
- package/dist/agent/action-dispatcher.js +4 -5
- package/dist/agent/actions.d.ts +32 -23
- package/dist/agent/actions.js +38 -24
- package/dist/agent/ai-sdk.js +5 -4
- package/dist/agent/error.js +3 -3
- package/dist/agent/invocation.js +3 -4
- package/dist/agent/model/sdk.js +2 -0
- package/dist/agent/model/tool-choice.d.ts +9 -0
- package/dist/agent/model/tool-choice.js +14 -0
- package/dist/agent/observation-feed.d.ts +5 -6
- package/dist/agent/observation-feed.js +8 -11
- package/dist/agent/observation.d.ts +40 -11
- package/dist/agent/observation.js +109 -59
- package/dist/agent/phases.d.ts +7 -1
- package/dist/agent/phases.js +12 -3
- package/dist/agent/point-tap.d.ts +8 -8
- package/dist/agent/point-tap.js +1 -1
- package/dist/agent/prompts.js +3 -0
- package/dist/agent/replay.d.ts +19 -8
- package/dist/agent/replay.js +29 -29
- package/dist/agent/scroll-target.d.ts +2 -3
- package/dist/agent/step-cache.d.ts +43 -15
- package/dist/agent/step-cache.js +156 -67
- package/dist/agent/tool-loop.js +94 -5
- package/dist/cache/anchors.d.ts +72 -23
- package/dist/cache/anchors.js +233 -60
- package/dist/cache/context.d.ts +25 -14
- package/dist/cache/context.js +50 -8
- package/dist/cache/decide.d.ts +2 -2
- package/dist/cache/decide.js +4 -6
- package/dist/cache/identity.d.ts +18 -4
- package/dist/cache/identity.js +9 -4
- package/dist/cache/recorder.d.ts +15 -1
- package/dist/cache/recorder.js +36 -23
- package/dist/cache/rekeyed.d.ts +46 -0
- package/dist/cache/rekeyed.js +89 -0
- package/dist/cache/relocate.d.ts +14 -14
- package/dist/cache/relocate.js +33 -25
- package/dist/cache/route.d.ts +32 -33
- package/dist/cache/route.js +94 -50
- package/dist/cache/store.d.ts +6 -0
- package/dist/cache/store.js +21 -1
- package/dist/cache/trace.d.ts +53 -14
- package/dist/cache/trace.js +76 -30
- package/dist/cli/bin.js +2 -2
- package/dist/cli/index.js +43 -32
- package/dist/cli/init/engines.d.ts +1 -3
- package/dist/cli/init/engines.js +2 -2
- package/dist/cli/init/gateways.d.ts +1 -1
- package/dist/cli/init/gateways.js +11 -1
- package/dist/cli/init/mcp-config.d.ts +2 -1
- package/dist/cli/init/mcp-config.js +8 -3
- package/dist/cli/init/sibling-versions.json +1 -1
- package/dist/cli/init/versions.d.ts +0 -8
- package/dist/cli/init/versions.js +1 -10
- package/dist/cli/mcp.d.ts +3 -2
- package/dist/cli/mcp.js +52 -16
- package/dist/cli/run-output.d.ts +17 -0
- package/dist/cli/run-output.js +45 -0
- package/dist/collect/collect.js +8 -4
- package/dist/collect/registry.js +27 -16
- package/dist/config/compiled-files.d.ts +36 -0
- package/dist/config/compiled-files.js +50 -0
- package/dist/config/diagnose.js +21 -19
- package/dist/config/esm-hooks.d.ts +39 -44
- package/dist/config/esm-hooks.js +238 -63
- package/dist/config/load.d.ts +1 -7
- package/dist/config/load.js +23 -66
- package/dist/config/register.d.ts +8 -0
- package/dist/config/register.js +9 -0
- package/dist/config/resolve.d.ts +1 -0
- package/dist/config/resolve.js +33 -1
- package/dist/config/specifiers.d.ts +19 -0
- package/dist/config/specifiers.js +33 -0
- package/dist/config/tsconfig.d.ts +56 -0
- package/dist/config/tsconfig.js +115 -0
- package/dist/config/typescript.d.ts +19 -0
- package/dist/config/typescript.js +253 -0
- package/dist/engine/contract.d.ts +1 -0
- package/dist/engine/index.d.ts +4 -2
- package/dist/engine/session.js +2 -2
- package/dist/expect/async.js +4 -2
- package/dist/expect/poll-scope.d.ts +52 -0
- package/dist/expect/poll-scope.js +113 -0
- package/dist/expect/poll.js +33 -6
- package/dist/expect/values.js +11 -4
- package/dist/internal/abandoned.d.ts +11 -0
- package/dist/internal/abandoned.js +23 -0
- package/dist/internal/brands.d.ts +2 -0
- package/dist/internal/brands.js +2 -0
- package/dist/internal/client-identity.d.ts +9 -0
- package/dist/internal/client-identity.js +14 -0
- package/dist/internal/errors.d.ts +10 -3
- package/dist/internal/errors.js +23 -8
- package/dist/internal/node-version.d.ts +16 -7
- package/dist/internal/node-version.js +50 -15
- package/dist/internal/package-manager.js +1 -1
- package/dist/internal/realm-slot.d.ts +5 -4
- package/dist/internal/redact.d.ts +9 -4
- package/dist/internal/redact.js +120 -23
- package/dist/internal/time.d.ts +26 -4
- package/dist/internal/time.js +58 -14
- package/dist/internal/urls.d.ts +9 -5
- package/dist/internal/urls.js +19 -10
- package/dist/locator/engine.d.ts +12 -2
- package/dist/locator/engine.js +55 -14
- package/dist/locator/screen.js +59 -9
- package/dist/mcp/catalog.d.ts +3 -2
- package/dist/mcp/catalog.js +6 -6
- package/dist/mcp/output.d.ts +59 -0
- package/dist/mcp/output.js +129 -0
- package/dist/mcp/server.d.ts +4 -0
- package/dist/mcp/server.js +9 -12
- package/dist/mcp/session.d.ts +8 -1
- package/dist/mcp/session.js +56 -15
- package/dist/mcp/tools.js +7 -5
- package/dist/oauth/chatgpt.js +3 -23
- package/dist/oauth/cli.js +1 -0
- package/dist/oauth/copilot.d.ts +5 -4
- package/dist/oauth/copilot.js +80 -7
- package/dist/oauth/grok.js +2 -1
- package/dist/oauth/models.js +2 -1
- package/dist/oauth/opencode-console.d.ts +10 -0
- package/dist/oauth/opencode-console.js +133 -0
- package/dist/oauth/providers/github-copilot.d.ts +12 -0
- package/dist/oauth/providers/github-copilot.js +77 -23
- package/dist/oauth/providers/opencode-console.d.ts +31 -0
- package/dist/oauth/providers/opencode-console.js +148 -0
- package/dist/oauth/providers.d.ts +2 -6
- package/dist/oauth/providers.js +3 -7
- package/dist/oauth/responses.d.ts +9 -0
- package/dist/oauth/responses.js +23 -0
- package/dist/report/build.d.ts +26 -0
- package/dist/report/build.js +16 -8
- package/dist/report/failure-text.d.ts +6 -1
- package/dist/report/failure-text.js +14 -4
- package/dist/report/format.d.ts +10 -5
- package/dist/report/format.js +12 -5
- package/dist/report/junit.js +11 -12
- package/dist/report/list.d.ts +12 -0
- package/dist/report/list.js +61 -24
- package/dist/report/markdown.d.ts +1 -1
- package/dist/report/markdown.js +37 -14
- package/dist/report/repeats.d.ts +6 -1
- package/dist/report/repeats.js +12 -3
- package/dist/run/artifacts.d.ts +22 -4
- package/dist/run/artifacts.js +71 -3
- package/dist/run/execute.d.ts +34 -5
- package/dist/run/execute.js +93 -44
- package/dist/run/extended-fixtures.js +6 -2
- package/dist/run/failure-evidence.js +10 -4
- package/dist/run/fixtures.js +6 -8
- package/dist/run/in-process.d.ts +2 -0
- package/dist/run/in-process.js +52 -10
- package/dist/run/last-run.d.ts +42 -8
- package/dist/run/last-run.js +121 -14
- package/dist/run/process-output.d.ts +73 -0
- package/dist/run/process-output.js +130 -0
- package/dist/run/provision.d.ts +2 -0
- package/dist/run/provision.js +1 -0
- package/dist/run/realm.d.ts +4 -1
- package/dist/run/realm.js +20 -6
- package/dist/run/records.d.ts +20 -4
- package/dist/run/records.js +26 -0
- package/dist/run/retry.d.ts +10 -1
- package/dist/run/retry.js +30 -12
- package/dist/run/runner.d.ts +9 -1
- package/dist/run/runner.js +101 -39
- package/dist/run/scheduler.js +255 -19
- package/dist/run/secrecy.d.ts +22 -19
- package/dist/run/secrecy.js +22 -18
- package/dist/run/serial.d.ts +75 -2
- package/dist/run/serial.js +243 -59
- package/dist/run/standalone.js +1 -1
- package/dist/run/steps.d.ts +0 -6
- package/dist/run/steps.js +3 -20
- package/dist/run/trace-redaction.d.ts +5 -6
- package/dist/run/trace-redaction.js +5 -6
- package/dist/run/worker/entry.js +40 -18
- package/dist/run/worker/handle.js +10 -5
- package/dist/run/worker/protocol.d.ts +85 -3
- package/dist/run/worker/session.d.ts +2 -0
- package/dist/run/worker/session.js +29 -0
- package/dist/telemetry/environment.d.ts +2 -1
- package/dist/telemetry/environment.js +12 -2
- package/dist/telemetry/events.js +1 -0
- package/dist/telemetry/project.d.ts +5 -6
- package/dist/telemetry/project.js +6 -11
- package/dist/telemetry/store.d.ts +5 -5
- package/dist/telemetry/store.js +5 -7
- package/dist/telemetry/telemetry.d.ts +2 -3
- package/dist/telemetry/telemetry.js +2 -4
- package/dist/types.d.ts +15 -6
- package/docs/agent-steps.mdx +1 -0
- package/docs/agents.mdx +7 -3
- package/docs/api-testing.mdx +2 -1
- package/docs/authentication.mdx +6 -3
- package/docs/browser.mdx +49 -12
- package/docs/bug-bash.mdx +2 -1
- package/docs/cache.mdx +66 -19
- package/docs/ci.mdx +14 -9
- package/docs/coding-agents.mdx +2 -1
- package/docs/core-concepts.mdx +208 -0
- package/docs/debugging.mdx +25 -12
- package/docs/decision-models.mdx +142 -0
- package/docs/example-projects.mdx +32 -0
- package/docs/executors.mdx +7 -2
- package/docs/explore.mdx +1 -0
- package/docs/github.mdx +14 -5
- package/docs/index.mdx +9 -5
- package/docs/integrations/eas.mdx +1 -0
- package/docs/integrations/index.mdx +1 -0
- package/docs/integrations/kernel.mdx +3 -2
- package/docs/migrate/cypress.mdx +26 -3
- package/docs/migrate/detox.mdx +24 -1
- package/docs/migrate/maestro.mdx +28 -4
- package/docs/migrate/playwright.mdx +44 -18
- package/docs/migrate/selenium.mdx +27 -3
- package/docs/mobile-ci.mdx +6 -5
- package/docs/mobile.mdx +96 -8
- package/docs/models.mdx +60 -9
- package/docs/quickstart.mdx +28 -16
- package/docs/reference/agent.mdx +13 -3
- package/docs/reference/app.mdx +2 -1
- package/docs/reference/cli.mdx +41 -18
- package/docs/reference/config.mdx +67 -13
- package/docs/reference/engine.mdx +25 -13
- package/docs/reference/environment.mdx +10 -2
- package/docs/reference/errors.mdx +13 -10
- package/docs/reference/expect.mdx +36 -13
- package/docs/reference/mcp.mdx +21 -8
- package/docs/reference/mobile.mdx +24 -13
- package/docs/reference/reporters.mdx +9 -2
- package/docs/reference/screen.mdx +49 -16
- package/docs/reference/test.mdx +32 -5
- package/docs/reference/web.mdx +133 -23
- package/docs/security.mdx +58 -42
- package/docs/subscriptions.mdx +92 -12
- package/docs/telemetry.mdx +9 -6
- package/docs/tools.mdx +1 -0
- package/docs/{starting-your-app.mdx → web.mdx} +157 -11
- package/docs/writing-an-engine.mdx +2 -1
- package/package.json +25 -8
- package/schema/report-v1.schema.json +14 -1
- package/skills/e2e/SKILL.md +5 -3
- package/skills/e2e/references/agent.md +33 -11
- package/skills/e2e/references/debugging.md +4 -4
- package/skills/e2e/references/mcp.md +8 -3
- package/skills/e2e/references/running.md +15 -9
- package/skills/e2e/references/setup.md +45 -16
- package/skills/e2e/references/writing-tests.md +35 -18
- package/docs/viewports.mdx +0 -146
- package/docs/writing-tests.mdx +0 -256
package/docs/debugging.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Debugging a run
|
|
3
3
|
description: Read the report, open the evidence, watch the browser, and see what the model was sent.
|
|
4
|
+
icon: "magnifying-glass"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
Start with the failure in the terminal. Every run also writes
|
|
@@ -63,7 +64,9 @@ URL: `http://127.0.0.1:3000/todos`
|
|
|
63
64
|
|
|
64
65
|
`Look at` points to the failing call. Assertion failures include expected
|
|
65
66
|
and observed values; locator failures include the requested locator and
|
|
66
|
-
nearby matches. Repeated failure details help you compare attempts.
|
|
67
|
+
nearby matches. Repeated failure details help you compare attempts. The
|
|
68
|
+
runner spends at most 5 s capturing the screen and screenshot once per failed
|
|
69
|
+
attempt; an app that does not answer in time gets no screen section.
|
|
67
70
|
|
|
68
71
|
## The report
|
|
69
72
|
|
|
@@ -91,7 +94,9 @@ failed result:
|
|
|
91
94
|
|
|
92
95
|
Artifacts land under `.e2e/artifacts/` (`<output>/artifacts/` when the
|
|
93
96
|
[`output`](/reference/config#output) is moved), one directory per attempt.
|
|
94
|
-
Each run empties it first, so it holds the latest run's evidence
|
|
97
|
+
Each run empties it first, so it holds the latest run's evidence. A
|
|
98
|
+
`--last-failed` rerun instead empties it down to the files the report it
|
|
99
|
+
reruns names, and writes its own under `rerun-<n>/`:
|
|
95
100
|
|
|
96
101
|
- **Screenshots.** Failed attempts capture one whenever the engine can and
|
|
97
102
|
pixels are allowed. The built-in `agent.assert` also captures one after
|
|
@@ -99,12 +104,13 @@ Each run empties it first, so it holds the latest run's evidence:
|
|
|
99
104
|
be withheld. Use `app.screenshot('label')` to capture one explicitly.
|
|
100
105
|
- **Playwright traces** on the attempts the [`trace`](/reference/config#trace)
|
|
101
106
|
mode records: every attempt locally, the first retry in CI. Open one with
|
|
102
|
-
`npx playwright show-trace <file
|
|
103
|
-
run, `--trace` records every attempt
|
|
107
|
+
`npx playwright-core@1.63.0 show-trace <file>` (`pnpm dlx` under pnpm);
|
|
108
|
+
`--trace off` skips the cost for one run, `--trace` records every attempt
|
|
109
|
+
of a CI run.
|
|
104
110
|
- **Downloads** saved through `browser.waitForDownload`, recorded with kind
|
|
105
|
-
`download`. They are the app's bytes: `redaction: "incomplete"` unless
|
|
106
|
-
secret
|
|
107
|
-
|
|
111
|
+
`download`. They are the app's bytes: `redaction: "incomplete"` unless the
|
|
112
|
+
session knows a secret value and the file is text, which the runner rewrites and
|
|
113
|
+
marks `complete`.
|
|
108
114
|
- **Videos** and **agent transcripts** when you ask for them; see below.
|
|
109
115
|
|
|
110
116
|
After a secret fill, no screenshots are taken for the rest of the attempt; see
|
|
@@ -193,9 +199,16 @@ stdout | web tests/todos.e2e.ts > adds a todo
|
|
|
193
199
|
hello from the test
|
|
194
200
|
```
|
|
195
201
|
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
202
|
+
Output from outside a test, such as a test file's top level or a reporter,
|
|
203
|
+
prints under `stdout | runner` or `stderr | runner`. Both streams print on
|
|
204
|
+
stdout, in a pipe or a CI log too. Without the `list` reporter, output
|
|
205
|
+
goes to its own stream as written.
|
|
206
|
+
|
|
207
|
+
Secret values are [redacted](/security#redaction). Output printed while the
|
|
208
|
+
config loads appears once it has loaded, before the run header, and is
|
|
209
|
+
dropped for a warning if the load fails. Every other line appears when it
|
|
210
|
+
is written. The output is not part of the report; use `expect` or an
|
|
211
|
+
artifact for anything a later reader needs.
|
|
199
212
|
|
|
200
213
|
## Agent steps
|
|
201
214
|
|
|
@@ -261,13 +274,13 @@ large raw JSON file.
|
|
|
261
274
|
|
|
262
275
|
Replayed steps make no model call and leave no AI trace. Add `--no-cache`
|
|
263
276
|
when you need to inspect the whole flow or rule out a stale recording. See
|
|
264
|
-
[Caching
|
|
277
|
+
[Caching](/cache).
|
|
265
278
|
|
|
266
279
|
## Common errors
|
|
267
280
|
|
|
268
281
|
| Code | Usual cause | Fix |
|
|
269
282
|
| --- | --- | --- |
|
|
270
|
-
| `APP_UNREACHABLE` | Nothing answers at `app.url`, or `app.command` never became ready; on a device, the message may name the iOS automation runner instead (busy, wedged, or a snapshot it could not present) | Set `app.command.log` and read it; check the port; raise `app.command.startupTimeout`. For the runner, follow the recovery in the message: `npx agent-device daemon stop`, reboot the simulator (see [
|
|
283
|
+
| `APP_UNREACHABLE` | Nothing answers at `app.url`, or `app.command` never became ready; on a device, the message may name the iOS automation runner instead (busy, wedged, or a snapshot it could not present) | Set `app.command.log` and read it; check the port; raise `app.command.startupTimeout`. For the runner, follow the recovery in the message: `npx agent-device daemon stop`, reboot the simulator (see [Mobile](/mobile#troubleshooting)) |
|
|
271
284
|
| `APP_ALREADY_RUNNING` | Something already serves `app.url` when the runner wanted to start `app.command` | Stop it, or set `reuseExisting: true` for local runs |
|
|
272
285
|
| `LOCATOR_NOT_FOUND` | Wrong role or name, text not exact, element inside an iframe, page not open | Read the markup for the accessible name; try `exact: false`; `browser.frameLocator` for iframes; `--headed` to look |
|
|
273
286
|
| `LOCATOR_AMBIGUOUS` | Two matches: a hidden duplicate, a repeated label | Add `{ name }`, scope under a container, `filter`, `first()`, or `{ visible: true }` |
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Decision models
|
|
3
|
+
description: Run bounded actions with a decision model and a small text model instead of a full LLM agent.
|
|
4
|
+
icon: "scale-balanced"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
`@e2e-dev/decision` runs `agent.act` and `agent.assert` through a decision
|
|
8
|
+
model instead of an LLM. For each action the decision model answers one
|
|
9
|
+
request: which operation, and which element for each operation that has
|
|
10
|
+
more than one, chosen from what the executor builds out of the latest
|
|
11
|
+
semantic tree. It never generates free text: when the choice is `type`, a
|
|
12
|
+
small language model writes the field value. Both models are
|
|
13
|
+
plain AI SDK models, so any decision model provider that answers `choice`
|
|
14
|
+
questions with probability distributions (Jev) and any language-model
|
|
15
|
+
provider works; the runner still authorizes every dispatched action.
|
|
16
|
+
|
|
17
|
+
## Install and configure
|
|
18
|
+
|
|
19
|
+
<CodeGroup>
|
|
20
|
+
```bash npm
|
|
21
|
+
npm install -D @e2e-dev/decision
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
```bash pnpm
|
|
25
|
+
pnpm add -D @e2e-dev/decision
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
```bash bun
|
|
29
|
+
bun add -d @e2e-dev/decision
|
|
30
|
+
```
|
|
31
|
+
</CodeGroup>
|
|
32
|
+
|
|
33
|
+
The decision model answers `choice` questions with probability
|
|
34
|
+
distributions; the text model writes field values. Each installs as its
|
|
35
|
+
own package. Decision models need `ai` 7, version 7.0.128 or later:
|
|
36
|
+
|
|
37
|
+
<CodeGroup>
|
|
38
|
+
```bash npm
|
|
39
|
+
npm install -D @ai-sdk/typesafe-ai @openrouter/ai-sdk-provider
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
```bash pnpm
|
|
43
|
+
pnpm add -D @ai-sdk/typesafe-ai @openrouter/ai-sdk-provider
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
```bash bun
|
|
47
|
+
bun add -d @ai-sdk/typesafe-ai @openrouter/ai-sdk-provider
|
|
48
|
+
```
|
|
49
|
+
</CodeGroup>
|
|
50
|
+
|
|
51
|
+
```ts title="e2e.config.ts"
|
|
52
|
+
import type { E2EConfig } from 'e2e';
|
|
53
|
+
import { web } from '@e2e-dev/web';
|
|
54
|
+
import { decisionExecutor } from '@e2e-dev/decision';
|
|
55
|
+
import { typeSafeAi } from '@ai-sdk/typesafe-ai';
|
|
56
|
+
import { openrouter } from '@openrouter/ai-sdk-provider';
|
|
57
|
+
|
|
58
|
+
export default {
|
|
59
|
+
targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
|
|
60
|
+
agents: {
|
|
61
|
+
default: {
|
|
62
|
+
executor: decisionExecutor({
|
|
63
|
+
model: typeSafeAi.decisionModel('jev-latest'),
|
|
64
|
+
textModel: openrouter('inception/mercury-2.5'),
|
|
65
|
+
}),
|
|
66
|
+
},
|
|
67
|
+
},
|
|
68
|
+
} satisfies E2EConfig;
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
`evaluationModel()` instances still work. A model id string or a language
|
|
72
|
+
model in `model` fails config load with `INVALID_CONFIG`.
|
|
73
|
+
|
|
74
|
+
Tests stay plain natural language, with no params needed:
|
|
75
|
+
|
|
76
|
+
```ts title="tests/todos.e2e.ts"
|
|
77
|
+
import { test, expect } from 'e2e';
|
|
78
|
+
|
|
79
|
+
test('adds a todo', async ({ app, screen, agent }) => {
|
|
80
|
+
await app.open('/todos');
|
|
81
|
+
await agent.act('Add a todo named Buy coffee, and verify it is listed.');
|
|
82
|
+
await expect(screen.getByTestId('todo')).toContainText('Buy coffee');
|
|
83
|
+
await agent.assert('The todo list contains Buy coffee.');
|
|
84
|
+
});
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
## How text gets typed
|
|
88
|
+
|
|
89
|
+
The decision model only ever names an element index. When it picks `type`,
|
|
90
|
+
the text model writes the value from the goal, the field's label, and the
|
|
91
|
+
page text, and the executor types exactly that string. Model output never
|
|
92
|
+
becomes a selector, a URL, or a key. When the goal supplies no value,
|
|
93
|
+
nothing is typed and the decision model picks another action. Password
|
|
94
|
+
fields never reach the text model; secrets stay declared handles that only
|
|
95
|
+
`typeSecret` fills.
|
|
96
|
+
|
|
97
|
+
The text model is also the agent's judgment tier: it is exposed as the
|
|
98
|
+
executor's `model`, so `waitFor` and `extract` use it too. It can come
|
|
99
|
+
from the agents entry's `model` instead. Without either, `type` is never
|
|
100
|
+
offered.
|
|
101
|
+
|
|
102
|
+
## Finishing a step
|
|
103
|
+
|
|
104
|
+
When the decision model answers `done` or `failed`, the executor checks
|
|
105
|
+
the claim on a fresh screen with one question: is the step's task complete?
|
|
106
|
+
It sees the step's non-secret params, the actions taken with the non-secret
|
|
107
|
+
values typed, and the page, including the text of named status regions, but
|
|
108
|
+
no model reasoning or secret. A confirmed `done` passes the
|
|
109
|
+
step and a confirmed `failed` fails it with `ACTION_FAILED`. An
|
|
110
|
+
unconfirmed claim goes back to the decision model as history; the second one
|
|
111
|
+
ends the step, as `ACTION_FAILED` when the screen contradicts the goal and
|
|
112
|
+
as `blocked` (`AUTOMATION_UNSUPPORTED`) otherwise.
|
|
113
|
+
|
|
114
|
+
## Gates
|
|
115
|
+
|
|
116
|
+
Both gates are off by default: the executor follows the most probable
|
|
117
|
+
choice. Set `minProbability` or `minConfidence` to block weaker actions,
|
|
118
|
+
reject weaker completion checks, and fail weaker assertions as
|
|
119
|
+
inconclusive instead:
|
|
120
|
+
|
|
121
|
+
```ts title="Executor setup"
|
|
122
|
+
const executor = decisionExecutor({ model, minProbability: 0.9, minConfidence: 0.8 });
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Probability and confidence are different provider fields: probability is the
|
|
126
|
+
selected option's share of the answer distribution, confidence is the
|
|
127
|
+
provider's separate statistic. A threshold is a policy choice; measure
|
|
128
|
+
accuracy on your own tests before changing it.
|
|
129
|
+
|
|
130
|
+
## Limits
|
|
131
|
+
|
|
132
|
+
- No vision: screenshots never reach the model; canvas-only controls and
|
|
133
|
+
image-based targets are not implemented.
|
|
134
|
+
- No `navigate`: in-app navigation is a `tap` on a link. URLs belong in
|
|
135
|
+
test code (`browser.goto`, `app.open`).
|
|
136
|
+
- No multi-select `listbox` in v1.
|
|
137
|
+
- The choice cap is 255 options per question (TypeSafe's limit). Larger
|
|
138
|
+
screens list viewport nodes first and count the rest as omitted, so the
|
|
139
|
+
model can scroll; the step never blocks just for being large.
|
|
140
|
+
- Drag, upload, and hover are not offered in v1.
|
|
141
|
+
- `waitFor` and `extract` use the text model through the runner's
|
|
142
|
+
built-in judgment tier.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Example projects
|
|
3
|
+
description: Run standalone e2e examples with Vite, Next.js, Expo, and SwiftUI.
|
|
4
|
+
icon: "folder-open"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Each example is a standalone greeting app with three locator tests and two
|
|
8
|
+
agent tests. Enter a name, press Greet, and check the result. The projects
|
|
9
|
+
install published e2e packages from npm and work outside this repository.
|
|
10
|
+
|
|
11
|
+
<CardGroup cols={2}>
|
|
12
|
+
<Card title="Vite" icon="/images/examples/vite.svg" href="https://github.com/tester-army/e2e/tree/main/examples/with-vite" cta="View example" arrow>
|
|
13
|
+
React with Vite. Run locator and agent tests in Chromium.
|
|
14
|
+
</Card>
|
|
15
|
+
<Card title="Next.js" icon="/images/examples/nextjs.svg" href="https://github.com/tester-army/e2e/tree/main/examples/with-next" cta="View example" arrow>
|
|
16
|
+
Next.js App Router. The runner starts the app and tests it in Chromium.
|
|
17
|
+
</Card>
|
|
18
|
+
<Card title="Expo" icon="/images/integrations/expo.svg" href="https://github.com/tester-army/e2e/tree/main/examples/with-expo" cta="View example" arrow>
|
|
19
|
+
React Native with Expo. Run the same tests on web, an iOS simulator, or an Android emulator.
|
|
20
|
+
</Card>
|
|
21
|
+
<Card title="SwiftUI" icon="/images/examples/swiftui.png" href="https://github.com/tester-army/e2e/tree/main/examples/with-swiftui" cta="View example" arrow>
|
|
22
|
+
A native iOS app with an Xcode project. Run tests on an iPhone simulator.
|
|
23
|
+
</Card>
|
|
24
|
+
</CardGroup>
|
|
25
|
+
|
|
26
|
+
Follow each project's README for installation, builds, and run commands.
|
|
27
|
+
The config and tests live beside the app. The SwiftUI project keeps them
|
|
28
|
+
in its `e2e/` folder.
|
|
29
|
+
|
|
30
|
+
Locator tests need no model. Set `AI_GATEWAY_API_KEY` to run the agent
|
|
31
|
+
tests; without it, those tests skip. See [Models](/models) for model setup
|
|
32
|
+
and the [Quickstart](/quickstart) to add e2e to your own app.
|
package/docs/executors.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Custom executors
|
|
3
3
|
description: Customize the agent loop or implement your own step executor.
|
|
4
|
+
icon: "microchip"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
Use a custom executor when a `system` prompt and [project tools](/tools) are
|
|
@@ -14,6 +15,8 @@ not enough. There are two options:
|
|
|
14
15
|
`ai` package installed.
|
|
15
16
|
|
|
16
17
|
Both receive a deadline, redacted observations, and checked action methods.
|
|
18
|
+
For a provider that returns bounded choices instead of generated text, see
|
|
19
|
+
the [Clef and Jev decision executor](/decision-models).
|
|
17
20
|
The runner records their verdicts. Either goes under `executor` in an agents
|
|
18
21
|
entry, beside the options that still apply: `model`, `judge`, `context`, and
|
|
19
22
|
the budgets. An entry with `executor` rejects `system` and `tools`; pass
|
|
@@ -215,8 +218,10 @@ default, follows the configured cache mode. A recorded step replays without
|
|
|
215
218
|
calling `runStep`. A replay that stops partway calls `runStep` with
|
|
216
219
|
`ctx.replayedPrefix`: continue from the current screen and do not redo the
|
|
217
220
|
replayed actions. What a step does through `ctx.actions` is recorded; a
|
|
218
|
-
mutating tool call leaves a gap the executor fills on replay. Entries are
|
|
219
|
-
keyed by
|
|
221
|
+
mutating tool call leaves a gap the executor fills on replay. Entries are
|
|
222
|
+
keyed by the configured agent and its context, so an entry one agent
|
|
223
|
+
recorded never replays for another; the executor itself is not part of the
|
|
224
|
+
key, so changing an agent's executor keeps its entries.
|
|
220
225
|
|
|
221
226
|
`'off'` never replays or records the executor's steps, so every step reaches
|
|
222
227
|
`runStep`. Use it for a brain whose own history or memory is the source of
|
package/docs/explore.mdx
CHANGED
package/docs/github.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Pull request comments
|
|
3
3
|
description: The @e2e-dev/github reporter posts one comment per run from GitHub Actions and keeps it current.
|
|
4
|
+
icon: "code-pull-request"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
`@e2e-dev/github` posts test results as a pull request comment, including
|
|
@@ -63,14 +64,17 @@ Upload artifacts even when a retry passes. Each failure's evidence links open
|
|
|
63
64
|
the run's Artifacts section; inside the download, every test has a directory
|
|
64
65
|
named after it and the agent that ran it
|
|
65
66
|
(`web/tests_billing.e2e.ts__upgrades/default/attempt-0/`), so keep
|
|
66
|
-
`path: .e2e/artifacts` and the download lines up with the report. A
|
|
67
|
+
`path: .e2e/artifacts` and the download lines up with the report. A
|
|
68
|
+
`--last-failed` rerun keeps the first run's directories and puts its own
|
|
69
|
+
under `rerun-1/`, so a flaky test's failed attempt still links to its own
|
|
70
|
+
files. A project
|
|
67
71
|
below the checkout root, such as `packages/e2e-tests`, gets source links under
|
|
68
72
|
that path. The checkout root is the nearest directory holding `.git` inside
|
|
69
73
|
`GITHUB_WORKSPACE`, so `actions/checkout` with a `path` input works too.
|
|
70
74
|
|
|
71
75
|
## The comment
|
|
72
76
|
|
|
73
|
-
```markdown
|
|
77
|
+
```markdown expandable
|
|
74
78
|
### 🔴 e2e: 1 failed, 1 flaky, 3 passed, 1 skipped
|
|
75
79
|
9 agent steps · 4 replayed from cache · 31 model calls · 128k tokens (62% cached) · $0.31
|
|
76
80
|
|
|
@@ -143,9 +147,14 @@ passes:
|
|
|
143
147
|
|
|
144
148
|
The comment covers the whole suite, not only the tests the rerun ran again.
|
|
145
149
|
A test that failed and then passed shows as flaky, a test that failed twice
|
|
146
|
-
stays failed,
|
|
147
|
-
|
|
148
|
-
|
|
150
|
+
stays failed, one that was interrupted and then passed shows as passed, and
|
|
151
|
+
the rest keep their first-pass results. The comment stays red while the
|
|
152
|
+
rerun left out something it owed: a failed test, one `--max-failures` stopped
|
|
153
|
+
before it ran, or a failed `beforeAll` or `afterAll` whose tests another
|
|
154
|
+
filter kept out of the rerun. It shows ⏹️ when the rerun left out a test the
|
|
155
|
+
first pass never finished. The rerun's own exit code counts only what it ran.
|
|
156
|
+
A failed attempt's evidence links to the first run's files, which the rerun
|
|
157
|
+
keeps.
|
|
149
158
|
|
|
150
159
|
## Matrix jobs
|
|
151
160
|
|
package/docs/index.mdx
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Introduction
|
|
3
3
|
description: Open source AI testing for web and mobile apps. Spend less time on tests and more time building.
|
|
4
|
+
icon: "house"
|
|
4
5
|
---
|
|
5
6
|
|
|
6
7
|
import { VideoCard } from './snippets/video-card.jsx';
|
|
@@ -9,9 +10,9 @@ e2e is a next-generation testing framework for web and mobile
|
|
|
9
10
|
apps, built to keep up with the speed we ship software today.
|
|
10
11
|
|
|
11
12
|
You choose how agentic each test is: fully deterministic with
|
|
12
|
-
[locators](/
|
|
13
|
+
[locators](/core-concepts#locators) and
|
|
13
14
|
[assertions](/reference/expect#locator-matchers), fully driven by an agent
|
|
14
|
-
from [goals](/
|
|
15
|
+
from [goals](/core-concepts#goals) in natural language, or a mix of
|
|
15
16
|
both in the same test.
|
|
16
17
|
|
|
17
18
|
<VideoCard
|
|
@@ -49,7 +50,7 @@ you shipped.
|
|
|
49
50
|
## How it works
|
|
50
51
|
|
|
51
52
|
- **Targets.** Each test runs once per target, such as Chromium or an iPhone
|
|
52
|
-
simulator. The [web engine](/
|
|
53
|
+
simulator. The [web engine](/web) drives browsers through
|
|
53
54
|
Playwright, the [mobile engine](/mobile) drives iOS simulators and Android
|
|
54
55
|
emulators through agent-device, and you can
|
|
55
56
|
[write your own](/writing-an-engine).
|
|
@@ -70,8 +71,11 @@ you shipped.
|
|
|
70
71
|
<Card title="Quickstart" icon="rocket" href="/quickstart">
|
|
71
72
|
One prompt for your coding agent, or set up by hand.
|
|
72
73
|
</Card>
|
|
73
|
-
<Card title="
|
|
74
|
-
|
|
74
|
+
<Card title="Example projects" icon="folder-open" href="/example-projects">
|
|
75
|
+
Run a standalone app with Vite, Next.js, Expo, or SwiftUI.
|
|
76
|
+
</Card>
|
|
77
|
+
<Card title="Core concepts" icon="pen-to-square" href="/core-concepts">
|
|
78
|
+
Goals, assertions, and locators.
|
|
75
79
|
</Card>
|
|
76
80
|
<Card title="Coding agents" icon="robot" href="/coding-agents">
|
|
77
81
|
Let your coding agent write, run, and debug tests.
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
title: EAS Simulators
|
|
3
3
|
sidebarTitle: EAS Simulators
|
|
4
4
|
description: Run mobile targets on EAS Simulators' hosted iOS simulators and Android emulators with the easSimulators() device provider from @e2e-dev/eas.
|
|
5
|
+
icon: "/images/icons/expo.svg"
|
|
5
6
|
---
|
|
6
7
|
|
|
7
8
|
`easSimulators()` from `@e2e-dev/eas` is a
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
title: Kernel browsers
|
|
3
3
|
sidebarTitle: Kernel
|
|
4
4
|
description: Run web targets in Kernel's hosted Chromium with the kernel() browser provider from @e2e-dev/kernel.
|
|
5
|
+
icon: "/images/icons/kernel.svg"
|
|
5
6
|
---
|
|
6
7
|
|
|
7
8
|
`kernel()` from `@e2e-dev/kernel` is a
|
|
@@ -60,7 +61,7 @@ web({
|
|
|
60
61
|
```
|
|
61
62
|
|
|
62
63
|
Each attempt's window then opens maximized, and the page
|
|
63
|
-
[fills it](/
|
|
64
|
+
[fills it](/web#fill-the-window): 1920 by 993 on a 1920 by 1080 screen,
|
|
64
65
|
after the tab strip and address bar. With a fixed web `viewport`, the page is
|
|
65
66
|
emulated at that size inside a smaller window, so Kernel's live view shows it
|
|
66
67
|
cut off or in part of the window.
|
|
@@ -90,7 +91,7 @@ cookies and storage are not there.
|
|
|
90
91
|
that browser's default context, on its own first tab, so a `profile` applies
|
|
91
92
|
and the live view shows one tab. It costs a browser start per test and
|
|
92
93
|
carries the limits of a [per-attempt lease](/browser#hosted-browsers): no
|
|
93
|
-
`headers`, `basicAuth`, or `
|
|
94
|
+
`headers`, `basicAuth`, `userAgent`, `locale`, or `timezoneId`, no `app.clearState()`, and no
|
|
94
95
|
[sessions](/authentication), which fail the run before any test.
|
|
95
96
|
|
|
96
97
|
## Recordings
|
package/docs/migrate/cypress.mdx
CHANGED
|
@@ -1,8 +1,31 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: From Cypress
|
|
3
3
|
description: Replace Cypress command chains with async tests and keep your data-cy attributes.
|
|
4
|
+
icon: "/images/icons/cypress.svg"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
8
|
+
|
|
9
|
+
export const migrationPrompt = `Migrate this project's Cypress tests to e2e, one spec at a time. Docs: \`https://e2e.tester.army/docs/migrate/cypress.md\`
|
|
10
|
+
|
|
11
|
+
1. Read the migration guide above. It maps Cypress config, commands, and assertions to e2e.
|
|
12
|
+
2. If the project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent). It writes a config and an example test, and installs nothing yet.
|
|
13
|
+
3. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
14
|
+
4. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
15
|
+
5. Port \`cypress.config.*\` into the e2e config: \`baseUrl\` and the dev server command become the target's \`app\`, and if the specs select by \`data-cy\`, set \`web({ testIdAttribute: 'data-cy' })\` so \`screen.getByTestId\` reads it. Keep Cypress installed while both suites run.
|
|
16
|
+
6. Install dependencies.
|
|
17
|
+
7. List the Cypress end-to-end specs and ask me which one to move first. Suggest one that breaks often or waits with \`cy.wait(ms)\`. Component tests have no e2e equivalent; leave them in Cypress.
|
|
18
|
+
8. Rewrite that spec as a \`.e2e.ts\` file under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default), with one awaited call per line and no chains. \`cy.contains\` matches a substring and e2e queries match the whole text, so pass \`{ exact: false }\` where the spec relied on a fragment. Replace each \`cy.wait(ms)\` with an assertion on the state the test waits for. Move \`cy.task\`, \`cy.exec\`, and \`cy.request\` into plain Node code, and custom commands into functions or fixtures.
|
|
19
|
+
9. Keep navigation and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted interactions breaks often, propose an \`agent.act\` goal for it instead.
|
|
20
|
+
10. Run the new file with \`npx e2e run <file>\` and fix it until it passes. Keep the Cypress spec until I confirm it can go.
|
|
21
|
+
11. Summarize which tests moved, which stayed in Cypress, and why.`;
|
|
22
|
+
|
|
23
|
+
<SetupPrompt>
|
|
24
|
+
|
|
25
|
+
<Prompt description={migrationPrompt} children={migrationPrompt} actions={["copy", "cursor"]} />
|
|
26
|
+
|
|
27
|
+
</SetupPrompt>
|
|
28
|
+
|
|
6
29
|
e2e tests run in Node. Use `await` for UI actions and call Node functions
|
|
7
30
|
directly for work previously handled by `cy.task`. Both suites can stay in
|
|
8
31
|
the project while you migrate one spec at a time.
|
|
@@ -15,7 +38,7 @@ for the agent calls shown below.
|
|
|
15
38
|
| Cypress | e2e |
|
|
16
39
|
| --- | --- |
|
|
17
40
|
| `baseUrl` | `app: { url }` on the target |
|
|
18
|
-
| `start-server-and-test` | `app: { url, command }`, see [Starting your app](/starting-your-app) |
|
|
41
|
+
| `start-server-and-test` | `app: { url, command }`, see [Starting your app](/web#starting-your-app) |
|
|
19
42
|
| `viewportWidth`, `viewportHeight` | `web({ viewport })` |
|
|
20
43
|
| `data-cy` selectors | `web({ testIdAttribute: 'data-cy' })`, then `screen.getByTestId` |
|
|
21
44
|
| `retries` | `retries` |
|
|
@@ -115,8 +138,8 @@ Start with a spec that often breaks and whose fixed `cy.wait` calls can
|
|
|
115
138
|
become assertions on the condition the test needs.
|
|
116
139
|
|
|
117
140
|
<CardGroup cols={2}>
|
|
118
|
-
<Card title="
|
|
119
|
-
Goals,
|
|
141
|
+
<Card title="Core concepts" icon="pen-to-square" href="/core-concepts">
|
|
142
|
+
Goals, assertions, and locators instead of scripted selectors.
|
|
120
143
|
</Card>
|
|
121
144
|
<Card title="Working with the browser" icon="globe" href="/browser">
|
|
122
145
|
Routes, dialogs, downloads, iframes, and page evaluation.
|
package/docs/migrate/detox.mdx
CHANGED
|
@@ -1,8 +1,31 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: From Detox
|
|
3
3
|
description: Use your existing app build and testIDs with screen, expect, and agent goals.
|
|
4
|
+
icon: "/images/icons/detox.svg"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
8
|
+
|
|
9
|
+
export const migrationPrompt = `Migrate this project's Detox tests to e2e, one spec at a time. Docs: \`https://e2e.tester.army/docs/migrate/detox.md\` and \`https://e2e.tester.army/docs/mobile.md\`
|
|
10
|
+
|
|
11
|
+
1. Read the migration guide above. It maps \`.detoxrc.js\` and the Detox API to e2e. The mobile guide covers devices and app builds.
|
|
12
|
+
2. If the project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent). It writes a web config and an example test, and installs nothing yet. Replace @e2e-dev/web with @e2e-dev/mobile, and rewrite \`tests/example.e2e.ts\` for the mobile engine or delete it.
|
|
13
|
+
3. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
14
|
+
4. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
15
|
+
5. Port \`.detoxrc.js\` into the e2e config: one target per platform with \`mobile({ platform })\`, the app's \`bundleId\` and \`appPath\` on the target, and the device type as \`mobile({ device })\`. e2e uses the normal debug or release build, with no Detox framework.
|
|
16
|
+
6. Install dependencies, then run \`npx agent-device doctor\` and fix what it reports.
|
|
17
|
+
7. List the Detox specs and ask me which one to move first. Suggest a flow that needs frequent maintenance.
|
|
18
|
+
8. Rewrite that spec as a \`.e2e.ts\` file under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default), not in the Detox folder. Start it with \`app.open()\`, since nothing launches on its own. \`testID\` props work with \`screen.getByTestId\`. Replace \`waitFor(...).withTimeout()\` with a locator assertion and its \`timeout\`. e2e fails when a query matches several nodes, so narrow those queries.
|
|
19
|
+
9. Keep the launch and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted interactions breaks often, propose an \`agent.act\` goal for it instead.
|
|
20
|
+
10. Build and install the app, run the new file with \`npx e2e run <file>\` on each platform target, and fix it until it passes. Keep the Detox spec until I confirm it can go.
|
|
21
|
+
11. Summarize which tests moved, which stayed in Detox, and why.`;
|
|
22
|
+
|
|
23
|
+
<SetupPrompt>
|
|
24
|
+
|
|
25
|
+
<Prompt description={migrationPrompt} children={migrationPrompt} actions={["copy", "cursor"]} />
|
|
26
|
+
|
|
27
|
+
</SetupPrompt>
|
|
28
|
+
|
|
6
29
|
`@e2e-dev/mobile` drives a simulator or emulator using your normal debug
|
|
7
30
|
or release build. It does not need the Detox framework. React Native
|
|
8
31
|
`testID` props work with `getByTestId` on both platforms.
|
|
@@ -131,7 +154,7 @@ to settle, configure the engine's
|
|
|
131
154
|
[`settle` and `transition` options](/reference/mobile#engine).
|
|
132
155
|
|
|
133
156
|
<CardGroup cols={2}>
|
|
134
|
-
<Card title="
|
|
157
|
+
<Card title="Mobile" icon="mobile-screen" href="/mobile">
|
|
135
158
|
Configure your app and devices.
|
|
136
159
|
</Card>
|
|
137
160
|
<Card title="Mobile engine reference" icon="book" href="/reference/mobile">
|
package/docs/migrate/maestro.mdx
CHANGED
|
@@ -1,8 +1,32 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: From Maestro
|
|
3
3
|
description: Move Maestro flows to TypeScript with locators, assertions, and agent goals.
|
|
4
|
+
icon: "/images/icons/maestro.svg"
|
|
4
5
|
---
|
|
5
6
|
|
|
7
|
+
import { SetupPrompt } from '../snippets/setup-prompt-card.jsx';
|
|
8
|
+
|
|
9
|
+
export const migrationPrompt = `Migrate this project's Maestro flows to e2e, one flow at a time. Docs: \`https://e2e.tester.army/docs/migrate/maestro.md\` and \`https://e2e.tester.army/docs/mobile.md\`
|
|
10
|
+
|
|
11
|
+
1. Read the migration guide above. It maps every Maestro config option and command to e2e and marks what e2e does not have yet. The mobile guide covers devices and app builds.
|
|
12
|
+
2. e2e tests are TypeScript. If the project has no \`package.json\`, as in a native iOS or Android project, create the test project in an \`e2e/\` folder beside the app.
|
|
13
|
+
3. If the test project has no e2e config (\`e2e.config.ts\` or \`e2e.config.mts\`), run \`npx e2e init --yes\` (or the pnpm or bun equivalent) there. It writes a web config and an example test, and installs nothing yet. Replace @e2e-dev/web with @e2e-dev/mobile, and rewrite \`tests/example.e2e.ts\` for the mobile engine or delete it.
|
|
14
|
+
4. Read the e2e skill (\`npx e2e guide\` prints it, \`npx e2e guide <topic>\` prints a topic) and follow it for every step below.
|
|
15
|
+
5. If you ran init, ask me which model to use for agent steps: a ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, an API key, or a local model. Configure it with its provider package. If it needs a sign-in, give me the \`npx e2e login\` command and wait until I confirm.
|
|
16
|
+
6. Port the flows' \`appId\` and \`.maestro/config.yaml\` into the e2e config: one target per platform with \`mobile({ platform })\`, \`app: { bundleId }\` on each target, and tags as \`{ tags }\` on tests. Move passwords and tokens from \`env\` into \`credentials\` or \`secrets\`.
|
|
17
|
+
7. Install dependencies, then run \`npx agent-device doctor\` and fix what it reports.
|
|
18
|
+
8. List the flows and ask me which one to move first. Suggest one whose \`optional\` steps and \`runFlow: { when }\` branches often need updating.
|
|
19
|
+
9. Rewrite that flow as a \`.e2e.ts\` test under \`tests/\` (e2e runs \`tests/**/*.e2e.ts\` by default), with one awaited call per command. Start it with \`app.open()\`, since nothing launches on its own. Maestro matches text and ids as regular expressions and taps the first of several matches; e2e matches the whole string and fails on several, so use a RegExp, \`.first()\`, or \`.nth()\` where the flow relied on that. \`optional\` and \`runFlow: { when }\` become \`if (await locator.isVisible())\`. Relative selectors such as \`below\` have no port; use a label, a test id, or a goal. Where the guide marks a command missing, use the workaround it gives, or leave that flow in Maestro and tell me why.
|
|
20
|
+
10. Keep the launch and the final assertions deterministic, with \`app.open()\` and locator assertions. Where a run of scripted taps breaks often, propose an \`agent.act\` goal for it instead.
|
|
21
|
+
11. Build and install the app, run the new file with \`npx e2e run <file>\` on each platform target, and fix it until it passes. Keep the Maestro flow until I confirm it can go.
|
|
22
|
+
12. Summarize which flows moved, which stayed in Maestro, and why.`;
|
|
23
|
+
|
|
24
|
+
<SetupPrompt>
|
|
25
|
+
|
|
26
|
+
<Prompt description={migrationPrompt} children={migrationPrompt} actions={["copy", "cursor"]} />
|
|
27
|
+
|
|
28
|
+
</SetupPrompt>
|
|
29
|
+
|
|
6
30
|
Translate Maestro commands into awaited TypeScript calls. Keep exact
|
|
7
31
|
interactions as locator calls, or give the agent a goal that covers several
|
|
8
32
|
of them. Both runners can use the same simulator or emulator.
|
|
@@ -167,7 +191,7 @@ declares `requires: ['device']` so other targets skip it. `Status` is `same`,
|
|
|
167
191
|
| `optional: true` on a command | the same `isVisible()` check | renamed |
|
|
168
192
|
| `repeat: { times: n }`, `repeat: { while }` | a `for` or `while` loop | renamed |
|
|
169
193
|
| `runScript`, `evalScript`, `${output.x}` | TypeScript in the test | renamed |
|
|
170
|
-
| `http.get`, `http.post`, `json()` | Node `fetch`, see [
|
|
194
|
+
| `http.get`, `http.post`, `json()` | Node `fetch`, see [API](/api-testing) | renamed |
|
|
171
195
|
| `onFlowStart`, `onFlowComplete` | `beforeEach`, `afterEach` | renamed |
|
|
172
196
|
| `startRecording`, `stopRecording` | `test('x', { video: 'on' }, ...)` records that test's whole attempts, on a device the mobile engine records; no start and stop inside a test | renamed |
|
|
173
197
|
|
|
@@ -220,10 +244,10 @@ Start with a flow whose `optional` steps and `runFlow: { when }` branches
|
|
|
220
244
|
often need updating.
|
|
221
245
|
|
|
222
246
|
<CardGroup cols={2}>
|
|
223
|
-
<Card title="
|
|
247
|
+
<Card title="Mobile" icon="mobile-screen" href="/mobile">
|
|
224
248
|
Configure your app and devices.
|
|
225
249
|
</Card>
|
|
226
|
-
<Card title="
|
|
227
|
-
Goals,
|
|
250
|
+
<Card title="Core concepts" icon="pen-to-square" href="/core-concepts">
|
|
251
|
+
Goals, assertions, and locators instead of scripted selectors.
|
|
228
252
|
</Card>
|
|
229
253
|
</CardGroup>
|