e2e 0.16.0 → 0.18.0

package/docs/telemetry.mdx

@@ -1,6 +1,7 @@
 ---
 title: Telemetry
 description: What the CLI sends, what it never sends, and how to turn it off.
+icon: "chart-line"
 ---
 
 The `e2e` CLI sends anonymous usage data so we can see which commands,
@@ -72,7 +73,7 @@ Who ran it:
 | Property | Value |
 | --- | --- |
 | `distinct_id` | A random id stored in the preferences file. CI uses `ci:<vendor>` instead, and a platform that runs e2e for its users uses `fleet:<name>` when it sets `E2E_TELEMETRY_FLEET` |
-| `project_id` | A hash derived from the repository's root commit, shared across full clones. Outside git or in a shallow clone, a hash of the project path with a random local salt. `null` in CI when no root commit is available |
+| `project_id` | A hash derived from the repository's root commit, shared across full clones. `null` outside git and in a shallow clone |
 | `session_id` | A random id for this one invocation |
 | `first_run`, `days_since_first_run` | Whether this command created the preferences file, and whole days since it was created. `null` in CI and in a fleet, where no machine persists, and for a preferences file from before the time was recorded |
 
@@ -119,7 +120,7 @@ Versions:
 | `command` | `run` or `explore` |
 | `status`, `exit_code`, `duration_ms` | The run's outcome |
 | `flags` | The names of the flags given to the run command, never their values |
-| `tests_discovered`, `tests_selected`, `tests_executed`, `tests_passed`, `tests_failed`, `tests_flaky`, `tests_skipped` | The summary counts |
+| `tests_discovered`, `tests_selected`, `tests_executed`, `tests_passed`, `tests_failed`, `tests_interrupted`, `tests_flaky`, `tests_skipped` | The summary counts |
 | `attempts_total`, `tests_retried` | How many attempts the run made across tests and serial groups, and how many of them needed more than one |
 | `targets`, `platforms`, `engines` | How many targets, on which platforms, and with which engines and versions, as your config declares them: `web`, `ios`, `web@0.6.1`, or your own engine's name and version. A name that is not a plain token of letters, digits, dots, dashes, and underscores is reported as `other` |
 | `steps_total`, `steps_agent`, `steps_locator`, `steps_assertion`, `steps_screen`, `steps_app`, `steps_session`, `steps_resource` | Step counts by kind |
@@ -181,16 +182,18 @@ listed above.
 ## In CI
 
 CI runs are attributed to the CI vendor rather than to a machine, no
-preferences file is written on the runner, and no notice is printed. Set
-`E2E_TELEMETRY_DISABLED: '1'` in the job's `env` to send nothing from CI.
+preferences file is written on the runner, and no notice is printed. A coding
+agent that sets `CI` in its shell, with no CI vendor around it, counts as the
+machine it runs on. Set `E2E_TELEMETRY_DISABLED: '1'` in the job's `env` to
+send nothing from CI.
 
 ## In a fleet
 
 A platform that runs e2e on its users' behalf starts each run on a fresh
 machine, and every fresh machine would otherwise count as a new user. Set
 `E2E_TELEMETRY_FLEET=<name>` in those machines and they are attributed to
-`fleet:<name>` instead, like CI: no preferences file, no notice, and no
-project id outside git. Or set `E2E_TELEMETRY_DISABLED=1` and send nothing.
+`fleet:<name>` instead, like CI: no preferences file and no notice. Or set
+`E2E_TELEMETRY_DISABLED=1` and send nothing.
 
 ## Where it goes
 

package/docs/tools.mdx

@@ -1,6 +1,7 @@
 ---
 title: Project tools
 description: Add tools for test APIs, data setup, and device operations.
+icon: "screwdriver-wrench"
 ---
 
 Put tools in an agent's `tools` so the agent can call your test API or perform

package/docs/{starting-your-app.mdx → web.mdx}

@@ -1,13 +1,21 @@
 ---
-title: Starting your app
-description: Point the runner at a running app, or let it start the app for the run.
+title: Web
+description: Point the web engine at your app or let the runner start it, and run a suite at several viewport sizes.
+icon: "globe"
 ---
 
+`@e2e-dev/web` runs tests in a browser against your app's URL. This page
+covers getting the app running for a run and testing it at different
+sizes. For routes, cookies, dialogs, downloads, and the rest of the
+`browser` fixture, see [Working with the browser](/browser).
+
+## Starting your app
+
 Set your app's URL in the target's `app`. If you also provide a start
 command, the runner starts the app, waits until it is ready, and stops it
 when the run ends.
 
-## Point at a running app
+### Point at a running app
 
 Import `web` from `@e2e-dev/web` and put this in the exported
 config object:
@@ -25,7 +33,7 @@ export default {
 you can change the address for a run without editing the file. URLs without
 a scheme use HTTPS, or HTTP for loopback hosts.
 
-## Let the runner start it
+### Let the runner start it
 
 Add `app.command` and the runner spawns the dev server, waits until `app.url`
 answers, and stops it when the run ends, fails, or is interrupted:
@@ -79,7 +87,7 @@ temp-directory variables, and `env`, so pass what the app needs explicitly.
 See the
 [command reference](/reference/config#command) for all options.
 
-### Let the runner pick the port
+#### Let the runner pick the port
 
 Use port `0` to run multiple checkouts without a port conflict. The runner
 chooses a free port and replaces `{port}` in the command's `args` and `env`.
@@ -98,7 +106,7 @@ valid when the port changes. Use `127.0.0.1` or `[::1]`, matching the address
 the command binds; `localhost:0` is rejected. See
 [free ports](/reference/config#free-ports) for details.
 
-### Next.js dev targets
+#### Next.js dev targets
 
 Next.js 16 blocks cross-origin requests to its dev resources. A target that
 opens `127.0.0.1` or `[::1]` while `next dev` identifies as `localhost`
@@ -122,7 +130,7 @@ Use `'[::1]'` in the list when `app.url` opens `http://[::1]:0`.
 A target on `localhost` needs no entry, because `next dev` identifies as
 `localhost`. A production `next start` target is unaffected either way.
 
-## Dependencies
+### Dependencies
 
 <Note>
 This version starts only `app.command` (one process for every target that
@@ -132,7 +140,7 @@ have `app.command` start a script that brings them up and serves the app. A
 services API returns in a later release.
 </Note>
 
-## Several targets
+### Several targets
 
 Two browsers on one app are two targets that each name it:
 
@@ -155,9 +163,9 @@ test runs on every target unless it is scoped with `platforms` or
 `requires`; `--target <name>` selects at run time. A device target sits
 beside a browser target in the same list; see the
 [Mobile engine reference](/reference/mobile). One target per screen size is
-how a suite runs at several [viewports](/viewports).
+how a suite runs at several [viewports](#viewports).
 
-## Protected previews
+### Protected previews
 
 Use `headers` to access protected previews or skip a tunnel's interstitial
 page. For example, a Vercel preview can use a protection bypass header:
@@ -221,11 +229,149 @@ workers. Playwright traces include request headers, so a protected-preview
 trace can contain your bypass secret. Restrict access to those traces.
 </Note>
 
+## Viewports
+
+Set a viewport on the target to run a suite at that size. Use
+`browser.setViewport()` to resize during a test. The agent uses the current size
+and cannot resize it.
+
+### One size per target
+
+`viewport` sets the initial size of each attempt, in pixels. The default is
+1280 by 720. Add one target for each size you want to test:
+
+```ts title="e2e.config.ts"
+import type { E2EConfig } from 'e2e';
+import { web } from '@e2e-dev/web';
+
+const app = { url: 'http://localhost:3000' };
+
+export default {
+  targets: [
+    { name: 'desktop', engine: web(), app },
+    { name: 'tablet', engine: web({ viewport: { width: 820, height: 1180 } }), app },
+    { name: 'phone', engine: web({ browser: 'webkit', viewport: { width: 390, height: 844 } }), app },
+  ],
+} satisfies E2EConfig;
+```
+
+Each test runs once per target, with a separate result labeled `desktop`,
+`tablet`, or `phone`. Run one size with `npx e2e run --target phone`, or list
+the selected test-target pairs with `npx e2e list`.
+
+To have the runner start one dev server for every size, give every target
+the same `app.command`: targets declaring one command share one process; see
+[several targets](#several-targets). Every retry starts at
+the target's configured size.
+
+### Fill the window
+
+`viewport: null` sets no size, so the page fills the browser window. The
+agent, screenshots, and videos use that size. Use it in a headed run, or
+with a hosted browser whose live view would show a fixed 1280 by 720 page
+in one corner:
+
+```ts title="e2e.config.ts"
+{ name: 'window', engine: web({ viewport: null }), app }
+```
+
+The page takes the window's size, minus the toolbar in a headed browser. A
+local headed launch opens a small window; a hosted browser uses the window
+size you requested from the service. `browser.setViewport` still works.
+
+### Resize inside a test
+
+`browser.setViewport` resizes the page for the rest of the attempt, through
+`app.restart()` and `app.clearState()`:
+
+```ts
+import { test } from '@e2e-dev/web';
+import { expect } from 'e2e';
+
+test('the menu collapses on a narrow screen', async ({ app, screen, browser }) => {
+  await app.open('/');
+  await expect(screen.getByRole('navigation')).toBeVisible();
+
+  await browser.setViewport({ width: 390, height: 844 });
+  await expect(screen.getByRole('button', 'Menu')).toBeVisible();
+  await expect(screen.getByRole('navigation')).toBeHidden();
+});
+```
+
+Use this when one test needs to check a layout before and after resizing.
+For a whole suite, configure separate targets instead.
+
+Call it before `app.open` to load the first page at that size, so the app
+never renders its desktop layout first:
+
+```ts
+await browser.setViewport({ width: 390, height: 844 });
+await app.open('/');
+```
+
+The resize appears as a step in the report. To check the exact width, read
+it from the page:
+
+```ts
+const width = await browser.evaluate(() => window.innerWidth);
+expect(width).toBe(390);
+```
+
+### What the agent sees
+
+The agent receives the current viewport size with each observation.
+Screenshots cover that viewport. Off-screen elements can still appear in
+the tree, and tapping them scrolls them into view. The agent can scroll to
+reveal lazy-loaded content.
+
+Resize before calling `agent.act`:
+
+```ts
+test('the mobile menu reaches pricing', async ({ app, agent, browser }) => {
+  await browser.setViewport({ width: 390, height: 844 });
+  await app.open('/');
+  await agent.act('open the navigation menu and go to Pricing');
+  await expect(browser).toHaveURL('/pricing');
+});
+```
+
+### Scoping a test to one size
+
+`platforms` and `requires` cannot select a viewport; all Playwright targets
+use platform `web`. To skip a test at some sizes, check the width in the
+test:
+
+```ts
+test('the sidebar stays open', async ({ app, screen, browser }) => {
+  await app.open('/');
+  test.skip((await browser.evaluate(() => window.innerWidth)) < 768, 'desktop layout only');
+  await expect(screen.getByRole('complementary')).toBeVisible();
+});
+```
+
+To keep a whole group of size-specific tests apart, give them their own
+config and run it with `--config`.
+
+### Limits
+
+A narrow viewport tests responsive layout. It still uses a desktop browser
+with a scale factor of 1, desktop pointer behavior, and its usual user agent.
+It does not emulate touch, color scheme, or reduced-motion preferences.
+Use the [device engine](/mobile) for tests on a simulator or emulator.
+
+A video keeps the size the page had when it started recording; set
+`web({ screencast: { size } })` for one fixed size. A browser provider that
+records, such as [Kernel](/integrations/kernel#recordings), records the whole
+screen instead.
+
 <CardGroup cols={2}>
+  <Card title="Working with the browser" icon="globe" href="/browser">
+    Routes, cookies, dialogs, downloads, and the rest of the `browser` fixture.
+  </Card>
   <Card title="Config reference" icon="sliders" href="/reference/config#the-app-under-test">
     `app`, `command`, and every app key.
   </Card>
   <Card title="Web engine reference" icon="masks-theater" href="/reference/web">
-    Browser, viewport, `connect`, `headers`, `basicAuth`.
+    Browser, viewport, `connect`, `headers`, `basicAuth`, and every `browser` method.
   </Card>
 </CardGroup>

package/docs/writing-an-engine.mdx

@@ -1,6 +1,7 @@
 ---
 title: Writing an engine
 description: Implement the e2e engine contract for another UI platform.
+icon: "gear"
 ---
 
 An engine connects `screen`, `app`, and agent calls to a UI platform through
@@ -40,7 +41,7 @@ the engine's capabilities when the config loads.
 This skeleton assumes a `createClient` adapter for your platform. Its nodes
 must follow the semantic tree contract described below.
 
-```ts title="src/index.ts"
+```ts title="src/index.ts" expandable
 import { ConfigurationError, defineEngine, resolveExpression, type EngineHandle } from 'e2e/engine';
 
 export function hyperdrive(options: { device: string }): EngineHandle {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "e2e",
-  "version": "0.16.0",
+  "version": "0.18.0",
   "description": "SDK, runner, and CLI for agentic end-to-end testing",
   "keywords": [
     "e2e",
@@ -62,6 +62,10 @@
     "./oauth/grok": {
       "types": "./dist/oauth/grok.d.ts",
       "default": "./dist/oauth/grok.js"
+    },
+    "./oauth/opencode-console": {
+      "types": "./dist/oauth/opencode-console.d.ts",
+      "default": "./dist/oauth/opencode-console.js"
     }
   },
   "bin": {
@@ -75,23 +79,35 @@
     "NOTICE"
   ],
   "dependencies": {
-    "@ai-sdk/provider": "4.0.18",
+    "@ai-sdk/provider": "4.0.21",
     "@clack/prompts": "1.8.1",
-    "@modelcontextprotocol/server": "2.2.0",
-    "@vitest/expect": "5.0.2",
+    "@jridgewell/remapping": "2.3.5",
+    "@modelcontextprotocol/server": "2.3.0",
+    "@oxc-project/runtime": "0.152.0",
+    "@vitest/expect": "5.0.3",
     "commander": "15.0.0",
+    "get-tsconfig": "4.14.3",
+    "oxc-parser": "0.152.0",
+    "oxc-transform": "0.152.0",
     "picocolors": "1.1.1",
     "pngjs": "7.0.0",
-    "tsx": "4.23.14",
     "zod": "4.6.1"
   },
   "peerDependencies": {
+    "@ai-sdk/anthropic": "^4.0.0",
+    "@ai-sdk/google": "^4.0.0",
     "@ai-sdk/openai": "^4.0.0",
     "@ai-sdk/openai-compatible": "^3.0.0",
     "@ai-sdk/xai": "^5.0.0",
     "ai": "^7.0.0"
   },
   "peerDependenciesMeta": {
+    "@ai-sdk/anthropic": {
+      "optional": true
+    },
+    "@ai-sdk/google": {
+      "optional": true
+    },
     "@ai-sdk/openai": {
       "optional": true
     },
@@ -106,10 +122,12 @@
     }
   },
   "devDependencies": {
+    "@ai-sdk/anthropic": "4.0.58",
+    "@ai-sdk/google": "4.0.77",
     "@ai-sdk/openai": "4.0.71",
     "@ai-sdk/openai-compatible": "3.0.53",
     "@ai-sdk/xai": "5.0.4",
-    "@e2e-dev/web": "0.11.2",
+    "@e2e-dev/web": "0.13.0",
     "@modelcontextprotocol/client": "2.2.0",
     "@openrouter/ai-sdk-provider": "3.1.0",
     "@types/node": "26.6.2",
@@ -117,12 +135,11 @@
     "ai": "7.0.107",
     "ajv": "8.20.0",
     "ajv-formats": "3.0.1",
-    "playwright": "1.63.0",
     "typescript": "7.0.2",
     "vitest": "5.0.1"
   },
   "engines": {
-    "node": ">=22.12.0"
+    "node": "^22.22.3 || >=24.8.0"
   },
   "scripts": {
     "build": "node scripts/prepare-build.ts && tsc --project tsconfig.build.json && node scripts/write-sibling-versions.ts && node ../../scripts/stamp-dist.ts",

package/schema/report-v1.schema.json

@@ -570,13 +570,14 @@
     "summary": {
       "type": "object",
       "additionalProperties": false,
-      "required": ["discovered", "selected", "executed", "passed", "failed", "flaky", "skipped"],
+      "required": ["discovered", "selected", "executed", "passed", "failed", "interrupted", "flaky", "skipped"],
       "properties": {
         "discovered": { "type": "integer", "minimum": 0 },
         "selected": { "type": "integer", "minimum": 0 },
         "executed": { "type": "integer", "minimum": 0 },
         "passed": { "type": "integer", "minimum": 0 },
         "failed": { "type": "integer", "minimum": 0 },
+        "interrupted": { "type": "integer", "minimum": 0 },
         "flaky": { "type": "integer", "minimum": 0 },
         "skipped": { "type": "integer", "minimum": 0 }
       }
@@ -670,10 +671,22 @@
         "summary": { "$ref": "#/$defs/summary" },
         "limits": { "$ref": "#/$defs/limits" },
         "usage": { "$ref": "#/$defs/usage" },
+        "carried": { "$ref": "#/$defs/carried" },
         "explore": { "$ref": "#/$defs/explore" },
         "extensions": { "$ref": "#/$defs/extensions" }
       }
     },
+    "carried": {
+      "description": "What earlier runs left unresolved that a --last-failed rerun did not run again: their results as they reported them, the serial groups those results' attempts live on, and the suite hook failures whose scope still holds one of them. The next --last-failed selects these tests too. Not part of the run's status, exit code, or summary.",
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["results", "serialGroups", "errors"],
+      "properties": {
+        "results": { "type": "array", "minItems": 1, "items": { "$ref": "#/$defs/result" } },
+        "serialGroups": { "type": "array", "items": { "$ref": "#/$defs/serialGroup" } },
+        "errors": { "type": "array", "items": { "$ref": "#/$defs/error" } }
+      }
+    },
     "exploreFinding": {
       "type": "object",
       "additionalProperties": false,

package/skills/e2e/SKILL.md

@@ -10,8 +10,8 @@ one goal; `agent.assert`, `agent.waitFor`, and `agent.extract` judge the
 screen. `screen`, `app`, `browser`, and `expect` make exact interactions and
 checks. The replay cache reruns verified actions and checks their recorded end
 state without a model call; agent judgments still run live. UI targets use
-`@e2e-dev/web` for browsers or `@e2e-dev/mobile` for iOS simulators and
-Android emulators. A test that takes only `app` can check an API with `fetch`
+`@e2e-dev/web` for browsers or `@e2e-dev/mobile` for iOS simulators,
+Android emulators, and connected phones. A test that takes only `app` can check an API with `fetch`
 and `expect` (topic `writing-tests`). Model sign-in commands are in
 [setup](references/setup.md#subscriptions-and-api-keys).
 
@@ -61,7 +61,9 @@ one; the installed CLI prints the same text with `npx e2e guide <topic>`
 (`e2e guide` alone prints this page). For anything the topics do not cover,
 the full documentation ships in the `docs/` directory of the installed `e2e`
 package (`node_modules/e2e/docs` in a single-package project); a link such as
-`/reference/cli` is `docs/reference/cli.mdx`.
+`/reference/cli` is `docs/reference/cli.mdx`. Complete projects for Vite,
+Next.js, Expo, and SwiftUI, each with its config, scripts, and a passing
+suite, are in https://github.com/tester-army/e2e/tree/main/examples.
 
 | Topic | File | Read it when |
 | --- | --- | --- |

package/skills/e2e/references/agent.md

@@ -6,8 +6,13 @@ model-call budget, no shared transcript. No agent step, no model calls.
 
 ## Configure a model
 
-Put an AI SDK model under `agents.default`. Vercel AI Gateway reads
-`AI_GATEWAY_API_KEY` or, without it, a Vercel OIDC token:
+Put an AI SDK model under `agents.default`. You can use all AI SDK providers
+that support tool calls and the language model specification v2 or later
+(https://ai-sdk.dev/providers). Examples: `@ai-sdk/openai`,
+`@ai-sdk/anthropic`, `@ai-sdk/google`, `@ai-sdk/amazon-bedrock`,
+`@openrouter/ai-sdk-provider`, `ollama-ai-provider-v2`. For a server with
+`/v1/chat/completions`, use `@ai-sdk/openai-compatible`. Vercel AI Gateway
+reads `AI_GATEWAY_API_KEY` or, without it, a Vercel OIDC token:
 
 ```ts
 import type { E2EConfig } from 'e2e';
@@ -24,6 +29,14 @@ Subscription logins and local models:
 [setup](setup.md#subscriptions-and-api-keys). Keep `ai@^7` installed with
 any provider.
 
+Decision models pick actions from choices the executor builds: install
+`@e2e-dev/decision` and put `decisionExecutor({ model:
+typeSafeAi.decisionModel('jev-latest'), textModel: openrouter('inception/mercury-2.5') })`
+under `executor`. Tests stay plain language with no params; the text model
+writes field values. See the shipped `docs/decision-models.mdx` or
+[the online guide](https://e2e.tester.army/docs/decision-models) for setup,
+gates, and limits.
+
 - Pass a model instance, not a string (`INVALID_CONFIG`).
 - An agents entry is one plain object of `model`, `judge`, `system`,
   `context`, `tools`, `maxSteps`, `maxModelCalls`, `judgmentTimeout`,
@@ -32,7 +45,8 @@ any provider.
 - `model` drives `agent.act`. Judgments use `judge` when set, else `model`.
 - A missing model for the built-in agent raises one run-level
   `MODEL_UNAVAILABLE` when the first test acquires `agent`, exit 2; auth
-  failures surface on the first model call as `MODEL_PROVIDER_FAILED`.
+  failures surface on the first model call as `MODEL_PROVIDER_FAILED`, or as
+  `CONFIG_LOAD_FAILED` for a provider that reads its key when the config loads.
 - `context` is what the app calls things, sent to every model call, judges
   included; `system` is how the acting agent works, read only by the act
   loop. Both live on the agents entry; `agentContext` on a test or group
@@ -50,8 +64,8 @@ Tests use `agents.default` unless selected otherwise:
 - `{ agent: 'name' }` on an `agent.*` call overrides the test's choice.
 
 For signed-in personas, pair an agent with a `session` in a describe block
-repeated per persona; the cache records per agent step, so a specialised
-agent replays too.
+repeated per persona; the cache keys every entry by the agent and its
+context, so each persona records and replays its own steps.
 
 ## act: one goal
 
@@ -156,6 +170,9 @@ expect(data.titles).toContain('Buy milk');
 - Judgments see the assertion and the current screen only, never prior
   steps or the act loop's summaries; malformed output gets one repair
   round, then `MODEL_OUTPUT_INVALID`.
+- A value shown in more than one place (a total in the summary and on the
+  pay button) must agree everywhere, or the judgment fails. Name the one
+  you mean (`'the order summary total is $42.00'`) when only it matters.
 - `waitFor` observes every `interval` (default 3 s), judges only when the
   screen changed, and is `STEP_TIMEOUT` after `timeout` (default 30 s).
 - `extract` takes any Standard Schema validator (zod works); the model sees
@@ -236,14 +253,19 @@ matches. Misses and hand-offs use the model; `agent.assert`,
   `agent.assert`, `agent.waitFor`), so an unchecked `act` never replays; a
   plain-value `expect`, `expect.poll`, `agent.extract`, another `act`, or
   the attempt passing confirms nothing.
-- A replay needs the app on the recorded path (unless the recording opens
-  with a navigation), re-finds each control by role, name, test id,
-  placeholder, and input purpose, and passes alone only when the recorded
-  end path and the controls seen during the step are back; otherwise the
-  agent takes over mid-step. `step.cache.reason` says why: `no-entry`,
+- A replay needs the app on the recorded route (origin, path, and query,
+  ids and tokens aside; unless the recording opens with a navigation),
+  re-finds each control by role, name, test id, placeholder, input purpose,
+  and its named row (an unnamed control with twins and no named container
+  hands off), and passes alone only when the recorded end route is back,
+  every control that appeared (with its checked or selected state) is
+  there, every one that went away is gone, at least one of those changed
+  during the replay, and no new alert showed; otherwise the agent takes
+  over mid-step. Entries are keyed per agent and per `agentContext`. `step.cache.reason` says why: `no-entry`,
   `wrong-context`, `target-not-found`, `target-ambiguous`, `end-mismatch`,
   and so on.
-- A step recording no actions creates no entry; one whose `unique()` value
+- A step recording no actions, or changing nothing on screen or in the
+  route, creates no entry; one whose `unique()` value
   equals, is spelled inside, or is the encoded form of another param's value
   is not recorded either (`step.cache.notRecorded`: `param-collision`).
 - `e2e init` gitignores `.e2e/cache/`; remove that line to commit entries

package/skills/e2e/references/debugging.md

@@ -28,7 +28,7 @@ jq '.run.results[] | select(.selected and .status != "passed") | .attempts[-1]
    (recorded as `skipped`).
 4. Artifacts, under `.e2e/artifacts/`: `failure/screen.txt`
    and the engine's screenshot per failed attempt; a Playwright trace per
-   traced attempt (`npx playwright show-trace <file>`); downloads; with
+   traced attempt (`npx playwright-core@1.63.0 show-trace <file>`; `pnpm dlx` under pnpm); downloads; with
    `--video` the recording (`video/video.webm` in a local browser, each
    later page `video/video-part<n>.webm` with its own `startedAt`;
    `video/video.mp4` on a device; a provider's file or link); with
@@ -46,7 +46,7 @@ jq '.run.results[] | select(.selected and .status != "passed") | .attempts[-1]
 | `COLLECTION_ERROR` | `async` describe body, `test.setup` inside `describe`, an option forbidden in a serial group, registration outside collection | Restructure per `writing-tests` |
 | `HOOK_FAILED` | `beforeAll` or `afterAll` threw; its scope's tests skip | Fix the hook; the report carries its error |
 | `UNSUPPORTED_ARTIFACT` | A test's or target's `trace` or `video` on an engine that cannot record | Drop it there, or set it at the config root or CLI (such targets skip with a notice) |
-| `BROWSER_INSTALL_FAILED`, `LAUNCH_TIMEOUT` | Browser download failed; engine init or attempt start exceeded `launchTimeout` | Run the quoted `npx playwright install <names>` (`--with-deps` on bare Linux); raise the root `launchTimeout` (60 s default) |
+| `BROWSER_INSTALL_FAILED`, `LAUNCH_TIMEOUT` | Browser download failed; engine init or attempt start exceeded `launchTimeout` | Run the quoted `npx @e2e-dev/web install <names>` or `pnpm exec e2e-web install <names>` (`--with-deps` on bare Linux); raise the root `launchTimeout` (60 s default) |
 | `APP_UNREACHABLE` | `app.command` never answered `readyUrl` within `startupTimeout`; on a device, a message naming the iOS automation runner: the runner failed, not the app | Read the quoted log lines; check the port, `app.url`, `app.command.env`, `app.command.startupTimeout`. Runner: rerun, else `npx agent-device daemon stop` and reboot the simulator |
 | `APP_ALREADY_RUNNING` | Something already serves `url` when `command` should start | Stop it, or `reuseExisting: true` locally |
 | `APP_URL_REQUIRED`, `APP_NOT_OPEN` | A navigation on a target without `app.url`; a `screen` call before `app.open()` | Add `app.url` to the target; open the app first |
@@ -58,14 +58,14 @@ jq '.run.results[] | select(.selected and .status != "passed") | .attempts[-1]
 | `MODEL_OUTPUT_INVALID` | An `extract` or `assert` answer failed the schema after one repair round | Simplify the schema or question; pick a stronger model |
 | `ACTION_FAILED` | Element not actionable (covered, disabled, detached), or an operation timed out | `expect` the condition first; close overlays; check `actionTimeout` |
 | `TEST_TIMEOUT` | The attempt exceeded `timeout` (120 s) | Split the test, or raise `timeout` |
-| `STEP_NOT_AWAITED` | The body returned while a step still ran: a call without `await` | `await` the call the code frame names (every `app`, `agent`, `screen`, `expect` call) |
+| `STEP_NOT_AWAITED` | The body returned while a step or `expect.poll` still ran, or an `afterEach` hook or fixture teardown returned while an `expect.poll` still ran: a call without `await` | `await` the call the code frame names (every `app`, `agent`, `screen`, `expect`, `expect.poll` call) |
 | `MODEL_UNAVAILABLE` | `agents.<name>.model` holds no AI SDK instance: reported once under `run.errors` at the first `agent` fixture; the run stops. A model-less custom executor fails only `waitFor` and `extract` this way | Construct one, e.g. `gateway('openai/gpt-6-luna-fast')` from `ai`, and export its key (`AI_GATEWAY_API_KEY`) |
 | `MODEL_PROVIDER_FAILED` | Network, 5xx, rate limit, no credits, or no response in 120 s, after the transport retries | Check the credential (the key; for keyless `gateway()` the Vercel CLI login and `.vercel/project.json`) and quota; retry (exit 3) |
 | `REPLAY_STALE` | `--strict-cache` or `cache.strict`, and a committed recording no longer replays; `step.cache.reason` says why | Rerun read-write with the knob the message names off; commit the changed entry under the directory it names (a custom `cache.store` is written by that run) |
 | `AUTOMATION_UNSUPPORTED` | The step needs an interaction the engine's toolset lacks (drag on a device, say); blocked, exit 1 | Do that step with `screen` actions, or run the test on a target that supports it |
 | `STEP_TIMEOUT`, `STEP_BUDGET_EXHAUSTED` | Goal too big or ambiguous, or a slow provider | Split the goal, use on-screen wording, add vocabulary via `agents.<name>.context` or `agentContext`; raise `maxSteps` or `maxModelCalls` (budget), the config `timeout` (`act`), `judgmentTimeout` (judgments); `--debug` |
 | `CONTEXT_OVERFLOW` | Screen plus step history did not fit the model's context window (`act` after one shrink-and-retry; a judgment on the first overflow) | Lower `agents.<name>.maxObservationBytes`, split the step, or pick a larger-window model |
-| `POLICY_DENIED` | A `file:`, `data:`, or `javascript:` URL; a password `Secret` into a non-password sink; reading a secure field, `toHaveValue`, `toHaveText`, `toContainText`, `toHaveAttribute` included; `app.screenshot()` after a secret fill | http(s) only; passwords into password inputs only; assert the outcome, not the value; screenshot before filling secrets |
+| `POLICY_DENIED` | A URL whose scheme is not `http:` or `https:` (`file:`, `view-source:`, `data:`); a password `Secret` into a non-password sink; reading a secure field, `toHaveValue`, `toHaveText`, `toContainText`, `toHaveAttribute` included; `app.screenshot()` after a secret fill, or an `act` or `vision: true` step whose tree read timed out after one | http(s) or `about:blank` only; passwords into password inputs only; assert the outcome, not the value; screenshot before filling secrets |
 | `UNSUPPORTED_CAPABILITY` | A fixture the engine lacks (`browser` on a device), `schema` or `vision` on `act`, an action the surface lacks | Declare `requires: ['browser']`; drop the option or action |
 | `SESSION_UNAVAILABLE`, `SESSION_CONTRACT` | `session: 'x'` with no setup saving `x`; a setup that skipped a declared name. `SESSION_MISMATCH`, `SESSION_EXPIRED`, `SESSION_INVALID`: the stored session is another run's or app's, expired, or corrupt | Add or fix the `test.setup`; rerun it |
 | `ONLY_IN_CI` | `test.only` reached CI | Remove it |

package/skills/e2e/references/mcp.md

@@ -24,8 +24,9 @@ No model is needed: without the optional `ai` package, sessions open and the
 deterministic tools work.
 
 Flags: `--config <path>` names the default config file, `--target <name>`
-fixes the target every session opens on, `--headless` hides the browser or
-simulator (sessions are headed by default outside CI), `--max-sessions <n>`
+fixes the target every session opens on, `--headed` shows the browser or
+simulator when the engine supports it in sessions that do not set `headed`
+(sessions are headless by default, as in `e2e run`), `--max-sessions <n>`
 sets how many sessions may be open at once (default 4, 1 through 16).
 
 ## Tools
@@ -34,7 +35,7 @@ Four tools; everything a session can do is a catalog behind `call`.
 
 | Tool | Does |
 | --- | --- |
-| `open_session` | Loads the config (`config` names another file; default the nearest `e2e.config.ts`), starts the declared app command if any, boots the engine, opens the app URL, and returns the session id, the catalog, and the first observation. `target` is required when the config declares several. Each session has its own browser or device. |
+| `open_session` | Loads the config (`config` names another file; default the nearest `e2e.config.ts`), starts the declared app command if any, boots the engine, opens the app URL, and returns the session id, the catalog, and the first observation. `target` is required when the config declares several. `headed: true` shows the browser or simulator when the user wants to watch; without it the session follows the server's `--headed`, headless by default. Each session has its own browser or device. |
 | `tools` | The catalog: one line per tool with its argument names (`?` marks optional), the first sentence of its description, and `[read-only]` where it changes nothing. `tools {tool}` shows the full description and the JSON Schema of its arguments. |
 | `call` | Runs one catalog tool: `call {tool: "tap", args: {target: "n42"}}`. Arguments are checked against the tool's schema first; a wrong one fails with `INVALID_ARGUMENT` naming the field. |
 | `close_session` | Saves a recording still running, ends the attempt, disposes the engine, stops the app processes the session started once no other session uses them. |
@@ -57,6 +58,10 @@ The catalog, per session:
 | `stop_recording` | Stops it and returns the absolute path of each video file, under `<output>/videos/<session>/` (`.e2e` by default), or the URL of a provider's own recording. |
 | Project tools | Every `defineTool` in the agent's `tools` that applies to the target's platform, under its own name; an engine pack such as `mobileTools` adds `open_app`, `swipe`, `alert`. |
 
+Project tools use their `toModelOutput` renderer for strings as well as
+structured results. Its `error-text` and `error-json` outputs set the MCP
+result's `isError: true`; treat them as failed calls.
+
 Once a secret is filled in the session, `screenshot` and the point tools
 (`tap_at`, `hover_at`, `type_at`, `press_at`, `select_at`) answer with a
 `PIXEL_TAINTED` line for the rest of it; act on listed nodes by id (topic