e2e 0.15.1 → 0.16.0

package/docs/writing-an-engine.mdx

@@ -0,0 +1,316 @@
+---
+title: Writing an engine
+description: Implement the e2e engine contract for another UI platform.
+---
+
+An engine connects `screen`, `app`, and agent calls to a UI platform through
+the `e2e/engine` contract. Use it to support a desktop app, a TV, or another
+platform.
+
+The existing `@e2e-dev/web` and `@e2e-dev/mobile` packages use the
+same public `defineEngine` API. Read their implementations alongside this
+guide for complete examples.
+
+## The ownership boundary
+
+The runner manages tests and policy. The engine observes and controls the UI.
+
+| Concern | Owner |
+| --- | --- |
+| Collection, targets, retries, hooks, sessions | runner |
+| Query polling and strict single-match cardinality | runner |
+| Assertion polling, step recording, timeouts | runner |
+| One immediate query, one node read | engine |
+| One node's actionability and input dispatch | engine |
+| Agent planning, prompts, budgets, ledger | runner |
+| Tool authorization and secret resolution | runner |
+| Process, page, and device mechanics | engine |
+| Source masking of secure observations | engine |
+| Defense-in-depth redaction | runner |
+
+`locate` must return the current matches immediately, including an empty
+array when nothing matches. Do not add query retries or reject multiple
+matches. The runner handles polling and single-match checks.
+
+## The shape of an engine
+
+Return a handle from `defineEngine`. It validates the manifest and determines
+the engine's capabilities when the config loads.
+
+This skeleton assumes a `createClient` adapter for your platform. Its nodes
+must follow the semantic tree contract described below.
+
+```ts title="src/index.ts"
+import { ConfigurationError, defineEngine, resolveExpression, type EngineHandle } from 'e2e/engine';
+
+export function hyperdrive(options: { device: string }): EngineHandle {
+  const client = createClient(options);
+  let bundleId = '';
+  return defineEngine({
+    name: 'hyperdrive',
+    version: '1.0.0',
+    spiVersion: 1,
+    platform: 'ios',
+    validateApp(app, { targetName }) {
+      if (app.bundleId === undefined) {
+        throw new ConfigurationError('INVALID_CONFIG', `target "${targetName}" needs app.bundleId`);
+      }
+    },
+    async init(info) {
+      bundleId = info.app.bundleId!;
+      await client.boot();
+    },
+    async dispose(context) { await client.shutdown({ signal: context.signal }); },
+    async observe() {
+      const screen = await client.snapshot();
+      return {
+        location: screen.title,
+        // One root with a stable id: `perform(root, swipe)` is the viewport swipe.
+        root: { ref: { id: 'root', revision: '' }, role: 'window', children: screen.nodes },
+        viewport: screen.viewport,
+      };
+    },
+    async locate(expression) {
+      const screen = await client.snapshot();
+      return resolveExpression(expression, screen.nodes);
+    },
+    actions: ['tap', 'fill', 'swipe'],
+    async perform(ref, action) {
+      switch (action.kind) {
+        case 'tap': return client.tap(ref.id);
+        case 'fill': return client.type(ref.id, action.value);
+        case 'swipe': return client.scroll(ref.id === 'root' ? undefined : ref.id, action.direction);
+      }
+    },
+    session: {
+      async back() { await client.back(); },
+      async restart() { await client.relaunch(bundleId); },
+    },
+  });
+}
+```
+
+```ts title="e2e.config.ts"
+targets: [{ name: 'phone', engine: hyperdrive({ device: 'iPhone 16' }), app: { bundleId: 'com.example.app' } }],
+```
+
+`version` is required and forms part of the replay cache key. Change it when
+engine behavior changes. With `perform`, also declare `actions` to list the
+supported action kinds. `locate` answers `screen` queries; the exported
+`resolveExpression` applies the standard query rules to a semantic tree.
+The app under test is the target's `app`, not an engine option:
+`validateApp` checks it at config load, and `prepare` and `init` receive it
+as `info.app`. `session` provides methods to open or
+restart it.
+
+## What an engine is not
+
+Engines drive user interfaces. Use fixtures for supporting services such as
+an inbox, a database, or a test API. Add those through
+[`test.extend`](/reference/test#your-own-fixtures).
+
+<Note>
+Import shared types and helpers from `e2e/engine`, and declare `e2e` as a
+peer dependency so the engine and runner share their error classes.
+The public exports include errors, text and URL matching, assertion polling,
+JSON validation, and cancellation helpers. There is no internal subpath.
+</Note>
+
+## Capabilities
+
+Implement the optional methods your platform supports. Their presence
+determines the available capabilities. Calls to unsupported capabilities
+fail with `UNSUPPORTED_CAPABILITY`. A useful engine starts with these:
+
+| Member | Unlocks |
+| --- | --- |
+| `observe(context, options?)` | Agent observations and the replay cache |
+| `perform(ref, action, context)` + `actions` | Node actions of the declared kinds, including viewport scrolling through `swipe` |
+| `locate(expression, context)` | `screen`, `Locator`, and `expect(locator)` |
+| `validateApp(app, info)` | A config-load check of the target's `app`: throw `INVALID_CONFIG` naming `info.targetName` for a field the platform needs or cannot drive. [Reference](/reference/engine#app) |
+| `session.{open?,back?,restart?,reset?}` | `app.open()`, `app.back()`, `app.restart()`, `app.clearState()`, and the agent's `navigate` and `back` verbs |
+
+Pointer actions, the keyboard, saved state, artifacts, and contributed
+fixtures are optional capabilities of their own; the
+[engine reference](/reference/engine#capabilities) lists every member.
+
+`actions`, `location`, `pointer`, and `keyboard` require `observe`. The agent
+receives only tools supported by the engine and the step's declared
+credentials: each action kind you declare unlocks its verb (`hover` the
+`hover` tool, `dragTo` the `drag` tool, `setInputFiles` the `upload` tool,
+`check` with `uncheck` the `check` tool, and so on; the
+[engine reference](/reference/engine#capabilities) lists the mapping).
+Agent and locator actions both reach `perform` as a
+`LocatorAction`. Throw `UNSUPPORTED_CAPABILITY` if a particular node cannot
+support an otherwise declared action.
+
+The agent's `scroll` and `screen.swipe()` call
+`perform(root, { kind: 'swipe' })`. Keep the root id stable across observations.
+A swipe on another node scrolls within that node.
+
+Use the exported `parseKey` to interpret `press` keys such as `Enter` or
+`ControlOrMeta+a`. The runner rejects invalid spellings with `INVALID_ARGUMENT`.
+Map valid keys to your platform and throw `UNSUPPORTED_CAPABILITY` for keys
+it cannot send. See the [key grammar](/reference/engine#keys).
+
+An observation may include `location`, such as a URL, screen title, or window
+name. The runner anchors replay cache entries on it and applies origin
+checks when it parses as a URL.
+
+`session.open` receives a URL already resolved by the runner. After `restart`
+or `reset`, the runner reopens the configured address through `open`. If your
+engine has no `open` method, relaunch the app within those hooks.
+
+There is no separate secure-fill capability flag. Before dispatching a
+secret fill, the runner checks the declared credential, current
+observation, and compatible editable field. The engine receives a
+`fill` action with `sensitive: true` and must handle it securely or throw
+`UNSUPPORTED_CAPABILITY`. The runner then stops taking screenshots; see
+[screenshots after a secret fill](/security#screenshots-after-a-secret-fill).
+
+An engine option can take a secret too, such as a password the engine sends
+the app. Accept `string | Secret` (tell them apart with `isSecret` from
+`e2e/engine`), list each handle in the engine's `secrets`, and read the value
+in `startAttempt` through `context.resolveSecret(secret)`. The config load
+rejects a name no secret has, and the runner redacts the value from the
+moment it resolves. When the app sees the value in another form (a header
+that encodes it), return that form from `resolveSecret(secret, { derived })`
+so it is redacted under the same name. The protection is text only: a
+session whose engine resolved a secret keeps its screenshots and pixels. A config calls `secrets.get()` before any run exists, so
+the handle is a reference by name until then.
+
+### Lifecycle
+
+In the order they run:
+
+- `prepare`: once per target in the runner process. Install runtimes here;
+  the `env` it returns reaches every worker.
+- `init`: once per worker before its first attempt. Boot devices here.
+- `startAttempt`: before each attempt. Create the test's UI.
+- `settleAttempt`: after the body and again after teardown. Throw the error
+  of a test callback no step awaited, such as a dialog handler.
+- `endAttempt`: after each attempt. Clean up the test's UI; it also runs
+  after a failed start, so make it idempotent.
+- `dispose`: at worker shutdown, even if `init` never ran.
+- `finish`: once per target after the last worker is gone, whenever
+  `prepare` ran. Release what `prepare` acquired.
+
+Use `info.env` for spawned processes and `info.log` to report progress. The
+[engine reference](/reference/engine#lifecycle) has each hook's arguments and
+budgets.
+
+### Nodes and refs
+
+`observe` returns a `SemanticNode` tree. `locate` returns the nodes matching
+an expression. Assign ids that stay unique for the UI's lifetime and keep
+the root id stable. The runner assigns revisions and checks staleness.
+When `perform` receives `{ id, revision }`, look up the id and throw
+`NODE_STALE` if the node no longer exists. When a match leaves the UI while
+`locate` reads it, throw a retryable `NODE_STALE` too, never a node read from
+what is gone: a removed element reports no size, and a node read from it would
+pass `toBeHidden` while its replacement is on screen. The runner re-resolves
+within the caller's deadline.
+
+Limit snapshot size if the platform can return an unbounded tree. Set
+`truncated` when the limit cuts off nodes so the runner knows the listing
+is incomplete. Use the platform's accessible names for nodes.
+
+Put test identifiers in `testId`, whether they come from a document attribute,
+an iOS accessibility identifier, an Android resource id, or a desktop
+automation id. The `testId` query matches only that field.
+
+### Vision-only surfaces
+
+For a UI without a semantic tree, return a childless `root` and `pixels`
+from `observe`. Implement `performAt` and declare the supported kinds in
+`pointerActions`.
+
+The agent receives the screenshot and can act through tools such as `tap_at`.
+The runner passes points to `performAt` when no listed node occupies them.
+Coordinate actions replay only at the recorded viewport size, and the runner
+checks the recorded end state before accepting a replay.
+
+### Keyboard
+
+Implement `keyboard` to type into a control after the agent focuses it.
+`keyboard.type` must deliver the text as a person typing would, through the
+platform's keyboard path, so that key events reach the app where the platform
+has them: a browser engine dispatches `keydown`, `keypress`, `input`, and
+`keyup` per character; a device engine sends the text through the device
+keyboard. Inserting the value without keystrokes is not a compliant
+implementation. It must throw `NOT_ACTIONABLE` when no editable control has
+focus. Omit `dismiss` on platforms without an on-screen keyboard. The same
+hook, after the `focus` action, is what `locator.pressSequentially` types
+through; the harness composes the two and asks nothing more of the engine.
+
+### State is a credential
+
+A `state` snapshot can contain authentication data such as cookies and tokens.
+The runner encrypts it with a per-run key, binds it to the run, target,
+engine, and app identity, and deletes it after the run.
+
+Do not persist, cache, log, or include snapshots in errors. `restore` must
+replace the app's entire persisted state instead of merging into it.
+
+### Errors
+
+Throw `EngineError` with a code from the
+[engine reference](/reference/engine). Only `NODE_STALE` and `FRAME_NOT_FOUND`
+can be retryable. Use `ACTION_MAY_HAVE_COMMITTED` when an action may already
+have taken effect so the runner does not repeat it.
+
+`INVALID_STATE` maps to `APP_NOT_OPEN`, and `UNSUPPORTED_CAPABILITY` is a
+configuration error. Unknown errors become infrastructure failures.
+
+## Contributed fixtures
+
+Engines can contribute fixtures such as `browser` or `device`. Wrap their methods
+with `context.fixture` to record calls as `<fixture>.<method>` steps with
+their own artifacts and failures.
+
+```ts
+fixtures: {
+  device: (context) => context.fixture('device', {
+    async setNetwork(state: 'online' | 'offline') { await client.network(state); },
+    async home() { await client.home(); },
+  }, {
+    setNetwork: { kind: 'resource', label: (state) => state },
+    home: { kind: 'resource' },
+  }),
+},
+```
+
+```ts title="tests/fixtures.ts"
+import { test as base } from 'e2e';
+import type { Device } from 'e2e-engine-hyperdrive';
+
+export const test = base.extend<{ device: Device }>();
+```
+
+The fixture context provides URL policy checks, timeouts, operation budgets,
+artifact attachment, and factories for locators and assertions. For example,
+`expectable(target, factory)` adds matchers for `expect(fixture)`. See the
+[engine reference](/reference/engine) for the full context.
+
+## Ship a tool pack
+
+Platform operations the action vocabulary lacks can ship as a pack of
+[project tools](/tools) built with `defineTool`, with `platforms` set so each
+tool is offered only on compatible targets. `@e2e-dev/mobile/tools` is one
+example.
+
+## Verify it
+
+Run a test file against your engine. Check lifecycle order, observations,
+actions, errors, and unsupported capabilities.
+[`engine-contract.test.ts`](https://github.com/tester-army/e2e/blob/main/packages/e2e/tests/integration/engine-contract.test.ts)
+demonstrates the runner's contract and provides examples for these checks.
+
+<CardGroup cols={2}>
+  <Card title="Engine contract" icon="gear" href="/reference/engine">
+    Every type and hook of `e2e/engine`.
+  </Card>
+  <Card title="Mobile engine reference" icon="mobile-screen" href="/reference/mobile">
+    A complete device engine to read alongside.
+  </Card>
+</CardGroup>

package/docs/writing-tests.mdx

@@ -0,0 +1,256 @@
+---
+title: Writing tests
+description: Goals, checks, extraction, and the deterministic APIs.
+---
+
+Use agent calls for goals and questions about the screen. Use `screen` and
+`expect` when you know the exact action or value to check. A test can freely mix both.
+
+```ts title="tests/todos.e2e.ts"
+import { test, expect } from 'e2e';
+
+test('todos survive a filter round-trip', async ({ app, agent, screen }) => {
+  await app.open('/todos');
+
+  await agent.act('add todos "Buy milk" and "Walk the dog", complete the first');
+  await agent.assert('only one todo remains open');
+
+  await screen.getByRole('tab', 'Done').tap();
+  await expect(screen.getByTestId('todo')).toHaveCount(1);
+});
+```
+
+## Goals: `agent.act`
+
+Give `act` one goal. The agent chooses and performs the actions, then returns
+when the goal passes or throws when it fails.
+
+```ts
+await agent.act('complete checkout with the test card');
+await agent.act('invite {email} as an editor', { params: { email: 'ada@example.test' } });
+```
+
+Pass values through `params` so the agent uses your test data. For passwords,
+use a `Secret`, this prevents the model from seeing your secrets. See [Signing in](/authentication).
+
+```ts
+import { credentials } from 'e2e';
+
+const member = credentials.user('member');
+await agent.act('Sign in', {
+  params: { username: member.username, password: member.password },
+});
+```
+
+<Tip>
+**Write instructions as you would say them out loud**
+
+Name what is visible and prefer the exact wording on screen. One goal per
+call. The flow inside a goal is the model's job; the order of goals is yours.
+</Tip>
+
+For a value that changes on every run, wrap it in `unique()` so the
+[replay cache](/cache#what-must-match) can still replay the step:
+
+```ts
+import { expect, unique } from 'e2e';
+
+const email = `ada+${Date.now()}@example.test`;
+await agent.act('sign up with email {email}', { params: { email: unique(email) } });
+await expect(screen.getByText(email)).toBeVisible();
+```
+
+If a goal fails, check the report and make the instruction more specific.
+Use the labels on screen. Put vocabulary shared by several goals in the
+agent's `context`; see [Agents and personas](/agents).
+
+A step can pass, fail, or be blocked by something such as missing credentials.
+[How agent steps work](/agent-steps#how-a-step-ends) explains the verdicts and
+exit codes.
+
+## Checks: `assert`, `waitFor`, `extract`
+
+`assert` checks a statement against the current screen. A false statement
+fails with `ASSERTION_FAILED`. If the screen does not provide enough evidence,
+it fails with `ASSERTION_INCONCLUSIVE`.
+
+The judging model sees the statement and the current screen. It does not see
+earlier steps or the acting agent's summaries.
+
+```ts
+await agent.assert('the dashboard shows a trial badge');
+```
+
+`assert` checks the screen now. Use `waitFor` when the condition may become true
+later. It polls the screen and only calls the model when the observation changes.
+
+```ts
+await agent.waitFor('the export finishes and a download link appears', {
+  interval: 500,
+  timeout: 120_000,
+});
+```
+
+`extract` reads structured data from the screen. Provide a
+[Standard Schema](https://standardschema.dev) validator, such as Zod, to define
+the result.
+
+```ts
+import { z } from 'zod';
+
+const data = await agent.extract('every todo title and how many remain', {
+  schema: z.object({ titles: z.array(z.string()), remaining: z.number().int() }),
+});
+expect(data.titles).toContain('Buy milk');
+```
+
+<Tip>
+For model-generated descriptions, check the relevant meaning rather than the
+entire sentence. Use a locator assertion when the exact text matters.
+</Tip>
+
+### Judging pixels
+
+By default, judgments read a redacted text snapshot of the screen. Add
+`vision: true` for visual checks, such as charts or layout. Use
+`vision: 'only'` to judge the screenshot alone.
+
+```ts
+await agent.assert('the chart trends upward', { vision: true });
+await agent.assert('the search form is not covered by a banner', { vision: 'only' });
+```
+
+`act` requests screenshots itself when it needs them, so it has no `vision`
+option. On supported engines, it can tap and type at points in the image:
+
+```ts
+await agent.act('enter the code 3141 on the drawn keypad and press OK');
+```
+
+Images add input tokens. Screenshots are withheld for the rest of an attempt
+after a secret is filled. The [agent reference](/reference/agent#vision)
+lists the modes and restrictions.
+
+## Deterministic APIs: `screen` and `expect`
+
+Use locators to interact with known controls and check exact values.
+An `expect` after `act` checks the exact result. It is also what lets the
+[replay cache](/cache) record the step and keep it.
+
+```ts
+// Semantic queries: role and name first, label second, test ID last.
+await screen.getByRole('button', 'Sign in').tap();
+await screen.getByLabel('Email').fill('ada@example.test');
+await screen.getByTestId('todo').first().tap();
+
+// Scope into a row, gesture like a user:
+const row = screen.getByRole('listitem').filter({ hasText: 'Design review' });
+await row.getByRole('button', 'Archive').tap();
+await screen.swipe({ direction: 'up', momentum: 'fast' });
+
+// Coordinates, for what the tree does not list:
+await screen.getByRole('image', 'Seat map').tap({ position: { x: 120, y: 48 } });
+await screen.swipe({ from: { x: 300, y: 400 }, to: { x: 40, y: 400 } });
+
+// Polling assertions:
+await expect(screen.getByRole('status')).toHaveText('2 remaining');
+await expect(screen.getByRole('heading', 'Dashboard')).toBeVisible();
+```
+
+Actions and single-node reads or assertions fail when a query matches more
+than one node. Narrow it with `.filter()`, `.first()`, or `.nth()`.
+Locator assertions retry until they pass or time out. Direct reads such as
+`textContent()` do not retry a mismatched value.
+
+Use the `browser` fixture for browser features such as routes, cookies, dialogs,
+frames, and downloads; see [Working with the browser](/browser). The
+[screen reference](/reference/screen) lists every query and action.
+
+Share setup and cleanup through `test.extend`. A fixture can create a test
+workspace before the test and delete it afterward, including when the test
+fails. See [Your own fixtures](/reference/test#your-own-fixtures).
+
+## Tests without a screen
+
+A test that takes only `app` calls no model and drives no screen. Use it to
+check your API with `fetch` in the same run as UI tests. See
+[Testing APIs](/api-testing).
+
+## Structure
+
+```ts
+import { beforeEach, describe, test } from 'e2e';
+
+describe('billing', { tags: ['billing'] }, () => {
+  beforeEach(async ({ app }) => { await app.open('/billing'); });
+
+  test('upgrades', { retries: 2, timeout: 60_000 }, async ({ agent }) => {
+    await agent.act('upgrade to Pro');
+  });
+});
+
+test.skip('not ready yet', async () => {});
+test.only('just this one while I work', async () => {}); // rejected in CI
+
+test('needs two organizations', async ({ app }) => {
+  test.skip((await countOrganizations()) < 2, 'one organization: nothing to switch to');
+  await app.open('/organizations');
+});
+```
+
+`test.skip(condition, reason)` inside a body throws when the condition holds:
+the test stops at that line and is reported `skipped`, as in Playwright. Call
+it before the first step when it guards a precondition.
+
+`describe` and the hooks are also `test.describe`, `test.beforeEach`, and so
+on. A `beforeEach` imported from `e2e` sees the core fixtures; import
+`test`, `describe`, and the hooks from `@e2e-dev/web` (or `@e2e-dev/mobile`)
+to have `browser` (or `device`) typed from one import, and use
+`test.beforeEach` on a `test` you extended with your own fixtures.
+
+Common options include `timeout`, `retries`, and `tags`. Use `session` for
+state from a setup test, `agentContext` for extra instructions, and `agent`
+to choose a configured [agent](/agents). `trace` and `video` record one test
+without recording the suite; see [Watch it happen](/debugging#watch-it-happen).
+Group options apply to their tests. The [test reference](/reference/test)
+lists all options and defaults.
+
+Each test runs in its own attempt. Prepare the app data it needs through
+fixtures or hooks. When a flow must span several tests, mark the group
+`{ serial: true }`. Members share app state, run in order on one worker,
+and retry as a whole.
+
+## Run one test
+
+<CodeGroup>
+```bash npm
+npx e2e run tests/signup.e2e.ts:12   # the test declared at line 12
+npx e2e run --grep checkout          # tests whose title matches
+npx e2e run --target web             # one target, by name
+```
+
+```bash pnpm
+pnpm exec e2e run tests/signup.e2e.ts:12   # the test declared at line 12
+pnpm exec e2e run --grep checkout          # tests whose title matches
+pnpm exec e2e run --target web             # one target, by name
+```
+
+```bash bun
+bunx e2e run tests/signup.e2e.ts:12   # the test declared at line 12
+bunx e2e run --grep checkout          # tests whose title matches
+bunx e2e run --target web             # one target, by name
+```
+</CodeGroup>
+
+The [CLI reference](/reference/cli#e2e-run) lists every flag.
+
+## Next
+
+<CardGroup cols={2}>
+  <Card title="Signing in" icon="key" href="/authentication">
+    Sessions and secrets.
+  </Card>
+  <Card title="Agents and personas" icon="users" href="/agents">
+    Several agents in one suite, and running a flow as each of them.
+  </Card>
+</CardGroup>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "e2e",
-  "version": "0.15.1",
+  "version": "0.16.0",
   "description": "SDK, runner, and CLI for agentic end-to-end testing",
   "keywords": [
     "e2e",
@@ -69,6 +69,7 @@
   },
   "files": [
     "dist",
+    "docs",
     "schema/*.schema.json",
     "skills",
     "NOTICE"
@@ -76,7 +77,7 @@
   "dependencies": {
     "@ai-sdk/provider": "4.0.18",
     "@clack/prompts": "1.8.1",
-    "@modelcontextprotocol/sdk": "1.30.1",
+    "@modelcontextprotocol/server": "2.2.0",
     "@vitest/expect": "5.0.2",
     "commander": "15.0.0",
     "picocolors": "1.1.1",
@@ -108,7 +109,8 @@
     "@ai-sdk/openai": "4.0.71",
     "@ai-sdk/openai-compatible": "3.0.53",
     "@ai-sdk/xai": "5.0.4",
-    "@e2e-dev/web": "0.11.1",
+    "@e2e-dev/web": "0.11.2",
+    "@modelcontextprotocol/client": "2.2.0",
     "@openrouter/ai-sdk-provider": "3.1.0",
     "@types/node": "26.6.2",
     "@types/pngjs": "6.0.5",

package/skills/e2e/SKILL.md

@@ -58,7 +58,10 @@ test('a member upgrades to Pro', async ({ app, agent, screen, browser }) => {
 
 Read the topic for the job before writing code. The files sit next to this
 one; the installed CLI prints the same text with `npx e2e guide <topic>`
-(`e2e guide` alone prints this page).
+(`e2e guide` alone prints this page). For anything the topics do not cover,
+the full documentation ships in the `docs/` directory of the installed `e2e`
+package (`node_modules/e2e/docs` in a single-package project); a link such as
+`/reference/cli` is `docs/reference/cli.mdx`.
 
 | Topic | File | Read it when |
 | --- | --- | --- |

package/skills/e2e/references/agent.md

@@ -250,7 +250,9 @@ matches. Misses and hand-offs use the model; `agent.assert`,
   and share replays with CI and teammates (CI stays `read-only` unless
   `cache: 'read-write'` is set).
 - A failing run evicts the entries it implicates; `--no-cache` rules the
-  cache out of a failure.
+  cache out of a failure. A cancelled run and a failure where no model
+  answered (`MODEL_PROVIDER_FAILED`, `MODEL_UNAVAILABLE`) implicate nothing
+  and evict nothing.
 - With committed recordings, `--strict-cache` in CI fails a recording that
   no longer replays with `REPLAY_STALE` instead of quietly spending model
   calls every run; re-record locally and commit. Unrecorded steps still run

package/skills/e2e/references/mcp.md

@@ -20,6 +20,9 @@ Or declare it in the client's project config (`.mcp.json` for Claude Code,
 { "mcpServers": { "e2e": { "command": "npx", "args": ["e2e", "mcp"] } } }
 ```
 
+No model is needed: without the optional `ai` package, sessions open and the
+deterministic tools work.
+
 Flags: `--config <path>` names the default config file, `--target <name>`
 fixes the target every session opens on, `--headless` hides the browser or
 simulator (sessions are headed by default outside CI), `--max-sessions <n>`