e2e 0.15.1 → 0.16.0

package/docs/cache.mdx

@@ -0,0 +1,306 @@
+---
+title: Caching agent steps
+description: Replay verified agent actions without model calls, and let the agent take over when the app changes.
+---
+
+The replay cache records the actions in an `agent.act()` step after a later
+check verifies the result. On the next run, the runner replays those actions
+without calling the model. If a control or the expected end state no longer
+matches, the agent takes over from the current screen.
+
+`agent.assert`, `agent.waitFor`, and `agent.extract` always run live.
+
+The run summary reports the outcomes:
+
+```text
+           AI  4.1k tokens · 2 model calls · anthropic/claude-sonnet-4.5
+        Cache  4 replayed · 1 handed off · 1 missed
+```
+
+| Run summary | `step.cache.mode` | Meaning |
+| --- | --- | --- |
+| `replayed` | `self-finalized` | The recording completed without a model call. |
+| `handed off` | `agent-concluded` | Replay started, then the agent took over. |
+| `missed` | `missed` | The agent ran the step from the beginning. |
+
+Misses and hand-offs also include `step.cache.reason`. `agent.act()` returns
+the same cache object as `result.cache`. The cache object is absent when
+caching is off for the step.
+
+To rule the cache out of a failure, run once without it:
+
+<CodeGroup>
+```bash npm
+npx e2e run --no-cache
+```
+
+```bash pnpm
+pnpm exec e2e run --no-cache
+```
+
+```bash bun
+bunx e2e run --no-cache
+```
+</CodeGroup>
+
+## How a replay works
+
+1. Check the starting screen: the same route, whatever the record id, the
+   query, or the fragment. Retries always run live.
+2. Find each recorded control and repeat its action. The runner matches
+   controls by their role, name, test id, and surrounding context. It waits
+   for missing controls and stops if a match is ambiguous. A control with no
+   name is found by its place among the unnamed controls of its kind.
+3. Check the final path and controls that appeared during the original step.
+   Text that changes with the data, a count, a date, an id, is not used.
+4. Finish without a model call, or let the agent continue with a record of
+   what ran and why replay stopped.
+
+Typed values replay as recorded, even when an earlier run left the same
+value on screen. Secret fills resolve the current value by name and run
+through authorization again.
+
+## What must match
+
+A cache entry belongs to a specific test, target, instruction, and set of
+params. Renaming the test or target, changing the instruction or an ordinary
+param, or upgrading the engine's major or minor version causes a miss.
+Changing the model does not.
+
+A timestamp or a fresh email address would cause a miss on every run. Wrap
+values like these in `unique()` so the runner substitutes the current value
+when replaying:
+
+```ts
+import { unique } from 'e2e';
+
+const name = `E2E ${Date.now()} Company`;
+await agent.act('create a company named {name}', { params: { name: unique(name) } });
+await expect(screen.getByRole('heading', { name })).toBeVisible();
+```
+
+Use `unique()` for data the flow enters or checks. Keep choices that change
+the flow, such as a plan name, as ordinary params. The substitution covers
+the value as a URL spells it too, `%20` in a path or `+` in a search query.
+
+When a `unique()` value equals another param's value or appears inside one,
+the step passes but is not recorded, and `step.cache.notRecorded` reads
+`param-collision`. The next run, with a fresh value, records it.
+
+For preview URLs that change on each deploy, set
+[`app.identity`](/reference/config#the-app-under-test) to keep the same cache
+identity. On devices, replay uses the
+[screen title](/reference/mobile#replay-cache-on-a-device) as the path.
+
+## When a recording is written
+
+In `read-write` mode, the runner saves a recording at the end of the
+attempt when a later verification step passed. A step that took no action
+records nothing.
+
+| Counts as verification | Does not count |
+| --- | --- |
+| `expect(locator).toHaveText(...)` and other locator matchers | `expect(value).toBe(...)` on a plain value |
+| `expect(browser).toHaveURL(...)` and other engine assertions | `expect.poll(...)` |
+| `locator.waitFor()`, `browser.waitForURL()` | `locator.textContent()` and other reads |
+| `agent.assert(...)`, `agent.waitFor(...)` | `agent.extract(...)`, another `act`, the attempt passing |
+
+Follow each `act` with a check of its outcome:
+
+```ts title="tests/cart.e2e.ts"
+import { expect, test } from 'e2e';
+
+test('adds an item', async ({ app, agent, screen }) => {
+  await app.open('/cart');
+  await agent.act('add one item to the cart');
+  await expect(screen.getByRole('status')).toHaveText('1 item');
+});
+```
+
+A failing attempt can still save steps verified before the failure. It
+deletes unverified entries and replays implicated in the failure, including
+an `end-mismatch`. Custom stores need a `delete` method for this cleanup.
+
+Cancelling a run leaves the cache unchanged. An attempt whose only failures
+are ones where no model answered leaves every entry it did not confirm
+unchanged: `MODEL_UNAVAILABLE`
+(no model configured) and `MODEL_PROVIDER_FAILED` (the provider returned an
+error such as a missing or rejected key, a rate limit, or a 5xx, or a request
+got no response within 120 seconds). Those say nothing about the app, so a
+read-write run without model access keeps its recordings, including one a
+step replayed before an `agent.assert` that could not reach its model, and
+still saves steps verified before the failure. A model that answered with
+something unusable (`MODEL_OUTPUT_INVALID`, `CONTEXT_OVERFLOW`) or a step that
+ran out of time (`STEP_TIMEOUT`, even while a request was still open) counts
+as a failure of the flow, and so does any other failure in the same attempt,
+such as an `afterEach` assertion.
+
+## Why a step was not replayed
+
+`step.cache.reason` in the report says why:
+
+| Reason | What happened | What to do |
+| --- | --- | --- |
+| `no-entry` | No recording matches this step and its params. | The next verified pass records it. |
+| `retry` | Retries never replay. | Nothing. |
+| `invalid-entry` | The file is unreadable, over 1 MiB, or from another schema version. | `npx e2e cache clear`. |
+| `truncated` | The recording hit a limit, see [below](#limits-and-gaps). | Split the step, or pass the secret as a `Secret`. |
+| `wrong-context` | The app was on another screen than when the step was recorded: a different route, not a different record, query, or fragment. | Open the same screen before the step. |
+| `target-not-found` | A recorded control did not appear within 15 seconds. | The UI changed. The next passing run re-records. |
+| `target-ambiguous` | The number of matching controls changed. | Give repeated controls distinct names or use unique test data. |
+| `gap` | An action cannot be replayed, see [below](#limits-and-gaps). | The agent continues this part live. Put stable values in `params`. |
+| `action-failed` | The app rejected a replayed action. | Check the app. |
+| `action-uncertain` | A replayed action failed with an unknown commit state. The agent is told to check before acting again. | Check the app. |
+| `viewport-changed` | A bare-point tap or hover was recorded on a viewport of another size. | Keep the viewport fixed. |
+| `end-mismatch` | Every action ran, but the recorded path or a control that should have appeared did not come back. | Leftover data from an earlier run, a banner that comes and goes, or a regression. |
+
+A stop before any action ran is a miss. After replay starts, it is a
+hand-off. Unique test data and cleanup in `afterEach` help avoid stale
+state between runs.
+
+### Limits and gaps
+
+A recording is `truncated` when the step took more than 50 actions (only
+possible with `maxSteps` raised above 50), typed a value or URL over 4,096
+characters or one holding a registered secret, or acted on a control with no
+usable description.
+
+A `gap` is an action the cache cannot repeat: a mutating
+[project tool](/tools), or a value the agent read off the screen, typed after
+seeing a screenshot, or worked out from the date. `step.cache.derived` names
+the rule: `whole-node` (a control's whole name or text), `minted-token` (a
+code or id the app showed as its own word), `date`, or `pixels`. Values from
+the instruction or `params`, or ones the agent composed itself, replay.
+
+## What a recording holds
+
+Each entry is a JSON file under `.e2e/cache/`, named by a hash of its key.
+It contains actions, target descriptions, typed text, a step summary, and
+checks for the final state. Those checks include the path and up to eight
+controls that appeared during the step.
+
+Entries contain no prompts, conversations, screenshots, observed trees, or
+secret values. A secret fill stores only the secret's name. Other typed
+values are stored verbatim, so review recordings before sharing them; see
+[cache trust](/security#cache-trust).
+
+## Modes
+
+| Mode | Replays | Records | Default |
+| --- | --- | --- | --- |
+| `'read-write'` | yes | yes | locally |
+| `'read-only'` | yes | no | in CI when `cache` is unset |
+| `'off'` | no | no | `--no-cache` for one run |
+
+```ts title="e2e.config.ts fragment"
+cache: 'off',
+// or
+cache: { mode: 'read-write', dir: '.e2e/cache' },
+```
+
+`--no-cache` overrides the config. To let CI update recordings, explicitly
+set `cache: 'read-write'`. In `read-only` mode, stale entries remain until
+a read-write run replaces them.
+
+A custom `cache.store` replaces the file store and does not default to
+read-only in CI. `e2e explore` and `e2e mcp` do not use the cache. See the
+[config reference](/reference/config#cache).
+
+## Commit the replay cache
+
+<span id="commit-your-traces" />
+
+`e2e init` adds `.e2e/cache/` to `.gitignore`, so every machine records its
+own and CI starts with none. To share replays, remove that line and commit
+the directory. CI stays `read-only` unless you set `cache: 'read-write'`.
+Treat committed entries as test data and review them in the pull request
+that re-recorded them.
+
+Local runs leave a committed directory clean unless something changed. A
+replayed step never rewrites its entry, and a live re-recording with the same
+actions, paths, and anchors is not written: the summary, the measured end
+wait, and the rule that flagged a typed run-time value differ between two
+live runs of the same flow and do not count. A file changes when:
+
+- a step records for the first time,
+- the agent finishes a step after a hand-off, re-recording what it saw,
+- a step fails after replaying, or ends in an `end-mismatch` the agent had
+  to repair, which deletes the entry,
+- no verification step confirmed the entry by the end of the attempt, which
+  deletes it.
+
+A replay that still finds every control refreshes nothing, so an entry for a
+drifting UI is repaired only once a control can no longer be found.
+
+## Fail on a stale recording
+
+A committed recording that stops replaying is quiet by default: the agent
+takes the step over, the test passes, and every CI run pays for the model
+calls until someone re-records. `--strict-cache` (or `cache: { strict: true }`)
+makes it loud. A step whose recording exists but no longer replays fails
+with `REPLAY_STALE` and makes no model call:
+
+<CodeGroup>
+```bash npm
+CI=1 npx e2e run --strict-cache
+```
+
+```bash pnpm
+CI=1 pnpm exec e2e run --strict-cache
+```
+
+```bash bun
+CI=1 bunx e2e run --strict-cache
+```
+</CodeGroup>
+
+The reasons that fail are the ones that mean the app or the entry changed:
+`invalid-entry`, `wrong-context`, `target-not-found`, `target-ambiguous`,
+`viewport-changed`, `action-failed`, `action-uncertain`, and `end-mismatch`.
+A step with no recording (`no-entry`), a retry, a `gap`, and a `truncated`
+recording still run live. `REPLAY_STALE` exits 2 and is not retried, since a
+retry never replays and would pass live. The entry stays where it is, even
+in `read-write` mode, so every strict run fails on it until it is re-recorded:
+run the test in `read-write` mode without the flag, and with `cache.strict`
+off when the config set it, then commit the changed entry; a custom
+`cache.store` receives the new entry from that run instead. The message
+names the knob that turned strict on and where the entry lives: the cache
+directory, or the configured `cache.store`. A store whose read fails is not
+a stale recording; the step runs live.
+
+## Inspect the cache
+
+<CodeGroup>
+```bash npm
+npx e2e cache ls      # one row per entry: test, target, instruction digest, age, actions
+npx e2e cache stats   # entry count and size
+npx e2e cache clear   # delete every entry
+```
+
+```bash pnpm
+pnpm exec e2e cache ls      # one row per entry: test, target, instruction digest, age, actions
+pnpm exec e2e cache stats   # entry count and size
+pnpm exec e2e cache clear   # delete every entry
+```
+
+```bash bun
+bunx e2e cache ls      # one row per entry: test, target, instruction digest, age, actions
+bunx e2e cache stats   # entry count and size
+bunx e2e cache clear   # delete every entry
+```
+</CodeGroup>
+
+<Note>
+The `cached` percentage in AI usage is the provider's prompt cache. It
+reduces the cost of a model call. The replay cache described here skips the
+call entirely.
+</Note>
+
+<CardGroup cols={2}>
+  <Card title="Config reference" icon="sliders" href="/reference/config#cache">
+    `mode`, `store`, `dir`.
+  </Card>
+  <Card title="CLI reference" icon="terminal" href="/reference/cli#e2e-cache">
+    `e2e cache ls`, `stats`, `clear`.
+  </Card>
+</CardGroup>

package/docs/ci.mdx

@@ -0,0 +1,210 @@
+---
+title: Continuous integration
+description: Run tests on pull requests, upload reports, and configure model credentials in CI.
+---
+
+Run deterministic and agent tests in the same CI job. This workflow assumes
+your config includes a [command](/starting-your-app) to start the app and
+uses AI Gateway for model calls:
+
+```yaml title=".github/workflows/e2e.yml"
+name: e2e
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  e2e:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: pnpm/action-setup@9fd676a19091d4595eefd76e4bd31c97133911f1 # v4.2.0
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 26
+          cache: pnpm
+      - run: pnpm install --frozen-lockfile
+
+      - name: Install browsers
+        run: npx playwright install chromium --with-deps
+
+      - name: Run e2e
+        run: npx e2e run --reporter list,junit
+        env:
+          APP_URL: http://127.0.0.1:3000
+          AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
+          E2E_USER_ADMIN_USERNAME: ${{ secrets.E2E_USER_ADMIN_USERNAME }}
+          E2E_USER_ADMIN_PASSWORD: ${{ secrets.E2E_USER_ADMIN_PASSWORD }}
+
+      - name: Upload report
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: e2e-report
+          path: |
+            .e2e/report.json
+            .e2e/junit.xml
+          if-no-files-found: warn
+
+      - name: Upload artifacts
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: e2e-artifacts
+          path: .e2e/artifacts
+          if-no-files-found: warn
+          retention-days: 7
+```
+
+The workflow pins actions to commit SHAs and installs dependencies from the
+lockfile. `npx e2e` runs the installed version.
+
+Artifacts upload on every run that is not cancelled, so a test that failed
+and then passed on retry still has its evidence.
+
+For iOS and Android suites, see [Mobile CI](/mobile-ci): GitHub Actions, EAS
+Workflows, Bitrise, and Codemagic.
+
+Configure your CI's test-report integration to read `.e2e/junit.xml`. It
+includes startup failures such as `APP_UNREACHABLE` in a `run` suite. Add
+the [GitHub reporter](/github) to post results on the pull request. Make the
+e2e job a required check in your repository settings if it should block merges.
+
+## What CI changes
+
+CI mode is on when `CI` is set to a value other than an empty string, `0`,
+or `false`. Use `CI=1 npx e2e run` to reproduce these defaults locally:
+
+| Setting | Local | CI |
+| --- | --- | --- |
+| `retries` | `0` | `1` |
+| `workers` | half your cores | `1` |
+| `cache` when unset | `read-write` | `read-only` |
+| `.only` | allowed | rejected with `ONLY_IN_CI` |
+| `command.reuseExisting` | honored | ignored, with a notice |
+
+Override worker and retry defaults with flags:
+
+<CodeGroup>
+```bash npm
+npx e2e run --workers 4 --retries 0
+```
+
+```bash pnpm
+pnpm exec e2e run --workers 4 --retries 0
+```
+
+```bash bun
+bunx e2e run --workers 4 --retries 0
+```
+</CodeGroup>
+
+Add `E2E_TELEMETRY_DISABLED: '1'` to the job's `env` to send no
+[telemetry](/telemetry#in-ci) from CI.
+
+## Sharding
+
+`--shard <index/total>` runs one slice of the suite per job. Every job runs
+the same command with its own index, and the same suite shards the same way
+on every machine. A shard that gets no test fails with `NO_TESTS`; add
+`--pass-with-no-tests` when a matrix may have more shards than tests. The
+[CLI reference](/reference/cli#flags) has the slicing rules.
+
+```yaml
+jobs:
+  e2e:
+    strategy:
+      fail-fast: false
+      matrix:
+        shard: [1, 2, 3]
+    steps:
+      # ...
+      - run: npx e2e run --shard ${{ matrix.shard }}/3 --reporter list,junit
+```
+
+Each shard writes its own `report.json` and `junit.xml`; upload them under a
+name that includes the shard index. Combining the documents into one is not
+implemented yet.
+
+## Rerunning failures
+
+`--last-failed` runs only the tests the previous run did not pass, read from
+`report.json` in the [`output`](/reference/config#output) directory (`.e2e`
+by default). The rerun empties `artifacts/` there when it starts, so
+upload the first run's artifacts before rerunning in the same job if you want
+both. The [CLI reference](/reference/cli#flags) lists what counts as not
+passed. Locally it is the loop after a red run:
+
+<CodeGroup>
+```bash npm
+npx e2e run
+npx e2e run --last-failed
+```
+
+```bash pnpm
+pnpm exec e2e run
+pnpm exec e2e run --last-failed
+```
+
+```bash bun
+bunx e2e run
+bunx e2e run --last-failed
+```
+</CodeGroup>
+
+In CI the report is a job artifact, not a file on the next runner; download
+it into `.e2e/` before the rerun step, or rerun the failed job whole.
+
+## Model credentials
+
+Pass the credential your configured provider needs, such as
+`AI_GATEWAY_API_KEY` in the workflow above. See [Models](/models)
+for provider setup. Tests that never use `agent` need no model credentials.
+
+Store keys in CI secrets and use spending limits where available. The code
+running in the job can read those keys, including test code and dependencies.
+GitHub withholds secrets from fork pull requests. Fully replayed action
+steps need no model call, but live actions and agent judgments still do.
+Run untrusted code without secrets or write tokens. See the
+[security model](/security).
+
+## Gating pull requests on agent steps
+
+A failed agent step fails the job like a failed `expect`. Keep the checks
+predictable by giving each `agent.act` one goal and checking its outcome
+with a locator assertion or `agent.assert`.
+
+To reuse verified actions in CI, [share the replay cache](/cache#commit-the-replay-cache).
+A cache hit avoids a model call for that action step. Agent judgments still
+run live, and a cache miss or hand-off uses the model.
+
+CI retries once by default. A test that passes only on retry is reported as
+flaky, so you can identify flows that need investigation.
+
+`--max-failures <n>` stops the run once that many tests have failed, so a
+broken deploy does not spend a model call on every remaining agent step. The
+tests not started are skipped with cause `failure-limit`, the ones running
+end as `interrupted`, and the job exits 1.
+
+## Exit codes
+
+Any non-zero exit fails the job. [Debugging a run](/debugging#read-the-failure)
+lists each code and whether a retry of the job can help.
+
+<CardGroup cols={2}>
+  <Card title="Pull request comments" icon="code-pull-request" href="/github">
+    One comment per run, edited in place on a rerun.
+  </Card>
+  <Card title="CLI reference" icon="terminal" href="/reference/cli">
+    Flags, reporters, and every exit code.
+  </Card>
+</CardGroup>

package/docs/coding-agents.mdx

@@ -0,0 +1,183 @@
+---
+title: Coding agents
+description: Give a coding agent the skill, a live app session, and reports it can use to write and debug tests.
+---
+
+`e2e init` installs a skill for writing and running tests, plus MCP
+configuration for inspecting the app. Coding agents use the CLI to run
+tests and the reports to investigate failures.
+
+## The skill
+
+<CodeGroup>
+```bash npm
+npx e2e init
+```
+
+```bash pnpm
+pnpm dlx e2e init
+```
+
+```bash bun
+bunx e2e init
+```
+</CodeGroup>
+
+| Path | Used by |
+| --- | --- |
+| `.agents/skills/e2e/` | Agents that read `.agents/skills` |
+| `.claude/skills/e2e/` | Claude Code; a symlink to `.agents/skills/e2e/` when both are installed |
+
+The copy holds `SKILL.md` and topic references; the link is the layout
+`npx skills add` produces, so one copy serves every agent. Run `init` again
+after upgrading e2e to refresh it.
+
+Without `init`, `npx skills add tester-army/e2e` installs the skill from the
+repository, and `npx e2e guide [topic]` prints it from the
+installed package.
+
+The package also ships these docs as `.mdx` pages in its `docs/` directory
+(`node_modules/e2e/docs` in a single-package project), so an agent can read
+any page offline. A link such as `/reference/cli` is `docs/reference/cli.mdx`.
+
+For agents that need an explicit pointer, add this line to your project
+instructions. `init` leaves those files unchanged:
+
+```md title="AGENTS.md"
+End-to-end tests use e2e; read .agents/skills/e2e/SKILL.md before writing or running one.
+```
+
+## How an agent looks at the app
+
+`e2e mcp` lets the coding agent open and interact with the app. Each agent,
+subagents included, opens its own session, so several can drive the app at
+once. Its `locate` tool checks a `screen.*` locator against the current app and returns
+the test code when exactly one element matches.
+
+`init` registers the server in `.mcp.json` for Claude Code and
+`.cursor/mcp.json` for Cursor. To register it with Claude Code by hand:
+
+<CodeGroup>
+```bash npm
+claude mcp add e2e -- npx e2e mcp
+```
+
+```bash pnpm
+claude mcp add e2e -- pnpm exec e2e mcp
+```
+
+```bash bun
+claude mcp add e2e -- bunx e2e mcp
+```
+</CodeGroup>
+
+See the [MCP server reference](/reference/mcp) for its tools, session rules,
+and flags. Run tests and read reports through the CLI.
+
+## How an agent writes a test
+
+The skill tells the coding agent to read the config and an existing test
+first. It uses `agent.act` for one goal at a time and checks the outcome
+with `expect`. Known fields and exact values can use `screen` directly:
+
+```ts title="tests/checkout.e2e.ts"
+import { test, expect, credentials } from 'e2e';
+
+test('a signed-in user reaches checkout', async ({ screen, app, agent }) => {
+  const user = credentials.user('customer');
+  await app.open('/login');
+  await screen.getByLabel('Email').fill(user.username);
+  await screen.getByLabel('Password').fill(user.password);
+  await screen.getByRole('button', 'Sign in').click();
+  await expect(screen.getByRole('heading', 'Welcome back')).toBeVisible();
+
+  await agent.act('Add the cheapest item on the page to the cart and open the cart');
+  await expect(screen.getByRole('heading', 'Your cart')).toBeVisible();
+});
+```
+
+The main rules are:
+
+- One `agent.act` per goal, followed by an `expect`. The assertion makes the
+  step's cache entry eligible for replay, and it is what fails when the agent
+  did the wrong thing.
+- No sleeps. Locators poll, actions wait for their target, and `expect`
+  retries until its timeout.
+- Secrets through `credentials.user(name)`, declared in the config, never
+  literal in a test.
+
+## How an agent runs it
+
+<CodeGroup>
+```bash npm
+npx e2e run tests/checkout.e2e.ts
+```
+
+```bash pnpm
+pnpm exec e2e run tests/checkout.e2e.ts
+```
+
+```bash bun
+bunx e2e run tests/checkout.e2e.ts
+```
+</CodeGroup>
+
+Run one file while iterating. For model configuration and credentials, see
+[Models](/models). Fix CLI, config, and collection errors before
+retrying a run.
+
+## How an agent reads the result
+
+Every run that gets to its tests writes `.e2e/report.json`. For a report that is easier to read,
+run:
+
+<CodeGroup>
+```bash npm
+npx e2e run tests/checkout.e2e.ts --reporter list,markdown
+```
+
+```bash pnpm
+pnpm exec e2e run tests/checkout.e2e.ts --reporter list,markdown
+```
+
+```bash bun
+bunx e2e run tests/checkout.e2e.ts --reporter list,markdown
+```
+</CodeGroup>
+
+This writes `.e2e/summary.md` and a page for each failed or flaky test under
+`.e2e/failures/`. Start with the failure page. It includes the failing line,
+steps, recent model turns, and the screen at failure.
+
+When reading JSON, check `run.errors` for run-level failures, then the last
+attempt of each failed result. Its `error`, `steps`, and `artifacts` show
+what failed and how far the test got. `--reporter json` also prints the JSON
+report to stdout.
+
+For model-call details, use `--ai-trace` and read the trace with
+[unbox-ai](/debugging#agent-steps). Avoid opening the raw trace in an agent's
+context; it repeats prompts and can be several megabytes.
+
+## How an agent treats the cache
+
+A verified `agent.act` can replay from `.e2e/cache/` on later runs. Use
+`--no-cache` to check the live agent behavior after changing a test or to
+investigate a replay failure. Pair it with `--ai-trace` to capture every
+model call in the flow.
+
+Do not edit recordings by hand. A failed read-write run removes implicated
+entries so a later verified run can replace them. A failure where no model
+answered implicates nothing and removes none. See
+[Caching agent steps](/cache).
+
+<CardGroup cols={2}>
+  <Card title="CLI reference" icon="terminal" href="/reference/cli">
+    init, guide, mcp, and run, flag by flag.
+  </Card>
+  <Card title="MCP server" icon="server" href="/reference/mcp">
+    Inspect the app before writing a test.
+  </Card>
+  <Card title="Debugging a run" icon="bug" href="/debugging">
+    The report, the artifacts, and the agent flags.
+  </Card>
+</CardGroup>