codeceptjs 4.2.0 → 4.2.1-beta.1

package/README.md

@@ -11,6 +11,7 @@
 | 🌐 Web    | Puppeteer  | [![Puppeteer Tests](https://github.com/codeceptjs/CodeceptJS/actions/workflows/puppeteer.yml/badge.svg)](https://github.com/codeceptjs/CodeceptJS/actions/workflows/puppeteer.yml)                  |
 | 🌐 Web    | WebDriver  | [![WebDriver Tests](https://github.com/codeceptjs/CodeceptJS/actions/workflows/webdriver.yml/badge.svg)](https://github.com/codeceptjs/CodeceptJS/actions/workflows/webdriver.yml)                  |
 | 🌐 Web    | Obscura    | [![Obscura Helper Tests](https://github.com/codeceptjs/CodeceptJS/actions/workflows/obscura.yml/badge.svg)](https://github.com/codeceptjs/CodeceptJS/actions/workflows/obscura.yml)               |
+| 🌐 Web    | Lightpanda | [![Lightpanda Helper Tests](https://github.com/codeceptjs/CodeceptJS/actions/workflows/lightpanda.yml/badge.svg)](https://github.com/codeceptjs/CodeceptJS/actions/workflows/lightpanda.yml)      |
 | 📱 Mobile | Appium     | [![Appium Tests - Android](https://github.com/codeceptjs/CodeceptJS/actions/workflows/appium_Android.yml/badge.svg)](https://github.com/codeceptjs/CodeceptJS/actions/workflows/appium_Android.yml) |
 
 # CodeceptJS [![Made in Ukraine](https://img.shields.io/badge/made_in-ukraine-ffd700.svg?labelColor=0057b7)](https://stand-with-ukraine.pp.ua)
@@ -44,6 +45,7 @@ CodeceptJS uses **Helper** modules to provide actions to `I` object. Currently,
 - [**Puppeteer**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Puppeteer.md) - uses Google Chrome's Puppeteer for fast headless testing.
 - [**WebDriver**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/WebDriver.md) - uses [webdriverio](http://webdriver.io/) to run tests via WebDriver or Devtools protocol.
 - [**Obscura**](https://codecept.io/helpers/Obscura) - drives the lightweight Obscura browser through Chrome DevTools Protocol. See [Alternative Browser Engines](https://codecept.io/alternative-browsers).
+- [**Lightpanda**](https://codecept.io/helpers/Lightpanda) - drives the Lightpanda headless browser through Chrome DevTools Protocol. See [Alternative Browser Engines](https://codecept.io/alternative-browsers).
 - [**Appium**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Appium.md) - for **mobile testing** with Appium
 - [**Detox**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Detox.md) - This is a wrapper on top of Detox library, aimed to unify testing experience for CodeceptJS framework. Detox provides a grey box testing for mobile applications, playing especially well for React Native apps.
 

package/bin/codecept.js

@@ -78,8 +78,16 @@ program
   .command('init [path]')
   .description('Creates dummy config in current dir or [path]')
   .option('-y, --yes', 'skip prompts and use defaults (Playwright + chromium, BASE_URL env for url)')
+  .option('--agent <name>', 'set up an AI coding agent: claude, codex, cursor, opencode')
   .action(commandHandler('../lib/command/init.js'))
 
+program
+  .command('init:agent [agent]')
+  .alias('init-agent')
+  .description('Set up CodeceptJS MCP server and skills for a coding agent: claude, codex, cursor, opencode')
+  .option('-y, --yes', 'skip prompts, requires [agent]')
+  .action(commandHandler('../lib/command/initAgent.js'))
+
 program
   .command('check')
   .option(commandFlags.config.flag, commandFlags.config.description)
@@ -178,6 +186,7 @@ program
   .option('-p, --plugins <k=v,k2=v2,...>', 'enable plugins, comma-separated')
   .option('--shuffle', 'Shuffle the order in which test files run')
   .option('--shard <index/total>', 'run only a fraction of tests (e.g., --shard 1/4)')
+  .option('--last-failed', 'run only tests that failed in the previous run')
 
   // mocha options
   .option('--colors', 'force enabling of colors')
@@ -207,8 +216,10 @@ program
   .option(commandFlags.config.flag, commandFlags.config.description)
   .option('-g, --grep <pattern>', 'only run tests matching <pattern>')
   .option('-i, --invert', 'inverts --grep matches')
+  .option('--last-failed', 'run only tests that failed in the previous run')
   .option('-o, --override [value]', 'override current config options')
   .option('--suites', 'parallel execution of suites not single tests')
+  .option('--serial <pattern>', 'run tests matching <pattern> one by one after parallel workers finish')
   .option('--by <strategy>', 'test distribution strategy: "test" (pre-assign individual tests), "suite" (pre-assign test suites), or "pool" (dynamic distribution for optimal load balancing, recommended)')
   .option(commandFlags.debug.flag, commandFlags.debug.description)
   .option(commandFlags.verbose.flag, commandFlags.verbose.description)

package/docs/agents.md

@@ -25,7 +25,15 @@ CodeceptJS is token-efficient: it stores HTML, ARIA, logs, and HTTP request data
 
 ## Essential Setup
 
-Two things make agent testing work: the **skills** that teach the agent CodeceptJS, and the **MCP server** that lets it drive the browser. Set both up once, from your project directory.
+Two things make agent testing work: the **skills** that teach the agent CodeceptJS, and the **MCP server** that lets it drive the browser. Set both up once, from your project directory:
+
+```bash
+npx codeceptjs init:agent claude
+```
+
+Use `codex`, `cursor`, or `opencode` instead of `claude` for other agents, or omit the name to pick from a list. `npx codeceptjs init` offers the same step at the end.
+
+### Manual setup
 
 Install the skills:
 
@@ -165,6 +173,23 @@ After a failed run, the agent reads every trace under `output/`, clusters failur
 
 If the fix held, the PR goes green. If it didn't, every edit is rolled back with `git checkout` and the report says which patterns the agent couldn't safely handle. No half-applied fixes left behind, no `retries: 3` masking the problem.
 
+## Assertions in plain language
+
+Some outcomes are hard to pin to a locator: "checkout form has all required fields", "success message is shown". Instead of building a fragile chain of `see*` checks, the agent can write the expected outcome as a statement with the [Decision helper](/assertions#decision-assertions):
+
+```js
+I.decide([
+  'order summary lists the purchased items',
+  'success message is shown',
+])
+```
+
+The agent runs the statement on the live page like any other command and keeps it only if it passes. The test then checks it on every run.
+
+To let the agent use decisions, [configure a decision model](/ai#configure-decision-models) and enable the Decision helper.
+
+Decision models like [Jev](https://openrouter.ai/typesafe/jev-1.13) are built for this. They answer in a fraction of a second, cost a fraction of a cent per request, and return a probability instead of free text. That makes them fast and cheap enough to run on every CI build, and predictable enough to keep in a test.
+
 ## Skills bundle
 
 Skills teach the agent best practices for using CodeceptJS. Plug them in when you develop tests with agents, and update them regularly to ensure you use CodeceptJS in the most effective way.

package/docs/ai.md

@@ -23,6 +23,7 @@ CodeceptJS AI can do the following:
 
 - 🏋️‍♀️ **assist writing tests** in `pause()` or interactive shell mode
 - 🚑 **self-heal failing tests** (can be used on CI)
+- ⚖️ **assert statements in plain language** with [decision models](#decision-assertions)
 
 ![](/img/fill_form.gif)
 
@@ -338,6 +339,62 @@ Run tests with both AI and analyze enabled:
 npx codeceptjs run --ai
 ```
 
+## Decision Assertions
+
+Some checks are hard to express with locators: "checkout form has all required fields", "success message is shown". The [Decision helper](/helpers/Decision) asserts them in plain language:
+
+```js
+I.decide([
+  'checkout form has all required fields',
+  'submit button enabled',
+])
+I.decideVisually('sidebar is shown')
+```
+
+It uses a [decision model](https://openrouter.ai/models?output_modalities=decisions) like [Jev](https://openrouter.ai/typesafe/jev-1.13) instead of a chat model. A decision model reads the page and returns the probability that a statement is true. It is fast, costs a fraction of a cent per request, and gives a probability instead of free text, so a step passes or fails on a confidence threshold you set.
+
+Decision models are separate from `ai.model`: decisions don't need `ai.model` or the `--ai` flag. CodeceptJS calls them with AI SDK [`experimental_decide`](https://ai-sdk.dev/docs/ai-sdk-core/decisions).
+
+### Configure Decision Models
+
+Decision models are served by the [OpenRouter Decisions API](https://openrouter.ai/models?output_modalities=decisions). Install the [OpenRouter provider](https://ai-sdk.dev/providers/community-providers/openrouter) `@openrouter/ai-sdk-provider`, set `OPENROUTER_API_KEY`, and create the decision model with `evaluationModel()` in `codecept.conf.js`:
+
+```js
+import { createOpenRouter } from '@openrouter/ai-sdk-provider'
+
+const openrouter = createOpenRouter({
+  apiKey: process.env.OPENROUTER_API_KEY,
+})
+
+export default {
+  ai: {
+    decisionModel: {
+      model: openrouter.evaluationModel('typesafe/jev-1.13'),
+      confidence: 0.8,
+    },
+  },
+  helpers: {
+    Playwright: { url: 'http://localhost' },
+    Decision: {},
+  },
+}
+```
+
+`model` is used by `I.decide`.
+
+### Visual Decision Model
+
+`I.decideVisually` sends a screenshot, so it needs a separate decision model with image input. Set it as `visualModel`:
+
+```js
+decisionModel: {
+  model: openrouter.evaluationModel('typesafe/jev-1.13'),
+  visualModel: openrouter.evaluationModel('cloudflare/clef'),
+},
+```
+
+See [Decision Assertions](/assertions#decision-assertions) for all options and usage.
+
 ## Advanced Configuration
 
 AI prompts and HTML compression can be configured inside `ai` section of `codecept.conf` file:

package/docs/alternative-browsers.md

@@ -6,153 +6,180 @@ title: Alternative Browser Engines
 # Alternative Browser Engines
 
 ::: warning Experimental
-The `CDPBrowser`, `Obscura`, and `Kitesurf` helpers are experimental in CodeceptJS 4.2. Pin browser versions in CI and retain Playwright or WebDriver coverage for compatibility-critical tests.
+The `Obscura`, `Lightpanda` and `Kitesurf` helpers are experimental. Keep a Playwright or WebDriver job for compatibility-critical tests.
 :::
 
-Playwright and Puppeteer drive full Chromium — the most accurate way to test what users see.
-But a new class of lightweight, agent-era browsers has appeared, and CodeceptJS can drive them
-through the `CDPBrowser` helper family:
-
-- **[Obscura](https://github.com/h4ckf0r0day/obscura)** — an open-source Rust browser with a real
-  V8 engine. From v0.2.0, the default release build also renders — real layout, computed styles,
-  and screenshots — with `-no-render` builds still available for pure-speed, nothing-painted
-  scraping mode. Release archives are available for Linux, macOS, and Windows.
-- **[Kitesurf](https://blog.cloudflare.com/kitesurf/)** — Cloudflare's browser that runs in V8
-  isolates on Cloudflare Workers, with a real layout and rendering pipeline. Cloud-only,
-  free in beta, planned to be open-sourced.
-
-Both speak Chrome DevTools Protocol. CodeceptJS drives them with raw CDP — one round-trip per
-action, no stale element handles — which is why suites on these browsers run fast and never hang
-on navigation races.
-
-## When are they better than Playwright?
-
-**Smoke suites where startup and execution time matter.** Obscura is distributed as a standalone
-binary and is designed for lightweight browser automation. Benchmark it against your own pages and
-CI environment before choosing it for a PR gate.
-
-**Massive parallel scale.** Kitesurf sessions are Cloudflare Workers — they spawn in about a
-second, cost nothing while idle, and there is no practical ceiling on how many you run at once.
-Combined with `run-workers`, every worker acquires its own cloud browser:
-
-    // codecept.conf.js — each worker independently loads the config,
-    // so each one gets its own Kitesurf session automatically
-    export const config = {
-      helpers: {
-        Kitesurf: {
-          url: 'https://staging.myapp.com',
-        },
-      },
-    }
-
-    npx codeceptjs run-workers 16
-
-Sixteen cloud browsers, zero local resources, feedback in the time of your slowest test.
-Scale the number up as far as your suite can split — the browsers are no longer
-the bottleneck, and your CI runner only coordinates.
-
-**Testing the DOM, not the pixels.** Most functional assertions — text appears, form submits,
-redirect happens, cookie is set — do not need a GPU raster pipeline. Obscura executes your
-app's real JavaScript in real V8; it only skips painting. For API-adjacent flows
-(login → dashboard data appears), that is exactly the right amount of browser.
-
-**Constrained environments.** ARM CI runners, thin containers, air-gapped machines:
-a static binary with no system dependencies goes where Chromium will not.
-
-**Optional stealth builds.** Obscura publishes separate `-stealth` archives. Treat their behaviour
-as an Obscura capability rather than a browser-compatibility guarantee from CodeceptJS.
-
-## When to stay with Playwright
-
-- Anything visual: visual regression, PDF (neither helper exposes PDF output). Obscura's v0.2.0+
-  rendering/CSS engine is new and independently implemented — expect edge cases and gaps versus a
-  real browser, especially around inherited properties and less common computed-style values.
-- Visibility semantics on `-no-render` Obscura builds (and v0.1.x): every element reports as
-  visible — `seeElement`/`dontSeeElement` throw and point you to `seeElementInDOM`. On v0.2.0+
-  default builds, `CDPBrowser` detects the real layout engine per binary and visibility works
-  normally.
-- Complex input: drag-and-drop, hover chains, file uploads, iframes, multi-tab, service workers.
-- Cross-browser coverage (Firefox, WebKit).
-- Testing local apps with Kitesurf: the cloud browser must reach your app; use a tunnel
-  (`cloudflared tunnel --url http://localhost:3000`) or a deployed environment.
-
-## Configuration
-
-CodeceptJS 4.2 is tested in CI with Obscura 0.2.2. Obscura 0.2.x is recommended; 0.1.x and
-`-no-render` builds operate without layout, visibility assertions, or screenshots. See
-[Installation](/installation#obscura-experimental) for platform-specific archive names.
-
-    helpers: {
-      Obscura: {
-        url: 'http://localhost:3000',
-      },
-    }
-
-    helpers: {
-      Kitesurf: {
-        url: 'https://staging.myapp.com',
-        accountId: process.env.CF_ACCOUNT_ID,
-        apiToken: process.env.CF_API_TOKEN,
-      },
-    }
-
-Any other CDP endpoint works through the base helper:
-
-    helpers: {
-      CDPBrowser: {
-        url: 'http://localhost:3000',
-        endpoint: 'http://127.0.0.1:9222',
-      },
-    }
-
-### Obscura's three connection modes
-
-Obscura manages its own `obscura serve` process, the same way Playwright manages its own browser
-process — there is nothing to start by hand in the common case:
-
-- **Self-launch (default)** — leave `endpoint` unset. The helper resolves a binary
-  (`binaryPath` in the config, then `OBSCURA_PATH`, then `obscura` on `PATH`), spawns
-  `obscura serve` on a free port, and kills it when the run ends. Not setting `port` is
-  intentional: a free port is picked automatically, which is what makes `run-workers`
-  collision-free — every worker gets its own instance without any config.
-
-      helpers: {
-        Obscura: {
-          url: 'http://localhost:3000',
-          binaryPath: '/usr/local/bin/obscura', // optional override, like Playwright's executablePath
-        },
-      }
-
-- **Attach** — set `endpoint` explicitly to connect to an Obscura instance you manage yourself
-  (already running locally, in a container, or on a remote host). The helper only connects; it
-  never spawns or kills anything.
-
-      helpers: {
-        Obscura: {
-          url: 'http://localhost:3000',
-          endpoint: 'http://127.0.0.1:9222',
-        },
-      }
-
-- **Courtesy-attach** — only relevant when `endpoint` is unset and no binary can be resolved
-  either. If something is already answering on the conventional `http://127.0.0.1:9222`, the
-  helper attaches to it (and, again, never kills it) instead of failing outright. This exists so
-  that "start Obscura by hand and just run the tests" keeps working without any config, while
-  self-launch is still the default for everyone else.
-
-## Capability matrix
-
-| | Playwright | Obscura | Kitesurf |
-|---|---|---|---|
-| Real JS execution (V8) | yes | yes | yes |
-| Layout / getBoundingClientRect | yes | yes (v0.2.0+ default builds); synthetic on `-no-render`/v0.1.x | yes |
-| Screenshots | yes | yes (v0.2.0+ default builds); no on `-no-render`/v0.1.x | yes |
-| Visibility assertions | yes | yes (v0.2.0+ default builds); no, DOM-presence only, on `-no-render`/v0.1.x | yes |
-| Screencast / video (`screencast` plugin) | yes — WebM via `page.screencast`, with caption burn-in | yes — APNG via CDP `Page.startScreencast`, assembled in-process (v0.2.0+ default builds; verified PNG frames on the live server); no caption burn-in | untested |
-| Startup model | local browser process | standalone local binary | remote cloud session |
-| Parallel scale | machine-bound | machine-bound (light) | near-unlimited (cloud) |
-| Where it runs | local/grid | local | Cloudflare only |
-| License / cost | open source | Apache-2.0 | proprietary, free beta |
-
-See helper reference pages: [CDPBrowser](/helpers/CDPBrowser), [Obscura](/helpers/Obscura),
-[Kitesurf](/helpers/Kitesurf).
+Playwright drives full Chromium, Firefox and WebKit: the most accurate way to test what users see, at the cost of a heavy browser per worker. Alternative engines trade part of that accuracy for speed, a small footprint, or cloud scale. They are a good fit for functional tests (text appears, form submits, redirect happens, cookie is set) and a poor one for anything visual.
+
+Your tests do not change. `I.amOnPage`, `I.click`, `I.fillField`, `I.see`, `I.seeElement` and the rest of the `I.*` web API work the same; you only swap the helper in the config. All engines below speak Chrome DevTools Protocol, so there is no `npx playwright install` step.
+
+Compared to Playwright, none of them supports iframes (`switchTo`), multiple tabs, popups, `dragAndDrop`, `moveCursorTo`, or Playwright-only APIs such as `usePlaywrightTo` and `mockRoute`. Tag scenarios that need those and skip them on the alternative engine:
+
+```sh
+npx codeceptjs run --grep @playwright-only --invert
+```
+
+To see which `I.*` actions the configured engine supports:
+
+```sh
+npx codeceptjs list
+```
+
+## Obscura
+
+[Obscura](https://github.com/h4ckf0r0day/obscura) is a lightweight open-source browser with a real V8 engine and its own rendering engine, distributed as a single binary for Linux, macOS and Windows. Unlike Lightpanda it renders: layout, visibility checks, screenshots and the `screencast` plugin work.
+
+Limitations:
+
+- `attachFile` does not upload files.
+- Clicks inside nested shadow DOM do not work.
+- `focus` and `blur` have no effect.
+- Clipboard actions do not work.
+- A `<select multiple>` submits only its first selected option.
+- An element inside a `visibility: hidden` parent is reported as visible.
+- Some computed styles (`cursor`, `user-select`) differ from Chromium.
+- Rich text editors TinyMCE, Trix, Monaco and ProseMirror do not work.
+
+### Setup
+
+Install Obscura following the [official instructions](https://github.com/h4ckf0r0day/obscura#installation) and put `obscura` on `PATH`. Then enable the helper:
+
+```js
+// codecept.conf.js
+export const config = {
+  helpers: {
+    Obscura: {
+      url: 'http://localhost:3000',
+    },
+  },
+}
+```
+
+If the binary is not on `PATH`, set `binaryPath` in the config or the `OBSCURA_PATH` environment variable.
+
+### Usage
+
+```sh
+npx codeceptjs run
+```
+
+The helper starts `obscura serve` on a free port before the first test and stops it after the last one, like Playwright manages Chromium. Each worker gets its own instance:
+
+```sh
+npx codeceptjs run-workers 8
+```
+
+To connect to an Obscura you started yourself, set `endpoint`; the helper then never starts or stops it:
+
+```sh
+obscura serve --port 9222 --allow-private-network --allow-file-access
+```
+
+```js
+Obscura: {
+  url: 'http://localhost:3000',
+  endpoint: 'http://127.0.0.1:9222',
+}
+```
+
+All options: [Obscura helper reference](/helpers/Obscura).
+
+## Lightpanda
+
+[Lightpanda](https://lightpanda.io) is an open-source headless browser built from scratch for automation. It runs your app's JavaScript in V8 but never paints anything, which makes it start in milliseconds and use a fraction of Chromium's memory. It computes enough layout for `seeElement`, `waitForVisible` and other visibility checks.
+
+Limitations:
+
+- No screenshots: `saveScreenshot`, failure screenshots and the `screencast` plugin do not work.
+- No scrolling: `scrollTo` and `scrollPageToBottom` have no effect.
+- `clickXY` does not work.
+- Clipboard actions do not work.
+- Computed styles cover `display`, `visibility` and `opacity`; other CSS properties are unreliable.
+- Rich text editors TinyMCE, Trix and Monaco do not work.
+- Linux and macOS only; on Windows use WSL2.
+- Licensed under AGPL-3.0. It runs as a separate process, so it does not affect the license of your tests.
+
+### Setup
+
+Install Lightpanda following the [official instructions](https://lightpanda.io/docs/open-source/installation) and put `lightpanda` on `PATH`. Then enable the helper:
+
+```js
+// codecept.conf.js
+export const config = {
+  helpers: {
+    Lightpanda: {
+      url: 'http://localhost:3000',
+    },
+  },
+}
+```
+
+If the binary is not on `PATH`, set `binaryPath` in the config or the `LIGHTPANDA_PATH` environment variable.
+
+### Usage
+
+```sh
+npx codeceptjs run
+```
+
+The helper starts `lightpanda serve` on a free port with telemetry turned off, and stops it after the last test. Each worker gets its own instance:
+
+```sh
+npx codeceptjs run-workers 8
+```
+
+To connect to a Lightpanda you started yourself, set `endpoint`; the helper then never starts or stops it:
+
+```sh
+LIGHTPANDA_DISABLE_TELEMETRY=true lightpanda serve --host 127.0.0.1 --port 9222
+```
+
+```js
+Lightpanda: {
+  url: 'http://localhost:3000',
+  endpoint: 'http://127.0.0.1:9222',
+}
+```
+
+All options: [Lightpanda helper reference](/helpers/Lightpanda).
+
+## Kitesurf
+
+[Kitesurf](https://blog.cloudflare.com/kitesurf/) is Cloudflare's browser running on Cloudflare Workers, with a real layout and rendering pipeline. Sessions start in about a second and nothing runs on your machine, so the number of parallel browsers is limited only by how far your suite splits.
+
+Limitations:
+
+- Cloud only, currently a free beta.
+- The browser cannot reach `localhost`: test a deployed environment or expose your app with a tunnel such as `cloudflared tunnel --url http://localhost:3000`.
+
+### Setup
+
+Enable [Cloudflare Browser Run](https://developers.cloudflare.com/browser-run/) and create an API token with the **Browser Rendering → Edit** permission. Export the credentials:
+
+```sh
+export CF_ACCOUNT_ID=your-account-id
+export CF_API_TOKEN=your-api-token
+```
+
+Then enable the helper:
+
+```js
+// codecept.conf.js
+export const config = {
+  helpers: {
+    Kitesurf: {
+      url: 'https://staging.myapp.com',
+    },
+  },
+}
+```
+
+### Usage
+
+Every worker opens its own cloud session:
+
+```sh
+npx codeceptjs run-workers 16
+```
+
+All options: [Kitesurf helper reference](/helpers/Kitesurf).

package/docs/assertions.md

@@ -397,6 +397,79 @@ If two checks failed, the scenario fails with a single aggregated message like:
 expected soft assertions '[expected web application to include "You must accept the terms", expected element (.summary-error) to be visible]' to be empty
 ```
 
+## Decision Assertions
+
+Some checks are hard to express with locators: "checkout form has all required fields", "success message is shown", "sidebar is shown". The [Decision helper](/helpers/Decision) asserts such statements in plain language with a decision model.
+
+A [decision model](https://openrouter.ai/models?output_modalities=decisions), like [Jev](https://openrouter.ai/typesafe/jev-1.13), does not generate text. It reads the page and returns the probability that a statement is true. This makes it a good fit for assertions:
+
+- **Fast.** One request returns in a fraction of a second, so a decision step runs about as fast as a regular browser step.
+- **Cost-efficient.** A request costs a fraction of a cent. You can run decision assertions in every CI build.
+- **Reliable.** The answer is a probability, not free text. There is nothing to parse, and you choose how confident the model must be for the step to pass.
+
+Configure the decision model in `ai.decisionModel` as described in [Configure Decision Models](/ai#configure-decision-models), and enable the helper next to your browser helper:
+
+```js
+helpers: {
+  Playwright: { url: 'http://localhost' },
+  Decision: {},
+}
+```
+
+`ai.decisionModel` accepts:
+
+| Option | Default | Description |
+|---|---|---|
+| `model` | | decision model for `I.decide` |
+| `visualModel` | | decision model with image input for `I.decideVisually` |
+| `confidence` | `0.7` | minimal probability for a statement to pass |
+| `timeout` | `15000` | request timeout in ms |
+| `maxLength` | `12000` | maximal length of ARIA snapshot or HTML sent to the model |
+
+Any model from the [OpenRouter decision models](https://openrouter.ai/models?output_modalities=decisions) list can be used with `openrouter.evaluationModel()`. Decisions don't need the `--ai` flag, and `ai.model` is not required.
+
+Then assert statements about the current page:
+
+```js
+I.decide('top level navigation is available')
+
+I.decide([
+  'checkout form has all required fields',
+  'success message is shown',
+  'submit button enabled',
+  'cancel button present',
+])
+
+I.decideVisually('sidebar is shown')
+```
+
+`I.decide` sends the page URL, title and ARIA snapshot to the model. A statement passes when its probability reaches `confidence`. A list of statements is checked in one request, and every statement must pass. The failure message lists the statements that failed, with their probabilities:
+
+```
+expected page to satisfy "success message is shown" (12%) with confidence of 70%
+```
+
+`I.decideVisually` also sends a screenshot, so it needs a [visual decision model](/ai#visual-decision-model) with image input, like [Clef](https://openrouter.ai/cloudflare/clef), set as `visualModel`. Visual decisions are experimental.
+
+To turn decisions off without removing the helper, set its `mode`:
+
+| Mode | Behavior |
+|---|---|
+| `assert` | default, fails the step when a statement is not confirmed |
+| `report` | requests the model but never fails, even on API errors; results and errors are added to the step as a comment |
+| `skip` | does not request the model, every decision passes |
+
+```js
+helpers: {
+  Playwright: { url: 'http://localhost' },
+  Decision: { mode: process.env.CI ? 'assert' : 'skip' },
+}
+```
+
+A failed decision is not retried by the [retryFailedStep](/plugins/retryFailedStep) plugin: asking again would cost another request and return the same answer. Only connection errors and timeouts are retried.
+
+Use decision assertions for what a page means, and built-in assertions for exact values. `I.see('Total: $42.00')` is still the right check for a price.
+
 ## Choosing an Approach
 
 | You want to check… | Use |
@@ -412,4 +485,5 @@ expected soft assertions '[expected web application to include "You must accept
 | A matcher the above do not cover | `grab*` + `chai` / `jest` / `node:assert` |
 | A **reusable, project-specific** check | [Custom helper](/custom-helpers) with `see*` method using `codeceptjs/assertions` |
 | Many independent checks in one run | `hopeThat` from `codeceptjs/effects` |
+| A statement that is hard to express with locators | [Decision helper](#decision-assertions) — `I.decide`, `I.decideVisually` |
 | Hiding values from logs | `secret()` |

package/docs/bootstrap.md

@@ -10,6 +10,8 @@ you can use the `bootstrap` and `teardown` config. Use it to start and stop a we
 
 When using the [parallel execution](/parallel) mode, there are two additional hooks available; `bootstrapAll` and `teardownAll`. See [bootstrapAll & teardownAll](#bootstrapall-teardownall) for more information.
 
+> To start and stop the application under test, use the [`webServer`](/configuration#web-server) option instead of writing bootstrap code.
+
 
 ### Example: Bootstrap & Teardown