codeceptjs 4.0.0-rc.2 → 4.0.0-rc.20

package/docs/debugging.md

@@ -0,0 +1,332 @@
+---
+permalink: /debugging
+title: Debugging Tests
+---
+
+# Debugging Tests
+
+CodeceptJS provides powerful tools for debugging your tests at every level: from verbose output to interactive breakpoints with step-by-step execution.
+
+## Output Verbosity
+
+When something goes wrong, start by increasing the output level:
+
+```bash
+npx codeceptjs run --steps     # print each step
+npx codeceptjs run --debug     # print steps + debug info
+npx codeceptjs run --verbose   # print everything
+```
+
+| Flag | What you see |
+|------|-------------|
+| `--steps` | Each executed step (`I click`, `I see`, etc.) |
+| `--debug` | Steps + helper/plugin info, URLs, loaded config |
+| `--verbose` | Everything above + internal promise queue, retries, timeouts |
+
+We recommend **always using `--debug`** when developing tests.
+
+For full internal logs, use the `DEBUG` environment variable:
+
+```bash
+DEBUG=codeceptjs:* npx codeceptjs run
+```
+
+You can narrow it down to specific modules:
+
+```bash
+DEBUG=codeceptjs:recorder npx codeceptjs run    # promise chain only
+DEBUG=codeceptjs:pause npx codeceptjs run       # pause session only
+```
+
+## Interactive Pause
+
+The most powerful debugging tool in CodeceptJS is **interactive pause**. It stops test execution and opens a live shell where you can run steps directly in the browser.
+
+Add `pause()` anywhere in your test:
+
+```js
+Scenario('debug checkout', ({ I }) => {
+  I.amOnPage('/products')
+  I.click('Add to Cart')
+  pause()  // <-- test stops here, shell opens
+  I.click('Checkout')
+})
+```
+
+When the shell opens, you'll see:
+
+```
+ Interactive shell started
+ Use JavaScript syntax to try steps in action
+ - Press ENTER to run the next step
+ - Press TAB twice to see all available commands
+ - Type exit + Enter to exit the interactive shell
+ - Prefix => to run js commands
+```
+
+### Available Commands
+
+| Input | What it does |
+|-------|-------------|
+| `click('Login')` | Runs `I.click('Login')` — the `I.` prefix is added automatically |
+| `see('Welcome')` | Runs `I.see('Welcome')` |
+| `grabCurrentUrl()` | Runs a grab method and prints the result |
+| `=> myVar` | Evaluates JavaScript expression |
+| *ENTER* (empty) | Runs the next test step and pauses again |
+| `exit` or `resume` | Exits the shell and continues the test |
+| *TAB TAB* | Shows all available I.* methods |
+
+You can also pass variables into the shell:
+
+```js
+const userData = { name: 'John', email: 'john@test.com' }
+pause({ userData })
+// in shell: => userData.name
+```
+
+### Using Pause as a Stop Point
+
+`pause()` works as a **breakpoint** in your test. Place it before a failing step to inspect the page state:
+
+```js
+Scenario('fix login bug', ({ I }) => {
+  I.amOnPage('/login')
+  I.fillField('Email', 'user@test.com')
+  I.fillField('Password', 'secret')
+  pause()  // stop here, inspect the form before submitting
+  I.click('Sign In')
+  I.see('Dashboard')
+})
+```
+
+You can also add it in hooks:
+
+```js
+After(({ I }) => {
+  pause()  // pause after every test to inspect final state
+})
+```
+
+### Pause Modes
+
+`pause()` adapts to who's driving the test:
+
+- **TTY (humans)** — when `process.stdin` is a terminal (running `npx codeceptjs run --debug` yourself), the readline REPL described above opens.
+- **MCP server (agent-driven debug)** — the MCP server registers an in-process pause handler before running tests, so when `pause()` fires inside a `run_test` invocation, control yields back to the agent. The agent drives the REPL through the [`pause` MCP tool](/mcp#pause). The same `I` container the test uses runs the agent's code, so artifacts (URL, ARIA, HTML, screenshot, console, storage) are captured against the live page.
+
+## Pause Plugin
+
+For automated debugging without modifying test code, use the `pause` plugin. It pauses tests based on different triggers, controlled entirely from the command line. The default is `on=fail`.
+
+### Pause on Failure
+
+Automatically enters interactive pause when a step fails:
+
+```bash
+npx codeceptjs run -p pause
+# or, explicit:
+npx codeceptjs run -p pause:on=fail
+```
+
+This is the most common debug workflow — run your tests, and when one fails, you land in the interactive shell with the browser in the exact state of the failure. You can inspect elements, try different selectors, and figure out what went wrong.
+
+> The legacy `pauseOnFail` plugin still works as a deprecated alias.
+
+### Pause on Every Step
+
+Enters interactive pause at the start of the test. Use *ENTER* to advance step by step:
+
+```bash
+npx codeceptjs run -p pause:on=step
+```
+
+This gives you full step-by-step execution. After each step, you're back in the interactive shell where you can inspect the page before pressing ENTER to continue.
+
+### Pause on File (Breakpoint)
+
+Pauses when execution reaches a specific file:
+
+```bash
+npx codeceptjs run -p pause:on=file:path=tests/login_test.js
+```
+
+With a specific line number:
+
+```bash
+npx codeceptjs run -p pause:on=file:path=tests/login_test.js;line=43
+```
+
+This works like a breakpoint — the test runs normally until it hits a step defined at that file and line, then opens the interactive shell.
+
+### Pause on URL
+
+Pauses when the browser navigates to a matching URL:
+
+```bash
+npx codeceptjs run -p pause:on=url:pattern=/users/1
+```
+
+Supports `*` wildcards:
+
+```bash
+npx codeceptjs run -p pause:on=url:pattern=/api/*/edit
+npx codeceptjs run -p pause:on=url:pattern=/checkout/*
+```
+
+This is useful when you want to inspect a specific page regardless of which test step navigates there.
+
+## Browser Control
+
+For ad-hoc overrides of browser helper config without editing `codecept.conf`, use the `browser` plugin via `-p`. Works for Playwright, Puppeteer, WebDriver and Appium in one call.
+
+Force a visible browser:
+
+```bash
+npx codeceptjs run -p browser:show
+```
+
+Force headless (also injects `--headless` into WebDriver chrome/firefox capability args):
+
+```bash
+npx codeceptjs run -p browser:hide
+```
+
+Switch the browser engine for Playwright / Puppeteer / WebDriver / TestCafe in one shot — no per-helper config gymnastics:
+
+```bash
+npx codeceptjs run -p browser:browser=firefox
+npx codeceptjs run -p browser:browser=webkit:hide
+```
+
+Pass any other helper config as `key=value`. Values are coerced (`true`/`false` → boolean, digits → Number, otherwise string). Tokens are colon-chained on a single `-p`:
+
+```bash
+npx codeceptjs run -p browser:windowSize=1024x768:video=false
+npx codeceptjs run -p browser:hide:video=true
+```
+
+`browser=<name>` routes through `setBrowser` (so Puppeteer correctly receives `product`, Playwright receives `browser`, etc.); `windowSize=WxH` routes through `setWindowSize` (which also injects `--window-size=W,H` into chromium/chrome args). Anything else is shallow-merged onto every browser helper present in config.
+
+## IDE Debugging
+
+### VS Code
+
+You can use the built-in Node.js debugger in VS Code to set breakpoints in test files.
+
+Add this configuration to `.vscode/launch.json`:
+
+```json
+{
+  "type": "node",
+  "request": "launch",
+  "name": "codeceptjs",
+  "args": ["run", "--grep", "@your_test_tag", "--debug"],
+  "program": "${workspaceFolder}/node_modules/codeceptjs/bin/codecept.js"
+}
+```
+
+Set breakpoints in your test files, then press F5 to start debugging. You'll be able to step through code, inspect variables, and use the VS Code debug console — all while the browser is open and controlled by the test.
+
+Combine with `pause()` for the best experience: set a VS Code breakpoint to inspect JavaScript state, then add `pause()` to interact with the browser.
+
+### WebStorm
+
+```bash
+node $NODE_DEBUG_OPTION ./node_modules/.bin/codeceptjs run
+```
+
+### Node.js Inspector
+
+```bash
+node --inspect ./node_modules/.bin/codeceptjs run
+node --inspect-brk ./node_modules/.bin/codeceptjs run  # break on first line
+```
+
+## Debugging on Failure
+
+Several built-in plugins capture information when tests fail. These are most useful on CI where you can't use interactive debugging.
+
+### Screenshots on Failure
+
+Enabled by default. Saves a screenshot when a test fails:
+
+```js
+plugins: {
+  screenshot: {
+    enabled: true,
+    on: 'fail',
+    uniqueScreenshotNames: true,
+    fullPageScreenshots: true,
+  }
+}
+```
+
+Screenshots are saved in the `output` directory. The same plugin also supports `on=test`, `on=step`, `on=file`, and `on=url` to capture screenshots in other situations.
+
+### Page Info on Failure
+
+Captures URL, HTML errors, and browser console logs on failure:
+
+```js
+plugins: {
+  pageInfo: {
+    enabled: true,
+  }
+}
+```
+
+### Step-by-Step Report
+
+Generates a slideshow of screenshots taken after every step — a visual replay of what the test did. Set `slides: true` on the `screenshot` plugin (with `on=step`):
+
+```js
+plugins: {
+  screenshot: {
+    enabled: true,
+    on: 'step',
+    slides: true,
+    deleteSuccessful: true,   // keep only failed tests
+    fullPageScreenshots: true,
+  }
+}
+```
+
+```bash
+npx codeceptjs run -p screenshot:on=step;slides=true
+```
+
+After the run, open `output/records.html` to browse through the slideshows.
+
+## AI-Powered Debugging
+
+### AI in Interactive Pause
+
+When AI is configured, the interactive pause shell accepts natural language commands:
+
+```
+ AI is enabled! (experimental) Write what you want and make AI run it
+
+I.$ fill the login form with test credentials and submit
+```
+
+AI reads the current page HTML and generates CodeceptJS steps. See [Testing with AI](/ai) for setup.
+
+### AI Trace Plugin
+
+The `aiTrace` plugin captures rich execution traces for AI analysis — screenshots, HTML snapshots, ARIA trees, browser logs, and network requests at every step:
+
+```js
+plugins: {
+  aiTrace: {
+    enabled: true,
+  }
+}
+```
+
+After a test run, trace files are generated in `output/trace_*/trace.md`. Feed these to an AI assistant (like Claude Code) for automated failure analysis. See [AI Trace Plugin](/aitrace) for details.
+
+### MCP Server
+
+CodeceptJS includes an [MCP server](/mcp) that allows AI agents to control tests programmatically — list tests, run them step by step, capture artifacts, and analyze results. This enables AI-driven debugging workflows where an agent can investigate failures autonomously.
+
+> AI agent integration and MCP server will be covered in detail on a dedicated page.

package/docs/detox.md

@@ -0,0 +1,235 @@
+---
+permalink: /detox
+title: Testing React Native with Detox
+---
+
+> Warning! Detox support in CodeceptJS is experimental. Please try it and help us to test it and improve it. [See Detox helper repository](https://github.com/codeceptjs/detox-helper).
+
+Automated mobile testing can be slow, hard, and ineffective. The price of it goes high, if we take into account fragility of applications, slowness of emulators, and the complexity of debug. [Appium](/mobile) helps writing mobile tests but not all apps can be tested effectively with it. That's why you should consider using an alternative approach.
+
+Meet [Detox](https://github.com/wix/Detox) - grey-box testing solution for mobile testing by Wix.
+
+Unlike, Appium, Detox requires to update mobile application to include test instrumentation, so an application could receive commands from a test, and act accordingly. Let's see pros and cons of such approach:
+
+
+**Pros**:
+
+* faster tests
+* synchronization with application
+* plays nicely with React Native
+
+**Cons**:
+
+* application should be built in a special way
+* no hybrid applications
+* native Android apps not supported (except React Native)
+* no cloud testing with SauceLabs or BrowserStack
+
+CodeceptJS allows you to try different options and choose the one which works best for you. Both Appium and Detox helpers share the same syntax for testing mobile applications, interactive pause, automatic retries, and other useful features.
+
+CodeceptJS provides next features over standard Detox:
+
+* **Unified API**. The same test can be executed in Appium or Detox. Unified API helps different teams to use the same syntax and easy port tests from one engine to another.
+* [Interactive pause](/basics#pause). When starting/stopping an application takes a long time, debugging and writing tests can be hard. CodeceptJS solves this by pausing an execution and letting you try different commands and locators. With this feature a test can be writtern during one running session.
+* [Auto-retries](/basics#retries) using `retryFailedStepPlugin` and `I.retry()`
+* **Cross-Platform testing** - one test can be executed on different engines. When needed, platform-specific actions and locators can be easily applied.
+
+## A Test
+
+Compare a test written for Detox using its native syntax:
+
+```js
+await expect(element(by.text('Welcome'))).toBeVisible();
+await expect(element(by.id('createdAndVisibleText'))).toNotExist();
+await element(by.id('GoButton')).tap();
+await waitFor(element(by.id('createdAndVisibleText'))).toExist().withTimeout(20000);
+await expect(element(by.id('createdAndVisibleText'))).toExist();
+```
+
+with the same test written using CodeceptJS syntax:
+
+```js
+I.see('Welcome');
+I.dontSeeElement('#createdAndVisibleText');
+I.click('#GoButton');
+I.waitForElement('#createdAndVisibleText', 20);
+I.seeElement('#createdAndVisibleText');
+```
+
+As you see, CodeceptJS test is shorter and easier to follow. By simplifying the code and reducing visual noise we make tests easier to follow. But before writing a test we need to prepare an application to be testable with Detox.
+
+## Setup
+
+It is important to follow [Detox guide](https://github.com/wix/Detox/blob/master/docs/Introduction.GettingStarted.md) to build an application with Detox test instrouments included.
+
+After you install Detox, create configuration and build an application using `detox build` command, you are ready to integrate Detox with CodeceptJS.
+
+You need to install Detox CodeceptJS helper:
+
+```
+npm i @codeceptjs/detox-helper --save-dev
+```
+
+Then enable this helper in `codecept.conf.js`:
+
+```js
+helpers: {
+   Detox: {
+     require: '@codeceptjs/detox-helper',
+     configuration: '<detox app configuration name>',
+     reloadReactNative: true,
+   }
+}
+```
+
+Enable `reloadReactNative: true` if you test React Native application.
+
+## Actions
+
+Create test as usual, by running command:
+
+```
+npx codeceptjs gt
+```
+
+A test starts when emulator starts and loads an application. So you can begin testing an application.
+
+```js
+// inside a created test
+Scenario('test React Native app', ({ I }) => {
+  I.see('Welcome');
+  I.tap('Start');
+  I.see('Started!');
+});
+```
+
+The most common actions are:
+
+* `tap` (or `click`)
+* `multiTap` - perform multiple taps on element
+* `longPress` - longer press
+* `fillField` - fill in value of text field
+* `clearField` - clear value in text field
+* `appendField` - append value in a field
+* `swipeUp`, `swipeDown`, `swipeLeft`, `swipeRigth`
+
+There are also common assertions:
+
+* `see` - to check visibility of text
+* `seeElement` - to check visibility of element
+* `seeElementExists` - to check that element exists
+
+> For more details on actions refer to the [API reference of Detox helper](https://github.com/codeceptjs/detox-helper#api).
+
+## Locators
+
+To write a test you need to learn about available locators in Detox.
+Unlike, Appium there are no XPath locators. Possible locators limited to `text`, `id`, `accessibility id`, and element `type`. Thus, again, an application should be prepared for testing, to ensure that all active elements are accessible.
+
+Let's see how they can be used:
+
+* For **ID locators** use `#` prefix (same as in CSS). Example:
+
+```js
+I.seeElement('#WelcomeScreen')
+```
+* For **accessibility ID** use `~` prefix (as in Appium helper). Example:
+
+```js
+I.seeElement('~welcome')
+```
+
+* Locating elements **by text** requires no prefix, but can be applied only for actions accepting semantic locators.
+
+```js
+I.tap('Start')
+I.fillField('Username', 'davert')
+I.clearField('Username')
+I.see('Welcome, davert')
+```
+
+* Locating elements **by type** also use no prefix but can be used only where for elements.
+
+```js
+I.seeElement('Button')
+```
+
+Text locators can be combined with others, as methods `tap`, `click` and `see` can receive a second paramater, which defines a context where to perfrom a search.
+
+```js
+// tap "Start" inside "#Layout"
+I.tap('Start', '#Layout');
+// see text "Welcome" inside "~msg"
+I.see('Welcome', '~msg');
+```
+
+Alternatively, you can use specify locator by using *strict locator*, passing an object as a locator:
+
+```js
+I.click({ type: 'Button' });
+I.seeElement({ label: 'welcome' });
+```
+
+> If you are familiar with Detox API, this is how locators are actually translated: `{label: 'welcome'}` => `by.label('welcome')`.
+
+### Cross Platform Testing
+
+If element differs on on iOS and Android you can use **cross platform locators**.
+
+```js
+// locate element by text on Android
+// locate element by accessibility id on iOS
+I.click({ android: 'Start', ios: '~start' });
+```
+
+When application behavior differs on Android and iOS use platform-specific actions:
+
+```js
+I.runOnIOS(() => {
+  // this will be executed only on IOS
+  I.see('Hello iOS');
+});
+I.runOnAndroid(() => {
+  // this will be executed only on Android
+  I.see('Hello Android');
+});
+```
+
+### Sample Test
+
+Finally, you can get a test looking like this
+
+```js
+Feature('My Detox App');
+
+Scenario('save in application', ({ I }) => {
+  I.setLandscapeOrientation();
+  I.fillField('#text', 'a new text');
+  I.see('a new text', '#textValue');
+  I.dontSeeElement('#createdAndVisibleText');
+  I.click({ ios: '#GoButton', android: 'Button' });
+  I.waitForElement('#createdAndVisibleText', 20);
+  I.seeElement('#createdAndVisibleText');
+  I.runOnAndroid(() => {
+    I.click('Save');
+    I.see('Text Saved', '#message');
+  });
+  I.runOnIOS(() => {
+    I.click('SAVE');
+    I.see('SAVED!');
+  });
+});
+```
+
+To execute it use `codeceptjs run` command
+
+```
+npx codeceptjs run
+```
+If you want to use detox configuration other than is set in `codecept.conf.js` use `--configuration` argument:
+
+```
+npx codeceptjs run --configuration android.test.ci
+```
+
+You can also pass all [other arguments that Detox CLI supports](https://github.com/wix/Detox/blob/master/docs/APIRef.DetoxCLI.md#test).

package/docs/docker.md

@@ -0,0 +1,136 @@
+# Codeceptjs Docker
+
+CodeceptJS has an [official docker image](https://hub.docker.com/r/codeceptjs/codeceptjs) based on Playwright image. Image supports Playwright, Puppeteer, and WebDriver engines.
+
+## How to Use
+
+This image comes with the necessary dependencies and packages to execute CodeceptJS tests.
+Mount in your CodeceptJS config directory into the `/tests` directory in the docker container.
+
+Sample mount: `-v path/to/codecept.conf.js:/tests`
+
+CodeceptJS runner is available inside container as `codeceptjs`.
+
+### Locally
+
+You can execute CodeceptJS with Puppeteer locally with no extra configuration.
+
+```sh
+docker run --net=host -v $PWD:/tests codeceptjs/codeceptjs
+```
+
+To customize execution call `codeceptjs` command:
+
+```sh
+# run tests with steps
+docker run --net=host -v $PWD:/tests codeceptjs/codeceptjs codeceptjs run --steps
+
+# run tests with @user in a name
+docker run --net=host -v $PWD:/tests codeceptjs/codeceptjs codeceptjs run --grep "@user"
+```
+
+
+### Docker Compose
+
+```yaml
+version: '2'
+services:
+  codeceptjs:
+    image: codeceptjs/codeceptjs
+    depends_on:
+      - firefox
+      - web
+    volumes:
+      - .:/tests
+  web:
+    image: node
+    command: node app/server.js
+    volumes:
+      - .:/app
+  firefox:
+    image: selenium/standalone-firefox-debug:2.53.0
+    ports:
+      - '4444'
+      - '5900'
+```
+
+### Linking Containers
+
+If using the WebDriver driver, link the container with a Selenium Standalone docker container with an alias of `selenium`. Additionally, make sure your `codeceptjs.conf.js` contains the following to allow CodeceptJS to identify where Selenium is running.
+
+```javascript
+  ...
+  helpers: {
+    WebDriver: {
+      ...
+      host: process.env.HOST
+      ...
+    }
+  }
+  ...
+```
+
+```sh
+$ docker run -d -P --name selenium-chrome selenium/standalone-chrome
+
+# Alternatively, selenium/standalone-firefox can be used
+
+$ docker run -it --rm -v /<path_to_codeceptjs_test_dir>/:/tests/ --link selenium-chrome:selenium codeceptjs/codeceptjs
+```
+
+You may run use `-v $(pwd)/:tests/` if running this from the root of your CodeceptJS tests directory.
+_Note: The output of your test run will appear in your local directory if your output path is `./output` in the CodeceptJS config_
+
+### Build
+
+To build this image:
+
+```sh
+docker build -t codeceptjs/codeceptjs .
+```
+
+* this directory will be added as `/codecept` insde container
+* tests directory is expected to be mounted as `/tests`
+* `codeceptjs` is a synlink to `/codecept/bin/codecept.js`
+
+To build this image with your desired Node version:
+
+```sh
+docker build -t codeceptjs/codeceptjs . --build-arg NODE_VERSION=12.10.0
+```
+
+### Passing Options
+
+Options can be passed by calling `codeceptjs`:
+
+```
+docker run -v $PWD:/tests codeceptjs/codeceptjs codeceptjs run --debug
+```
+
+Alternatively arguments to `codecept run` command can be passed via `CODECEPT_ARGS` environment variable. For example to run your tests with debug
+output:
+
+```yaml
+version: '2'
+services:
+  codeceptjs:
+    image: codeceptjs/codeceptjs
+    environment:
+      - CODECEPT_ARGS=--debug
+    volumes:
+      - .:/tests
+```
+
+You can also use `run-workers`to run tests by passing `NO_OF_WORKERS`, additionally, you can pass more params like showing the debug info as the following example:
+
+```yaml
+version: '2'
+services:
+  codeceptjs:
+    image: codeceptjs/codeceptjs
+    environment:
+      - NO_OF_WORKERS=3
+      - CODECEPT_ARGS=--debug
+    volumes:
+      - .:/tests
+```