codeceptjs 4.0.2-beta.8 → 4.0.2

package/docs/commands.md

@@ -0,0 +1,352 @@
+---
+permalink: /commands
+title: Commands
+---
+
+# Commands
+
+## Run
+
+Executes tests. Requires `codecept.conf.js` config to be present in provided path.
+
+---
+
+Run all tests from current dir
+
+```sh
+npx codeceptjs run
+```
+
+Load config and run tests from `test` dir
+
+```sh
+npx codeceptjs run -c test
+```
+
+Run only tests with "signin" word in name
+
+```sh
+npx codeceptjs run --grep "signin"
+```
+
+Run all tests without "@IEOnly" word in name
+
+```sh
+npx codeceptjs run --grep "@IEOnly" --invert
+```
+
+Run single test [path to codecept.js] [test filename]
+
+```sh
+npx codeceptjs run github_test.js
+```
+
+Run single test with steps printed
+
+```sh
+npx codeceptjs run github_test.js --steps
+```
+
+Run test files in shuffled order
+
+```sh
+npx codeceptjs run --shuffle
+```
+
+Run single test in debug mode (see more in [debugging](#Debugging) section)
+
+```sh
+npx codeceptjs run github_test.js --debug
+```
+
+Select config file manually (`-c` or `--config` option)
+
+```sh
+npx codeceptjs run -c my.codecept.conf.js
+npx codeceptjs run --config path/to/codecept.conf.js
+```
+
+Override config on the fly. Provide valid JSON which will be merged into current config:
+
+```sh
+npx codeceptjs run --override '{ "helpers": {"WebDriver": {"browser": "chrome"}}}'
+```
+
+Run tests and produce xunit report:
+
+```sh
+npx codeceptjs run --reporter xunit
+```
+
+Use any of [Mocha reporters](https://github.com/mochajs/mocha/tree/master/lib/reporters) used.
+
+#### Debugging 
+
+Run single test in debug mode
+
+```sh
+npx codeceptjs run --debug
+```
+
+Run test with internal logs printed.
+
+```sh
+npx codeceptjs run --verbose
+```
+
+Display complete debug output including scheduled promises
+
+```
+DEBUG=codeceptjs:* npx codeceptjs run
+```
+
+## Plugin Arguments
+
+`run`, `run-workers`, `run-multiple`, `run-rerun` and `dry-run` accept a `-p` (`--plugins`) flag to enable plugins on the command line, with optional arguments per plugin. Tokens are colon-chained per plugin, comma-separated across plugins:
+
+```sh
+npx codeceptjs run -p <name>                       # enable plugin
+npx codeceptjs run -p <name>:<arg1>:<arg2>         # enable + pass args
+npx codeceptjs run -p <plugin1>,<plugin2>:<arg>    # multiple plugins
+```
+
+Plugins listed via `-p` are activated even when their config has `enabled: false` (or no `enabled` flag). This is the supported way to switch a plugin on for a single run without editing `codecept.conf`.
+
+A few examples:
+
+```sh
+npx codeceptjs run -p pause                              # pause on first failure (default on=fail)
+npx codeceptjs run -p pause:on=step                      # pause before every step
+npx codeceptjs run -p pause:on=url:pattern=/checkout/*   # pause on URL match
+npx codeceptjs run -p "screenshot:on=step;slides=true"   # produce a step-by-step HTML report
+```
+
+### Browser Control
+
+The built-in `browser` plugin overrides browser-helper config from the CLI — works for Playwright, Puppeteer, WebDriver and Appium without editing `codecept.conf`.
+
+```sh
+npx codeceptjs run -p browser:show                       # force visible browser
+npx codeceptjs run -p browser:hide                       # force headless
+npx codeceptjs run -p browser:browser=firefox            # switch engine
+npx codeceptjs run -p browser:windowSize=1024x768        # set viewport
+npx codeceptjs run -p browser:hide:browser=webkit:windowSize=800x600
+```
+
+Tokens after `browser:` are either flags (`show`, `hide`) or `key=value` pairs. Three keys get per-helper translation:
+
+- `browser=<name>` — Puppeteer receives `product`, Playwright/WebDriver receive `browser`. Validated per helper (`chromium`/`webkit`/`firefox` for Playwright, `chrome`/`firefox` for Puppeteer).
+- `show=true|false` (or the `show`/`hide` flag) — sets `show` on Playwright/Puppeteer; injects/strips `--headless` in WebDriver chrome/firefox capability args.
+- `windowSize=WxH` — sets `windowSize` on every helper; also adds `--window-size=W,H` to chromium/chrome args for Playwright/Puppeteer.
+
+Anything else (`-p browser:video=false:waitForTimeout=10000`) is shallow-merged onto every browser helper present in config. Values are coerced (`true`/`false` → boolean, digits → Number, otherwise string).
+
+## Run Workers
+
+Run tests in parallel threads. CodeceptJS supports different distribution strategies for optimal performance.
+
+```bash
+# Run with 3 workers using default strategy (pre-assign tests)
+npx codeceptjs run-workers 3
+
+# Run with pool mode for dynamic test distribution (recommended)
+npx codeceptjs run-workers 3 --by pool
+
+# Run with suite distribution
+npx codeceptjs run-workers 3 --by suite
+
+# Pool mode with filtering
+npx codeceptjs run-workers 4 --by pool --grep "@smoke"
+```
+
+**Test Distribution Strategies:**
+
+- `--by test` (default): Pre-assigns individual tests to workers
+- `--by suite`: Pre-assigns entire test suites to workers  
+- `--by pool`: Dynamic distribution for optimal load balancing (recommended for best performance)
+
+The pool mode provides the best load balancing by maintaining tests in a shared pool and distributing them dynamically as workers become available. This prevents workers from sitting idle and ensures optimal CPU utilization, especially when tests have varying execution times.
+
+See [Parallel Execution](/parallel) documentation for more details.
+
+## Run Rerun <Badge text="Since 3.3.6" type="warning"/>
+
+Run tests multiple times to detect and fix flaky tests.
+
+```
+npx codeceptjs run-rerun
+```
+
+For this command configuration is required:
+
+```js
+{
+  // inside codecept.conf.js
+  rerun: {
+    // how many times all tests should pass
+    minSuccess: 2,
+
+    // how many times to try to rerun all tests
+    maxReruns: 4,
+  }
+}
+```
+
+Use Cases:
+
+* `minSuccess: 1, maxReruns: 5` - run all tests no more than 5 times, until first successful run.
+* `minSuccess: 3, maxReruns: 5` - run all tests no more than 5 times, until reaching 3 successfull runs.
+* `minSuccess: 10, maxReruns: 10` - run all tests exactly 10 times, to check their stability.
+
+
+## Dry Run
+
+Prints test scenarios without executing them
+
+```
+npx codeceptjs dry-run
+```
+
+When passed `--steps` or `--debug` option runs tests, disabling all plugins and helpers, so you can get step-by-step report with no tests actually executed.
+
+```
+npx codeceptjs dry-run --steps
+```
+
+If a plugin needs to be enabled in `dry-run` mode, pass its name in `-p` option:
+
+```
+npx codeceptjs dry-run --steps -p allure
+```
+
+If some plugins need to be enabled in `dry-run` mode, pass its name in `-p` option:
+
+```
+npx codeceptjs dry-run --steps -p allure,customLocator
+```
+
+If all plugins need to be enabled in `dry-run` mode, pass its name in `-p` option:
+
+```
+npx codeceptjs dry-run --steps -p all
+```
+
+To enable bootstrap script in dry-run mode, pass in `--bootstrap` option when running with `--steps` or `--debug`
+
+```
+npx codeceptjs dry-run --steps --bootstrap
+```
+
+## Run Multiple
+
+> ⚠️ prefer using run-workers instead
+
+Run multiple suites. Unlike `run-workers` spawns processes to execute tests.
+[Requires additional configuration](/advanced#multiple-browsers-execution) and can be used to execute tests in multiple browsers.
+
+```sh
+npx codeceptjs run-multiple smoke:chrome regression:firefox
+```
+
+## Init
+
+Creates `codecept.conf.js` file in current directory:
+
+```sh
+npx codeceptjs init
+```
+
+Or in provided path
+
+```sh
+npx codecept init test
+```
+
+## Migrate
+
+Migrate your current `codecept.json` to `codecept.conf.js`
+
+```sh
+npx codeceptjs migrate
+```
+
+## Shell
+
+Interactive shell. Allows to try `I.` commands in runtime
+
+```sh
+npx codeceptjs shell
+```
+
+## Generators
+
+Create new test
+
+```sh
+npx codeceptjs generate:test
+```
+
+Create new pageobject
+
+```sh
+npx codeceptjs generate:pageobject
+```
+
+Create new helper
+
+```sh
+npx codeceptjs generate:helper
+```
+
+## TypeScript Definitions
+
+TypeScript Definitions allows IDEs to provide autocompletion when writing tests.
+
+```sh
+npx codeceptjs def
+npx codeceptjs def --config path/to/codecept.conf.js
+```
+
+After doing that IDE should provide autocompletion for `I` object inside `Scenario` and `within` blocks.
+
+Add optional parameter `output` (or shortcut `-o`), if you want to place your definition file in specific folder:
+
+```sh
+npx codeceptjs def --output ./tests/typings
+npx codeceptjs def -o ./tests/typings
+```
+
+## List Commands
+
+Prints all available methods of `I` to console.
+
+```sh
+npx codeceptjs list
+```
+
+Use `-c` to point at a specific config (same as `run`):
+
+```sh
+npx codeceptjs list -c ./test/acceptance/codecept.Playwright.js
+```
+
+Add `--docs` to print full documentation (description, examples, `@param` annotations) below each action — pulled from helper JSDoc and `docs/webapi/*` snippets:
+
+```sh
+npx codeceptjs list --docs
+```
+
+Use `--action` to show docs for a single action. The `I.` prefix is optional and `--docs` is implied:
+
+```sh
+npx codeceptjs list --action amOnPage
+npx codeceptjs list --action I.click -c ./test/acceptance/codecept.Playwright.js
+```
+
+## Local Environment Information
+
+Prints debugging information concerning the local environment
+
+```sh
+npx codeceptjs info
+```

package/docs/community-helpers.md

@@ -0,0 +1,63 @@
+---
+permalink: /community-helpers
+title: Community Helpers
+editLink: false
+---
+
+# Community Helpers
+> Share your helpers at our [Wiki Page](https://github.com/codeceptjs/CodeceptJS/wiki/Community-Helpers)
+
+Here is the list of helpers created by our community.
+Please **add your own** by editing this page.
+## Webhooks
+
+* [codeceptjs-webhook-helper](https://github.com/onemolegames/codeceptjs-webhook-helper) - to check webhook calls during the tests.
+
+## Email Checking
+
+* [MailCatcher](https://gist.github.com/schmkr/026732dfa1627b927ff3a08dc31ee884) - to check emails via Mailcatcher locally.
+* [codeceptjs-mailhog-helper](https://github.com/tsuemura/codeceptjs-mailhog-helper) - to check emails via Mailhog locally.
+* [codeceptjs-testmailapp-helper](https://github.com/pavkam/codeceptjs-testmailapp-helper) - to check emails via Testmail.app service.
+* [codeceptjs-mailosaurhelper](https://github.com/yurkovychv/codeceptjs-mailosaur) - to check emails via [Mailosaur](https://mailosaur.com/) service.
+
+## Data Sources
+
+* [codeceptjs-httpmock](https://github.com/testphony/codeceptjs-httpMock) -  a helper which wraps mockttp library to manage http mock in tests.
+* [codeceptjs-http](https://github.com/testphony/codeceptjs-http) - a helper which wraps then-request library to process HTTP requests. It's alternative helper that provides more flexible request management.
+* [codeceptjs-dbhelper](https://github.com/thiagodp/codeceptjs-dbhelper) - allows you to execute queries or commands to databases using database-js. 
+
+## Cloud Providers
+* [codeceptjs-saucehelper](https://github.com/puneet0191/codeceptjs-saucehelper/) - a helper which updates `Test Names` & `Test Results` on Saucelabs 
+* [codeceptjs-bshelper](https://github.com/PeterNgTr/codeceptjs-bshelper) - a helper which updates `Test Names` & `Test Results` on Browserstack 
+* [codeceptjs-tbhelper](https://github.com/testingbot/codeceptjs-tbhelper) - a helper which updates `Test Names` & `Test Results` on TestingBot 
+
+## Visual-Testing
+* [codeceptjs-resemblehelper](https://github.com/puneet0191/codeceptjs-resemblehelper) - a helper which helps with visual testing using resemble.js. 
+* [codeceptjs-applitoolshelper](https://www.npmjs.com/package/codeceptjs-applitoolshelper) - a helper which helps interaction with [Applitools](https://applitools.com)
+* [codeceptjs-pixelmatchhelper](https://github.com/stracker-phil/codeceptjs-pixelmatchhelper) - a helper that integrates pixelmatch for visual testing. 
+
+## Reporters
+* [codeceptjs-rphelper](https://github.com/reportportal/agent-js-codecept) is a CodeceptJS helper which can publish tests results on ReportPortal after execution.
+* [codeceptjs-xray-helper](https://www.npmjs.com/package/codeceptjs-xray-helper) is a CodeceptJS helper which can publish tests results on [XRAY](https://confluence.xpand-it.com/display/XRAYCLOUD/Import+Execution+Results+-+REST).
+* [codeceptjs-xray-cloud-helper](https://www.npmjs.com/package/codeceptjs-xray-cloud-helper) is a helper that automatically retrieves the result of CodeceptJS tests and sends them to XRAY/JIRA(cloud version) via [XRAY Cloud API](https://docs.getxray.app/display/XRAYCLOUD/Import+Execution+Results+-+REST+v2#ImportExecutionResultsRESTv2-XrayJSONresults).
+* [codeceptjs-slack-reporter](https://www.npmjs.com/package/codeceptjs-slack-reporter) Get a Slack notification when one or more scenarios fail.
+* [codeceptjs-browserlogs-plugin](https://github.com/pavkam/codeceptjs-browserlogs-plugin) Record the browser logs for failed tests.
+* [codeceptjs-testrail](https://github.com/PeterNgTr/codeceptjs-testrail) - a plugin to integrate with [Testrail](https://www.gurock.com/testrail)
+* [codeceptjs-monocart-coverage](https://github.com/cenfun/codeceptjs-monocart-coverage) - a plugin to generate coverage reports, it integrate with [monocart coverage reports](https://github.com/cenfun/monocart-coverage-reports)
+
+## Browser request control
+* [codeceptjs-resources-check](https://github.com/luarmr/codeceptjs-resources-check) Load a URL with Puppeteer and listen to the requests while the page is loading. Enabling count the number or check the sizes of the requests.
+
+## Assertion & Validations
+* [codeceptjs-chai](https://www.npmjs.com/package/codeceptjs-chai) is a CodeceptJS helper which wraps
+[chai](https://www.chaijs.com/) library to complete chai assertion steps with CodeceptJS logging.
+
+## Other
+
+* [codeceptjs-cmdhelper](https://github.com/thiagodp/codeceptjs-cmdhelper) allows you to run commands in the terminal/console
+* [eslint-plugin-codeceptjs](https://www.npmjs.com/package/eslint-plugin-codeceptjs) Eslint rules for CodeceptJS.
+* [codeceptjs-datalayer-helper](https://github.com/kobenguyent/codeceptjs-datalayer-helper) CodeceptJS DataLayer helper helps you to get the datalayer JavaScript array that is used to store information and send this data to the tag manager.
+* [codeceptjs-a11y-helper](https://github.com/kobenguyent/codeceptjs-a11y-helper) accessibility tests integrated with CodeceptJS - Playwright-axe
+* [codeceptjs-lighthouse-helper](https://github.com/kobenguyent/codeceptjs-lighthouse-helper) lighthouse audit integrated with CodeceptJS - Playwright
+* [Snowplow Data analytics](https://www.npmjs.com/package/@viasat/codeceptjs-snowplow-helper) - Test your Snowplow events implementations with CodeceptJS and Snowplow Micro.
+* [codeceptjs-failure-logger](https://github.com/kobenguyent/codeceptjs-failure-logger) - Log failed CodeceptJS tests to file

package/docs/configuration.md

@@ -0,0 +1,185 @@
+---
+permalink: /configuration
+title: Configuration
+---
+
+# Configuration
+
+`codeceptjs init` creates a `codecept.conf.js` (or `codecept.conf.ts`) in your test root. It exports a `config` object:
+
+```js
+export const config = {
+  tests: './**/*_test.js',
+  output: './output',
+  helpers: {
+    Playwright: { url: 'http://localhost', browser: 'chromium' },
+  },
+  include: {
+    I: './steps_file.js',
+  },
+}
+```
+
+## Options
+
+**Tests and files**
+
+- `tests` — glob pattern (or array of patterns) locating your test files, e.g. `'tests/**/*_test.js'` (or `*_test.ts` for TypeScript).
+- `output` — directory for failure screenshots, artifacts, and temporary files. Default `./output`.
+- `include` — page objects and support objects exposed via dependency injection: `{ I: './steps_file.js', loginPage: './pages/Login.js' }`. They can then be injected by name — `Scenario('test', ({ I, loginPage }) => …)`.
+- `require` — extra modules to load before tests run: assertion libraries, TypeScript loaders, setup files. See [Require](#require).
+- `grep` — run only tests whose name matches this pattern, e.g. `grep: '@firefox'`. Handy when you keep separate configs per environment.
+
+**Helpers and plugins**
+
+- `helpers` — enable and configure [helpers](/helpers): `{ Playwright: { url: 'https://mysite.com', browser: 'firefox' } }`.
+- `plugins` — enable [plugins](/plugins): `{ autoDelay: { enabled: true } }`.
+
+**Hooks**
+
+- `bootstrap` / `teardown` — run code before / after the whole run; an async function or a path to a JS module. See [Bootstrap](/bootstrap).
+- `bootstrapAll` / `teardownAll` — run once around a parallel run (before any worker starts / after all finish). See [bootstrapAll / teardownAll](/bootstrap#bootstrapall-teardownall).
+
+**Test runner**
+
+- `timeout` — default per-test timeout in seconds; a test is killed if it stops responding.
+- `mocha` — [Mocha options](https://mochajs.org/#configuring-mocha-nodejs), including extra reporters. See [Reporters](/reports).
+
+**BDD**
+
+- `gherkin` — enable [BDD features](/bdd#configuration): `{ features: './features/*.feature', steps: ['./step_definitions/steps.js'] }`.
+  - `gherkin.features` — feature file glob, or an array of globs.
+  - `gherkin.steps` — JS files with step definitions.
+
+**TypeScript**
+
+- `fullPromiseBased` — generate typings where `I.*` methods return promises, so you can `await` each command. See [TypeScript](/typescript#promise-based-typings).
+
+**Other**
+
+- `translation` — enable [localized commands](/translation).
+- `maskSensitiveData` — mask secrets in console output.
+- `noGlobals` — don't register the global functions (`Feature`, `Scenario`, `Before`, …). Not recommended.
+
+## Require
+
+`require` loads modules before tests run — assertion libraries (`'should'`), setup files (`'./lib/setup'`), and TypeScript loaders. Modules load in the order listed.
+
+For TypeScript test files in CodeceptJS 4.x, use the [`tsx`](https://tsx.is) loader:
+
+```ts
+// codecept.conf.ts
+export const config = {
+  tests: './**/*_test.ts',
+  require: ['tsx/cjs'],
+  helpers: {},
+  include: {},
+}
+```
+
+Combine several modules:
+
+```ts
+require: ['tsx/cjs', 'should', './lib/testSetup']
+```
+
+The config file itself (`codecept.conf.ts`) and helpers are transpiled automatically — only test files need the loader. See [TypeScript](/typescript) for the full setup.
+
+## Dynamic configuration
+
+A JS/TS config file is plain code, so you can read environment variables and build the config at runtime:
+
+```js
+export const config = {
+  helpers: {
+    WebDriver: {
+      url: process.env.CODECEPT_URL || 'http://localhost:3000',
+      user: process.env.CLOUDSERVICE_USER,
+      key: process.env.CLOUDSERVICE_KEY,
+      waitForTimeout: 10000,
+    },
+  },
+  include: {
+    I: './src/steps_file.js',
+    loginPage: './src/pages/login_page.js',
+  },
+}
+```
+
+Override config values at runtime with `--override` / `-o`, passing JSON:
+
+```sh
+npx codeceptjs run -o '{ "helpers": { "WebDriver": { "browser": "firefox" } } }'
+```
+
+Point at a non-default config file with `--config` / `-c`:
+
+```sh
+npx codeceptjs run --config ./path/to/config.js
+```
+
+## Common configuration patterns
+
+[`@codeceptjs/configure`](https://github.com/codeceptjs/configure) ships with CodeceptJS as a dependency. It holds shared recipes for config that's independent of the active helper.
+
+Toggle headless mode, set window size, and so on:
+
+```js
+import { setHeadlessWhen, setWindowSize } from '@codeceptjs/configure'
+
+setHeadlessWhen(process.env.HEADLESS || process.env.CI)
+setWindowSize(1600, 1200)
+
+export const config = {
+  // ...
+}
+```
+
+For one-shot bundles use `setBrowserConfig` — pass any subset of `{ browser, show, windowSize, url }` and the right per-helper translation happens automatically (Puppeteer gets `product`, WebDriver gets `--headless` injected, …). Keys whose value is `undefined` are skipped, so an unset env var won't clobber existing config:
+
+```js
+import { setBrowserConfig } from '@codeceptjs/configure'
+
+setBrowserConfig({
+  browser: process.env.BROWSER,     // optional engine override
+  show: !process.env.HEADLESS,      // headed unless HEADLESS is set
+  windowSize: '1280x720',
+  url: process.env.URL,             // overrides helper.url when set
+})
+```
+
+`setCommonPlugins()` enables a curated set of plugins and registers a few more as discoverable, so they can be switched on ad-hoc with [`-p` plugin arguments](/commands#plugin-arguments) without editing the config:
+
+```js
+import { setCommonPlugins } from '@codeceptjs/configure'
+
+setCommonPlugins()
+```
+
+- `retryFailedStep` — *enabled*. Retries steps that fail with transient errors.
+- `screenshot` — *enabled*. Screenshot on `fail` (default), or `test` / `step` / `file` / `url`.
+- `pause` — *registered*. Pause on failure / step / file / URL: `-p pause:on=fail`, `-p pause:on=step`, `-p pause:on=file:path=tests/login_test.js`, `-p pause:on=url:pattern=/checkout/*`.
+- `browser` — *registered*. CLI overrides for browser helpers: `-p browser:show`, `-p browser:browser=firefox`. See [browser control](/commands#browser-control).
+- `aiTrace` — *registered*. Capture AI traces: `-p aiTrace`, narrow with `on=fail|test|step|file|url`.
+- `heal` — *registered*. Self-heal failing steps: `-p heal`, narrow with `on=file|url`.
+
+> `eachElement`, `tryTo`, and `retryTo` are no longer plugins in 4.x — import them from `codeceptjs/effects`.
+
+## Profile
+
+`process.env.profile` carries the value of the `--profile` CLI option, so you can branch the config on it:
+
+```sh
+npx codeceptjs run --profile firefox
+```
+
+```js
+export const config = {
+  helpers: {
+    WebDriver: {
+      url: 'http://localhost:3000',
+      browser: process.env.profile || 'chrome',
+    },
+  },
+}
+```