codeceptjs 4.0.2-beta.8 → 4.0.2

package/docs/retry.md

@@ -0,0 +1,311 @@
+---
+permalink: /retry
+title: Retry Mechanisms
+---
+
+# Retry Mechanisms
+
+CodeceptJS provides flexible retry mechanisms to handle flaky tests. Use retries when dealing with unstable environments, network delays, or timing issues — not to mask bugs in your code.
+
+## Retry Levels
+
+* [helper retries](#helper-retries) — happen low level on browser interaction (resolves rendering of elements)
+* [failed step retries](#failed-step-retries) — performed by CodeceptJS on step fail
+* [manual step retries](#manual-step-retries) — on known flaky steps
+* [multiple steps retry](#multiple-steps-retry) — retry a group of steps together as a single operation
+* [self-healing steps](#self-healing-steps) — AI-powered recovery that continues tests without changing test code
+* [scenario retry](#scenario-retry) — retry entire test scenarios on failure
+* [feature retry](#feature-retry) — retry all scenarios within a feature
+* [hook retries](#hook-retries) — retry `Before`/`After` hooks on failure
+
+## Helper Retries
+
+Plawright has a built-in retry mechanism for element interactions. When you call `I.click('Button')`, after the element is located Playwright keeps retrying until it is actionable — up to `timeout` (default 5s).
+
+> WebDriver has a different auto-retry option: [smartWait](/webdriver#smartwait)
+
+Even though the handle exists (from `.all()`), Playwright still waits for it to become visible, stable (not mid-animation), enabled, not covered by an overlay/modal, and not rerendering.
+
+```js
+helpers: {
+  Playwright: {
+    timeout: 5000,      // retry the action until the element is actionable
+    waitForAction: 100  // fixed pause AFTER click/doubleClick/pressKey
+  }
+}
+```
+
+What each setting does:
+
+```
+find element (no wait — fails instantly if locator matches nothing)
+  → wait up to `timeout` for it to become actionable   ← timeout
+  → perform action
+  → sleep `waitForAction` ms                            ← waitForAction (settle pause, not a wait)
+```
+
+`timeout` covers the action. If the locator matches nothing yet, the step fails immediately. Use [Failed Step Retries](#failed-step-retries) to cover that gap.
+
+
+## Failed Step Retries
+
+CodeceptJS retries all failed steps by default by using the `retryFailedStep` plugin.
+
+```js
+plugins: {
+  retryFailedStep: {
+    enabled: true,
+    retries: 3
+  }
+}
+```
+
+Steps matching `amOnPage`, `wait*`, `send*`, `execute*`, `run*`, `have*` are skipped by default.
+
+When a scenario has its own retries, step retries are disabled by default (`deferToScenarioRetries: true`). This prevents excessive execution time:
+
+```js
+Scenario('test', { retries: 2 }, ({ I }) => {
+  I.click('Button')  // step retries disabled; scenario retries run instead
+})
+```
+
+To disable step retries for a specific test:
+
+```js
+Scenario('manual retries only', { disableRetryFailedStep: true }, ({ I }) => {
+  I.click('Button', step.retry(5))
+})
+```
+
+Defaults: `minTimeout: 150`, `factor: 1.5`, `maxTimeout: 10000`.
+
+
+> See [plugin reference](/plugins/retry-failed-step) for more options
+
+Retries are calculated via this formula:
+
+```
+gap(N) = min(minTimeout × factor^(N-1), maxTimeout)
+```
+
+Practically if step fails it will trigger a retry with increasing delay until `maxTimeout` is reached:
+
+```
+retries: 2                              => 0.15s-0.4s (150,225ms)
+retries: 3                              => 0.15s-0.7s (150,225,338ms)
+retries: 3, minTimeout: 1000            => 1s-4.75s   (1s,1.5s,2.25s)
+retries: 3, minTimeout: 1000, factor: 2 => 1s-7s      (1s,2s,4s)
+retries: 5, minTimeout: 1000, factor: 2 => 1s-25s     (1s,2s,4s,8s,10s)
+```
+
+Playwright `timeout` adds to each attempt only when the element is found:
+
+- `Playwright.timeout: 5000`
+- `retries: 2, minTimeout: 1000`
+
+```
+element not found => 0 + (1s+1s)    = 2s
+element found but not interactable  => 3×5s + (1s+1s) = 17s
+```
+
+## Manual Step Retries
+
+Retry a specific step known to be flaky:
+
+```js
+import step from 'codeceptjs/steps'
+
+Scenario('checkout', ({ I }) => {
+  I.amOnPage('/cart')
+  I.click('Proceed to Checkout', step.retry(5))  // retry up to 5 times
+  I.see('Payment')
+})
+```
+
+Configure timing with exponential backoff:
+
+```js
+I.click('Submit', step.retry({
+  retries: 3,
+  minTimeout: 1000,  // wait 1 second before first retry
+  maxTimeout: 5000,  // max 5 seconds between retries
+  factor: 1.5        // exponential backoff multiplier
+}))
+```
+
+Pass `0` for infinite retries.
+
+## Multiple Steps Retry
+
+Retry a group of steps together as a single operation:
+
+```js
+import { retryTo } from 'codeceptjs/effects'
+
+await retryTo(() => {
+  I.click('Load More')
+  I.see('New Content')
+}, 3)
+```
+
+If any step inside fails, the entire block retries. Use this for sequences that must succeed together — switching into an iframe and filling a form, for example.
+
+**Learn more:** [Effects](/effects#retryto)
+
+## Self-Healing Steps
+
+When a step fails, a healing recipe runs recovery actions and continues the test — without touching test code. With AI healing enabled:
+
+```js
+Scenario('checkout', ({ I }) => {
+  I.click('Proceed to Checkout')
+  I.see('Payment')
+})
+```
+
+- `I.click('Proceed to Checkout')` fails — button was renamed or moved
+  - failed step, error message, and page HTML are sent to an LLM
+  - AI scans page elements and suggests valid replacement actions
+  - CodeceptJS executes the suggestions until one succeeds
+- test continues with `I.see('Payment')`
+
+Run with `--ai` to activate:
+
+```bash
+npx codeceptjs run --ai
+```
+
+You can also write custom recipes for non-UI failures — network errors, data glitches, UI migrations.
+
+**Learn more:** [Self-Healing Tests](/heal), [AI Configuration](/ai)
+
+## Scenario Retry
+
+Retry an entire test when it fails:
+
+```js
+Scenario('API integration', { retries: 3 }, ({ I }) => {
+  I.sendGetRequest('/api/users')
+  I.seeResponseCodeIs(200)
+})
+```
+
+Retry all scenarios globally, or by grep pattern:
+
+```js
+export const config = {
+  retry: [
+    { Scenario: 3, grep: 'API' },       // retry scenarios containing "API" 3 times
+    { Scenario: 5, grep: '@flaky' }      // retry @flaky-tagged scenarios 5 times
+  ]
+}
+```
+
+## Feature Retry
+
+Retry all scenarios within a feature:
+
+```js
+Feature('Payment Processing', { retries: 2 })
+
+Scenario('credit card payment', ({ I }) => { ... })  // retries 2 times
+Scenario('paypal payment', ({ I }) => { ... })        // retries 2 times
+```
+
+Or target features by pattern in config:
+
+```js
+export const config = {
+  retry: [
+    { Feature: 3, grep: 'Integration' }
+  ]
+}
+```
+
+## Hook Retries
+
+Retry `Before`/`After` hooks when they fail:
+
+```js
+Before(({ I }) => {
+  I.amOnPage('/')
+}).retry(2)
+```
+
+Set per feature:
+
+```js
+Feature('My Suite', {
+  retryBefore: 2,
+  retryAfter: 1,
+  retryBeforeSuite: 3,
+  retryAfterSuite: 1
+})
+```
+
+Or globally:
+
+```js
+export const config = {
+  retry: [
+    { BeforeSuite: 2, Before: 1, After: 1 }
+  ]
+}
+```
+
+## Retry Priority
+
+When multiple retry configurations exist, higher-priority retries take precedence:
+
+| Priority | Type | Description |
+|----------|------|-------------|
+| **Highest** | Manual Step (`step.retry()`) | Explicit retries in test code |
+| | Automatic Step | `retryFailedStep` plugin |
+| | Multiple Steps (`retryTo`) | Retry groups of steps together |
+| | Scenario Config | Retry entire scenarios |
+| | Feature Config | Retry all scenarios in a feature |
+| **Lowest** | Hook Config | Retry failed hooks |
+
+`retryTo` operates independently from step-level retries. If a step inside `retryTo` fails, the entire block retries.
+
+## Best Practices
+
+1. **Understand helper retries first** — Playwright/Puppeteer/WebDriver already retry actions internally
+2. **Start with scenario retries** — simpler and less likely to cause issues
+3. **Use manual retries for known flaky steps** — most predictable behavior
+4. **Enable `deferToScenarioRetries`** — prevents excessive retries (default)
+5. **Don't over-retry** — if tests fail consistently, fix the root cause
+6. **Use grep patterns** — apply retries only where needed
+7. **Retry timeouts, not bugs** — retries handle environmental issues, not code defects
+8. **Consider healing for complex recovery** — see [Self-Healing Tests](/heal)
+
+## Troubleshooting
+
+### Tests running too long
+
+- Confirm `deferToScenarioRetries: true` (the default)
+- Reduce retry counts
+- Use `grep` patterns to target specific tests
+- Add problematic steps to `ignoredSteps`
+
+### Retries not working
+
+1. Check configuration syntax
+2. Check the priority table — a higher-priority retry may be overriding
+3. Confirm `disableRetryFailedStep: true` is not set on the scenario
+4. Confirm the step isn't in `ignoredSteps`
+
+Debug with:
+
+```bash
+DEBUG_RETRY_PLUGIN=1 npx codeceptjs run
+```
+
+### `step.retry()` is undefined
+
+Import `step` from codeceptjs:
+
+```js
+import step from 'codeceptjs/steps'
+```

package/docs/secrets.md

@@ -0,0 +1,150 @@
+# Secrets
+
+It is possible to **mask out sensitive data** when passing it to steps. This is important when filling password fields, or sending secure keys to API endpoint. CodeceptJS provides two approaches for masking sensitive data:
+
+## 1. Using the `secret()` Function
+
+Wrap data in `secret` function to mask sensitive values in output and logs.
+
+For basic string `secret` just wrap a value into a string:
+
+```js
+I.fillField('password', secret('123456'))
+```
+
+When executed it will be printed like this:
+
+```
+I fill field "password" "*****"
+```
+
+**Other Examples**
+
+```js
+I.fillField('password', secret('123456'))
+I.append('password', secret('123456'))
+I.type('password', secret('123456'))
+```
+
+For an object, which can be a payload to POST request, specify which fields should be masked:
+
+```js
+I.sendPostRequest(
+  '/login',
+  secret(
+    {
+      name: 'davert',
+      password: '123456',
+    },
+    'password',
+  ),
+)
+```
+
+The object created from `secret` is as Proxy to the object passed in. When printed password will be replaced with \*\*\*\*.
+
+> ⚠️ Only direct properties of the object can be masked via `secret`
+
+## 2. Global Sensitive Data Masking
+
+CodeceptJS can automatically mask sensitive data in all output (logs, steps, debug messages, errors) using configurable patterns. This feature uses the `maskSensitiveData` configuration option.
+
+### Basic Usage (Boolean)
+
+Enable basic masking with predefined patterns:
+
+```js
+// codecept.conf.js
+export const config = {
+  // ... other config
+  maskSensitiveData: true,
+}
+```
+
+This will mask common sensitive data patterns like:
+
+- Authorization headers
+- API keys
+- Passwords
+- Tokens
+- Client secrets
+
+### Advanced Usage (Custom Patterns)
+
+Define your own masking patterns:
+
+```js
+// codecept.conf.js
+export const config = {
+  // ... other config
+  maskSensitiveData: {
+    enabled: true,
+    patterns: [
+      {
+        name: 'Email',
+        regex: /(\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b)/gi,
+        mask: '[MASKED_EMAIL]',
+      },
+      {
+        name: 'Credit Card',
+        regex: /\b(?:\d{4}[- ]?){3}\d{4}\b/g,
+        mask: '[MASKED_CARD]',
+      },
+      {
+        name: 'Phone Number',
+        regex: /(\+?1[-.\s]?)?\(?([0-9]{3})\)?[-.\s]?([0-9]{3})[-.\s]?([0-9]{4})/g,
+        mask: '[MASKED_PHONE]',
+      },
+      {
+        name: 'SSN',
+        regex: /\b\d{3}-\d{2}-\d{4}\b/g,
+        mask: '[MASKED_SSN]',
+      },
+    ],
+  },
+}
+```
+
+### Pattern Configuration
+
+Each custom pattern object should have:
+
+- `name`: A descriptive name for the pattern
+- `regex`: A JavaScript regular expression to match the sensitive data
+- `mask`: The replacement string to show instead of the sensitive data
+
+### Examples
+
+With the above configuration:
+
+**Input:**
+
+```
+User email: john.doe@company.com
+Credit card: 4111 1111 1111 1111
+Phone: +1-555-123-4567
+```
+
+**Output:**
+
+```
+User email: [MASKED_EMAIL]
+Credit card: [MASKED_CARD]
+Phone: [MASKED_PHONE]
+```
+
+### Where Masking Applies
+
+Global sensitive data masking is applied to:
+
+- Step descriptions and output
+- Debug messages (`--debug` mode)
+- Log messages (`--verbose` mode)
+- Error messages
+- Success messages
+
+> ⚠️ Direct `console.log()` calls in helper functions are not masked. Use CodeceptJS output functions instead.
+
+### Combining Both Approaches
+
+You can use both `secret()` function and global masking together. The `secret()` function is applied first, then global patterns are applied to the remaining output.

package/docs/sessions.md

@@ -0,0 +1,80 @@
+---
+permalink: /sessions
+title: Multiple Sessions
+---
+
+# Multiple Sessions
+
+CodeceptJS can run several browser sessions in a single test. This is useful for testing real-time features like chat, notifications, or multi-user workflows.
+
+```js
+Scenario('two users chat', ({ I }) => {
+  I.amOnPage('/chat')
+  I.fillField('name', 'davert')
+  I.click('Sign In')
+  I.see('Hello, davert')
+
+  session('john', () => {
+    I.amOnPage('/chat')
+    I.fillField('name', 'john')
+    I.click('Sign In')
+    I.see('Hello, john')
+  })
+
+  // back in default session (davert)
+  I.fillField('message', 'Hi, john')
+  I.see('me: Hi, john', '.messages')
+
+  session('john', () => {
+    I.see('davert: Hi, john', '.messages')
+  })
+})
+```
+
+Switch back to a session at any point by using the same name.
+
+## Session Configuration
+
+Override the browser config for a specific session:
+
+```js
+session('mobile', { browser: 'webkit', viewport: { width: 375, height: 812 } }, () => {
+  I.amOnPage('/')
+  I.see('Mobile View')
+})
+```
+
+## Opening Sessions Without Switching
+
+Open sessions in the background without switching to them immediately:
+
+```js
+Scenario('multi-user test', ({ I }) => {
+  session('john')
+  session('mary')
+  session('jane')
+
+  I.amOnPage('/')
+
+  session('mary', () => {
+    I.amOnPage('/login')
+  })
+})
+```
+
+## Returning Values
+
+`session` can return a value for use in the main scenario:
+
+```js
+const profileName = await session('john', () => {
+  I.amOnPage('/profile')
+  return I.grabTextFrom({ css: 'h1' })
+})
+I.fillField('Recipient', profileName)
+```
+
+## Notes
+
+- Sessions can use the `I` object, page objects, and any other objects declared for the scenario
+- `within` can be used inside a session, but `session` cannot be called from inside `within`

package/docs/shadow.md

@@ -0,0 +1,68 @@
+---
+permalink: /shadow
+title: Locating Shadow Dom Elements
+---
+
+# Locating Shadow Dom Elements
+
+> ℹ Shadow DOM locators is supported only in WebDriver helper
+
+Shadow DOM is one of the key browser features that make up web components. Web components are a really great way to build reusable elements, and are able to scale all the way up to complete web applications. Style encapsulation, the feature that gives shadow DOM it's power, has been a bit of a pain when it comes to E2E or UI testing. Things just got a little easier though, as CodeceptJS introduced built-in support for shadow DOM via locators of type `shadow`. Let's dig into what they're all about.
+
+Generated HTML code may often look like this (ref: [Salesforce's Lighting Web Components](https://github.com/salesforce/lwc)):
+
+```js
+<body>
+  <my-app>
+    <recipe-hello>
+      <button>Click Me!</button>
+    </recipe-hello>
+    <recipe-hello-binding>
+      <ui-input>
+        <input type="text" class="input">
+      </ui-input>
+    </recipe-hello-binding>
+  </my-app>
+</body>
+```
+
+This uses custom elements, `my-app`, `recipe-hello`, `recipe-hello-binding` and `ui-input`. It's quite common that clickable elements are not actual `a` or `button` elements but custom elements. This way `I.click('Click Me!');` won't work, as well as `fillField('.input', 'value)`. Finding a correct locator for such cases turns to be almost impossible until `shadow` element support is added to CodeceptJS.
+
+## Locate Shadow Dom
+
+For Web Components or [Salesforce's Lighting Web Components](https://github.com/salesforce/lwc) with Shadow DOM's, a special `shadow` locator is available. It allows to select an element by its shadow dom sequences and sequences are defined as an Array of `elements`. Elements defined in the array of `elements` must be in the ordered the shadow elements appear in the DOM.
+
+```js
+{ shadow: ['my-app', 'recipe-hello', 'button'] }
+{ shadow: ['my-app', 'recipe-hello-binding', 'ui-input', 'input.input'] }
+```
+
+In WebDriver, you can use shadow locators in any method where locator is required.
+
+For example, to fill value in `input` field or to click the `Click Me!` button, in above HTML code:
+
+```js
+I.fillField({ shadow: ['my-app', 'recipe-hello-binding', 'ui-input', 'input.input'] }, 'value');
+I.click({ shadow: ['my-app', 'recipe-hello', 'button'] });
+```
+
+## Example
+
+```js
+Feature('Shadow Dom Locators');
+
+Scenario('should fill input field within shadow elements', ({I}) => {
+
+  // navigate to LWC webpage containing shadow dom
+  I.amOnPage('https://recipes.lwc.dev/');
+
+  // click Click Me! button
+  I.click({ shadow: ['my-app', 'recipe-hello', 'button'] });
+
+  // fill the input field
+  I.fillField({ shadow: ['my-app', 'recipe-hello-binding', 'ui-input', 'input.input'] }, 'value');
+
+});
+
+
+```

package/docs/store.md

@@ -0,0 +1,94 @@
+---
+permalink: /store
+title: Store
+---
+
+# Store
+
+`store` is a global object that holds the state of the current run — which test and step are executing, which modes are enabled, and where the project lives on disk. Listeners, plugins, and helpers read it to find out *where in the lifecycle* they are without threading that information through every call.
+
+```js
+import { store } from 'codeceptjs'
+
+event.dispatcher.on(event.step.before, () => {
+  if (store.dryRun) return            // skip side effects on a dry run
+  console.log(`running ${store.currentTest?.title}`)
+})
+```
+
+It is a single shared object for the whole process. Read from it freely; write to it only if you are deliberately driving execution state (see [Writing to the store](#writing-to-the-store)).
+
+## Fields
+
+### Required
+
+Set once when the runner starts and immutable afterwards. Resolved from the test root, falling back to the legacy `global.codecept_dir` / `global.output_dir`.
+
+| Field | Type | What it is |
+| --- | --- | --- |
+| `store.codeceptDir` | `string` | root directory of the tests |
+| `store.outputDir` | `string` | resolved output directory for artifacts |
+| `store.workerMode` | `boolean` | `true` when running inside a [parallel](/parallel) worker |
+
+### Session config
+
+Set at session start and stable for that session. They reflect how the run was invoked.
+
+| Field | Type | What it is |
+| --- | --- | --- |
+| `store.debugMode` | `boolean` | run started with `--debug` |
+| `store.timeouts` | `boolean` | [timeouts](/timeouts) are enabled |
+| `store.autoRetries` | `boolean` | step auto-retries are active (the `tryTo` [effect](/effects) turns this off while it runs) |
+| `store.dryRun` | `boolean` | run is a `--dry-run`; steps must not cause side effects |
+| `store.featureOnly` | `boolean` | a `Feature.only()` was used |
+| `store.scenarioOnly` | `boolean` | a `Scenario.only()` was used |
+| `store.maskSensitiveData` | `boolean \| object` | mask [secret](/secrets) values in output |
+| `store.noGlobals` | `boolean` | `noGlobals` mode — the user imports everything instead of relying on globals |
+
+### State
+
+Changes constantly as the run progresses. The most useful fields for plugins and listeners:
+
+| Field | Type | What it is |
+| --- | --- | --- |
+| `store.onPause` | `boolean` | execution is paused inside [`pause()`](/basics#pause) |
+| `store.currentTest` | `Test \| null` | the running test, or `null` between tests |
+| `store.currentStep` | `Step \| null` | the running step, or `null` |
+| `store.currentSuite` | `Suite \| null` | the running suite, or `null` between suites |
+| `store.tsFileMapping` | `Map \| null` | TypeScript source-map lookup, when running `.ts` tests |
+
+`currentTest`, `currentStep`, and `currentSuite` carry the same objects passed to lifecycle events — see the [test and step object](/architecture#test-object) fields.
+
+## Reading the store
+
+A typical use is gating expensive or unsafe work by run mode:
+
+```js
+import { store, recorder } from 'codeceptjs'
+
+event.dispatcher.on(event.test.before, () => {
+  if (store.dryRun || store.workerMode) return
+
+  recorder.add('seed fixture data', async () => {
+    await api.post('/users', { name: 'john' })
+  })
+})
+```
+
+`store.currentTest` is the simplest way to attach context to the current test from anywhere — for example tagging an artifact:
+
+```js
+import { store } from 'codeceptjs'
+
+store.currentTest?.artifacts.push({ name: 'log', path: '/tmp/run.log' })
+```
+
+## Writing to the store
+
+CodeceptJS keeps the state fields current for you — the built-in [`store` listener](https://github.com/codeceptjs/CodeceptJS/blob/master/lib/listener/store.js) sets `currentSuite` and `currentTest` around each suite and test. You rarely need to write to it.
+
+Write only when you are deliberately driving execution — for example, a tool that opens `pause()` sets `store.onPause`. The required fields (`codeceptDir`, `outputDir`) are locked after `store.initialize()` and cannot be reassigned.
+
+---
+
+**See also:** [Architecture](/architecture) · [Extending CodeceptJS](/hooks) · [Events](/architecture#events)