codeceptjs 4.0.0-rc.2 → 4.0.0-rc.20

package/docs/retry.md

@@ -0,0 +1,274 @@
+---
+permalink: /retry
+title: Retry Mechanisms
+---
+
+# Retry Mechanisms
+
+CodeceptJS provides flexible retry mechanisms to handle flaky tests. Use retries when dealing with unstable environments, network delays, or timing issues — not to mask bugs in your code.
+
+## Helper Retries
+
+Browser automation helpers (Playwright, Puppeteer, WebDriver) have **built-in retry mechanisms** for element interactions. When you call `I.click('Button')`, Playwright automatically waits for the element to exist, be visible, stable, and enabled — retrying for up to 5 seconds.
+
+Configure the timeout in your helper settings:
+
+```js
+helpers: {
+  Playwright: {
+    timeout: 5000,      // retry actions for up to 5 seconds
+    waitForAction: 100  // wait 100ms before each action
+  }
+}
+```
+
+**Learn more:** [Playwright Helper](/helpers/Playwright), [Timeouts](/timeouts)
+
+## CodeceptJS Retry Levels
+
+When helper retries aren't enough, CodeceptJS adds retry layers on top.
+
+### 1. Manual Step Retry
+
+Retry a specific step known to be flaky:
+
+```js
+import step from 'codeceptjs/steps'
+
+Scenario('checkout', ({ I }) => {
+  I.amOnPage('/cart')
+  I.click('Proceed to Checkout', step.retry(5))  // retry up to 5 times
+  I.see('Payment')
+})
+```
+
+Configure timing with exponential backoff:
+
+```js
+I.click('Submit', step.retry({
+  retries: 3,
+  minTimeout: 1000,  // wait 1 second before first retry
+  maxTimeout: 5000,  // max 5 seconds between retries
+  factor: 1.5        // exponential backoff multiplier
+}))
+```
+
+Pass `0` for infinite retries.
+
+### 2. Automatic Step Retry
+
+Automatically retry all failed steps without modifying test code:
+
+```js
+plugins: {
+  retryFailedStep: {
+    enabled: true,
+    retries: 3
+  }
+}
+```
+
+Steps matching `amOnPage`, `wait*`, `send*`, `execute*`, `run*`, `have*` are skipped by default.
+
+When a scenario has its own retries, step retries are disabled by default (`deferToScenarioRetries: true`). This prevents excessive execution time:
+
+```js
+Scenario('test', { retries: 2 }, ({ I }) => {
+  I.click('Button')  // step retries disabled; scenario retries run instead
+})
+```
+
+To disable step retries for a specific test:
+
+```js
+Scenario('manual retries only', { disableRetryFailedStep: true }, ({ I }) => {
+  I.click('Button', step.retry(5))
+})
+```
+
+Full plugin options:
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `retries` | — | Retries per step |
+| `minTimeout` | — | Milliseconds before first retry |
+| `maxTimeout` | `Infinity` | Max milliseconds between retries |
+| `factor` | — | Exponential backoff multiplier |
+| `randomize` | `false` | Randomize timeout intervals |
+| `ignoredSteps` | `[]` | Patterns/regex of steps to never retry |
+| `deferToScenarioRetries` | `true` | Disable step retries when scenario retries exist |
+| `when` | `() => true` | Function receiving error; return `true` to retry |
+
+### 3. Multiple Steps Retry (retryTo)
+
+Retry a group of steps together as a single operation:
+
+```js
+import { retryTo } from 'codeceptjs/effects'
+
+await retryTo(() => {
+  I.click('Load More')
+  I.see('New Content')
+}, 3)
+```
+
+If any step inside fails, the entire block retries. Use this for sequences that must succeed together — switching into an iframe and filling a form, for example.
+
+**Learn more:** [Effects](/effects#retryto)
+
+### 4. Self-Healing Steps
+
+When a step fails, a healing recipe runs recovery actions and continues the test — without touching test code. With AI healing enabled:
+
+```js
+Scenario('checkout', ({ I }) => {
+  I.click('Proceed to Checkout')
+  I.see('Payment')
+})
+```
+
+- `I.click('Proceed to Checkout')` fails — button was renamed or moved
+  - failed step, error message, and page HTML are sent to an LLM
+  - AI scans page elements and suggests valid replacement actions
+  - CodeceptJS executes the suggestions until one succeeds
+- test continues with `I.see('Payment')`
+
+Run with `--ai` to activate:
+
+```bash
+npx codeceptjs run --ai
+```
+
+You can also write custom recipes for non-UI failures — network errors, data glitches, UI migrations.
+
+**Learn more:** [Self-Healing Tests](/heal), [AI Configuration](/ai)
+
+### 5. Scenario Retry
+
+Retry an entire test when it fails:
+
+```js
+Scenario('API integration', { retries: 3 }, ({ I }) => {
+  I.sendGetRequest('/api/users')
+  I.seeResponseCodeIs(200)
+})
+```
+
+Retry all scenarios globally, or by grep pattern:
+
+```js
+export const config = {
+  retry: [
+    { Scenario: 3, grep: 'API' },       // retry scenarios containing "API" 3 times
+    { Scenario: 5, grep: '@flaky' }      // retry @flaky-tagged scenarios 5 times
+  ]
+}
+```
+
+### 6. Feature Retry
+
+Retry all scenarios within a feature:
+
+```js
+Feature('Payment Processing', { retries: 2 })
+
+Scenario('credit card payment', ({ I }) => { ... })  // retries 2 times
+Scenario('paypal payment', ({ I }) => { ... })        // retries 2 times
+```
+
+Or target features by pattern in config:
+
+```js
+export const config = {
+  retry: [
+    { Feature: 3, grep: 'Integration' }
+  ]
+}
+```
+
+### 7. Hook Retries
+
+Retry `Before`/`After` hooks when they fail:
+
+```js
+Before(({ I }) => {
+  I.amOnPage('/')
+}).retry(2)
+```
+
+Set per feature:
+
+```js
+Feature('My Suite', {
+  retryBefore: 2,
+  retryAfter: 1,
+  retryBeforeSuite: 3,
+  retryAfterSuite: 1
+})
+```
+
+Or globally:
+
+```js
+export const config = {
+  retry: [
+    { BeforeSuite: 2, Before: 1, After: 1 }
+  ]
+}
+```
+
+## Retry Priority
+
+When multiple retry configurations exist, higher-priority retries take precedence:
+
+| Priority | Type | Description |
+|----------|------|-------------|
+| **Highest** | Manual Step (`step.retry()`) | Explicit retries in test code |
+| | Automatic Step | `retryFailedStep` plugin |
+| | Multiple Steps (`retryTo`) | Retry groups of steps together |
+| | Scenario Config | Retry entire scenarios |
+| | Feature Config | Retry all scenarios in a feature |
+| **Lowest** | Hook Config | Retry failed hooks |
+
+`retryTo` operates independently from step-level retries. If a step inside `retryTo` fails, the entire block retries.
+
+## Best Practices
+
+1. **Understand helper retries first** — Playwright/Puppeteer/WebDriver already retry actions internally
+2. **Start with scenario retries** — simpler and less likely to cause issues
+3. **Use manual retries for known flaky steps** — most predictable behavior
+4. **Enable `deferToScenarioRetries`** — prevents excessive retries (default)
+5. **Don't over-retry** — if tests fail consistently, fix the root cause
+6. **Use grep patterns** — apply retries only where needed
+7. **Retry timeouts, not bugs** — retries handle environmental issues, not code defects
+8. **Consider healing for complex recovery** — see [Self-Healing Tests](/heal)
+
+## Troubleshooting
+
+### Tests running too long
+
+- Confirm `deferToScenarioRetries: true` (the default)
+- Reduce retry counts
+- Use `grep` patterns to target specific tests
+- Add problematic steps to `ignoredSteps`
+
+### Retries not working
+
+1. Check configuration syntax
+2. Check the priority table — a higher-priority retry may be overriding
+3. Confirm `disableRetryFailedStep: true` is not set on the scenario
+4. Confirm the step isn't in `ignoredSteps`
+
+Debug with:
+
+```bash
+DEBUG_RETRY_PLUGIN=1 npx codeceptjs run
+```
+
+### `step.retry()` is undefined
+
+Import `step` from codeceptjs:
+
+```js
+import step from 'codeceptjs/steps'
+```

package/docs/secrets.md

@@ -0,0 +1,150 @@
+# Secrets
+
+It is possible to **mask out sensitive data** when passing it to steps. This is important when filling password fields, or sending secure keys to API endpoint. CodeceptJS provides two approaches for masking sensitive data:
+
+## 1. Using the `secret()` Function
+
+Wrap data in `secret` function to mask sensitive values in output and logs.
+
+For basic string `secret` just wrap a value into a string:
+
+```js
+I.fillField('password', secret('123456'))
+```
+
+When executed it will be printed like this:
+
+```
+I fill field "password" "*****"
+```
+
+**Other Examples**
+
+```js
+I.fillField('password', secret('123456'))
+I.append('password', secret('123456'))
+I.type('password', secret('123456'))
+```
+
+For an object, which can be a payload to POST request, specify which fields should be masked:
+
+```js
+I.sendPostRequest(
+  '/login',
+  secret(
+    {
+      name: 'davert',
+      password: '123456',
+    },
+    'password',
+  ),
+)
+```
+
+The object created from `secret` is as Proxy to the object passed in. When printed password will be replaced with \*\*\*\*.
+
+> ⚠️ Only direct properties of the object can be masked via `secret`
+
+## 2. Global Sensitive Data Masking
+
+CodeceptJS can automatically mask sensitive data in all output (logs, steps, debug messages, errors) using configurable patterns. This feature uses the `maskSensitiveData` configuration option.
+
+### Basic Usage (Boolean)
+
+Enable basic masking with predefined patterns:
+
+```js
+// codecept.conf.js
+export const config = {
+  // ... other config
+  maskSensitiveData: true,
+}
+```
+
+This will mask common sensitive data patterns like:
+
+- Authorization headers
+- API keys
+- Passwords
+- Tokens
+- Client secrets
+
+### Advanced Usage (Custom Patterns)
+
+Define your own masking patterns:
+
+```js
+// codecept.conf.js
+export const config = {
+  // ... other config
+  maskSensitiveData: {
+    enabled: true,
+    patterns: [
+      {
+        name: 'Email',
+        regex: /(\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b)/gi,
+        mask: '[MASKED_EMAIL]',
+      },
+      {
+        name: 'Credit Card',
+        regex: /\b(?:\d{4}[- ]?){3}\d{4}\b/g,
+        mask: '[MASKED_CARD]',
+      },
+      {
+        name: 'Phone Number',
+        regex: /(\+?1[-.\s]?)?\(?([0-9]{3})\)?[-.\s]?([0-9]{3})[-.\s]?([0-9]{4})/g,
+        mask: '[MASKED_PHONE]',
+      },
+      {
+        name: 'SSN',
+        regex: /\b\d{3}-\d{2}-\d{4}\b/g,
+        mask: '[MASKED_SSN]',
+      },
+    ],
+  },
+}
+```
+
+### Pattern Configuration
+
+Each custom pattern object should have:
+
+- `name`: A descriptive name for the pattern
+- `regex`: A JavaScript regular expression to match the sensitive data
+- `mask`: The replacement string to show instead of the sensitive data
+
+### Examples
+
+With the above configuration:
+
+**Input:**
+
+```
+User email: john.doe@company.com
+Credit card: 4111 1111 1111 1111
+Phone: +1-555-123-4567
+```
+
+**Output:**
+
+```
+User email: [MASKED_EMAIL]
+Credit card: [MASKED_CARD]
+Phone: [MASKED_PHONE]
+```
+
+### Where Masking Applies
+
+Global sensitive data masking is applied to:
+
+- Step descriptions and output
+- Debug messages (`--debug` mode)
+- Log messages (`--verbose` mode)
+- Error messages
+- Success messages
+
+> ⚠️ Direct `console.log()` calls in helper functions are not masked. Use CodeceptJS output functions instead.
+
+### Combining Both Approaches
+
+You can use both `secret()` function and global masking together. The `secret()` function is applied first, then global patterns are applied to the remaining output.

package/docs/sessions.md

@@ -0,0 +1,80 @@
+---
+permalink: /sessions
+title: Multiple Sessions
+---
+
+# Multiple Sessions
+
+CodeceptJS can run several browser sessions in a single test. This is useful for testing real-time features like chat, notifications, or multi-user workflows.
+
+```js
+Scenario('two users chat', ({ I }) => {
+  I.amOnPage('/chat')
+  I.fillField('name', 'davert')
+  I.click('Sign In')
+  I.see('Hello, davert')
+
+  session('john', () => {
+    I.amOnPage('/chat')
+    I.fillField('name', 'john')
+    I.click('Sign In')
+    I.see('Hello, john')
+  })
+
+  // back in default session (davert)
+  I.fillField('message', 'Hi, john')
+  I.see('me: Hi, john', '.messages')
+
+  session('john', () => {
+    I.see('davert: Hi, john', '.messages')
+  })
+})
+```
+
+Switch back to a session at any point by using the same name.
+
+## Session Configuration
+
+Override the browser config for a specific session:
+
+```js
+session('mobile', { browser: 'webkit', viewport: { width: 375, height: 812 } }, () => {
+  I.amOnPage('/')
+  I.see('Mobile View')
+})
+```
+
+## Opening Sessions Without Switching
+
+Open sessions in the background without switching to them immediately:
+
+```js
+Scenario('multi-user test', ({ I }) => {
+  session('john')
+  session('mary')
+  session('jane')
+
+  I.amOnPage('/')
+
+  session('mary', () => {
+    I.amOnPage('/login')
+  })
+})
+```
+
+## Returning Values
+
+`session` can return a value for use in the main scenario:
+
+```js
+const profileName = await session('john', () => {
+  I.amOnPage('/profile')
+  return I.grabTextFrom({ css: 'h1' })
+})
+I.fillField('Recipient', profileName)
+```
+
+## Notes
+
+- Sessions can use the `I` object, page objects, and any other objects declared for the scenario
+- `within` can be used inside a session, but `session` cannot be called from inside `within`

package/docs/shadow.md

@@ -0,0 +1,68 @@
+---
+permalink: /shadow
+title: Locating Shadow Dom Elements
+---
+
+# Locating Shadow Dom Elements
+
+> ℹ Shadow DOM locators is supported only in WebDriver helper
+
+Shadow DOM is one of the key browser features that make up web components. Web components are a really great way to build reusable elements, and are able to scale all the way up to complete web applications. Style encapsulation, the feature that gives shadow DOM it's power, has been a bit of a pain when it comes to E2E or UI testing. Things just got a little easier though, as CodeceptJS introduced built-in support for shadow DOM via locators of type `shadow`. Let's dig into what they're all about.
+
+Generated HTML code may often look like this (ref: [Salesforce's Lighting Web Components](https://github.com/salesforce/lwc)):
+
+```js
+<body>
+  <my-app>
+    <recipe-hello>
+      <button>Click Me!</button>
+    </recipe-hello>
+    <recipe-hello-binding>
+      <ui-input>
+        <input type="text" class="input">
+      </ui-input>
+    </recipe-hello-binding>
+  </my-app>
+</body>
+```
+
+This uses custom elements, `my-app`, `recipe-hello`, `recipe-hello-binding` and `ui-input`. It's quite common that clickable elements are not actual `a` or `button` elements but custom elements. This way `I.click('Click Me!');` won't work, as well as `fillField('.input', 'value)`. Finding a correct locator for such cases turns to be almost impossible until `shadow` element support is added to CodeceptJS.
+
+## Locate Shadow Dom
+
+For Web Components or [Salesforce's Lighting Web Components](https://github.com/salesforce/lwc) with Shadow DOM's, a special `shadow` locator is available. It allows to select an element by its shadow dom sequences and sequences are defined as an Array of `elements`. Elements defined in the array of `elements` must be in the ordered the shadow elements appear in the DOM.
+
+```js
+{ shadow: ['my-app', 'recipe-hello', 'button'] }
+{ shadow: ['my-app', 'recipe-hello-binding', 'ui-input', 'input.input'] }
+```
+
+In WebDriver, you can use shadow locators in any method where locator is required.
+
+For example, to fill value in `input` field or to click the `Click Me!` button, in above HTML code:
+
+```js
+I.fillField({ shadow: ['my-app', 'recipe-hello-binding', 'ui-input', 'input.input'] }, 'value');
+I.click({ shadow: ['my-app', 'recipe-hello', 'button'] });
+```
+
+## Example
+
+```js
+Feature('Shadow Dom Locators');
+
+Scenario('should fill input field within shadow elements', ({I}) => {
+
+  // navigate to LWC webpage containing shadow dom
+  I.amOnPage('https://recipes.lwc.dev/');
+
+  // click Click Me! button
+  I.click({ shadow: ['my-app', 'recipe-hello', 'button'] });
+
+  // fill the input field
+  I.fillField({ shadow: ['my-app', 'recipe-hello-binding', 'ui-input', 'input.input'] }, 'value');
+
+});
+
+
+```