codeceptjs 4.0.0-rc.2 → 4.0.0-rc.20

package/docs/els.md

@@ -0,0 +1,328 @@
+## Element Access
+
+The `els` module provides low-level element manipulation functions for CodeceptJS tests, allowing for more granular control over element interactions and assertions. Elements are wrapped in a unified `WebElement` class that provides a consistent API across all helpers (Playwright, WebDriver, Puppeteer).
+
+> **Note:** For a comprehensive guide on element-based testing patterns and best practices, see [Element-Based Testing](element-based-testing.md).
+
+### Usage
+
+Import the els functions in your test file:
+
+```js
+import { element, eachElement, expectElement, expectAnyElement, expectAllElements } from 'codeceptjs/els'
+```
+
+## element
+
+The `element` function allows you to perform custom operations on the first matching element found by a locator. It provides a low-level way to interact with elements when the built-in helper methods aren't sufficient.
+
+### Syntax
+
+```js
+element(purpose, locator, fn);
+// or
+element(locator, fn);
+```
+
+### Parameters
+
+- `purpose` (optional) - A string describing the operation being performed. If omitted, a default purpose will be generated from the function.
+- `locator` - A locator string/object to find the element(s).
+- `fn` - An async function that receives the element as its argument and performs the desired operation. `el` argument is a `WebElement` wrapper providing a consistent API across all helpers.
+
+### Returns
+
+Returns the result of the provided async function executed on the first matching element.
+
+### Example
+
+```js
+Scenario('my test', async ({ I }) => {
+  // combine element function with standard steps:
+  I.amOnPage('/cart');
+
+  // but use await every time you use element function
+  await element(
+    // with explicit purpose
+    'check custom attribute',
+    '.button',
+    async el => await el.getAttribute('data-test'),
+  );
+
+  // or simply
+  await element('.button', async el => {
+    return await el.isEnabled();
+  });
+});
+```
+
+### Notes
+
+- Only works with helpers that implement the `_locate` method
+- The function will only operate on the first element found, even if multiple elements match the locator
+- The provided callback must be an async function
+- Throws an error if no helper with `_locate` method is enabled
+
+## eachElement
+
+The `eachElement` function allows you to perform operations on each element that matches a locator. It's useful for iterating through multiple elements and performing the same operation on each one.
+
+### Syntax
+
+```js
+eachElement(purpose, locator, fn);
+// or
+eachElement(locator, fn);
+```
+
+### Parameters
+
+- `purpose` (optional) - A string describing the operation being performed. If omitted, a default purpose will be generated from the function.
+- `locator` - A locator string/object to find the element(s).
+- `fn` - An async function that receives two arguments:
+  - `el` - The current element being processed
+  - `index` - The index of the current element in the collection
+
+### Returns
+
+Returns a promise that resolves when all elements have been processed. If any element operation fails, the function will throw the first encountered error.
+
+### Example
+
+```js
+Scenario('my test', async ({ I }) => {
+  // combine element function with standard steps:
+  I.click('/hotels');
+
+  // iterate over elements but don't forget to put await
+  await eachElement(
+    'validate list items', // explain your actions for future review
+    '.list-item', // locator
+    async (el, index) => {
+      const text = await el.getText();
+      console.log(`Item ${index}: ${text}`);
+    },
+  );
+
+  // Or simply check if all checkboxes are checked
+  await eachElement('input[type="checkbox"]', async el => {
+    const checked = await el.getProperty('checked');
+    if (!checked) {
+      throw new Error('Found unchecked checkbox');
+    }
+  });
+});
+```
+
+### Notes
+
+- Only works with helpers that implement the `_locate` method
+- The function will process all elements that match the locator
+- The provided callback must be an async function
+- If an operation fails on any element, the error is logged and the function continues processing remaining elements
+- After all elements are processed, if any errors occurred, the first error is thrown
+- Throws an error if no helper with `_locate` method is enabled
+
+## expectElement
+
+The `expectElement` function allows you to perform assertions on the first element that matches a locator. It's designed for validating element properties or states and will throw an assertion error if the condition is not met.
+
+### Syntax
+
+```js
+expectElement(locator, fn);
+```
+
+### Parameters
+
+- `locator` - A locator string/object to find the element(s).
+- `fn` - An async function that receives the element as its argument and should return a boolean value:
+  - `true` - The assertion passed
+  - `false` - The assertion failed
+
+### Returns
+
+Returns a promise that resolves when the assertion is complete. Throws an assertion error if the condition is not met.
+
+### Example
+
+```js
+// Check if a button is enabled
+await expectElement('.submit-button', async el => {
+  return await el.isEnabled();
+});
+
+// Verify element has specific text content
+await expectElement('.header', async el => {
+  const text = await el.getText();
+  return text === 'Welcome';
+});
+
+// Check for specific attribute value
+await expectElement('#user-profile', async el => {
+  const role = await el.getAttribute('role');
+  return role === 'button';
+});
+```
+
+### Notes
+
+- Only works with helpers that implement the `_locate` method
+- The function will only check the first element found, even if multiple elements match the locator
+- The provided callback must be an async function that returns a boolean
+- The assertion message will include both the locator and the function used for validation
+- Throws an error if no helper with `_locate` method is enabled
+
+## expectAnyElement
+
+The `expectAnyElement` function allows you to perform assertions where at least one element from a collection should satisfy the condition. It's useful when you need to verify that at least one element among many matches your criteria.
+
+### Syntax
+
+```js
+expectAnyElement(locator, fn);
+```
+
+### Parameters
+
+- `locator` - A locator string/object to find the element(s).
+- `fn` - An async function that receives the element as its argument and should return a boolean value:
+  - `true` - The assertion passed for this element
+  - `false` - The assertion failed for this element
+
+### Returns
+
+Returns a promise that resolves when the assertion is complete. Throws an assertion error if no elements satisfy the condition.
+
+### Example
+
+```js
+Scenario('validate any element matches criteria', async ({ I }) => {
+  // Navigate to the page
+  I.amOnPage('/products');
+
+  // Check if any product is marked as "in stock"
+  await expectAnyElement('.product-item', async el => {
+    const status = await el.getAttribute('data-status');
+    return status === 'in-stock';
+  });
+
+  // Verify at least one price is below $100
+  await expectAnyElement('.price-tag', async el => {
+    const price = await el.getText();
+    return parseFloat(price.replace('$', '')) < 100;
+  });
+
+  // Check if any button in the list is enabled
+  await expectAnyElement('.action-button', async el => {
+    return await el.isEnabled();
+  });
+});
+```
+
+### Notes
+
+- Only works with helpers that implement the `_locate` method
+- The function will check all matching elements until it finds one that satisfies the condition
+- Stops checking elements once the first matching condition is found
+- The provided callback must be an async function that returns a boolean
+- Throws an assertion error if no elements satisfy the condition
+- Throws an error if no helper with `_locate` method is enabled
+
+## expectAllElements
+
+The `expectAllElements` function verifies that every element matching the locator satisfies the given condition. It's useful when you need to ensure that all elements in a collection meet specific criteria.
+
+### Syntax
+
+```js
+expectAllElements(locator, fn);
+```
+
+### Parameters
+
+- `locator` - A locator string/object to find the element(s).
+- `fn` - An async function that receives the element as its argument and should return a boolean value:
+  - `true` - The assertion passed for this element
+  - `false` - The assertion failed for this element
+
+### Returns
+
+Returns a promise that resolves when all assertions are complete. Throws an assertion error as soon as any element fails the condition.
+
+### Example
+
+```js
+Scenario('validate all elements meet criteria', async ({ I }) => {
+  // Navigate to the page
+  I.amOnPage('/dashboard');
+
+  // Verify all required fields have the required attribute
+  await expectAllElements('.required-field', async el => {
+    const required = await el.getAttribute('required');
+    return required !== null;
+  });
+
+  // Check if all checkboxes in a form are checked
+  await expectAllElements('input[type="checkbox"]', async el => {
+    const checked = await el.getProperty('checked');
+    return checked === true;
+  });
+
+  // Verify all items in a list have non-empty text
+  await expectAllElements('.list-item', async el => {
+    const text = await el.getText();
+    return text.trim().length > 0;
+  });
+
+  // Ensure all buttons in a section are enabled
+  await expectAllElements('#action-section button', async el => {
+    return await el.isEnabled();
+  });
+});
+```
+
+### Notes
+
+- Only works with helpers that implement the `_locate` method
+- The function checks every element that matches the locator
+- Fails fast: stops checking elements as soon as one fails the condition
+- The provided callback must be an async function that returns a boolean
+- The assertion message will include which element number failed (e.g., "element #2 of...")
+- Throws an error if no helper with `_locate` method is enabled
+
+## WebElement API
+
+Elements passed to your callbacks are wrapped in a `WebElement` class that provides a consistent API across all helpers. For complete documentation of the WebElement API, see [WebElement](WebElement.md).
+
+Quick reference of available methods:
+
+```js
+await element('.my-element', async el => {
+  // Get element information
+  const text = await el.getText()
+  const attr = await el.getAttribute('data-value')
+  const prop = await el.getProperty('value')
+  const html = await el.getInnerHTML()
+
+  // Check state
+  const visible = await el.isVisible()
+  const enabled = await el.isEnabled()
+  const exists = await el.exists()
+
+  // Interactions
+  await el.click()
+  await el.type('text')
+
+  // Child elements
+  const child = await el.$('.child')
+  const children = await el.$$('.child')
+
+  // Position
+  const box = await el.getBoundingBox()
+
+  // Native access
+  const helper = el.getHelper()
+  const native = el.getNativeElement()
+})
+```

package/docs/examples.md

@@ -0,0 +1,161 @@
+---
+permalink: /examples
+layout: Section
+sidebar: false
+title: Examples
+editLink: false
+---
+
+# Examples
+
+> Add your own examples to our [Wiki Page](https://github.com/codeceptjs/CodeceptJS/wiki/Examples)
+
+## [TodoMVC Examples](https://github.com/codecept-js/examples)
+
+![](https://github.com/codecept-js/examples/raw/master/todo.png)
+
+Playground repository where you can run tests in different helpers on a basic single-page website.
+
+Tests repository demonstrate usage of
+
+- Playwright helper
+- Puppeteer helper
+- WebDriver helper
+- Toggle headless mode with env variables
+- PageObjects
+- Cucumber syntax
+
+## [Basic Examples](https://github.com/Codeception/CodeceptJS/tree/master/examples)
+
+CodeceptJS repo contains basic tests (both failing and passing) just to show how it works.
+Our team uses it to test new features and run simple scenarios.
+
+## [CodeceptJS Cucumber E2E Framework](https://github.com/gkushang/codeceptjs-e2e)
+
+This repository contains complete E2E framework for CodeceptJS with Cucumber and SauceLabs Integration
+
+- CodecepJS-Cucumber E2E Framework
+- Saucelabs Integration
+- Run Cross Browser tests in Parallel on SauceLabs with a simple command
+- Run tests on `chrome:headless`
+- Page Objects
+- `Should.js` Assertion Library
+- Uses `wdio` service (selenium-standalone, sauce)
+- Allure HTML Reports
+- Uses shared Master configuration
+- Sample example and feature files of GitHub Features
+
+## [Enterprise Grade Tests](https://github.com/uc-cdis/gen3-qa)
+
+Complex testing solution by [Gen3](https://github.com/uc-cdis/gen3-qa)
+
+Includes
+
+- classical CodeceptJS tests
+- BDD tests
+- Jenkins integration
+- Complex Before/BeforeSuite scripts and more
+
+## [Testing Single Page Application](https://github.com/bugiratracker/codeceptjs-demo)
+
+End 2 end tests for Task management app (currently offline).
+
+Tests repository demonstrate usage of
+
+- Puppeteer helper
+- ApiDataFactory helper
+- autoLogin plugin
+- Dynamic config with profiles
+
+## [Practical E2E Tests](https://gitlab.com/paulvincent/codeceptjs-e2e-testing)
+
+Examples from the book [Practical End 2 End Testing with CodeceptJS](https://leanpub.com/codeceptjs/) by **Paul Vincent Beigang**.
+
+This repository demonstrates usage of:
+
+- dynamic config with profiles
+- testing WYSIWYG editor
+- GitLab CI
+
+## [Amazon Tests v2](https://gitlab.com/thanhnguyendh/codeceptjs-wdio-services)
+
+Testing Amazon website using Selenium WebDriver.
+
+This repository demonstrates usage of:
+
+- WebDriver helper
+- Page Objects
+- wdio services (selenium-standalone)
+- Parallel execution
+- GitLab CI setup
+
+## [Tests with Docker Compose](https://github.com/mathesouza/codeceptjs-docker-compose)
+
+Running CodeceptJS tests with Docker Compose
+
+This repository demonstrates usage of:
+
+- CodeceptJS Docker image
+- WebDriver helper
+- Allure plugin
+
+## [AngularJS Example Tests](https://github.com/armno/angular-e2e-codeceptjs-example)
+
+Based on [Setting up End-to-End Testing in Angular Project with CodeceptJS](https://medium.com/@armno/setting-up-end-to-end-testing-in-angular-project-with-codeceptjs-ac1784de3420) post by Armno Prommarak.
+
+This repository demonstrates usage of
+
+- Puppeteer helper
+- Working with Angular CLI
+- Reports with Mochawesome helper
+
+## [REST Example Tests](https://github.com/PeterNgTr/codeceptjs-rest-demo)
+
+This repository demonstrates usage of
+
+- REST helper
+
+## [Automation Starter](https://github.com/sjorrillo/automation-starter)
+
+The purpose of this application is for learning the basics and how to use good practices and useful tools in automation.
+
+- Puppeteer helper
+- Working with gherkin, also it has type definitions and to be able to use them inside when, given and then make sure you add `declare function inject(): { I: CodeceptJS.I, [key: string]: any; };`in the `steps.d.ts`file
+- Linting `airbnb-base`, `codeceptjs/codeceptjs` and full ES6 support
+
+## [Example for using: Puppeteer, Gherkin, Allure with parallel execution](https://github.com/SchnuckySchuster/codeceptJSExample)
+
+This is a ready to use example that shows how to integrate CodeceptJS with Puppeteer and Allure as reporting tool.
+
+- detailed ReadMe
+- tests written in cucumber alongside tests written in the codeceptJS DSL
+- puppeteer helper example
+- test steps, pages, fragments
+- examples for sequential and parallel execution
+- generation of allure test results
+
+## [Example for Advanced REST API testing: TypeScript, Axios, CodeceptJS, Jest Expect, Docker, Allure, Mock-Server, Prettier + Eslint, pre-commit, Jest Unit Tests ](https://github.com/EgorBodnar/rest-axios-codeceptjs-allure-docker-test-example)
+
+One button example with built-in mocked backend.
+
+If you already have a UI testing solution based on the CodeceptJS and you need to implement advanced REST API testing you can just extend your existing framework. Use this implementation as an example.
+This is necessary if all integrations with TMS and CI/CD are already configured, and you do not want to reconnect and configure the plugins and libraries used for the new test runner. Use CodeceptJS!
+
+- Easy run
+- Detailed README
+- Well documented mocked backend's REST API endpoints
+- HTTP request client with session support and unit tests
+- Exemplary code control
+- Ready to launch in a CI/CD system as is
+- OOP, Test data models and builders, endpoint decorators
+
+## [Playwright fun with CodeceptJS](https://github.com/PeterNgTr/codeceptjs-playwright-fun)
+
+- Tests are written in TS
+- CI/CD with Github Actions
+- Page Object Model is applied
+- ReportPortal Integration
+
+## How to
+
+- Create a plugin with TS [link](https://github.com/reutenkoivan/codeceptjs-plugins/tree/main/packages/html-snapshot-on-fail)

package/docs/heal.md

@@ -0,0 +1,213 @@
+# Self-Healing Tests
+
+Browser and Mobile tests can fail for vareity of reasons. However, on a big projects there are about 5-10 causes of flaky tests. The more you work and understand your end-to-end tests the more you learn patterns of failure. And after the research you understand how a test could have been fixed: to reload a page, to click that button once again, restart API request. If by looking into a failure you understand what, as a user, you would do to fix that error, then maybe you could teach your tests to heal themselves.
+
+## What is Healing
+
+**Healing defines the way how a test reacts to failure**. You can define multiple healing recipes that could take all needed information: error message, failed test, step, page URL, HTML, etc. A healing recipe can perform some action to fix the failing test on the fly and continue its execution.
+
+![](/img/healing.png)
+
+Let's start with a common scenario. If a user suddenly becomes unauthorized and is moved to a sign-in page, or receives an unauthorized message on the page, we can heal by navigating to the `/login` page and trying to enter credentials again.
+
+```js
+heal.addRecipe('loginOnUnauthorized', {
+  priority: 10,
+  steps: ['click', 'see', 'amOnPage'],
+  prepare: {
+    url: ({ I }) => I.grabCurrentUrl(),
+    html: ({ I }) => I.grabHTMLFrom('body'),
+  },
+  fn: async ({ url, error, step, html }) => {
+    if (!url.includes('/login') && !error.message.toLowerCase().includes('unauthorized') && !html.toLowerCase().includes('unauthorized')) return;
+    
+    return ({ I }) => {
+      I.amOnPage('/login');
+      I.fillField('Email', 'test@example.com');
+      I.fillField('Password', '123456');
+      I.click('Sign in');
+      I[step.name](...step.args);
+    };
+  },
+});
+```
+
+Another example is a very basic healing recipe. If after a click test has failed, try to reload page, and continue.
+
+```js
+heal.addRecipe('reload', {
+  priority: 10,
+  steps: ['click'],
+  fn: async ({ step }) => {
+    return ({ I }) => {
+      I.refreshPage();
+      I[step.name](...step.args);
+    };
+  },
+});
+```
+
+Sure, this won't always work and probably won't be useful on every project. But let's follow the idea: if a click has failed, probably the button is not on a page, maybe it is an issue of rendering, maybe some other element overlapped our button, so if we try to reload page we can continue test execution. At least, this is what manual QA would do if they will run the following test in a browser. They will try to reload a page before reporting "it has failed".
+
+So if it is a long end-2-end test that implements user journey, it is more valuable to continue its execution when possible, then fixing a minor issues like overlapping elements. Healing like this can improve the stability of a test.
+
+The example above is only one way a test can be healed. But you can define as many heal recipes as you like. What heal recipe would be effective in your case is depends on a system you test, so **there are no pre-defined heal recipes**. 
+
+## Healing Patterns
+
+There are some ideas where healing can be useful to you:
+
+* **Authorization**. If a user suddenly becomes unauthorized and is moved to a sign-in page, or receives an unauthorized message on the page, we can heal by navigating to the `/login` page and trying to enter credentials.
+* **Networking**. If a test depends on a remote resource, and fails because this resource is not available, you may try to send API request to restore that resource before throwing an error.
+* **Data Consistency**. A test may fail because you noticed the data glitch in a system. Instead of failing a test you may try to clean up the data and try again to proceed.
+* **UI Change**. If there is a planned UI migration of a component, for instance Button was changed to Dropdown. You can prepare test so if it fails clicking Button it can try to do so with Dropdown.
+* **Do it again**. If you know, that going one step back and trying to do same actions may solve the issue, you can do so from healers. For instance, a modal didn't render correctly, so you can close it and try to click to open it again.
+
+## Healing vs Retries
+
+Unlike retries heal recipes has following benefits:
+
+* Heal recipes are **declarative**, they are not added directly into into the test code. This keeps test clean and scenario-focused,
+* Retry can only re-run failed step(s), but heal recipe can **perform wide set of actions**
+* Heal recipe **can react to any step of any test**. So if you catch a common error and you can heal it, you won't need to guess where it can be thrown.
+
+## How to Start Healing
+
+To enable healing, you need to define healing recipes and enable heal plugin.
+
+Create basic healing recipes using this command:
+
+```
+npx codeceptjs generate:heal
+```
+
+or
+
+```
+npx codeceptjs gr
+```
+
+this will generate `recipes.js` (or `recipes.ts`) in the root directory. Provided default recipe include [AI healing](#ai-healing) and `clickAndType` recipe that replaces `fillField` with `click`+`type`. Use them as examples to write your own heal recipes that will fit for application you are testing.
+
+Require `recipes` file and add `heal` plugin to `codecept.conf` file:
+
+```js
+
+import './heal';
+
+export const config = {
+  // ...
+  plugins: {
+    heal: {
+      enabled: true
+    }
+  }
+}
+```
+
+> Please note that, healing has no sense while developing tests, so it won't work in `--debug` mode. 
+
+## Writing Recipes
+
+Custom heal recipes can be added by running `heal.addRecipe()` function. By default it should be added to `recipes.js` (or `recipes.ts`) file. 
+
+Let's see what recipe consist of:
+
+```js
+heal.addRecipe('reloadPageOnUserAccount', {
+  // recipe priority
+  // which recipe should be tried first 
+  priority: 10,
+
+  // an array of steps which may cause the error
+  // after which a recipe should be activate
+  steps: [
+    'click',
+  ],
+
+  // if you need some additional information like URL of a page,
+  // or its HTML, you can add this context to healing function by 
+  // defining `prepare` list of variable
+  prepare: {
+    url: ({ I }) => I.grabCurrentUrl(),
+    html: ({ I }) => I.grabHTMLFrom('body'),
+    // don't add variables that you won't use inside the recipe
+  },
+
+  // probably we want to execute recipes only on some tests
+  // so you can set a string or regex which will check if a test title matches the name
+  // in this case we execute recipe only on tests that have "@flaky" in their name
+  grep: '@flaky',
+
+  // function to launch healing process 
+  fn: async ({ 
+    // standard context variables
+    step, test, error, prevSteps,
+
+    // variables coming from prepare function
+    html, url,
+    
+    }) => {
+    const stepArgs = step.args;
+
+    // at this point we can decide, should we provide a healing recipe or not
+    // for instance, if URL is not the one we can heal at, we should not provide any recipes
+    if (!url.includes('/user/acccount')) return;
+  
+    // otherwise we return a function that will be executed
+    return ({ I }) => {
+      // this is a very basic example action
+      // probably you should do something more sophisticated
+      // to heal the test
+      I.reloadPage();
+      I.wait(1);
+      I[step.name](...stepArgs);
+    };
+  },
+});
+```
+
+Let's briefly sum up the properties of a recipe:
+
+* `grep` - selects tests by their name to apply heal to
+* `steps` - defines on which steps a recipe should react
+* `priority` - sets the order of recipes being applied
+* `prepare` - declare variables from a context, which can be used for healing
+* `fn` - a function to be applied for healing. It takes all context params: `test`, `step`, `error`, `prevSteps` and returns return either a function or a markdown text with recipes (used by AI healers). If no recipes match the context should not return anything;
+
+
+## AI Healing
+
+AI can be used to heal failed tests. Large Language Models can analyze HTML of a failed test and provide a suggestion what actions should be performed instead. This can be helpful when running tests on CI as AI can make basic decisions to stabilize failing tests. 
+
+> Use **OpenAI, Azure OpenAI, Claude**, or any of other LLM that can take a prompt, analyze request and provide valid JS code which can be executed by CodeceptJS as a healing suggestion.
+
+AI healing recipe is created within `recipes.js` file:
+
+```js
+heal.addRecipe('ai', {
+  priority: 10,
+  prepare: {
+    html: ({ I }) => I.grabHTMLFrom('body'),
+  },
+  steps: [
+    'click',
+    'fillField',
+    'appendField',
+    'selectOption',
+    'attachFile',
+    'checkOption',
+    'uncheckOption',
+    'doubleClick',
+  ],
+  fn: async (args) => {
+    return ai.healFailedStep(args);
+  },
+});
+```
+
+As you use, it will be activated on failed steps and will use HTML of a page as additional information. The prompt, error, and the HTML will be sent to AI provider you configured. 
+
+Learn more how you can [configure AI provider](./ai).
+
+To activate the AI healer don't forget to run tests with `--ai` flag.