codeceptjs 3.1.0 → 3.2.0

package/CHANGELOG.md

@@ -1,10 +1,134 @@
+## 3.2.0
+
+🛩️ Features:
+
+**Timeouts implemented**
+  * global timeouts (via `timeout` config option). 
+    * _Breaking change:_ timeout option expects **timeout in seconds**, not in miliseconds as it was previously.
+  * test timeouts (via `Scenario` and `Feature` options)
+    * _Breaking change:_ `Feature().timeout()` and `Scenario().timeout()` calls has no effect and are deprecated
+
+```js
+// set timeout for every test in suite to 10 secs
+Feature('tests with timeout', { timeout: 10 });
+
+// set timeout for this test to 20 secs
+Scenario('a test with timeout', { timeout: 20 }, ({ I }) => {});
+```  
+
+  * step timeouts (See #3059 by @nikocanvacom)
+
+```js
+// set step timeout to 5 secs
+I.limitTime(5).click('Link');
+```  
+ * `stepTimeout` plugin introduced to automatically add timeouts for each step (#3059 by @nikocanvacom).
+
+[**retryTo**](/plugins/#retryto) plugin introduced to rerun a set of steps on failure:
+
+```js
+// editing in text in iframe
+// if iframe was not loaded - retry 5 times
+await retryTo(() => {
+  I.switchTo('#editor frame');
+  I.fillField('textarea', 'value');
+}, 5);
+```
+
+* [Playwright] added `locale` configuration
+* [WebDriver] upgraded to webdriverio v7
+
+🐛 Bugfixes:
+
+* Fixed  allure plugin "Unexpected endStep()" error in #3098 by @abhimanyupandian 
+* [Puppeteer] always close remote browser on test end. See #3054 by @mattonem
+* stepbyStepReport Plugin: Disabled screenshots after test has failed. See #3119 by @ioannisChalkias
+
+
+## 3.1.3
+
+🛩️ Features:
+
+* BDD Improvement. Added `DataTableArgument` class to work with table data structures. 
+
+```js
+const { DataTableArgument } = require('codeceptjs');
+//...
+Given('I have an employee card', (table) => {
+  const dataTableArgument = new DataTableArgument(table);
+  const hashes = dataTableArgument.hashes(); 
+  // hashes = [{ name: 'Harry', surname: 'Potter', position: 'Seeker' }];
+  const rows = dataTableArgument.rows();
+  // rows = [['Harry', 'Potter', Seeker]];
+  }
+```
+See updated [BDD section](https://codecept.io/bdd/) for more API options. Thanks to @EgorBodnar
+
+* Support `cjs` file extensions for config file: `codecept.conf.cjs`. See #3052 by @kalvenschraut
+* API updates: Added `test.file` and `suite.file` properties to `test` and `suite` objects to use in helpers and plugins. 
+
+🐛 Bugfixes:
+
+* [Playwright] Fixed resetting `test.artifacts` for failing tests. See #3033 by @jancorvus. Fixes #3032
+* [Playwright] Apply `basicAuth` credentials to all opened browser contexts. See #3036 by @nikocanvacom. Fixes #3035
+* [WebDriver] Updated `webdriverio` default version to `^6.12.1`. See #3043 by @sridhareaswaran
+* [Playwright] `I.haveRequestHeaders` affects all tabs. See #3049 by @jancorvus
+* BDD: Fixed unhandled empty feature files. Fix #3046 by @abhimanyupandian 
+* Fixed `RangeError: Invalid string length` in `recorder.js` when running huge amount of tests.  
+* [Appium] Fixed definitions for `touchPerform`, `hideDeviceKeyboard`, `removeApp` by @mirao 
+
+📖 Documentation:
+
+* Added Testrail reporter [Reports Docs](https://codecept.io/reports/#testrail)
+
+
+## 3.1.2
+
+🛩️ Features:
+
+* Added `coverage` plugin to generate code coverage for Playwright & Puppeteer. By @anirudh-modi
+* Added `subtitle` plugin to generate subtitles for videos recorded with Playwright. By @anirudh-modi
+* Configuration: `config.tests` to accept array of file patterns. See #2994 by @monsteramba
+
+```js
+exports.config = {
+  tests: ['./*_test.js','./sampleTest.js'],
+  // ... 
+}
+```
+* Notification is shown for test files without `Feature()`. See #3011 by @PeterNgTr
+
+🐛 Bugfixes:
+
+* [Playwright] Fixed #2986 error is thrown when deleting a missing video. Fix by @hatufacci 
+* Fixed false positive result when invalid function is called in a helper. See #2997 by @abhimanyupandian
+* [Appium] Removed full page mode for `saveScreenshot`. See #3002 by @nlespiaucq
+* [Playwright] Fixed #3003 saving trace for a test with a long name. Fix by @hatufacci 
+
+🎱 Other:
+
+* Deprecated `puppeteerCoverage` plugin in favor of `coverage` plugin.
+
+## 3.1.1
+
+* [Appium] Fixed #2759
+ `grabNumberOfVisibleElements`, `grabAttributeFrom`, `grabAttributeFromAll` to allow id locators.
+
 ## 3.1.0
 
 * [Plawyright] Updated to Playwright 1.13
+* [Playwright] **Possible breaking change**: `BrowserContext` is initialized before each test and closed after. This behavior matches recommendation from Playwright team to use different contexts for tests.
+* [Puppeteer] Updated to Puppeteer 10.2.
+* [Protractor] Helper deprecated
+
+🛩️ Features:
+
 * [Playwright] Added recording of [video](https://codecept.io/playwright/#video) and [traces](https://codecept.io/playwright/#trace) by @davertmik
 * [Playwritght] [Mocking requests](https://codecept.io/playwright/#mocking-network-requests) implemented via `route` API of Playwright by @davertmik
 * [Playwright] Added **support for [React locators](https://codecept.io/react/#locators)** in #2912 by @AAAstorga
-* [Puppeteer] Updated to Puppeteer 10.2.
+
+🐛 Bugfixes:
+
 * [Puppeteer] Fixed #2244 `els[0]._clickablePoint is not a function` by @karunandrii.
 * [Puppeteer] Fixed `fillField` to check for invisible elements. See #2916 by @anne-open-xchange 
 * [Playwright] Reset of dialog event listener before registration of new one. #2946 by @nikocanvacom 
@@ -16,14 +140,16 @@
 
 ## 3.0.7
 
-Documentation fixes:
+📖 Documentation fixes:
+
 * Remove broken link from `Nightmare helper`. See #2860 by @Arhell
 * Fixed broken links in `playwright.md`. See #2848 by @johnhoodjr
 * Fix mocha-multi config example. See #2881 by @rimesc
 * Fix small errors in email documentation file. See #2884 by @mkrtchian
 * Improve documentation for `Sharing Data Between Workers` section. See #2891 by @ngraf
 
-Features:
+🛩️ Features:
+
 * [WebDriver] Shadow DOM Support for `Webdriver`. See #2741 by @gkushang
 * [Release management] Introduce the versioning automatically, it follows the semantics versioning. See #2883 by @PeterNgTr
 * Adding opts into `Scenario.skip` that it would be useful for building reports. See #2867 by @AlexKo4

package/README.md

@@ -29,7 +29,7 @@ Scenario('check Welcome page on site', ({ I }) => {
 
 CodeceptJS tests are:
 
-* **Synchronous**. You don't need to care about callbacks, or promises, test scenarios are linear, your test should be too.
+* **Synchronous**. You don't need to care about callbacks or promises or test scenarios which are linear. But, your tests should be linear.
 * Written from **user's perspective**. Every action is a method of `I`. That makes test easy to read, write and maintain even for non-tech persons.
 * Backend **API agnostic**. We don't know which WebDriver implementation is running this test. We can easily switch from WebDriverIO to Protractor or PhantomJS.
 
@@ -38,11 +38,10 @@ CodeceptJS uses **Helper** modules to provide actions to `I` object. Currently C
 * [**Playwright**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Playwright.md) - is a Node library to automate the Chromium, WebKit and Firefox browsers with a single API.
 * [**Puppeteer**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Puppeteer.md) - uses Google Chrome's Puppeteer for fast headless testing.
 * [**WebDriver**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/WebDriver.md) - uses [webdriverio](http://webdriver.io/) to run tests via WebDriver protocol.
-* [**Protractor**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Protractor.md) - helper empowered by [Protractor](http://protractortest.org/) to run tests via WebDriver protocol.
 * [**TestCafe**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/TestCafe.md) - cheap and fast cross-browser test automation.
 * [**Nightmare**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Nightmare.md) - uses Electron and NightmareJS to run tests.
 * [**Appium**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Appium.md) - for **mobile testing** with Appium
-* [**Detox**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Detox.md) - This is a wrapper on top of Detox library, aimied to unify testing experience for CodeceptJS framework. Detox provides a grey box testing for mobile applications, playing especially good for React Native apps.
+* [**Detox**](https://github.com/codeceptjs/CodeceptJS/blob/master/docs/helpers/Detox.md) - This is a wrapper on top of Detox library, aimed to unify testing experience for CodeceptJS framework. Detox provides a grey box testing for mobile applications, playing especially well for React Native apps.
 
 And more to come...
 

package/bin/codecept.js

@@ -105,6 +105,7 @@ program.command('run [test]')
   .option('-c, --config [file]', 'configuration file to be used')
   .option('--features', 'run only *.feature files and skip tests')
   .option('--tests', 'run only JS test files and skip features')
+  .option('--no-timeouts', 'disable all timeouts')
   .option('-p, --plugins <k=v,k2=v2,...>', 'enable plugins, comma-separated')
 
   // mocha options

package/docs/advanced.md

@@ -100,7 +100,7 @@ Scenario('update user profile', ({  }) => {
 All tests with `@tag` could be executed with `--grep @tag` option.
 
 ```sh
-codeceptjs run --grep @slow
+npx codeceptjs run --grep @slow
 ```
 
 Use regex for more flexible filtering:
@@ -119,24 +119,30 @@ CodeceptJS provides a debug mode in which additional information is printed.
 It can be turned on with `--debug` flag.
 
 ```sh
-codeceptjs run --debug
+npx codeceptjs run --debug
 ```
 
 to receive even more information turn on `--verbose` flag:
 
 ```sh
-codeceptjs run --verbose
+npx codeceptjs run --verbose
 ```
 
-And don't forget that you can pause execution and enter **interactive console** mode by calling `pause()` inside your test.
+> You can pause execution and enter **interactive console** mode by calling `pause()` inside your test.
 
-For advanced debugging use NodeJS debugger. In WebStorm IDE:
+To see a complete internal debug of CodeceptJS use `DEBUG` env variable:
+
+```sh
+DEBUG=codeceptjs:* npx codeceptjs run
+```
+
+For an interactive debugging use NodeJS debugger. In **WebStorm**:
 
 ```sh
 node $NODE_DEBUG_OPTION ./node_modules/.bin/codeceptjs run
 ```
 
-For Visual Studio Code, add the following configuration in launch.json:
+For **Visual Studio Code**, add the following configuration in launch.json:
 
 ```json
 {
@@ -180,29 +186,99 @@ You can use this options for build your own [plugins](https://codecept.io/hooks/
   });
 ```
 
-### Timeout
+### Timeout <Badge text="Updated in 3.2" type="warning"/>
 
-By default there is no timeout for tests, however you can change this value for a specific suite:
+Tests can get stuck due to various reasons such as network connection issues, crashed browser, etc.
+This can make tests process hang. To prevent these situations timeouts can be used. Timeouts can be set explicitly for flaky parts of code, or implicitly in a config.
+
+> Previous timeout implementation was disabled as it had no effect when dealing with steps and promises. 
+
+### Steps Timeout
+
+It is possible to limit a step execution to specified time with `I.limitTime` command. 
+It will set timeout in seconds for the next executed step:
 
 ```js
-Feature('Stop me').timeout(5000); // set timeout to 5s
+// limit clicking to 5 seconds
+I.limitTime(5).click('Link')
 ```
 
-or for the test:
+It is possible to set a timeout for all steps implicitly (except waiters) using [stepTimeout plugin](/plugins/#steptimeout).
+
+### Tests Timeout
+
+Test timeout can be set in seconds via Scenario options:
 
 ```js
-// set timeout to 1s
-Scenario("Stop me faster",({ I }) => {
-  // test goes here
-}).timeout(1000);
+// limit test to 20 seconds
+Scenario('slow test that should be stopped', { timeout: 20 }, ({ I }) => {
+  // ...
+})
+```
 
-// alternative
-Scenario("Stop me faster", {timeout: 1000},({ I }) => {});
+This timeout can be set globally in `codecept.conf.js` in seconds:
 
-// disable timeout for this scenario
-Scenario("Don't stop me", {timeout: 0},({ I }) => {});
+```js
+exports.config = {
+
+  // each test must not run longer than 5 mins
+  timeout: 300,
+
+}
 ```
 
+### Suites Timeout
+
+A timeout for a group of tests can be set on Feature level via options. 
+
+```js
+// limit all tests in this suite to 30 seconds
+Feature('flaky tests', { timeout: 30 })
+```
+
+### Sum Up
+
+Let's list all available timeout options.
+
+Timeouts can be set globally in config:
+
+```js
+// in codecept.confg.js:
+{ // ...
+   timeout: 30, // limit all tests in all suites to 30 secs
+
+   plugins: {
+     stepTimeout: {
+       enabled: true,
+       timeout: 10, // limit all steps except waiters to 10 secs
+     }
+   }
+} 
+
+```
+
+or inside a test file:
+
+```js
+// limit all tests in this suite to 10 secs
+Feature('tests with timeout', { timeout: 10 });
+
+// limit this test to 20 secs
+Scenario('a test with timeout', { timeout: 20 }, ({ I }) => {
+   // limit step to 5 seconds
+   I.limitTime(5).click('Link');
+});
+```
+
+Global timeouts will be overridden by explicit timeouts of a test or steps.
+
+### Disable Timeouts
+
+To execute tests ignoring all timeout settings use `--no-timeouts` option:
+
+```
+npx codeceptjs run --no-timeouts
+```
 
 ## Dynamic Configuration
 
@@ -249,45 +325,3 @@ Please note that some config changes can't be applied on the fly. For instance,
 
 Configuration changes will be reverted after a test or a suite.
 
-
-### Rerunning Flaky Tests Multiple Times <Badge text="Since 2.4" type="warning"/>
-
-End to end tests can be flaky for various reasons. Even when we can't do anything to solve this problem it we can do next two things:
-
-* Detect flaky tests in our suite
-* Fix flaky tests by rerunning them.
-
-Both tasks can be achieved with [`run-rerun` command](/commands/#run-rerun) which runs tests multiple times until all tests are passed.
-
-You should set min and max runs boundaries so when few tests fail in a row you can rerun them until they are succeeded.
-
-```js
-// inside to codecept.conf.js
-exports.config = { // ...
-  rerun: {
-    // run 4 times until 1st success
-    minSuccess: 1,
-    maxReruns: 4,
-  }
-}
-```
-
-If you want to check all your tests for stability you can set high boundaries for minimal success:
-
-```js
-// inside to codecept.conf.js
-exports.config = { // ...
-  rerun: {
-    // run all tests must pass exactly 5 times
-    minSuccess: 5,
-    maxReruns: 5,
-  }
-}
-```
-
-Now execute tests with `run-rerun` command:
-
-```
-npx codeceptjs run-rerun
-```
-

package/docs/basics.md

@@ -108,7 +108,7 @@ I.seeElement({name: 'password'});
 I.seeElement({react: 'user-profile', props: {name: 'davert'}});
 ```
 
-In [mobile testing](http://codecept.io/mobile/#locating-elements) you can use `~` to specify the accessibility id to locate an element. In web application you can locate elements by their `aria-label` value.
+In [mobile testing](https://codecept.io/mobile/#locating-elements) you can use `~` to specify the accessibility id to locate an element. In web application you can locate elements by their `aria-label` value.
 
 ```js
 // locate element by [aria-label] attribute in web

package/docs/bdd.md

@@ -264,8 +264,10 @@ You can also use the `parse()` method to obtain an object that allow you to get
 - `raw()` - returns the table as a 2-D array
 - `rows()` - returns the table as a 2-D array, without the first row
 - `hashes()` - returns an array of objects where each row is converted to an object (column header is the key)
+- `rowsHash()` - returns an object where each row corresponds to an entry(first column is the key, second column is the value)
+- `transpose()` - transpose the data, returns nothing. To work with the transposed table use the methods above.
 
-If we use hashes() with the previous exemple :
+If we use hashes() with the previous example :
 
 ```js
 Given('I have products in my cart', (table) => { // eslint-disable-line
@@ -281,7 +283,59 @@ Given('I have products in my cart', (table) => { // eslint-disable-line
   }
 });
 ```
+Examples of tables using: 
 
+```gherkin
+  Given I have a short employees card
+    | Harry | Potter  |
+    | Chuck | Norris  |
+```
+```js
+const { DataTableArgument } = require('codeceptjs');
+//...
+Given('I have a short employees card', (table) => {
+  const dataTableArgument = new DataTableArgument(table);
+  const raw = dataTableArgument.raw(); 
+  // row = [['Harry', 'Potter'], ['Chuck', 'Norris']]
+  dataTableArgument.transpose();
+  const transposedRaw = dataTableArgument.raw();
+  // transposedRaw = [['Harry', 'Chuck'], ['Potter', 'Norris']];
+  }
+);
+```
+```gherkin
+  Given I have an employee card
+    | name  | surname | position |
+    | Harry | Potter  | Seeker   |
+```
+```js
+const { DataTableArgument } = require('codeceptjs');
+//...
+Given('I have an employee card', (table) => {
+  const dataTableArgument = new DataTableArgument(table);
+  const hashes = dataTableArgument.hashes(); 
+  // hashes = [{ name: 'Harry', surname: 'Potter', position: 'Seeker' }];
+  const rows = dataTableArgument.rows();
+  // rows = [['Harry', 'Potter', Seeker]];
+  }
+);
+```
+```gherkin
+  Given I have a formatted employee card
+    | name     | Harry  |
+    | surname  | Potter |
+    | position | Seeker |
+```
+```js
+const { DataTableArgument } = require('codeceptjs');
+//...
+Given('I have a formatted employee card', (table) => {
+  const dataTableArgument = new DataTableArgument(table);
+  const rawHash = dataTableArgument.rowsHash();
+  // rawHash = { name: 'Harry', surname: 'Potter', position: 'Seeker' };
+  }
+);
+```
 ### Examples
 
 In case scenarios represent the same logic but differ on data, we can use *Scenario Outline* to provide different examples for the same behavior. Scenario outline is just like a basic scenario with some values replaced with placeholders, which are filled from a table. Each set of values is executed as a different test.