testplane 0.1.0-rc.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,2149 +1,104 @@
1
- # Testplane (ex-Hermione)
1
+ <p align="center">
2
+ <picture>
3
+ <img alt="Testplane" src="docs/images/testplane.svg" width="220" style="max-width: 100%;">
4
+ </picture>
5
+ </p>
2
6
 
3
- Testplane is a utility for integration testing of web pages using [WebdriverIO](https://webdriver.io/docs/api) and [Mocha](https://mochajs.org).
7
+ <p align="center">
8
+ Fast, scalable and robust testing solution for the ever-evolving web landscape.
9
+ </p>
4
10
 
5
- <!-- START doctoc generated TOC please keep comment here to allow auto update -->
6
- <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
11
+ <p align="center">
12
+ <a href="https://www.npmjs.com/package/testplane"><img src="https://img.shields.io/npm/dt/hermione.svg" alt="Total Downloads"></a>
13
+ <a href="https://github.com/gemini-testing/testplane/releases"><img src="https://img.shields.io/npm/v/testplane.svg" alt="Latest Release"></a>
14
+ <a href="https://github.com/gemini-testing/testplane/blob/master/LICENSE"><img src="https://img.shields.io/npm/l/hermione.svg" alt="License"></a>
15
+ </p>
7
16
 
8
- - [Why you should choose Testplane](#why-you-should-choose-testplane)
9
- - [Easy to use](#easy-to-use)
10
- - [Runs tests in parallel](#runs-tests-in-parallel)
11
- - [Runs tests in subprocesses](#runs-tests-in-subprocesses)
12
- - [Extensible](#extensible)
13
- - [Built-in assert library](#built-in-assert-library)
14
- - [Retries failed tests](#retries-failed-tests)
15
- - [Executes separate tests](#executes-separate-tests)
16
- - [Skips tests in specific browsers](#skips-tests-in-specific-browsers)
17
- - [Override browser configuration for test](#override-browser-configuration-for-test)
18
- - [`version(<browserVersion>)`](#versionbrowserversion)
19
- - [Offers flexible test configuration](#offers-flexible-test-configuration)
20
- - [Automatically initializes and closes grid sessions](#automatically-initializes-and-closes-grid-sessions)
21
- - [Fairly waits for screen rotate](#fairly-waits-for-screen-rotate)
22
- - [Prerequisites](#prerequisites)
23
- - [Selenium-standalone](#selenium-standalone)
24
- - [Quick start](#quick-start)
25
- - [Using npm init Testplane (a quick way)](#using-npm-init-testplane-a-quick-way)
26
- - [Configuring .testplane.conf.js by yourself (a slow way)](#configuring-testplaneconfjs-by-yourself-a-slow-way)
27
- - [Chrome Devtools Protocol](#chrome-devtools-protocol)
28
- - [Webdriver protocol](#webdriver-protocol)
29
- - [Commands API](#commands-api)
30
- - [Browser commands](#browser-commands)
31
- - [clearSession](#clearsession)
32
- - [Element commands](#element-commands)
33
- - [moveCursorTo](#movecursorto)
34
- - [Tests API](#tests-api)
35
- - [Arguments](#arguments)
36
- - [Hooks](#hooks)
37
- - [Skip](#skip)
38
- - [Only](#only)
39
- - [Config overriding](#config-overriding)
40
- - [testTimeout](#testtimeout)
41
- - [WebdriverIO extensions](#webdriverio-extensions)
42
- - [Sharable meta info](#sharable-meta-info)
43
- - [Execution context](#execution-context)
44
- - [AssertView](#assertview)
45
- - [RunStep](#runstep)
46
- - [OpenAndWait](#openandwait)
47
- - [Typescript usage](#typescript-usage)
48
- - [testplane.ctx typings](#testplanectx-typings)
49
- - [.testplane.conf.js](#testplaneconfjs)
50
- - [sets](#sets)
51
- - [browsers](#browsers)
52
- - [desiredCapabilities](#desiredcapabilities)
53
- - [gridUrl](#gridurl)
54
- - [baseUrl](#baseurl)
55
- - [browserWSEndpoint](#browserwsendpoint)
56
- - [automationProtocol](#automationprotocol)
57
- - [sessionEnvFlags](#sessionenvflags)
58
- - [httpTimeout](#httptimeout)
59
- - [urlHttpTimeout](#urlhttptimeout)
60
- - [pageLoadTimeout](#pageloadtimeout)
61
- - [sessionRequestTimeout](#sessionrequesttimeout)
62
- - [sessionQuitTimeout](#sessionquittimeout)
63
- - [testTimeout](#testtimeout-1)
64
- - [waitTimeout](#waittimeout)
65
- - [waitInterval](#waitinterval)
66
- - [sessionsPerBrowser](#sessionsperbrowser)
67
- - [takeScreenshotOnFails](#takescreenshotonfails)
68
- - [takeScreenshotOnFailsMode](#takescreenshotonfailsmode)
69
- - [takeScreenshotOnFailsTimeout](#takescreenshotonfailstimeout)
70
- - [testsPerSession](#testspersession)
71
- - [retry](#retry)
72
- - [shouldRetry](#shouldretry)
73
- - [calibrate](#calibrate)
74
- - [meta](#meta)
75
- - [windowSize](#windowsize)
76
- - [screenshotDelay](#screenshotdelay)
77
- - [orientation](#orientation)
78
- - [waitOrientationChange](#waitorientationchange)
79
- - [resetCursor](#resetcursor)
80
- - [tolerance](#tolerance)
81
- - [antialiasingTolerance](#antialiasingtolerance)
82
- - [compareOpts](#compareopts)
83
- - [buildDiffOpts](#builddiffopts)
84
- - [assertViewOpts](#assertviewopts)
85
- - [openAndWaitOpts](#openandwaitopts)
86
- - [screenshotsDir](#screenshotsdir)
87
- - [strictTestsOrder](#stricttestsorder)
88
- - [compositeImage](#compositeimage)
89
- - [screenshotMode](#screenshotmode)
90
- - [saveHistoryMode](#savehistorymode)
91
- - [agent](#agent)
92
- - [headers](#headers)
93
- - [transformRequest](#transformrequest)
94
- - [transformResponse](#transformresponse)
95
- - [strictSSL](#strictssl)
96
- - [user](#user)
97
- - [key](#key)
98
- - [region](#region)
99
- - [headless](#headless)
100
- - [isolation](#isolation)
101
- - [system](#system)
102
- - [debug](#debug)
103
- - [mochaOpts](#mochaopts)
104
- - [expectOpts](#expectopts)
105
- - [ctx](#ctx)
106
- - [patternsOnReject](#patternsonreject)
107
- - [workers](#workers)
108
- - [testsPerWorker](#testsperworker)
109
- - [parallelLimit](#parallellimit)
110
- - [fileExtensions](#fileextensions)
111
- - [plugins](#plugins)
112
- - [Parallel execution plugin code](#parallel-execution-plugin-code)
113
- - [List of useful plugins](#list-of-useful-plugins)
114
- - [prepareBrowser](#preparebrowser)
115
- - [prepareEnvironment](#prepareenvironment)
116
- - [CLI](#cli)
117
- - [Reporters](#reporters)
118
- - [Require modules](#require-modules)
119
- - [Overriding settings](#overriding-settings)
120
- - [Debug mode](#debug-mode)
121
- - [REPL mode](#repl-mode)
122
- - [switchToRepl](#switchtorepl)
123
- - [Test development in runtime](#test-development-in-runtime)
124
- - [How to set up using VSCode](#how-to-set-up-using-vscode)
125
- - [How to set up using Webstorm](#how-to-set-up-using-webstorm)
126
- - [Environment variables](#environment-variables)
127
- - [TESTPLANE_SKIP_BROWSERS](#testplane_skip_browsers)
128
- - [TESTPLANE_SETS](#testplane_sets)
129
- - [Programmatic API](#programmatic-api)
130
- - [config](#config)
131
- - [events](#events)
132
- - [errors](#errors)
133
- - [CoreError](#coreerror)
134
- - [CancelledError](#cancellederror)
135
- - [ClientBridgeError](#clientbridgeerror)
136
- - [HeightViewportError](#heightviewporterror)
137
- - [OffsetViewportError](#offsetviewporterror)
138
- - [AssertViewError](#assertviewerror)
139
- - [ImageDiffError](#imagedifferror)
140
- - [NoRefImageError](#norefimageerror)
141
- - [intercept](#intercept)
142
- - [run](#run)
143
- - [addTestToRun](#addtesttorun)
144
- - [readTests](#readtests)
145
- - [isFailed](#isfailed)
146
- - [isWorker](#isworker)
147
- - [halt](#halt)
148
- - [Test Collection](#test-collection)
149
- - [Test Parser API](#test-parser-api)
150
- - [setController(name, methods)](#setcontrollername-methods)
17
+ ---
151
18
 
152
- <!-- END doctoc generated TOC please keep comment here to allow auto update -->
19
+ Testplane (ex-Hermione) is a battle-hardened framework for testing web apps at any scale, any browser and any platform.
153
20
 
154
- ## Why you should choose Testplane
155
- Testplane provides several features that `WebdriverIO` doesn't, and makes the testing process easier.
21
+ 🧑‍💻 **Developer Friendly:** Enjoy a hassle-free start with our installation wizard, TypeScript support, instant feedback via live test editing, advanced HTML-reporter, and smart features like auto-wait and retries.
156
22
 
157
- ### Easy to use
158
- If you are familiar with [WebdriverIO](http://webdriver.io/) and [Mocha](https://mochajs.org), you can start writing and running tests in 5 minutes! You need to install Testplane via npm and add a tiny config to your project. For details, see the [Quick start](#quick-start) section.
23
+ 📸 **Visual Testing Redefined:** Capture anything from specific details to whole pages, manage diffs with a streamlined UI, explore a variety of diff modes and let Testplane tackle flakiness.
159
24
 
160
- ### Runs tests in parallel
161
- When tests are run one by one, it takes a lot of time. Testplane can run tests in parallel sessions in different browsers out of the box.
25
+ 🌐 **Test Across Environments:** Forget being tied to a couple of latest Chrome builds. Testplane goes beyond that, offering testing on real devices and broad automation protocol support, mirroring your users' actual environments.
162
26
 
163
- ### Runs tests in subprocesses
164
- Running of too many tests in parallel can lead to the overloading of the main process CPU usage which causes degradation in test passing time, so Testplane runs all tests in subprocesses in order to solve this problem.
27
+ 📈 **Scale Effortlessly:** Run thousands of tests on a remote browser grid or benefit from ultra-fast local execution. Testplane offers sharding, parallel test execution, and isolated browser contexts.
165
28
 
166
- ### Extensible
167
- `WebdriverIO` provides built-in commands for browser and page manipulation. Often projects need to store some common code and reuse it throughout all tests, so the developer needs to create some helpers and include them in the tests.
29
+ **Infinite Extensibility:** Testplane offers a versatile plugin system with dozens of open-source plugins on GitHub, along with custom reporters, commands, and execution logic.
168
30
 
169
- With Testplane this is very simple and straightforward. You can add any number of custom commands in the Testplane config and use them as `browser.myCustomCommand` in tests.
31
+ ## Getting started
170
32
 
171
- Moreover, Testplane provides plugins that work like hooks. They allow the developer to prepare the testing environment and react properly to test execution events.
33
+ > Note: if you prefer manual installation, you can run `npm i -D testplane`. Check out [the Docs](docs/quick-start.md) for details.
172
34
 
173
- ### Built-in assert library
174
- when writing integration tests, we need to check for various conditions, such as that a button element has certain attributes. So Testplane provides global `expect` variable that gives you access to a number of `matchers` that let you validate different things on the browser, an element or mock object. More about using `expect` and its API you can find [here](https://webdriver.io/docs/api/expect-webdriverio/).
35
+ 1. Use the CLI wizard to set up testplane and generate basic configuration:
175
36
 
176
- ### Retries failed tests
177
- Integration tests use a dynamic environment with a lot of dependencies, where any of them could be unstable from time to time. As a result, integration tests turn red randomly, which makes them imprecise. This spoils the entire testing process.
178
-
179
- To prevent incidental fails, Testplane retries a failed test before marking it as failed. This makes it possible to get rid of a majority of incidental fails. The number of retries can be specified for all browsers or for a specific browser.
180
-
181
- :warning: Testplane reruns tests in a new browser session to exclude situations when the browser environment is the cause of the failure.
182
-
183
- ### Executes separate tests
184
- Sometimes you only need to run specific tests, not all the tests in a set. Testplane makes this possible. You can specify the path to the test file
185
- ```
186
- testplane tests/func/mytest.js
187
- ```
188
-
189
- or filter describes by using the `--grep` option
190
-
191
- ```
192
- testplane --grep login
193
- ```
194
-
195
- or simply use the `mocha` `only()` API in the test
196
-
197
- ```js
198
- describe.only('user login', function() {...});
199
- ```
200
-
201
- ### Skips tests in specific browsers
202
- Sometimes you need to skip a test just in a specific browser, not in all browsers. For example, you don't need to run
203
- some test in ~~ugly~~ IE browsers. In Testplane you can do this with [Testplane helper](#skip). For example,
204
- you can skip some tests in a specific browser
205
- ```js
206
- describe('feature', function() {
207
- testplane.skip.in('ie8', 'it cannot work in this browser');
208
- it('nowaday functionality', function() {...});
209
- });
210
- ```
211
-
212
- or run tests in just one browser
213
- ```js
214
- describe('feature', function() {
215
- // will be skipped in all browsers except Chrome
216
- testplane.skip.notIn('chrome', 'it should work only in Chrome');
217
- it('specific functionality', function() {...});
218
- });
219
- ```
220
-
221
- In these cases you will see messages in reports with the reason for skipping.
222
-
223
- To skip a suite or test silently (without any messages in reports), you can pass the third argument with the silent flag:
224
- ```js
225
- testplane.skip.in('ie8', 'skipReason', {silent: true});
226
- // or
227
- testplane.skip.notIn('chrome', 'skipReason', {silent: true});
228
- ```
229
-
230
- Or you can use another Testplane helper, [only](#only), which is silent by default:
231
- ```js
232
- testplane.only.in('chrome');
233
- // or
234
- testplane.only.notIn('ie8');
235
- ```
236
-
237
- `testplane.only.in` will run tests only in the specified browsers and skip the rest silently.
238
-
239
- `testplane.only.notIn` will run tests in all browsers except the specified ones.
240
-
241
- ### Override browser configuration for test
242
- Testplane allows you to override a browser configuration for each test or a whole suite.
243
- Each method of `testplane.browser(<browserName>)` provide chaining.
244
-
245
- #### `version(<browserVersion>)`
246
-
247
- ```js
248
- // change the browser version for all chindren
249
- testplane.browser('chrome').version('70.3');
250
- describe('suite', function() {
251
- // ...
252
- });
253
- ```
254
-
255
- ```js
256
- // change the browser version for a specific test
257
- testplane.browser('chrome').version('70.3');
258
- it('test', function() {...});
259
- ```
260
-
261
- ```js
262
- testplane.browser('chrome').version('70.3');
263
- describe('suite', function() {
264
- it('test 1', function() {...});
265
-
266
- // this call will override the version only for test below
267
- testplane.browser('chrome').version('70.1');
268
- it('test 2', function() {...});
269
- });
270
- ```
271
-
272
- ### Offers flexible test configuration
273
- Testplane lets you configure running some set of tests in specific browsers. For example,
274
- ```js
275
- sets: {
276
- desktop: {
277
- files: 'tests/desktop',
278
- browsers: ['ie8', 'opera']
279
- },
280
- touch: {
281
- files: 'tests/touch',
282
- browsers: ['iphone', 'android']
283
- }
284
- }
285
- ```
286
- See [sets](#sets) for more details.
287
-
288
-
289
- ### Automatically initializes and closes grid sessions
290
- All work with the grid client is encapsulated in Testplane. Forget about `client.init` and `client.end` in your tests ;)
291
-
292
- ### Fairly waits for screen rotate
293
- Request `/session/:sessionId/orientation` is not a part of the official Webdriver specification, so commands `orientation` and `setOrientation` which are provided by client `webdriverio` from the box do not guarantee screen rotate before the next command will start to execute, but Testplane solves this problem.
294
-
295
- ## Prerequisites
296
- All you need are browsers that Testplane could use for testing. To do this you need to install some browsers, such as [chrome](https://www.google.com/chrome/) (to automate this process you can use the [@testplane/headless-chrome](https://github.com/gemini-testing/testplane-headless-chrome) plugin).
297
-
298
- Next, you have two ways to configure Testplane to work with browsers:
299
-
300
- * Using the devtools protocol (available only for `Chromium`-based browsers). This method does not need to be pre-configured. Just go to the [quick start](#quick-start).
301
- * Using the webdriver protocol. In this case you need to set up [Selenium](http://www.seleniumhq.org/) grid. The simplest way to get started is to use one of the NPM selenium standalone packages, such as [vvo/selenium-standalone](https://github.com/vvo/selenium-standalone). For more information about setting up, see [selenium-standalone](#selenium-standalone).
302
-
303
- ### Selenium-standalone
304
- Install `selenium-standalone` by command:
305
-
306
- ```
307
- npm i -g selenium-standalone
308
- ```
309
-
310
- Next you need to install browser drivers
311
-
312
- ```
313
- selenium-standalone install
314
- ```
315
-
316
- and run your server by executing
317
-
318
- ```
319
- selenium-standalone start
320
- ```
321
-
322
- :warning: If you will get error like `No Java runtime present, requesting install.` you should install [Java Development Kit (JDK)](https://www.oracle.com/technetwork/java/javase/downloads/index.html) for your OS.
323
-
324
- ## Quick start
325
- First of all, make sure that all [prerequisites](#prerequisites) are satisfied.
326
-
327
- Now you have two ways to configure project.
328
-
329
- ### Using npm init Testplane (a quick way)
330
-
331
- You just need to run the cli command from [create-testplane](https://github.com/gemini-testing/create-testplane) tool and answer a few questions:
332
- ```
333
- npm init testplane YOUR_PROJECT_PATH
334
- ```
335
-
336
- To skip all questions just add the option `-y` at the end.
337
-
338
- ### Configuring .testplane.conf.js by yourself (a slow way)
339
-
340
- Create Testplane config file with name `.testplane.conf.js` in the project root. There are two configuration options depending on the method selected in the `prerequisites` section.
341
-
342
- #### Chrome Devtools Protocol
343
-
344
- ```javascript
345
- module.exports = {
346
- sets: {
347
- desktop: {
348
- files: 'tests/desktop/**/*.testplane.js'
349
- }
350
- },
351
-
352
- browsers: {
353
- chrome: {
354
- automationProtocol: 'devtools',
355
- desiredCapabilities: {
356
- browserName: 'chrome'
357
- }
358
- }
359
- }
360
- };
361
- ```
362
-
363
- #### Webdriver protocol
364
-
365
- ```javascript
366
- module.exports = {
367
- gridUrl: 'http://localhost:4444/wd/hub',
368
-
369
- sets: {
370
- desktop: {
371
- files: 'tests/desktop/*.testplane.js'
372
- }
373
- },
374
-
375
- browsers: {
376
- chrome: {
377
- automationProtocol: 'webdriver', // default value
378
- desiredCapabilities: {
379
- browserName: 'chrome'
380
- }
381
- }
382
- }
383
- };
384
- ```
385
-
386
- Write your first test in `tests/desktop/github.testplane.js` file.
387
- ```javascript
388
- describe('github', function() {
389
- it('should check repository name', async ({ browser }) => {
390
- await browser.url('https://github.com/gemini-testing/testplane');
391
-
392
- await expect(browser.$('#readme h1')).toHaveText('Testplane (ex-Hermione)');
393
- });
394
- });
395
- ```
396
-
397
- Finally, run tests (be sure that you have already run `selenium-standalone start` command in next tab).
398
- ```
399
- node_modules/.bin/testplane
400
- ```
401
-
402
- ## Commands API
403
-
404
- Since Testplane is based on [WebdriverIO v8](https://webdriver.io/docs/api/), all the commands provided by WebdriverIO are available in it. But Testplane also has her own commands.
405
-
406
- ### Browser commands
407
-
408
- #### clearSession
409
-
410
- Browser command that clears session state (deletes cookies, clears local and session storages). For example:
411
-
412
- ```js
413
- it('test', async ({ browser }) => {
414
- await browser.url('https://github.com/gemini-testing/testplane');
415
-
416
- (await browser.getCookies()).length; // 5
417
- await browser.execute(() => localStorage.length); // 2
418
- await browser.execute(() => sessionStorage.length); // 1
419
-
420
- await browser.clearSession();
421
-
422
- (await browser.getCookies()).length; // 0
423
- await browser.execute(() => localStorage.length); // 0
424
- await browser.execute(() => sessionStorage.length); // 0
425
- });
426
- ```
427
-
428
- ### Element commands
429
-
430
- #### moveCursorTo
431
-
432
- > This command is temporary and will be removed in the next major (`testplane@1`). Differs from the standard [moveTo](https://webdriver.io/docs/api/element/moveTo/) in that it moves the cursor relative to the top-left corner of the element (like it was in `hermione@7`).
433
-
434
- Move the mouse by an offset of the specified element. If offset is not specified then mouse will be moved to the top-left corder of the element.
435
-
436
- Usage:
437
-
438
- ```typescript
439
- await browser.$(selector).moveCursorTo({ xOffset, yOffset });
440
- ```
441
-
442
- Available parameters:
443
-
444
- * **xOffset** (optional) `Number` – X offset to move to, relative to the top-left corner of the element;
445
- * **yOffset** (optional) `Number` – Y offset to move to, relative to the top-left corner of the element.
446
-
447
- ## Tests API
448
-
449
- ### Arguments
450
- Testplane calls test and hook callback with one argument. This argument is an object with the next properties:
451
- - **browser** - browser client
452
- - **currentTest** - current executing test
453
-
454
- Example:
455
- ```js
456
- beforeEach(async ({ browser, currentTest }) => {
457
- await browser.url(`/foo/bar?baz=${currentTest.id}`);
458
- });
459
-
460
- afterEach(async ({ browser, currentTest }) => {
461
- // Do some post actions with browser
462
- });
463
-
464
- it('some test', async ({ browser, currentTest }) => {
465
- await browser.click('.some-button');
466
-
467
- // Do some actions and asserts
468
- });
469
- ```
470
-
471
- You also can pass any data into your test through parameters:
472
- ```js
473
- beforeEach(async (opts) => {
474
- opts.bar = 'bar';
475
- });
476
-
477
- it('some test', async ({ browser, bar }) => {
478
- await browser.url(`/foo/${bar}`);
479
-
480
- // Do some actions and asserts
481
- });
482
- ```
483
-
484
- **browser** and **currentTest** also available through callback context:
485
- ```js
486
- beforeEach(async function() {
487
- await this.browser.url(`/foo/bar?baz=${this.currentTest.id}`);
488
- });
489
-
490
- afterEach(async function() {
491
- // Do some post actions with this.browser
492
- });
493
-
494
- it('some test', async function() {
495
- await this.browser.click('.some-button');
496
-
497
- // Do some actions and asserts
498
- });
499
- ```
500
-
501
- ### Hooks
502
-
503
- `before` and `after` hooks **are forbidden** in Testplane, you should use `beforeEach` and `afterEach` hooks instead. This feature was implemented in order to ensure better stability while running tests and make them independent of each other.
504
-
505
- ### Skip
506
- This feature allows you to ignore the specified suite or test in any browser, with an additional comment.
507
- You can do this by using the global `testplane.skip` helper. It supports the following methods:
508
-
509
- - `.in` – Adds matchers for browsers with the additional comment.
510
- - `.notIn` – `.in` method with the reverted value.
511
-
512
- Each of these methods takes the following arguments:
513
-
514
- - browser {String|RegExp|Array<String|RegExp>} – Matcher for browser(s) to skip.
515
- - [comment] {String} – Comment for skipped test.
516
- - [options] {Object} – Additional options.
517
-
518
- **Note that matchers will be compared with `browserId` specified in the config file, e.g. `chrome-desktop`.**
519
-
520
- For example,
521
- ```js
522
- describe('feature', function() {
523
- testplane.skip.in('chrome', "It shouldn't work this way in Chrome");
524
- it('should work this way', function() {
525
- return runTestThisWay();
526
- });
527
-
528
- it('should work that way', function() {
529
- return runTestThatWay();
530
- });
531
-
532
- testplane.skip.in(['chrome', 'firefox', /ie\d*/], 'Unstable test, see ticket TEST-487');
533
- it('should have done some tricky things', function() {
534
- return runTrickyTest();
535
- });
536
- });
537
- ```
538
-
539
- In this case, the behaviour `it should work this way` will be skipped only in `chrome` browser, but will be run in other browsers. `It should work that way` will not be ignored. So only the nearest test will be skipped. If you need to skip all tests within a suite, you can apply the `skip` helper to a `describe` so all tests within this suite will be skipped with the same comment.
540
- ```js
541
- testplane.skip.in('chrome', 'skip comment');
542
- describe('some feature', function() {
543
- it(...);
544
- it(...);
545
- });
546
- ```
547
-
548
- You can also use the `.notIn` method to invert matching. For example,
549
- ```js
550
- // ...
551
- testplane.skip.notIn('chrome', 'some comment');
552
- it('should work this way', function() {
553
- return doSomething();
554
- });
555
- // ...
556
- ```
557
-
558
- In this case, the test will be skipped in all browsers except `chrome`.
559
-
560
- All of these methods are chainable, so you can skip a test in several browsers with different comments. For example,
561
- ```js
562
- // ...
563
- testplane.skip
564
- .in('chrome', 'some comment')
565
- .notIn('ie9', 'another comment');
566
- it('test1', function() {
567
- return doSomething();
568
- });
569
- // ...
570
- ```
571
-
572
- If you need to skip a test in all browsers without a comment, you can use [mocha `.skip` method](http://mochajs.org/#inclusive-tests) instead of `testplane.skip.in(/.*/);`. The result will be the same.
573
-
574
- ### Only
575
- This feature allows you to ignore the specified suite or test in any browser silently (without any messages in reports).
576
- You can do this by using the global `testplane.only` helper. It supports two methods:
577
-
578
- - `.in` — The `testplane.skip.notIn` method with the silent flag,
579
- - `.notIn` — The `testplane.skip.in` with the silent flag.
580
-
581
- These methods take the following arguments:
582
-
583
- - browser {String|RegExp|Array<String|RegExp>} — A matcher for browser(s) to skip.
584
-
585
- For example:
586
- ```js
587
- // ...
588
- testplane.only.in('chrome');
589
-
590
- it('should work this way', function() {
591
- return doSomething();
592
- });
593
- ```
594
- The test will be skipped all browsers **silently** except in `chrome`.
595
-
596
- ```js
597
- testplane.only.notIn('ie9');
598
- it('should work another way', function() {
599
- return doSomething();
600
- });
601
- ```
602
- The test will be processed in all browsers and **silently** skipped in `ie9`.
603
-
604
- ### Config overriding
605
- You can override some config settings for specific test, suite or hook via `testplane.config.*` notation.
606
-
607
- #### testTimeout
608
- Overrides [testTimeout](#testtimeout-1) config setting. Can be set for tests and suites.
609
-
610
- ```js
611
- testplane.config.testTimeout(100500);
612
- it('some test', function() {
613
- return doSomething();
614
- });
615
- ```
616
-
617
- ### WebdriverIO extensions
618
- Testplane adds some useful methods and properties to the `webdriverio` session after its initialization.
619
-
620
- #### Sharable meta info
621
- Implemented via two commands:
622
-
623
- * setMeta(key, value)
624
- * getMeta([key])
625
-
626
- These methods allow you to store some information between webdriver calls and it can then be used in custom commands, for instance. This meta information will be shown in the [html-reporter](https://github.com/gemini-testing/html-reporter).
627
-
628
- **Note**: Testplane saves the last URL opened in the browser in meta info.
629
-
630
- Example:
631
- ```js
632
- it('test1', async ({ browser }) => {
633
- await browser.setMeta('foo', 'bar');
634
- await browser.url('/foo/bar?baz=qux');
635
-
636
- const val = await browser.getMeta('foo');
637
- console.log(val); // prints 'bar'
638
-
639
- const url = await browser.getMeta('url');
640
- console.log(url); // prints '/foo/bar?baz=qux'
641
-
642
- const meta = await browser.getMeta();
643
- console.log(meta); // prints `{foo: 'bar', url: '/foo/bar?baz=qux'}`
644
- });
645
- ```
646
-
647
- #### Execution context
648
- The execution context can be accessed by the `browser.executionContext` property, which contains the current test/hook object extended with the browser id.
649
-
650
- Example:
651
- ```js
652
- it('some test', async ({ browser }) => {
653
- await browser.url('/foo/bar');
654
- console.log('test', browser.executionContext);
655
- });
656
- ```
657
- will print something like this
658
- ```
659
- test: {
660
- "title": "some test",
661
- "async": 0,
662
- "sync": true,
663
- "timedOut": false,
664
- "pending": false,
665
- "type": "test",
666
- "body": "...",
667
- "file": "/foo/bar/baz/qux.js",
668
- "parent": "#<Suite>",
669
- "ctx": "#<Context>",
670
- "browserId": "chrome",
671
- "meta": {},
672
- "timer": {}
673
- }
674
- ```
675
-
676
- ### AssertView
677
-
678
- Command that adds ability to take screenshot for test state. Each state should have his own unique name. For example:
679
-
680
- ```js
681
- it('some test', async ({ browser }) => {
682
- await browser.url('some/url');
683
- await browser.assertView('plain', '.button');
684
-
685
- await browser.click('.button');
686
- await browser.assertView('clicked', '.button');
687
- });
688
- ```
689
-
690
- Could also be used as element's method:
691
-
692
- ```js
693
- it('some test', async ({ browser }) => {
694
- await browser.url('some/url');
695
-
696
- const elem = await browser.$('.button');
697
-
698
- await elem.assertView('plain');
699
- await elem.click();
700
- await elem.assertView('clicked');
701
- });
702
- ```
703
-
704
- *Note: assertView will trigger [waitForExist](https://webdriver.io/docs/api/element/waitForExist/) with [waitTimeout](#waittimeout) and [waitInterval](#waitinterval)*
705
-
706
- Parameters:
707
-
708
- - state (required) `String` – state name; should be unique within one test
709
- - selector (required) `String|String[]` – DOM-node selector that you need to capture
710
- - opts (optional) `Object`:
711
- - ignoreElements (optional) `String|String[]` – elements, matching specified selectors will be ignored when comparing images
712
- - tolerance (optional) `Number` – overrides config [browsers](#browsers).[tolerance](#tolerance) value
713
- - antialiasingTolerance (optional) `Number` – overrides config [browsers](#browsers).[antialiasingTolerance](#antialiasingTolerance) value
714
- - ignoreDiffPixelCount (optional) `Number | string` - the maximum amount of different pixels to still consider screenshots "the same". For example, when set to 5, it means that if there are 5 or fewer different pixels between two screenshots, they will still be considered the same. Alternatively, you can also define the maximum difference as a percentage (for example, 3%) of the image size. This option is useful when you encounter a few pixels difference that cannot be eliminated using the tolerance and antialiasingTolerance settings. The default value is 0.
715
- - allowViewportOverflow (optional) `Boolean` – by default Testplane throws an error if element is outside the viewport bounds. This option disables check that element is outside of the viewport left, top, right or bottom bounds. And in this case if browser option [compositeImage](#compositeimage) set to `false`, then only visible part of the element will be captured. But if [compositeImage](#compositeimage) set to `true` (default), then in the resulting screenshot will appear the whole element with not visible parts outside of the bottom bounds of viewport.
716
- - captureElementFromTop (optional) `Boolean` - ability to set capture element from the top area or from current position. In the first case viewport will be scrolled to the top of the element. Default value is `true`
717
- - compositeImage (optional) `Boolean` - overrides config [browsers](#browsers).[compositeImage](#compositeImage) value
718
- - screenshotDelay (optional) `Number` - overrides config [browsers](#browsers).[screenshotDelay](#screenshotDelay) value
719
- - selectorToScroll (optional) `String` - DOM-node selector which should be scroll when the captured element does not completely fit on the screen. Useful when you capture the modal (popup). In this case a duplicate of the modal appears on the screenshot. That happens because we scroll the page using `window` selector, which scroll only the background of the modal, and the modal itself remains in place. Works only when `compositeImage` is `true`.
720
- - disableAnimation (optional): `Boolean` - ability to disable animations and transitions while capturing a screenshot.
721
-
722
- All options inside `assertView` command override the same options in the [browsers](#browsers).[assertViewOpts](#assertViewOpts).
723
-
724
- Full example:
725
-
726
- ```js
727
- it('some test', async ({ browser }) => {
728
- await browser.url('some/url');
729
- await browser.assertView(
730
- 'plain', '.form',
731
- {
732
- ignoreElements: ['.link'],
733
- tolerance: 5,
734
- antialiasingTolerance: 4,
735
- allowViewportOverflow: true,
736
- captureElementFromTop: true,
737
- compositeImage: true,
738
- screenshotDelay: 10,
739
- selectorToScroll: '.modal'
740
- }
741
- );
742
- });
743
- ```
744
-
745
- For tests which have been just written using `assertView` command you need to update reference images, so for the first time `testplane` should be run with option `--update-refs` or via command `gui` which is provided by plugin [html-reporter](https://github.com/gemini-testing/html-reporter) (we highly recommend to use `gui` command instead of option `--update-refs`).
746
-
747
- ### RunStep
748
-
749
- Command that allows to add human-readable description for commands in `history`, when it is enabled ([html-reporter](https://github.com/gemini-testing/html-reporter) required) with [saveHistoryMode](#savehistorymode). For example:
750
-
751
- ```js
752
- it('some test', async ({browser}) => {
753
- await browser.runStep('some step name', async () => {
754
- await browser.url('some/url');
755
- await browser.$('some-selector').click();
756
- });
757
-
758
- await browser.runStep('other step name', async () => {
759
- await browser.runStep('some nested step', async () => {
760
- await browser.$('not-exist').click();
761
- });
762
- });
763
-
764
- await browser.runStep('another step', async () => {
765
- ...
766
- });
767
- });
768
- ```
769
-
770
- Will produce the following history, if test fails on 'Can't call click on element with selector "not-exist" because element wasn't found':
771
-
772
- - testplane: init browser
773
- - some step name
774
- - other step name
775
- - some nested step
776
- - $("not-exist")
777
- - click()
778
- - waitForExist
779
-
780
- In this example step `some step name` is collapsed, because it is completed successfully.
781
-
782
- You can also return values from step:
783
-
784
- ```js
785
- const parsedPage = await browser.runStep('parse page', async () => {
786
- ...
787
- return someData;
788
- });
789
- ```
790
-
791
- Parameters:
792
-
793
- - stepName (required) `String` – step name
794
- - stepCb (required) `Function` – step callback
795
-
796
- *Note: [html-reporter](https://github.com/gemini-testing/html-reporter) v9.7.7+ provides better history representation.*
797
-
798
- ### OpenAndWait
799
-
800
- Command that allows to open page and wait until it loads (by combination of the specified factors). For example:
801
-
802
- ```js
803
- it('some test', async ({browser}) => {
804
- await browser.openAndWait('some/url', {
805
- selector: ['.some', '.selector'],
806
- predicate: () => document.isReady,
807
- ignoreNetworkErrorsPatterns: ['https://mc.yandex.ru'],
808
- waitNetworkIdle: true,
809
- waitNetworkIdleTimeout: 500,
810
- failOnNetworkError: true,
811
- timeout: 20000,
812
- });
813
- });
814
- ```
815
-
816
- In this example, page will be considered as loaded, if elements with selectors `.some` and `.selector` exists, `document.isReady` is `true`, it has been 500 ms since the last network request succeeded, and there were no errors, while trying to load images, fonts and stylesheets.
817
-
818
- Parameters:
819
-
820
- - url (required) `String` – page url
821
- - waitOpts (optional) `Object`:
822
- - selector (optional) `String|String[]` – Selector(s) to element(s), which should exist on the page.
823
- - predicate (optional) `() => Promise<bool> | bool` – Predicate, which should return `true` if page is loaded. Predicate is being executed in the browser context: [waitUntil](https://webdriver.io/docs/api/element/waitUntil).
824
- - waitNetworkIdle (optional) `Boolean` – Waits until all network requests are done. `true` by default. Only works in chrome browser or when using [Chrome Devtools Protocol](#chrome-devtools-protocol).
825
- - waitNetworkIdleTimeout (optional) `Number` - Time (ms) after all network requests are resolved to consider network idle. 500 by default.
826
- - failOnNetworkError (optional) `Boolean` – If `true`, throws an error when network requests are failed. `true` by default. Only works in chrome browser or when using [Chrome Devtools Protocol](#chrome-devtools-protocol).
827
- - shouldThrowError (optional) `(match) => Boolean` - Predicate, which should return `true` on [Match](https://webdriver.io/docs/api/mock#match), if network error is considered critical for page load. By default, throws an error on image, stylesheet and font load error.
828
- - ignoreNetworkErrorsPatterns (optional) `Array<String | RegExp>` - Array of url patterns to ignore network requests errors. Has a priority over `shouldThrowError`
829
- - timeout (optional) `Number` - Page load timeout. [pageLoadTimeout](#pageloadtimeout) by default. Throws an error, if selectors are still not exist after timeout, or predicate is still resolving false.
830
-
831
- ## Typescript usage
832
-
833
- To write Testplane tests on typescript, you would need to install `ts-node`:
834
-
835
- ```bash
836
- npm i -D ts-node
837
- ```
838
-
839
- And include Testplane types in your `tsconfig.json` file:
840
-
841
- ```js
842
- // tsconfig.json
843
- {
844
- // other tsconfig options
845
- "compilerOptions": {
846
- // other compiler options
847
- "types": [
848
- // other types
849
- "testplane",
850
- ]
851
- }
852
- }
853
- ```
854
-
855
- Now you will be able to write Testplane tests using typescript.
856
-
857
- ### testplane.ctx typings
858
-
859
- If you want to extend testplane.ctx typings, you could use module augmentation:
860
-
861
- ```ts
862
- import type { TestplaneCtx } from "testplane";
863
-
864
- declare module "testplane" {
865
- interface TestplaneCtx {
866
- someVariable: string;
867
- }
868
- }
869
- ```
870
-
871
- Now `testplane.ctx` will have `someVariable` typings
872
-
873
- ## .testplane.conf.js
874
- Testplane is tuned using a configuration file. By default, it uses `.testplane.conf.js`, but you can use the `--config` option to specify a path to the configuration file.
875
-
876
- There is only one required field – `browsers`.
877
- ```javascript
878
- module.exports = {
879
- browsers: {
880
- chrome: {
881
- desiredCapabilities: {
882
- browserName: 'chrome'
883
- }
884
- }
885
- }
886
- };
887
- ```
888
-
889
- ### sets
890
- You can use sets to bind some set of tests to certain browsers.
891
-
892
- Format of the sets section:
893
- ```javascript
894
- sets: {
895
- common: { // run tests associated with this path in all browsers
896
- files: 'tests/common' // which are configured in the `browsers` option
897
- },
898
- desktop: {
899
- files: [
900
- 'tests/desktop/*.testplane.js',
901
- 'tests/common/*.testplane.js'
902
- ],
903
- ignoreFiles: ['tests/desktop/fixtures/**'], // exclude directories from reading while test finding
904
- browsers: ['browser'] // run tests which match the specified masks in the browser with the `browser` id
905
- }
906
- }
907
- ```
908
-
909
- * `files` – A list of test files or directories with test files. This can be a string if you want to specify just one file or directory. Also, you can use
910
- masks for this property.
911
-
912
- * `ignoreFiles` - A list of paths or masks to ignore from reading while test finding. When you write `!**/some-dir/**` it means that the directory will be read, but all the entries will not be included in the results. So using `ignoreFiles` you can speed up test reading for your project.
913
-
914
- * `browsers` – A list of browser IDs to run the tests specified in `files`. All browsers by default.
915
-
916
- You can specify sets to run using the CLI option `--set`.
917
-
918
- If sets are not specified in the config and paths were not passed from CLI, all files from the `testplane`
919
- directory are launched in all browsers specified in the config.
920
-
921
- Running tests using sets:
922
-
923
- ```
924
- testplane --set desktop
925
- ```
926
-
927
- ### browsers
928
- **Required.** The list of browsers to use for running tests.
929
-
930
- The browser section has the following format
931
- ```javascript
932
- browsers: {
933
- <browser_id>: {
934
- <option>:<value>
935
- <option>:<value>
936
- }
937
- }
938
- ```
939
- `<browser-id>` value is used for browser identification.
940
-
941
- Available browser options:
942
-
943
- Option name | Description
944
- ------------------------- | -------------
945
- `desiredCapabilities` | **Required.** Used WebDriver [DesiredCapabilities](https://github.com/SeleniumHQ/selenium/wiki/DesiredCapabilities)
946
- `gridUrl` | Selenium grid URL. Default value is `http://localhost:4444/wd/hub`.
947
- `baseUrl` | Base service-under-test URL. Default value is `http://localhost`.
948
- `browserWSEndpoint` | Websocket endpoint to connect to the browser in order to be able to use CDP protocol. Default value is `null`.
949
- `automationProtocol` | Browser automation protocol. Default value is `webdriver`.
950
- `sessionEnvFlags` | Environment flags that determine which protocol will be used in created browser session. Default value is `{}`.
951
- `waitTimeout` | Timeout for web page event. Default value is `1000` ms.
952
- `waitInterval` | Interval for web page event. Default value is `250` ms.
953
- `httpTimeout` | Timeout for any requests to Selenium server. Default value is `30000` ms.
954
- `urlHttpTimeout` | Timeout for `/url` request to Selenium server. Default value is `httpTimeout`.
955
- `pageLoadTimeout` | Timeout for the page loading to complete. Default value is `20000` ms.
956
- `sessionRequestTimeout` | Timeout for getting a browser session. Default value is `httpTimeout`.
957
- `sessionQuitTimeout` | Timeout for quitting a session. Default value is `5000`.
958
- `testTimeout` | Timeout for test execution (in milliseconds). Default value is `null`, in this case will be used common timeout for all browsers from `system.mochaOpts.timeout`.
959
- `sessionsPerBrowser` | Number of sessions which are run simultaneously. Default value is `1`.
960
- `takeScreenshotOnFails` | Options for setting up taking a screenshot of a test fail. Default value is `{testFail: true, assertViewFail: false}`.
961
- `takeScreenshotOnFailsMode` | Mode for taking a screenshot on test fail. Available options are `fullpage` and `viewport`. Default value is `fullpage`.
962
- `takeScreenshotOnFailsTimeout`| Timeout for taking screenshot on test fail. Default value is `5000`.
963
- `testsPerSession` | Maximum amount of tests (`it`s) to run in each web driver session.
964
- `retry` | How many times a test should be rerun. Default value is `0`.
965
- `shouldRetry` | Function that determines whether to make a retry. By default returns `true `if retry attempts are available otherwise returns `false`.
966
- `calibrate` | Allows to correctly capture the image. Default value is `false`.
967
- `meta` | Additional data that can be obtained via .getMeta() method.
968
- `windowSize` | Browser window dimensions. Default value is `null`.
969
- `screenshotDelay` | Allows to specify a delay (in milliseconds) before making any screenshot.
970
- `orientation` | Browser orientation that will be set before each test run. Default value is `null`.
971
- `waitOrientationChange` | Allows to wait until screen orientation is changed. Default value is `true`.
972
- `resetCursor` | Allows to configure whether to move mouse cursor to `body` coordinates `(0, 0)` before each test run.
973
- `tolerance` | Maximum allowed [CIEDE2000](http://en.wikipedia.org/wiki/Color_difference#CIEDE2000) difference between colors. Default value is `2.3`.
974
- `antialiasingTolerance` | Minimum difference in brightness between the darkest/lightest pixel (which is adjacent to the antiasing pixel) and theirs adjacent pixels. Default value is `4`.
975
- `compareOpts` | Options for comparing images.
976
- `buildDiffOpts` | Options for building diff image.
977
- `assertViewOpts` | Options for `assertView` command, used by default.
978
- `openAndWaitOpts` | Options for `openAndWaitOpts` command, used by default
979
- `screenshotsDir` | Directory to save reference images for command `assertView`. Default dir is `testplane/screens` which is relative to `process.cwd()`.
980
- `strictTestsOrder` | Testplane will guarantee tests order in [readTests](#readtests) results. `false` by default.
981
- `compositeImage` | Allows testing of regions which bottom bounds are outside of a viewport height. In the resulting screenshot the area which fits the viewport bounds will be joined with the area which is outside of the viewport height. `true` by default.
982
- `screenshotMode` | Image capture mode.
983
- `saveHistoryMode` | Allows to save history of executed commands. `all` by default.
984
- `agent` | Allows to use a custom `http`/`https`/`http2` [agent](https://www.npmjs.com/package/got#agent) to make requests. Default value is `null`.
985
- `headers` | Allows to set custom [headers](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#headers) to pass into every webdriver request. These headers aren't passed into browser request. Read more about this option in [wdio](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#headers). Default value is `null`.
986
- `transformRequest` | Allows to intercept [HTTP request options](https://github.com/sindresorhus/got#options) before a WebDriver request is made. Default value is `null`.
987
- `transformResponse` | Allows to intercept [HTTP response object](https://github.com/sindresorhus/got#response) after a WebDriver response has arrived. Default value is `null`.
988
- `strictSSL` | Whether it does require SSL certificate to be valid. Default value is `null` (it means that will be used [default value from wdio](https://webdriver.io/docs/options/#strictssl)).
989
- `user` | Cloud service username. Default value is `null`.
990
- `key` | Cloud service access key or secret key. Default value is `null`.
991
- `region` | Ability to choose different datacenters for run in cloud service. Default value is `null`.
992
- `headless` | Ability to run headless browser in cloud service. Default value is `null`.
993
- `isolation` | Ability to execute tests in isolated clean-state environment ([incognito browser context](https://chromedevtools.github.io/devtools-protocol/tot/Target/#method-createBrowserContext)). Default value is `false`, but `true` for chrome@93 and higher.
994
-
995
- #### desiredCapabilities
996
- **Required.** Used WebDriver [DesiredCapabilities](https://github.com/SeleniumHQ/selenium/wiki/DesiredCapabilities). For example,
997
-
998
- ```js
999
- 'chrome': {
1000
- desiredCapabilities: {
1001
- browserName: 'chrome',
1002
- version: '75.0', // or "browserVersion" if browser support w3c
1003
- chromeOptions: {...}
1004
- }
1005
- }
1006
- ```
1007
-
1008
- #### gridUrl
1009
- Selenium grid URL. Default value is `http://localhost:4444/wd/hub`.
1010
-
1011
- #### baseUrl
1012
- Base service-under-test URL. Default value is `http://localhost`.
1013
-
1014
- #### browserWSEndpoint
1015
- Websocket endpoint to connect to the browser in order to be able to use [CDP protocol](https://chromedevtools.github.io/devtools-protocol/). For example you specify `browserWSEndpoint: "ws://YOUR_HOST/devtools"` to which `sessionId` of the browser will be added to the end: `ws://YOUR_HOST/devtools/12345`, where `12345` is a `sessionId`. Default value is `null`, it means that `webdriverio` is trying to figure out the websocket endpoint on its own.
1016
-
1017
- #### automationProtocol
1018
- Browser automation protocol (`webdriver`, `devtools`) that will be used. Default value is `webdriver`.
1019
-
1020
- #### sessionEnvFlags
1021
- Environment flags that determine which protocol will be used in created browser session. By default environment flags are set automatically according to the used `desiredCapabilities` but in rare cases they are determined inaccurately and using this option they can be overriden explicitly.
1022
-
1023
- Available flags:
1024
- - isW3C - should apply [`WebDriverProtocol`](https://webdriver.io/docs/api/webdriver) or use default [`JsonWProtocol`](https://webdriver.io/docs/api/jsonwp);
1025
- - isChrome - should apply [`ChromiumProtocol`](https://webdriver.io/docs/api/chromium);
1026
- - isMobile - should apply [MJsonWProtocol](https://webdriver.io/docs/api/mjsonwp) and [AppiumProtocol](https://webdriver.io/docs/api/appium);
1027
- - isSauce - should apply [Sauce Labs specific vendor commands](https://webdriver.io/docs/api/saucelabs);
1028
- - isSeleniumStandalone - should apply [special commands when running tests using Selenium Grid or Selenium Standalone server](https://webdriver.io/docs/api/selenium);
1029
-
1030
- For example:
1031
-
1032
- ```js
1033
- 'chrome': {
1034
- sessionEnvFlags: {
1035
- isW3C: true,
1036
- isChrome: true
1037
- }
1038
- }
1039
- ```
1040
-
1041
- #### httpTimeout
1042
- Timeout for any requests to Selenium server. Default value is `30000` ms.
1043
-
1044
- #### urlHttpTimeout
1045
- Timeout for `/url` request to Selenium server. Default value is `httpTimeout`.
1046
- It may be useful when opening url takes a long time (for example a lot of logic is executed in middlewares), and you don't want to increase timeout for other commands.
1047
-
1048
- #### pageLoadTimeout
1049
- Timeout for the page loading to complete. Default value is `20000` ms.
1050
-
1051
- #### sessionRequestTimeout
1052
- Timeout for getting a browser session. Default value is `httpTimeout`.
1053
-
1054
- #### sessionQuitTimeout
1055
- Timeout for quitting a session. Default value is `5000`.
1056
-
1057
- #### testTimeout
1058
- Timeout for test execution (in milliseconds).
1059
- If applied to suite then timeout will be set for all tests and hooks inside this suite.
1060
- Default value is `null`, in this case will be used common timeout for all browsers from `system.mochaOpts.timeout`.
1061
-
1062
- #### waitTimeout
1063
- Timeout for [waitUntil](https://webdriver.io/docs/api/element/waitUntil) which is used to all `waitFor*` commands. It is used in the search for elements and web page events. Default value is `3000` ms.
1064
-
1065
- *For example: `browser.$('.element').click()` will wait up to 3000ms for element to exist before clicking it by default*
1066
-
1067
- #### waitInterval
1068
- Interval for [waitUntil](https://webdriver.io/docs/api/element/waitUntil) which is used to all `waitFor*` commands. It is used in element finding and web page events. Default value is `500` ms.
1069
-
1070
- *For example: `browser.$('.element').click()` will check element existence every 500ms by default*
1071
-
1072
- #### sessionsPerBrowser
1073
- Number of sessions which are run simultaneously. Global value for all browsers. Default value is `1`.
1074
-
1075
- #### takeScreenshotOnFails
1076
- Options for setting up taking a screenshot of a test fail. Can be an object with `testFail` and `assertViewFail` keys.
1077
-
1078
- * `testFail` (default: `true`) – takes a screenshot when an error occurs in the test, except `assertView` fail.
1079
- * `assertViewFail` (default: `true`) – takes a screenshot if the test fails on the `assetView` command.
1080
-
1081
- #### takeScreenshotOnFailsMode
1082
- Mode for taking a screenshot on test fail. There are two available options:
1083
-
1084
- * `fullpage` – Testplane will take a screenshot of the entire page from top. Default value.
1085
- * `viewport` – Testplane will take a screenshot of the current viewport.
1086
-
1087
- #### takeScreenshotOnFailsTimeout
1088
- Timeout for taking screenshot on test fail. Default value is `5000`.
1089
-
1090
- #### testsPerSession
1091
- Maximum amount of tests (`it`s) to run in each web driver session. After limit is reached, session will be closed and new one will be started.
1092
- By default is `Infinity` (no limit, all tests will be run in the same session). Set to smaller number in case of problems with stability.
1093
-
1094
- #### retry
1095
- How many times a test should be retried if it fails. Global value for all browsers. Default value is `0`.
1096
-
1097
- #### shouldRetry
1098
- Function that determines whether to make a retry. Must return boolean value. By default returns `true` if retry attempts are available otherwise returns `false`.
1099
- Argument of this function is object with fields:
1100
-
1101
- * `retriesLeft {Number}` — number of available retries
1102
- * `ctx` — in case of test `TEST_FAIL` it would be bound data, in case of `ERROR` it would be link to `Runnable`
1103
- * `[error]` — error type (available only in case of ERROR)
1104
-
1105
-
1106
- #### calibrate
1107
- Does this browser need to perform the calibration procedure. This procedure allows to correctly capture the image in case the particular WebDriver implementation captures browser UI along with web page. Default value is `false`.
1108
-
1109
- #### meta
1110
- Additional data that can be obtained via .getMeta() method
1111
-
1112
- #### windowSize
1113
- Browser window dimensions (i.e. `1600x1200`). If not specified, the size of the window depends on WebDriver. Can be specified as string with pattern `800x1000` or object with `width` and `height` keys (both keys should be number). For example,
1114
-
1115
- ```javascript
1116
- windowSize: '800x1000'
1117
- ```
1118
- and
1119
- ```javascript
1120
- windowSize: {
1121
- width: 800,
1122
- height: 1000
1123
- }
1124
- ```
1125
- are the same.
1126
-
1127
- :warning: You can't set specific resolutions for browser Opera or mobile platforms. They use only full-screen resolution.
1128
-
1129
- #### screenshotDelay
1130
- Allows to specify a delay (in milliseconds) before making any screenshot. This is useful when the page has elements which are animated or if you do not want to screen a scrollbar. Default value is `0`.
1131
-
1132
- #### orientation
1133
- Browser orientation (`landscape`, `portrait`) that will be set before each test run. It is necessary in order to return the browser orientation to the default state after test execution in which orientation is changed. Default value is `null`.
1134
-
1135
- #### waitOrientationChange
1136
- Allows to wait until screen orientation is changed. Works inside `webdriverio` commands `orientation` and `setOrientation`. This option guarantee that screen rotated before the next command will start to execute. Default value is `true`.
1137
-
1138
- #### resetCursor
1139
- Allows to configure whether to move mouse cursor to `body` coordinates `(0, 0)` before each test run. This can be useful to escape cases when a default position of a cursor affects your tests. We recommend to set this option *truthy* value for desktop browsers and *falsey* for mobile devices. Default value is `true`.
1140
-
1141
- #### tolerance
1142
- Indicates maximum allowed [CIEDE2000](http://en.wikipedia.org/wiki/Color_difference#CIEDE2000) difference between colors. Used only in non-strict mode. By default it's 2.3 which should be enough for the most cases. Increasing global default is not recommended, prefer changing tolerance for particular suites or states instead.
1143
-
1144
- #### antialiasingTolerance
1145
- Read more about this option in [looks-same](https://github.com/gemini-testing/looks-same#comparing-images-with-ignoring-antialiasing).
1146
-
1147
- #### compareOpts
1148
- Extra options for comparing images. See [looks-same](https://github.com/gemini-testing/looks-same#comparing-images) documentation for the list of available options. Default values are:
1149
- ```javascript
1150
- compareOpts: {
1151
- shouldCluster: false,
1152
- clustersSize: 10,
1153
- stopOnFirstFail: false
1154
- }
1155
- ```
1156
-
1157
- #### buildDiffOpts
1158
- Extra options for building diff image. See [looks-same](https://github.com/gemini-testing/looks-same#building-diff-image) documentation for the list of available options. Default values are:
1159
- ```javascript
1160
- buildDiffOpts: {
1161
- ignoreAntialiasing: true,
1162
- ignoreCaret: true
1163
- }
1164
- ```
1165
-
1166
- #### assertViewOpts
1167
- Default options used when calling [assertView](https://github.com/gemini-testing/testplane/#assertview), can be overridden by `assertView` options. Default values are:
1168
- ```javascript
1169
- ignoreElements: [],
1170
- captureElementFromTop: true,
1171
- allowViewportOverflow: false,
1172
- disableAnimation: true,
1173
- ignoreDiffPixelCount: 0,
1174
- ```
1175
-
1176
- #### openAndWaitOpts
1177
- Default options used when calling [openAndWait](https://github.com/gemini-testing/testplane/#openandwait), can be overriden by `openAndWait` options. Default values are:
1178
- ```javascript
1179
- waitNetworkIdle: true,
1180
- waitNetworkIdleTimeout: 500,
1181
- failOnNetworkError: true,
1182
- ignoreNetworkErrorsPatterns: []
1183
- ```
1184
-
1185
- #### screenshotsDir
1186
-
1187
- Directory to save reference images for command `assertView`. Default dir is `testplane/screens` which is relative to `process.cwd()`. The value of this option can also be a function which accepts one argument - an instance of a test within which comand `assertView` is called:
1188
-
1189
- ```javascript
1190
- screenshotsDir: (test) => `tests/screenshots/${test.parent.title}`
1191
- ```
1192
-
1193
- #### strictTestsOrder
1194
-
1195
- Testplane will guarantee tests order in [readTests](#readtests) results. `false` by default.
1196
-
1197
- #### compositeImage
1198
-
1199
- Allows testing of regions which bottom bounds are outside of a viewport height (default: true). In the resulting screenshot the area which fits the viewport bounds will be joined with the area which is outside of the viewport height.
1200
-
1201
- #### screenshotMode
1202
-
1203
- Image capture mode. There are 3 allowed values for this option:
1204
-
1205
- * `auto` (default). Mode will be obtained automatically;
1206
- * `fullpage`. Testplane will deal with screenshot of full page;
1207
- * `viewport`. Only viewport area will be used.
1208
-
1209
- By default, `screenshotMode` on android browsers is set to `viewport` to work around [the chromium bug](https://bugs.chromium.org/p/chromedriver/issues/detail?id=2853).
1210
-
1211
- #### saveHistoryMode
1212
-
1213
- Allows to save history of all executed commands. `'all'` by default.
1214
-
1215
- Available options:
1216
- - `'all'` - history is enabled
1217
- - `'none'` - history is disabled
1218
- - `'onlyFailed'` - history is saved for failed tests only
1219
-
1220
- Some plugins can rely on this history, for instance:
1221
- - [html-reporter](https://github.com/gemini-testing/html-reporter)
1222
- - [@testplane/profiler](https://github.com/gemini-testing/testplane-profiler)
1223
-
1224
- The history is available from such events: `TEST_END`, `TEST_PASS`, `TEST_FAIL` through payload:
1225
- ```js
1226
- // example of plugin code
1227
- module.exports = (testplane) => {
1228
- testplane.on(testplane.events.TEST_PASS, async (test) => {
1229
- console.log(test.history);
1230
- });
1231
- };
1232
- ```
1233
-
1234
- #### agent
1235
- Allows to use a custom `http`/`https`/`http2` [agent](https://www.npmjs.com/package/got#agent) to make requests. Default value is `null` (it means that will be used default http-agent from got).
1236
-
1237
- #### headers
1238
- Allows to set custom [headers](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#headers) to pass into every webdriver request. These headers aren't passed into browser request. Read more about this option in [wdio](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#headers). Default value is `null`.
1239
-
1240
- #### transformRequest
1241
- Allows to intercept [HTTP request options](https://github.com/sindresorhus/got#options) before a WebDriver request is made. Default value is `null`. If function is passed then it takes `RequestOptions` as the first argument and should return modified `RequestOptions`. Example:
1242
-
1243
- ```javascript
1244
- (RequestOptions) => RequestOptions
1245
- ```
1246
-
1247
- In runtime a unique `X-Request-ID` header is generated for each browser request which consists of `${FIRST_X_REQ_ID}__${LAST_X_REQ_ID}`, where:
1248
- - `FIRST_X_REQ_ID` - unique uuid for each test (different for each retry), allows to find all requests related to a single test run;
1249
- - `LAST_X_REQ_ID` - unique uuid for each browser request, allows to find one unique request in one test (together with `FIRST_X_REQ_ID`).
1250
-
1251
- Header `X-Request-ID` can be useful if you manage the cloud with browsers yourself and collect logs with requests. Real-world example: `2f31ffb7-369d-41f4-bbb8-77744615d2eb__e8d011d8-bb76-42b9-b80e-02f03b8d6fe1`.
1252
-
1253
- To override generated `X-Request-ID` to your own value you need specify it in `transformRequest` handler. Example:
1254
-
1255
- ```javascript
1256
- transformRequest: (req) => {
1257
- req.headers["X-Request-ID"] = "my_x_req_id";
1258
- }
1259
- ```
1260
-
1261
- #### transformResponse
1262
- Allows to intercept [HTTP response object](https://github.com/sindresorhus/got#response) after a WebDriver response has arrived. Default value is `null`. If function is passed then it takes `Response` (original response object) as the first and `RequestOptions` as the second argument. Should return modified `Response`. Example:
1263
-
1264
- ```javascript
1265
- (Response, RequestOptions) => Response
1266
- ```
1267
-
1268
- #### strictSSL
1269
- Whether it does require SSL certificate to be valid. Default value is `null` (it means that will be used [default value from wdio](https://webdriver.io/docs/options/#strictssl)).
1270
-
1271
- #### user
1272
- Cloud service username. Default value is `null`.
1273
-
1274
- #### key
1275
- Cloud service access key or secret key. Default value is `null`.
1276
-
1277
- #### region
1278
- Ability to choose different datacenters for run in cloud service. Default value is `null`.
1279
-
1280
- #### headless
1281
- Ability to run headless browser in cloud service. Default value is `null`. Can be set as a Boolean (the default value of the browser will be used). For Chrome browsers starting from version 112 also can be specified as a string with "new"|"old" values - this will enable the new headless mode (see [Chrome's blog post](https://developer.chrome.com/docs/chromium/new-headless)).
1282
-
1283
- #### isolation
1284
- Ability to execute tests in isolated clean-state environment ([incognito browser context](https://chromedevtools.github.io/devtools-protocol/tot/Target/#method-createBrowserContext)). It means that `testsPerSession` can be set to `Infinity` in order to speed up tests execution and save browser resources. Currently works only in chrome@93 and higher. Default value is `null`, but `true` for chrome@93 and higher.
1285
-
1286
- ### system
1287
-
1288
- #### debug
1289
- Turn webdriver debug mode on. Default value is `false`.
1290
-
1291
- #### mochaOpts
1292
-
1293
- Extra options for `mocha` which are passed to `mocha.setup`. See [Mocha](https://mochajs.org/) documentation for the list of options. Default values are:
1294
- ```javascript
1295
- mochaOpts: {
1296
- slow: 10000, // If test execution time is greater than this value, then the test is slow.
1297
- timeout: 60000 // timeout for test execution.
1298
- }
1299
- ```
1300
-
1301
- #### expectOpts
1302
-
1303
- Options for [expect-webdriverio](https://webdriver.io/docs/api/expect-webdriverio/) that allow you to change default wait timeout and interval between attempts. Default values are:
1304
-
1305
- ```javascript
1306
- expectOpts: {
1307
- wait: 3000, // wait timeout for expectation to succeed
1308
- interval: 100 // interval between attempts
1309
- }
1310
- ```
1311
-
1312
- #### ctx
1313
- A context which will be available in tests via method `testplane.ctx`:
1314
- ```javascript
1315
- ctx: {
1316
- foo: 'bar'
1317
- }
1318
- ```
1319
-
1320
- ```javascript
1321
- it('awesome test', function() {
1322
- console.log(testplane.ctx); // {foo: 'bar'}
1323
- });
1324
- ```
1325
-
1326
- **Recommendation**: use `ctx` in your tests in favor of global variables.
1327
-
1328
- #### patternsOnReject
1329
- Session would be rejected if test has been faile with error message which matches to specified patterns:
1330
- ```javascript
1331
- patternsOnReject: [
1332
- /some-pattern/,
1333
- 'other-pattern'
1334
- ]
1335
- ```
1336
-
1337
- #### workers
1338
- Testplane runs all tests in subprocesses in order to decrease the main process CPU usage. This options defines the numbers of subprocesses to start for running tests. Default value is `1`.
1339
-
1340
- #### testsPerWorker
1341
- The maximum number of tests to be run in one worker before it will be restarted.
1342
-
1343
- #### parallelLimit
1344
- By default, Testplane will run all browsers simultaneously. Sometimes (i.e. when using cloud services, such as SauceLabs) you have to limit the amount of browsers that can be run at the same time. This option effectively limits how many browsers Testplane will try to run in parallel. Default value is `Infinity`.
1345
-
1346
- #### fileExtensions
1347
- Ability to set file extensions, which Testplane will search on the file system. Default value is `[.js]`.
1348
-
1349
- ### plugins
1350
- Testplane plugins are commonly used to extend built-in functionality. For example, [html-reporter](https://github.com/gemini-testing/html-reporter) and [@testplane/safari-commands](https://github.com/gemini-testing/testplane-safari-commands).
1351
-
1352
- A plugin is a module that exports a single function. The function has two arguments:
1353
-
1354
- * The `testplane` instance
1355
- * Plugin options from the configuration file
1356
-
1357
- Plugins will be loaded before Testplane runs tests.
1358
-
1359
- It's strongly recommended to name Testplane plugins with the `testplane-` prefix. This makes searching for user plugins [very simple](https://github.com/search?l=JavaScript&q=testplane-&type=Repositories&utf8=%E2%9C%93).
1360
-
1361
- If a plugin name starts with `testplane-`, then the prefix can be ommited in the configuration file. If two modules with names `testplane-some-module` and `some-module` are specified, the module with the prefix will have higher priority.
1362
-
1363
- For example:
1364
- ```js
1365
- // .testplane.conf.js
1366
- // ...
1367
- plugins: {
1368
- 'my-cool-plugin': {
1369
- param: 'value'
1370
- }
1371
- }
1372
- // ...
1373
-
1374
- // testplane-my-cool-plugin/index.js
1375
- module.exports = function(testplane, opts) {
1376
- testplane.on(testplane.events.RUNNER_START, function(runner) {
1377
- return setUp(testplane.config, opts.param); // config can be mutated
1378
- });
1379
-
1380
- testplane.on(testplane.events.RUNNER_END, function() {
1381
- return tearDown();
1382
- });
1383
- }
1384
- ```
1385
-
1386
- **Properties of the `testplane` object**
1387
-
1388
- Property name | Description
1389
- ------------------------- | -------------
1390
- `config` | Config that is used in the test runner. Can be mutated.
1391
- `events` | Events list for subscription.
1392
-
1393
- **Available events which are triggered in the main process**
1394
-
1395
- Event | Description
1396
- ------------------------- | -------------
1397
- `INIT` | Will be triggered before any job start (`run` or `readTests`). If handler returns a promise then job will start only after the promise will be resolved. Emitted only once no matter how many times job will be performed.
1398
- `BEFORE_FILE_READ` | Will be triggered on test files parsing before reading the file. The handler accepts data object with `file`, `testplane` (helper which will be available in test file) and `testParser` (`TestParserAPI` object) fields.
1399
- `AFTER_FILE_READ` | Will be triggered on test files parsing right after reading the file. The handler accepts data object with `file` and `testplane` (helper which will be available in test file) fields.
1400
- `AFTER_TESTS_READ` | Will be triggered right after tests read via `readTests` or `run` methods with `TestCollection` object.
1401
- `RUNNER_START` | Will be triggered after worker farm initialization and before test execution. If a handler returns a promise, tests will be executed only after the promise is resolved. The handler accepts an instance of a runner as the first argument. You can use this instance to emit and subscribe to any other available events.
1402
- `RUNNER_END` | Will be triggered after test execution and before worker farm ends. If a handler returns a promise, worker farm will be ended only after the promise is resolved. The handler accepts an object with tests execution statistics.
1403
- `NEW_WORKER_PROCESS` | Will be triggered after new subprocess is spawned. The handler accepts a restricted process object with only `send` method.
1404
- `SESSION_START` | Will be triggered after browser session initialization. If a handler returns a promise, tests will be executed only after the promise is resolved. The handler accepts an instance of webdriverIO as the first argument and an object with a browser identifier as the second.
1405
- `SESSION_END` | Will be triggered after the browser session ends. If a handler returns a promise, tests will be executed only after the promise is resolved. The handler accepts an instance of webdriverIO as the first argument and an object with a browser identifier as the second.
1406
- `BEGIN` | Will be triggered before test execution, but after all the runners are initialized.
1407
- `END` | Will be triggered just before `RUNNER_END` event. The handler accepts a stats of tests execution.
1408
- `SUITE_BEGIN` | Test suite is about to execute.
1409
- `SUITE_END` | Test suite execution is finished.
1410
- `TEST_BEGIN` | Test is about to execute.
1411
- `TEST_END` | Test execution is finished.
1412
- `TEST_PASS` | Test passed.
1413
- `TEST_FAIL` | Test failed.
1414
- `TEST_PENDING` | Test is skipped.
1415
- `RETRY` | Test failed but went to retry.
1416
- `ERROR` | Generic (no tests) errors.
1417
- `INFO` | Reserved.
1418
- `WARNING` | Reserved.
1419
- `EXIT` | Will be triggered when SIGTERM is received (for example, Ctrl + C). The handler can return a promise.
1420
-
1421
- **Available events which are triggered in subprocesses**
1422
-
1423
- Event | Description
1424
- ------------------------- | -------------
1425
- `BEFORE_FILE_READ` | Will be triggered on test files parsing before reading the file. The handler accepts data object with `file`, `testplane` (helper which will be available in test file) and `testParser` (`TestParserAPI` object) fields.
1426
- `AFTER_FILE_READ` | Will be triggered on test files parsing right after reading the file. The handler accepts data object with `file` and `testplane` (helper which will be available in test file) fields.
1427
- `AFTER_TESTS_READ` | Will be triggered right after tests read each time some file is being reading during test run.
1428
- `NEW_BROWSER` | Will be triggered after new browser instance created. The handler accepts an instance of webdriverIO as the first argument and an object with a browser identifier and version as the second.
1429
- `UPDATE_REFERENCE` | Will be triggered after updating reference image.
1430
-
1431
- **REMARK!**
1432
-
1433
- Events which are triggered in the main process and subprocesses can not share information between each other, for example:
1434
-
1435
- ```js
1436
- module.exports = (testplane) => {
1437
- let flag = false;
1438
-
1439
- testplane.on(testplane.events.RUNNER_START, () => {
1440
- flag = true;
1441
- });
1442
-
1443
- testplane.on(testplane.events.NEW_BROWSER, () => {
1444
- // outputs `false`, because `NEW_BROWSER` event was triggered in a subprocess,
1445
- // but `RUNNER_START` was not
1446
- console.log(flag);
1447
- });
1448
-
1449
- testplane.on(testplane.events.RUNNER_END, () => {
1450
- // outputs `true`
1451
- console.log(flag);
1452
- });
1453
- };
1454
- ```
1455
-
1456
- But you can solve such problem this way:
1457
-
1458
- ```js
1459
- module.exports = (testplane, opts) => {
1460
- testplane.on(testplane.events.RUNNER_START, () => {
1461
- opts.flag = true;
1462
- });
1463
-
1464
- testplane.on(testplane.events.NEW_BROWSER, () => {
1465
- // outputs `true`, because properties in a config (variable `opts` is a part of a config)
1466
- // which have raw data types are passed to subprocesses after `RUNNER_START` event
1467
- console.log(opts.flag);
1468
- });
1469
- };
1470
- ```
1471
-
1472
- Besides, you have the ability to intercept events in plugins and translate them to other events:
1473
-
1474
- ```js
1475
- module.exports = (testplane) => {
1476
- testplane.intercept(testplane.events.TEST_FAIL, ({event, data: test}) => {
1477
- test.skip({reason: 'intercepted failure'});
1478
-
1479
- return {event: testplane.events.TEST_PENDING, test};
1480
- });
1481
-
1482
- testplane.on(testplane.events.TEST_FAIL, (test) => {
1483
- // this event handler will never be called
1484
- });
1485
-
1486
- testplane.on(testplane.evenst.TEST_PENDING, (test) => {
1487
- // this event handler will always be called instead of 'TEST_FAIL' one
1488
- });
1489
- };
1490
- ```
1491
-
1492
- If for some reason interceptor should not translate passed event to another one you can return the same object or some falsey value:
1493
-
1494
- ```js
1495
- module.exports = (testplane) => {
1496
- testplane.intercept(testplane.events.TEST_FAIL, ({event, data}) => {
1497
- return {event, data};
1498
- // return;
1499
- // return null;
1500
- // return false;
1501
- });
1502
-
1503
- testplane.on(testplane.events.TEST_FAIL, (test) => {
1504
- // this event handler will be called as usual because interceptor does not change event
1505
- });
1506
- };
1507
- ```
1508
-
1509
- If for some reason interceptor should ignore passed event and do not translate it to any other listeners you can return an empty object:
1510
-
1511
- ```js
1512
- module.exports = (testplane) => {
1513
- testplane.intercept(testplane.events.TEST_FAIL, ({event, data}) => {
1514
- return {};
1515
- });
1516
-
1517
- testplane.on(testplane.events.TEST_FAIL, (test) => {
1518
- // this event handler will NEVER be called because interceptor ignores it
1519
- });
1520
- };
1521
- ```
1522
-
1523
- The above feature can be used to delay triggering of some events, for example:
1524
-
1525
- ```js
1526
- module.exports = (testplane) => {
1527
- const intercepted = [];
1528
-
1529
- testplane.intercept(testplane.events.TEST_FAIL, ({event, data}) => {
1530
- intercepted.push({event, data});
1531
- return {};
1532
- });
1533
-
1534
- testplane.on(testplane.events.END, () => {
1535
- intercepted.forEach(({event, data}) => testplane.emit(event, data));
1536
- });
1537
- };
1538
- ```
1539
-
1540
- **Available events which can be intercepted**
1541
-
1542
- Event |
1543
- ------------------------- |
1544
- `SUITE_BEGIN` |
1545
- `SUITE_END` |
1546
- `TEST_BEGIN` |
1547
- `TEST_END` |
1548
- `TEST_PASS` |
1549
- `TEST_FAIL` |
1550
- `RETRY` |
1551
-
1552
- #### Parallel execution plugin code
1553
-
1554
- Runner has a method `registerWorkers` which register plugin's code for parallel execution in Testplane's worker farm. The method accepts parameters `workerFilepath` (string, absolute path), `exportedMethods` (array of string) and return object which contains async functions with names from `exportedMethods`. File with path `workerFilepath` should export object which contains async functions with names from `exportedMethods`.
1555
-
1556
- *Example*
1557
- ```js
1558
- // plugin code
1559
- let workers;
1560
-
1561
- module.exports = (testplane) => {
1562
- testplane.on(testplane.events.RUNNER_START, async (runner) => {
1563
- const workerFilepath = require.resolve('./worker');
1564
- const exportedMethods = ['foo'];
1565
- workers = runner.registerWorkers(workerFilepath, exportedMethods);
1566
-
1567
- // outputs `FOO_RUNNER_START`
1568
- console.log(await workers.foo('RUNNER_START'));
1569
- });
1570
-
1571
- testplane.on(testplane.events.RUNNER_END, async () => {
1572
- // outputs `FOO_RUNNER_END`
1573
- console.log(await workers.foo('RUNNER_END'));
1574
- });
1575
- };
1576
-
1577
- // worker.js
1578
- module.exports = {
1579
- foo: async function(event) {
1580
- return 'FOO_' + event;
1581
- }
1582
- };
1583
-
1584
- ```
1585
-
1586
- #### List of useful plugins
1587
- There are several plugins that may be useful:
1588
-
1589
- * [html-reporter](https://github.com/gemini-testing/html-reporter)
1590
- * [@testplane/safari-commands](https://github.com/gemini-testing/testplane-safari-commands)
1591
- * [@testplane/headless-chrome](https://github.com/gemini-testing/testplane-headless-chrome)
1592
- * ...and many others that you can find in [gemini-testing](https://github.com/search?q=topic%3Atestplane-plugin+org%3Agemini-testing&type=Repositories).
1593
-
1594
- ### prepareBrowser
1595
- Prepare the browser session before tests are run. For example, add custom user commands.
1596
- ```js
1597
- prepareBrowser: function(browser) {
1598
- browser.addCommand('commandName', require('./path/to/commands/commandName.js'));
1599
- }
1600
- ```
1601
-
1602
- The `browser` argument is a `WebdriverIO` session.
1603
-
1604
- ### prepareEnvironment
1605
- Configuration data can be changed depending on extra conditions in the `prepareEnvironment` function.
1606
-
1607
- ## CLI
1608
-
1609
- ```
1610
- testplane --help
1611
- ```
1612
-
1613
- shows the following
1614
-
1615
- ```
1616
- Usage: testplane [options] [paths...]
1617
-
1618
- Options:
1619
-
1620
- -V, --version output the version number
1621
- -c, --config <path> path to configuration file
1622
- -b, --browser <browser> run tests only in specified browser
1623
- -s, --set <set> run tests only in the specified set
1624
- -r, --require <module> require a module before running `testplane`
1625
- --reporter <reporter> test reporters
1626
- --grep <grep> run only tests matching the pattern
1627
- --update-refs update screenshot references or gather them if they do not exist ("assertView" command)
1628
- --inspect [inspect] nodejs inspector on [=[host:]port]
1629
- --inspect-brk [inspect-brk] nodejs inspector with break at the start
1630
- --repl [type] run one test, call `browser.switchToRepl` in test code to open repl interface (default: false)
1631
- --repl-before-test [type] open repl interface before test run (default: false)
1632
- --repl-on-fail [type] open repl interface on test fail only (default: false)
1633
- -h, --help output usage information
1634
- ```
1635
-
1636
- For example,
1637
- ```
1638
- testplane --config ./config.js --reporter flat --browser firefox --grep name
1639
- ```
1640
-
1641
- **Note.** All CLI options override config values.
1642
-
1643
- ### Reporters
1644
-
1645
- You can choose `flat`, `plain` or `jsonl` reporter by option `--reporter`. Default is `flat`.
1646
- Information about test results is displayed to the command line by default. But there is an ability to redirect the output to a file, for example:
1647
- ```
1648
- testplane --reporter '{"type": "jsonl", "path": "./some-path/result.jsonl"}'
1649
- ```
1650
-
1651
- In that example specified file path and all directories will be created automatically. Moreover you can use few reporters:
1652
- ```
1653
- testplane --reporter '{"type": "jsonl", "path": "./some-path/result.jsonl"}' --reporter flat
1654
- ```
1655
-
1656
- Information about each report type:
1657
- * `flat` – all information about failed and retried tests would be grouped by browsers at the end of the report;
1658
- * `plain` – information about fails and retries would be placed after each test;
1659
- * `jsonl` - displays detailed information about each test result in [jsonl](https://jsonlines.org/) format.
1660
-
1661
- ### Require modules
1662
-
1663
- Using `-r` or `--require` option you can load external modules, which exists in your local machine, before running `testplane`. This is useful for:
1664
-
1665
- - compilers such as TypeScript via [ts-node](https://www.npmjs.com/package/ts-node) (using `--require ts-node/register`) or Babel via [@babel/register](https://www.npmjs.com/package/@babel/register) (using `--require @babel/register`);
1666
- - loaders such as ECMAScript modules via [esm](https://www.npmjs.com/package/esm).
1667
-
1668
- Be sure to update [fileExtensions](#fileExtensions) apropriately, if you are planning to import anything other than `.js`.
1669
-
1670
- ### Overriding settings
1671
-
1672
- All options can also be overridden via command-line flags or environment variables. Priorities are the following:
1673
-
1674
- * A command-line option has the highest priority. It overrides the environment variable and config file value.
1675
-
1676
- * An environment variable has second priority. It overrides the config file value.
1677
-
1678
- * A config file value has the lowest priority.
1679
-
1680
- * If there isn't a command-line option, environment variable or config file option specified, the default is used.
1681
-
1682
- To override a config setting with a CLI option, convert the full option path to `--kebab-case`. For example, if you want to run tests against a different base URL, call:
1683
-
1684
- ```
1685
- testplane path/to/mytest.js --base-url http://example.com
1686
- ```
1687
-
1688
- To change the number of sessions for Firefox (assuming you have a browser with the `firefox` id in the config):
1689
-
1690
- ```
1691
- testplane path/to/mytest.js --browsers-firefox-sessions-per-browser 7
1692
- ```
1693
-
1694
- To override a setting with an environment variable, convert its full path to `snake_case` and add the `testplane_` prefix. The above examples can be rewritten to use environment variables instead of CLI options:
1695
-
1696
- ```
1697
- testplane_base_url=http://example.com testplane path/to/mytest.js
1698
- testplane_browsers_firefox_sessions_per_browser=7 testplane path/to/mytest.js
1699
- ```
1700
-
1701
- ### Debug mode
1702
-
1703
- In order to understand what is going on in the test step by step, there is a debug mode. You can run tests in this mode using these options: `--inspect` and `--inspect-brk`. The difference between them is that the second one stops before executing the code.
1704
-
1705
- Example:
1706
- ```
1707
- testplane path/to/mytest.js --inspect
1708
- ```
1709
-
1710
- **Note**: In the debugging mode, only one worker is started and all tests are performed only in it.
1711
- Use this mode with option `sessionsPerBrowser=1` in order to debug tests one at a time.
1712
-
1713
- ### REPL mode
1714
-
1715
- Testplane provides a [REPL](https://en.wikipedia.org/wiki/Read–eval–print_loop) implementation that helps you not only to learn the framework API, but also to debug and inspect your tests. In this mode, there is no timeout for the duration of the test (it means that there will be enough time to debug the test). It can be used by specifying the CLI options:
1716
-
1717
- - `--repl` - in this mode, only one test in one browser should be run, otherwise an error is thrown. REPL interface does not start automatically, so you need to call [switchToRepl](#switchtorepl) command in the test code. Disabled by default;
1718
- - `--repl-before-test` - the same as `--repl` option except that REPL interface opens automatically before run test. Disabled by default;
1719
- - `--repl-on-fail` - the same as `--repl` option except that REPL interface opens automatically on test fail. Disabled by default.
1720
-
1721
- #### switchToRepl
1722
-
1723
- Browser command that stops the test execution and opens REPL interface in order to communicate with browser. For example:
1724
-
1725
- ```js
1726
- it('foo', async ({browser}) => {
1727
- console.log('before open repl');
1728
-
1729
- await browser.switchToRepl();
1730
-
1731
- console.log('after open repl');
1732
- });
1733
- ```
1734
-
1735
- And run it using the command:
1736
-
1737
- ```bash
1738
- npx testplane --repl --grep "foo" -b "chrome"
1739
- ```
1740
-
1741
- In this case, we are running only one test in one browser (or you can use `testplane.only.in('chrome')` before `it` + `it.only`).
1742
- When executing the test, the text `before open repl` will be displayed in the console first, then test execution stops, REPL interface is opened and waits your commands. So we can write some command in the terminal:
1743
-
1744
- ```js
1745
- await browser.getUrl();
1746
- // about:blank
1747
- ```
1748
-
1749
- In the case when you need to execute a block of code, for example:
1750
-
1751
- ```js
1752
- for (const item of [...Array(3).keys]) {
1753
- await browser.$(`.selector_${item}`).isDisplayed();
1754
- }
1755
- ```
1756
-
1757
- You need to switch to editor mode by running the `.editor` command in REPL and insert the desired a block of code. Then execute it by pressing `Ctrl+D`.
1758
- It is worth considering that some of code can be executed without editor mode:
1759
- - one-line code like `await browser.getUrl().then(console.log)`;
1760
- - few lines of code without using block scope or chaining, for example:
1761
- ```js
1762
- await browser.url('http://localhost:3000');
1763
- await browser.getUrl();
1764
- // http://localhost:3000
37
+ ```shell
38
+ npm init testplane@latest new-testplane-project
1765
39
  ```
1766
40
 
1767
- After user closes the server, the test will continue to run (text `after open repl` will be displayed in the console and browser will close).
41
+ You may add `-- -y` option to quickly set up a project using defaults.
1768
42
 
1769
- Another command features:
1770
- - all `const` and `let` declarations called in REPL mode are modified to `var` in runtime. This is done in order to be able to redefine created variables;
1771
- - before switching to the REPL mode `process.cwd` is replaced with the path to the folder of the executed test. After exiting from the REPL mode `process.cwd` is restored. This feature allows you to import modules relative to the test correctly;
1772
- - ability to pass the context to the REPL interface. For example:
43
+ 2. Open the newly generated file `testplane-tests/example.testplane.ts`. We’ll modify the test to ensure the description includes expected text:
1773
44
 
1774
- ```js
1775
- it('foo', async ({browser}) => {
1776
- const foo = 1;
45
+ ```typescript
46
+ describe("test", () => {
47
+ it("example", async ({browser}) => {
48
+ await browser.url("https://example.com/");
49
+
50
+ const description = await browser.$("p");
1777
51
 
1778
- await browser.switchToRepl({foo});
52
+ expect(description).toHaveTextContaining("for use in illustrative examples in documents");
53
+ });
1779
54
  });
1780
55
  ```
1781
56
 
1782
- And now `foo` variable is available in REPL:
1783
-
1784
- ```bash
1785
- console.log("foo:", foo);
1786
- // foo: 1
1787
- ```
1788
-
1789
- #### Test development in runtime
1790
-
1791
- For quick test development without restarting the test or the browser, you can run the test in the terminal of IDE with enabled REPL mode:
1792
-
1793
- ```bash
1794
- npx testplane --repl-before-test --grep "foo" -b "chrome"
1795
- ```
1796
-
1797
- After that, you need to configure the hotkey in IDE to run the selected one or more lines of code in the terminal. As a result, each new written line can be sent to the terminal using a hotkey and due to this, you can write a test much faster.
1798
-
1799
- Also, during the test development process, it may be necessary to execute commands in a clean environment (without side effects from already executed commands). You can achieve this with the following commands:
1800
- - [clearSession](#clearsession) - clears session state (deletes cookies, clears local and session storages). In some cases, the environment may contain side effects from already executed commands;
1801
- - [reloadSession](https://webdriver.io/docs/api/browser/reloadSession/) - creates a new session with a completely clean environment.
1802
-
1803
- ##### How to set up using VSCode
1804
-
1805
- 1. Open `Code` -> `Settings...` -> `Keyboard Shortcuts` and print `run selected text` to search input. After that, you can specify the desired key combination
1806
- 2. Run `testplane` in repl mode (examples were above)
1807
- 3. Select one or mode lines of code and press created hotkey
1808
-
1809
- ##### How to set up using Webstorm
1810
-
1811
- Ability to run selected text in terminal will be available after this [issue](https://youtrack.jetbrains.com/issue/WEB-49916/Debug-JS-file-selection) will be resolved.
1812
-
1813
- ### Environment variables
1814
-
1815
- #### TESTPLANE_SKIP_BROWSERS
1816
- Skip the browsers specified in the config by passing the browser IDs. Multiple browser IDs should be separated by commas
1817
- (spaces after commas are allowed).
1818
-
1819
- For example,
1820
- ```
1821
- TESTPLANE_SKIP_BROWSERS=ie10,ie11 testplane
1822
- ```
1823
-
1824
- #### TESTPLANE_SETS
1825
- Specify sets to run using the environment variable as an alternative to using the CLI option `--set`.
1826
-
1827
- For example,
1828
- ```
1829
- TESTPLANE_SETS=desktop,touch testplane
1830
- ```
1831
-
1832
- ## Programmatic API
1833
-
1834
- With the API, you can use Testplane programmatically in your scripts or build tools. To do this, you must require `testplane` module and create instance:
1835
-
1836
- ```js
1837
- const Testplane = require('testplane');
1838
- const testplane = new Testplane(config);
1839
- ```
1840
-
1841
- * **config** (required) `Object|String` – Configuration object or path to the configuration file that will be read relative to `process.cwd`.
1842
-
1843
- Next, you will have access to the following parameters and methods:
1844
-
1845
- Name | Description
1846
- -------------- | -------------
1847
- `config` | Returns parsed Testplane config.
1848
- `events` | Returns Testplane events on which you can subscribe to.
1849
- `errors` | Errors which Testplane may return.
1850
- `intercept` | Method to intercept Testplane's events.
1851
- `run` | Starts running tests. By default run all tests from the config. Can also run only the specified tests.
1852
- `addTestToRun` | Adds test to the current run.
1853
- `readTests` | Starts reading tests. By default read all tests from the config. Can also read only the specified tests.
1854
- `isFailed` | Returns `true` or `false` depending on whether there has been an error or a test fail while running tests.
1855
- `isWorker` | Returns `true` or `false` depending on whether you call the method in one of the workers or in the master process.
1856
- `halt` | Abnormal termination of the test run in case of a terminal error.
1857
-
1858
- ### config
1859
-
1860
- Returns parsed Testplane config. Useful when you need to read some field from the config.
1861
-
1862
- ```js
1863
- // create testplane instance
1864
- console.log('plugins:', testplane.config.plugins);
1865
- ```
1866
-
1867
- ### events
1868
-
1869
- Returns Testplane events on which you can subscribe to. Useful when you need to subscribe to a specific event.
1870
-
1871
- ```js
1872
- // create testplane instance
1873
- testplane.on(testplane.events.INIT, async () => {
1874
- console.info('INIT event is being processed...');
1875
- });
1876
- ```
1877
-
1878
- ### errors
1879
-
1880
- Testplane may return errors of the following type:
1881
-
1882
- - [CoreError](#coreerror)
1883
- - [CancelledError](#cancellederror)
1884
- - [ClientBridgeError](#clientbridgeerror)
1885
- - [HeightViewportError](#heightviewporterror)
1886
- - [OffsetViewportError](#offsetviewporterror)
1887
- - [AssertViewError](#assertviewerror)
1888
- - [ImageDiffError](#imagedifferror)
1889
- - [NoRefImageError](#norefimageerror)
1890
-
1891
- #### CoreError
1892
-
1893
- A `CoreError` is returned if the browser fails to calibrate a blank page (about:blank). The error contains the following message:
1894
-
1895
- ```
1896
- Could not calibrate. This could be due to calibration page has failed to open properly
1897
- ```
1898
-
1899
- #### CancelledError
1900
-
1901
- The `CanceledEror` error is returned if the [halt](#halt) command terminates abnormally. The error contains the following message:
57
+ 3. Launch GUI:
1902
58
 
1903
- ```
1904
- Browser request was cancelled
1905
- ```
59
+ ```shell
60
+ npx testplane gui
61
+ ```
1906
62
 
1907
- #### ClientBridgeError
63
+ 4. Try running the test and watch it pass. Now, let's replace description text check with a visual assertion. Use the `assertView` command to carry out visual checks:
1908
64
 
1909
- A `ClientBridgeError` is returned when JavaScript injection on the client (browser) side fails. Testplane injects the code using the [execute](https://webdriver.io/docs/api/browser/execute/) WebDriverIO command. The error contains the following message:
65
+ ```diff
66
+ - expect(description).toHaveTextContaining("for use in illustrative examples in documents");
67
+ + await description.assertView("description"); // "description" is just a name of the assertion
68
+ ```
1910
69
 
1911
- ```
1912
- Unable to inject client script
1913
- ```
70
+ 5. Run the test again. It will fail, because a reference image for the heading is missing. You can accept the diff and re-run the test, it will then pass.
1914
71
 
1915
- #### HeightViewportError
72
+ Congratulations on writing your first Testplane test, which navigates to a page and executes a visual assertion. Dive into the Docs to discover more awesome features Testplane has to offer!
1916
73
 
1917
- The `HeightViewportError` is returned when trying to take a screenshot of an area whose bottom border does not fit into the viewport area. The error contains the following message:
74
+ ## Docs
1918
75
 
1919
- ```
1920
- Can not capture the specified region of the viewport.
1921
- The region bottom bound is outside of the viewport height.
1922
- Alternatively, you can test such cases by setting "true" value to option "compositeImage" in the config file
1923
- or setting "false" to "compositeImage" and "true" to option "allowViewportOverflow" in "assertView" command.
1924
- Element position: <cropArea.left>, <cropArea.top>; size: <cropArea.width>, <cropArea.height>.
1925
- Viewport size: <viewport.width>, <viewport.height>.
1926
- ```
76
+ The documentation is divided into several sections:
1927
77
 
1928
- In this case, the message prompts the Testplane user what settings need to be set in the Testplane config in order to be able to take a screenshot for the specified area.
78
+ - [Testplane Features Overview](docs/features.md)
79
+ - [Quick Start](docs/quick-start.md)
80
+ - [Dealing with Browsers](docs/dealing-with-browsers.md)
81
+ - [Writing Tests](docs/writing-tests.md)
82
+ - [Browser Commands Reference](docs/commands.md)
83
+ - [Testplane Config Reference](docs/config.md)
84
+ - [Testplane × Typescript](docs/typescript.md)
85
+ - [Testplane CLI](docs/cli.md)
86
+ - [Testplane Events](docs/events.md)
87
+ - [Testplane Programmatic API](docs/programmatic-api.md)
1929
88
 
1930
- #### OffsetViewportError
89
+ ## Rename from "Hermione"
1931
90
 
1932
- An `OffsetViewportError` is returned when attempting to take a screenshot of an area whose borders on the left, right, or top extend beyond the viewport. The error contains the following message:
91
+ This project was formerly known as "Hermione", but eventually some copyright and trademark issues surfaced, leading to the decision to rebrand. After some discussion, we settled on "Testplane" as the official new title. We've tailored Testplane `v1` to be a drop-in replacement for Hermione `v8`.
1933
92
 
1934
- ```
1935
- Can not capture the specified region of the viewport.
1936
- Position of the region is outside of the viewport left, top or right bounds.
1937
- Check that elements:
1938
- - does not overflow the document
1939
- - does not overflow browser viewport
1940
- Alternatively, you can increase browser window size using
1941
- "setWindowSize" or "windowSize" option in the config file.
1942
- But if viewport overflow is expected behavior then you can use
1943
- option "allowViewportOverflow" in "assertView" command.
1944
- ```
93
+ Learn more about about migration from Hermione to Testplane in [the Docs](docs/migration-from-hermione.md).
1945
94
 
1946
- In this case, the message prompts the Testplane user what settings need to be set in the Testplane config in order to be able to take a screenshot for the specified area.
95
+ ## Contributing
1947
96
 
1948
- #### AssertViewError
97
+ Our mission with this repository is to make the Testplane development process open, while continuing to improve upon its features, performance and ease of use. We hope other organizations find value in our project and benefit from our work.
1949
98
 
1950
- An `AssertViewError` is returned when an attempt to take a screenshot fails. The error may contain one of the following messages, depending on the cause of the crash:
99
+ We welcome and appreciate community contributions. To ensure our efforts are in sync, we recommend to raise an issue or leave a comment beforehand.
1951
100
 
1952
- ```
1953
- duplicate name for "<state>" state
1954
- ```
1955
-
1956
- ```
1957
- element ("<selector>") still not existing after <this.options.waitforTimeout> ms
1958
- ```
1959
-
1960
- ```
1961
- element ("<this.selector>") still not existing after <this.options.waitforTimeout> ms
1962
- ```
1963
-
1964
- #### ImageDiffError
1965
-
1966
- An `ImageDiffError` is returned from the [assertView](#assertview) command if a diff (difference in images) is detected when taking and comparing a screenshot with a reference screenshot. The error contains the following message:
1967
-
1968
- ```
1969
- images are different for "<stateName>" state
1970
- ```
1971
-
1972
- In addition, the `ImageDiffError` error contains the following data:
1973
-
1974
- * **stateName** `String` - Name of the state for which the screenshot was taken.
1975
- * **currImg** `Object` - Link to actual image.
1976
- * **refImg** `Object` - Link to reference image.
1977
- * **diffOpts** `Object` - Diff detection settings.
1978
- * **diffBounds** `Object` - Boundaries of areas with diffs in the image.
1979
- * **diffClusters** `Object` - Clusters with diffs in the image.
1980
-
1981
- Read more about [diffBounds](https://github.com/gemini-testing/looks-same/blob/master/README.md#getting-diff-bounds) and [diffClusters](https://github.com/gemini-testing/looks-same/blob/master/README.md#getting-diff-clusters) in the documentation of the [looks-same](https://github.com/gemini-testing/looks-same) package.
1982
-
1983
- #### NoRefImageError
1984
-
1985
- The `NoRefImageError` error is returned from the [assertView](#assertview) command if, when taking and comparing a screenshot, Testplane does not find a reference screenshot on the file system. The error contains the following message:
1986
-
1987
- ```
1988
- can not find reference image at <refImg.path> for "<stateName>" state
1989
- ```
1990
-
1991
- In addition, the `NoRefImageError` error contains the following data:
1992
-
1993
- * **stateName** `String` - Name of the state for which the screenshot was taken.
1994
- * **currImg** `Object` - Link to actual image.
1995
- * **refImg** `Object` - Link to reference image.
1996
-
1997
- ### intercept
1998
-
1999
- Method to intercept Testplane's events. The first argument of the method is the event that needs to be intercepted, and the second is the event handler. Example:
2000
-
2001
- ```js
2002
- // create testplane instance
2003
- testplane.intercept(testplane.events.TEST_FAIL, ({ event, data }) => {
2004
- return {event: testplane.events.TEST_PASS, test}; // make the test successful
2005
- });
2006
- ```
2007
-
2008
- Read more about event interception in the section [about plugins](#plugins).
2009
-
2010
- ### run
2011
-
2012
- Starts running tests. By default run all tests from the config. Can also run only the specified tests. Returns `true` if the test run succeeded, and `false` if it failed. Example:
2013
-
2014
- ```js
2015
- // create testplane instance
2016
- const success = await testplane.run(testPaths, options);
2017
- ```
2018
-
2019
- Available parameters:
2020
-
2021
- * **testPaths** (optional) `String[]|TestCollection` – Paths to tests relative to `process.cwd`. Also accepts test collection returned by `readTests`.
2022
- * **options** (optional) `Object`
2023
- * **reporters** (optional) `String[]` – Test result reporters.
2024
- * **browsers** (optional) `String[]` – Browsers to run tests in.
2025
- * **sets** (optional) `String[]`– Sets to run tests in.
2026
- * **grep** (optional) `RegExp` – Pattern that defines which tests to run.
2027
-
2028
- ### addTestToRun
2029
-
2030
- Adds test to the current run. Returns `false` if the current run has already ended or has been cancelled. Otherwise returns `true`. Example:
2031
-
2032
- ```js
2033
- // create testplane instance
2034
- const success = testplane.addTestToRun(test, browserId);
2035
- ```
2036
-
2037
- * **test** (required) `Test` – Test to run.
2038
- * **browserId** (required) `String` – Browser to run test in.
2039
-
2040
- ### readTests
2041
-
2042
- Starts reading tests. By default read all tests from the config. Can also read only the specified tests. Returns promise which resolves to the instance of [TestCollection](#test-collection) initialized by parsed tests. Example:
2043
-
2044
- ```js
2045
- // create testplane instance
2046
- await testplane.readTests(testPaths, options);
2047
- ```
2048
-
2049
- * **testPaths** (required) `String[]` – Paths to tests relative to `process.cwd`.
2050
- * **options** (optional) `Object`:
2051
- * **browsers** (optional) `String[]` – Read tests only for the specified browsers.
2052
- * **silent** (optional) `Boolean` – flag to disable events emitting while reading tests; default is `false`.
2053
- * **ignore** (optional) `String|Glob|Array<String|Glob>` - patterns to exclude paths from the test search.
2054
- * **sets** (optional) `String[]`– Sets to run tests in.
2055
- * **grep** (optional) `RegExp` – Pattern that defines which tests to run.
2056
-
2057
- ### isFailed
2058
-
2059
- Returns `true` or `false` depending on whether there has been an error or a test fail while running tests. Can be useful in plugins to determine Testplane's current status. Example:
2060
-
2061
- ```js
2062
- // create testplane instance
2063
- const failed = testplane.isFailed();
2064
- ```
2065
-
2066
- ### isWorker
2067
-
2068
- Returns `true` or `false` depending on whether you call the method in one of the workers or in the master process. Can be useful in plugins in order to distinguish the code execution context. Example:
2069
-
2070
- ```js
2071
- // implementation of some plugin
2072
- module.exports = (testplane) => {
2073
- if (testplane.isWorker()) {
2074
- // do some stuff only in workers
2075
- } else {
2076
- // do some stuff only in the master process
2077
- }
2078
- };
2079
- ```
2080
-
2081
- ### halt
2082
-
2083
- Abnormal termination of the test run in case of a terminal error. If process fails to gracefully shutdown in `timeout` milliseconds, it would be forcibly terminated (unless `timeout` is explicitly set to `0`).
2084
-
2085
- ```js
2086
- // create testplane instance
2087
- testplane.halt(error, [timeout=60000ms]);
2088
- ```
2089
-
2090
- ### Test Collection
2091
-
2092
- TestCollection object is returned by `testplane.readTests` method.
2093
-
2094
- TestCollection API:
2095
-
2096
- * `getBrowsers()` — returns list of browsers for which there are tests in collection.
2097
-
2098
- * `mapTests(browserId, (test, browserId) => ...)` - maps over tests for passed browser. If first argument (`browserId`) is omitted then method will map over tests for all browsers.
2099
-
2100
- * `sortTests(browserId, (currentTest, nextTest) => ...)` - sorts over tests for passed browser. If first argument (`browserId`) is omitted then method will sort over tests for all browsers.
2101
-
2102
- * `eachTest(browserId, (test, browserId) => ...)` - iterates over tests for passed browser. If first argument (`browserId`) is omitted then method will iterate over tests for all browsers.
2103
-
2104
- * `eachTestByVersion(browserId, (test, browserId, version) => ...)` - iterates over tests and over browser versions for passed browser. Implicitly sets browser version for each test in `browserVersion` property.
2105
-
2106
- * `disableAll([browserId])` - disables all tests. Disables tests for specific browser if `browserId` passed. Returns current test collection instance.
2107
-
2108
- * `enableAll([browserId])` - enables all previously disabled tests. Enables tests for specific browser if `browserId` passed. Returns current test collection instance.
2109
-
2110
- * `disableTest(fullTitle, [browserId])` - disables test with passed full title. Disables test only in specific browser if `browserId` passed. Returns current test collection instance.
2111
-
2112
- * `enableTest(fullTitle, [browserId])` - enables test with passed full title. Enables test only in specific browser if `browserId` passed. Returns current test collection instance.
2113
-
2114
- * `getRootSuite(browserId)` - returns root suite for passed browser. Returns `undefined` if there are no tests in collection for passed browser.
2115
-
2116
- * `eachRootSuite((root, browserId) => ...)` - iterates over all root suites in collection which have some tests.
2117
-
2118
- ### Test Parser API
2119
-
2120
- `TestParserAPI` object is emitted on `BEFORE_FILE_READ` event. It provides the ability to customize test parsing process.
2121
-
2122
- #### setController(name, methods)
2123
-
2124
- Adds controller to `testplane` object in test files.
2125
-
2126
- * `name` - controller name
2127
- * `methods` - an object with names as keys and callbacks as values describing controller methods. Each callback will be called on corresponding test or suite.
2128
-
2129
- Example:
2130
- ```js
2131
- // in plugin
2132
- testplane.on(testplane.events.BEFORE_FILE_READ, ({file, testParser}) => {
2133
- testParser.setController('logger', {
2134
- log: function(prefix) {
2135
- console.log(`${prefix}: Just parsed ${this.fullTitle()} from file ${file}`);
2136
- }
2137
- });
2138
- });
2139
-
2140
- // in test file
2141
- describe('foo', () => {
2142
- testplane.logger.log('some-prefix');
2143
- it('bar', function() {
2144
- // ...
2145
- });
2146
- });
2147
- ```
101
+ Visit our [contributing guide](CONTRIBUTING.md) to understand more about our development process and how to get involved.
2148
102
 
2149
- **Note**: controller will be removed as soon as current file will be parsed
103
+ ## License
104
+ Testplane is [MIT licensed](LICENSE).