codeceptjs 4.0.0-rc.22 → 4.0.0-rc.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -10
- package/docs/ai.md +3 -51
- package/docs/architecture.md +16 -0
- package/docs/bootstrap.md +1 -1
- package/docs/continuous-integration.md +16 -44
- package/docs/custom-helpers.md +1 -1
- package/docs/detox.md +1 -1
- package/docs/docker.md +1 -30
- package/docs/examples.md +0 -1
- package/docs/helpers/Appium.md +16 -2
- package/docs/helpers/Playwright.md +161 -160
- package/docs/helpers/Puppeteer.md +143 -250
- package/docs/helpers/WebDriver.md +134 -177
- package/docs/hooks.md +11 -1
- package/docs/index.md +1 -1
- package/docs/installation.md +2 -19
- package/docs/locators.md +1 -1
- package/docs/migrate-from-cypress.md +98 -0
- package/docs/migrate-from-java.md +108 -0
- package/docs/migrate-from-protractor.md +101 -0
- package/docs/migrate-from-testcafe.md +99 -0
- package/docs/migration-4.md +195 -8
- package/docs/plugins/aiTrace.md +49 -0
- package/docs/plugins/analyze.md +66 -0
- package/docs/plugins/auth.md +241 -0
- package/docs/plugins/autoDelay.md +48 -0
- package/docs/plugins/browser.md +41 -0
- package/docs/plugins/coverage.md +39 -0
- package/docs/plugins/customLocator.md +119 -0
- package/docs/plugins/customReporter.md +16 -0
- package/docs/plugins/expose.md +75 -0
- package/docs/plugins/heal.md +44 -0
- package/docs/plugins/junitReporter.md +51 -0
- package/docs/plugins/pageInfo.md +34 -0
- package/docs/plugins/pause.md +43 -0
- package/docs/plugins/pauseOnFail.md +18 -0
- package/docs/plugins/retryFailedStep.md +75 -0
- package/docs/plugins/screencast.md +55 -0
- package/docs/plugins/screenshot.md +58 -0
- package/docs/plugins/screenshotOnFail.md +18 -0
- package/docs/plugins/stepTimeout.md +65 -0
- package/docs/plugins.md +40 -862
- package/docs/reports.md +18 -4
- package/docs/retry.md +48 -18
- package/docs/store.md +94 -0
- package/docs/timeouts.md +1 -1
- package/docs/tutorial.md +207 -155
- package/docs/webdriver.md +6 -73
- package/lib/actor.js +1 -36
- package/lib/command/run-multiple.js +1 -2
- package/lib/heal.js +2 -2
- package/lib/helper/Playwright.js +1 -15
- package/lib/helper/Puppeteer.js +0 -103
- package/lib/helper/WebDriver.js +1 -28
- package/lib/helper/extras/PlaywrightLocator.js +10 -0
- package/lib/locator.js +0 -13
- package/lib/plugin/aiTrace.js +4 -3
- package/lib/plugin/analyze.js +3 -4
- package/lib/plugin/junitReporter.js +1 -1
- package/lib/plugin/pauseOnFail.js +3 -1
- package/lib/plugin/retryFailedStep.js +11 -10
- package/lib/plugin/screencast.js +1 -1
- package/lib/plugin/screenshot.js +2 -7
- package/lib/plugin/screenshotOnFail.js +3 -1
- package/lib/plugin/stepTimeout.js +3 -2
- package/lib/recorder.js +1 -1
- package/lib/step/base.js +7 -7
- package/lib/step/comment.js +2 -2
- package/lib/step/helper.js +4 -4
- package/lib/step/meta.js +3 -3
- package/lib/step/record.js +3 -3
- package/lib/workers.js +0 -4
- package/package.json +3 -4
- package/docs/helpers/Mochawesome.md +0 -8
- package/docs/helpers/MockServer.md +0 -212
- package/docs/helpers/Polly.md +0 -44
- package/docs/helpers/Protractor.md +0 -1769
- package/docs/helpers/SoftExpectHelper.md +0 -352
- package/docs/react.md +0 -70
- package/lib/helper/Mochawesome.js +0 -96
- package/lib/helper/extras/PlaywrightReactVueLocator.js +0 -61
- package/lib/helper/extras/React.js +0 -65
package/docs/migration-4.md
CHANGED
|
@@ -96,14 +96,27 @@ await Container.create(config, opts)
|
|
|
96
96
|
|
|
97
97
|
| Removed helper | What to do |
|
|
98
98
|
|----------------|------------|
|
|
99
|
-
| `Nightmare` | Switch to `Playwright`, `Puppeteer`, or `WebDriver`. |
|
|
100
|
-
| `Protractor` | Switch to `Playwright` or `WebDriver`. |
|
|
99
|
+
| `Nightmare` | Switch to `Playwright`, `Puppeteer`, or `WebDriver`. Nightmare is unmaintained; `Playwright` is the closest drop-in for headless flows. |
|
|
100
|
+
| `Protractor` | Switch to `Playwright` or `WebDriver`. Angular apps work without the Protractor-specific waits — use `waitForElement`/`waitForClickable`. |
|
|
101
101
|
| `TestCafe` | Switch to `Playwright`. |
|
|
102
102
|
| `AI` | Use the top-level `ai:` config option and the new `aiTrace` plugin. |
|
|
103
103
|
| `SoftExpectHelper` | Use the `hopeThat` effect instead — see below. |
|
|
104
|
+
| `Mochawesome` | Removed. Use the [Testomat.io Reporter](https://github.com/testomatio/reporter) HTML pipe for HTML reports — see below. |
|
|
104
105
|
|
|
105
106
|
`Container.STANDARD_ACTING_HELPERS` no longer lists `TestCafe`.
|
|
106
107
|
|
|
108
|
+
If you relied on a removed helper for behavior none of the built-in helpers cover, **write a custom helper**. A helper is a plain class extending `@codeceptjs/helper`; you can wrap any Node library and expose `I.*` actions. See [Custom Helpers](/helpers).
|
|
109
|
+
|
|
110
|
+
```js
|
|
111
|
+
import Helper from '@codeceptjs/helper'
|
|
112
|
+
|
|
113
|
+
class MyHelper extends Helper {
|
|
114
|
+
async doSomething() { /* wrap any library here */ }
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
export default MyHelper
|
|
118
|
+
```
|
|
119
|
+
|
|
107
120
|
### `SoftExpectHelper` → `hopeThat`
|
|
108
121
|
|
|
109
122
|
3.x shipped a `SoftExpectHelper` (`I.softAssert`, `I.softExpectEqual`, `I.flushSoftAssertions`, etc.) for soft assertions. It is gone in 4.x. Use the `hopeThat` effect — it works with **any** assertion that throws (built-in `I.see*`, your custom helper, `expect` from chai/jest, Node's `assert`).
|
|
@@ -131,6 +144,147 @@ hopeThat.noErrors()
|
|
|
131
144
|
|
|
132
145
|
Each `hopeThat()` call records the failure as a note on the test and lets the scenario continue; `hopeThat.noErrors()` throws once at the end with every recorded failure if any happened. See [Effects: hopeThat](/effects#hopethat).
|
|
133
146
|
|
|
147
|
+
### `Mochawesome` → Testomat.io Reporter
|
|
148
|
+
|
|
149
|
+
3.x bundled a `Mochawesome` helper that pushed steps and screenshots into a [`mochawesome`](https://www.npmjs.com/package/mochawesome) Mocha report. The helper, the `mochawesome` dependency, and the worker-level report-dir wiring are all gone in 4.x.
|
|
150
|
+
|
|
151
|
+
For an HTML report, use the [Testomat.io Reporter](https://github.com/testomatio/reporter) with the HTML pipe — it includes steps, screenshots, videos, and traces, works under `--workers`, and needs no helper in your config. See [Reports](/reports).
|
|
152
|
+
|
|
153
|
+
3.x:
|
|
154
|
+
|
|
155
|
+
```js
|
|
156
|
+
helpers: { Mochawesome: { uniqueScreenshotNames: true } }
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
npx codeceptjs run --reporter mochawesome --reporter-options reportDir=output
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
4.x:
|
|
164
|
+
|
|
165
|
+
```bash
|
|
166
|
+
npm install --save-dev @testomatio/reporter
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
```js
|
|
170
|
+
// codecept.conf.js
|
|
171
|
+
plugins: {
|
|
172
|
+
testomatio: {
|
|
173
|
+
enabled: true,
|
|
174
|
+
require: '@testomatio/reporter/codecept',
|
|
175
|
+
html: true,
|
|
176
|
+
},
|
|
177
|
+
}
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
npx codeceptjs run
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
The HTML report is written to `output/report/` by default. See [Reports > HTML](/reports) for pipe options.
|
|
185
|
+
|
|
186
|
+
Reporting notes:
|
|
187
|
+
|
|
188
|
+
- **JUnit XML** (CI servers, GitHub Actions test tab): enable the [`junitReporter`](/plugins#junitreporter) plugin instead of `mocha-junit-reporter`. It includes CodeceptJS steps. See [Reports → JUnit XML](/reports#junit-xml).
|
|
189
|
+
- **Multiple Mocha reporters** via `mocha-multi` / `cmr` is **not recommended** in 4.x — the Testomat.io HTML pipe plus the `junitReporter` plugin cover the HTML + JUnit combination without chaining reporters.
|
|
190
|
+
|
|
191
|
+
#### Keeping Mochawesome (not recommended)
|
|
192
|
+
|
|
193
|
+
The `mochawesome` reporter itself is a stock Mocha reporter — it never depended on CodeceptJS bundling it, so it keeps working. Only the bundled **helper** (which embedded failure screenshots into the report) was removed. If you must stay on Mochawesome, you own that glue now.
|
|
194
|
+
|
|
195
|
+
**Report only — no screenshots embedded.** This works as-is:
|
|
196
|
+
|
|
197
|
+
```bash
|
|
198
|
+
npm install --save-dev mochawesome
|
|
199
|
+
npx codeceptjs run --reporter mochawesome --reporter-options reportDir=output
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
**Report with embedded failure screenshots.** Re-create the old helper as a project-local custom helper. This is a faithful port of the 3.x helper:
|
|
203
|
+
|
|
204
|
+
```js
|
|
205
|
+
// helpers/Mochawesome.js
|
|
206
|
+
import Helper from '@codeceptjs/helper'
|
|
207
|
+
import { createRequire } from 'module'
|
|
208
|
+
// ⚠️ Internal, NOT semver-stable subpaths. Pin your codeceptjs version —
|
|
209
|
+
// a future minor may move these. This coupling is why core dropped the helper.
|
|
210
|
+
import { clearString } from 'codeceptjs/lib/utils.js'
|
|
211
|
+
import { testToFileName } from 'codeceptjs/lib/mocha/test.js'
|
|
212
|
+
|
|
213
|
+
const addContext = createRequire(import.meta.url)('mochawesome/addContext')
|
|
214
|
+
|
|
215
|
+
class Mochawesome extends Helper {
|
|
216
|
+
constructor(config) {
|
|
217
|
+
super(config)
|
|
218
|
+
this.options = { uniqueScreenshotNames: false, disableScreenshots: false, ...config }
|
|
219
|
+
this.currentTest = ''
|
|
220
|
+
this.currentSuite = null
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
_beforeSuite(suite) {
|
|
224
|
+
this.currentSuite = suite
|
|
225
|
+
this.currentTest = ''
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
_before() {
|
|
229
|
+
if (this.currentSuite?.ctx) {
|
|
230
|
+
this.currentTest = { test: this.currentSuite.ctx.currentTest }
|
|
231
|
+
}
|
|
232
|
+
}
|
|
233
|
+
|
|
234
|
+
_test(test) {
|
|
235
|
+
this.currentTest = { test }
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
_failed(test) {
|
|
239
|
+
if (this.options.disableScreenshots) return
|
|
240
|
+
let fileName
|
|
241
|
+
if (test.ctx?.test?.type === 'hook') {
|
|
242
|
+
this.currentTest = { test: test.ctx.test }
|
|
243
|
+
test._retries = -1
|
|
244
|
+
fileName = clearString(`${test.title}_${this.currentTest.test.title}`)
|
|
245
|
+
} else {
|
|
246
|
+
this.currentTest = { test }
|
|
247
|
+
fileName = testToFileName(test)
|
|
248
|
+
}
|
|
249
|
+
if (this.options.uniqueScreenshotNames) {
|
|
250
|
+
fileName = testToFileName(test, { unique: true })
|
|
251
|
+
}
|
|
252
|
+
if (test._retries < 1 || test._retries === test.retryNum) {
|
|
253
|
+
return addContext(this.currentTest, `${fileName}.failed.png`)
|
|
254
|
+
}
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
// exposed as I.addMochawesomeContext(...) for manual attachments
|
|
258
|
+
addMochawesomeContext(context) {
|
|
259
|
+
if (this.currentTest === '') this.currentTest = { test: this.currentSuite.ctx.test }
|
|
260
|
+
return addContext(this.currentTest, context)
|
|
261
|
+
}
|
|
262
|
+
}
|
|
263
|
+
|
|
264
|
+
export default Mochawesome
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
Register it and run with the reporter:
|
|
268
|
+
|
|
269
|
+
```js
|
|
270
|
+
// codecept.conf.js
|
|
271
|
+
helpers: {
|
|
272
|
+
Mochawesome: {
|
|
273
|
+
require: './helpers/Mochawesome.js',
|
|
274
|
+
uniqueScreenshotNames: true,
|
|
275
|
+
},
|
|
276
|
+
}
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
```bash
|
|
280
|
+
npx codeceptjs run --reporter mochawesome --reporter-options reportDir=output
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
Two caveats this port cannot fully solve:
|
|
284
|
+
|
|
285
|
+
- The internal imports (`codeceptjs/lib/utils.js`, `codeceptjs/lib/mocha/test.js`) are reachable but **not part of the public API** — pin `codeceptjs` and re-test on upgrades.
|
|
286
|
+
- The 3.x `screenshot`/`screenshotOnFail` plugin read `helpers.Mochawesome.config.uniqueScreenshotNames` to keep the embedded reference and the saved screenshot file in sync. Core no longer does this. Set `uniqueScreenshotNames` to the **same value** on both this helper and the `screenshot` plugin, or the embedded filename and the saved file can diverge on retries.
|
|
287
|
+
|
|
134
288
|
### Custom Assertion Libraries
|
|
135
289
|
|
|
136
290
|
No code changes are required for chai, expect, jest-style matchers, or Node's `assert` — just import them in your test files. With `noGlobals: true`, they work the same as before.
|
|
@@ -148,8 +302,8 @@ Heads up on chai: 3.x pinned `chai@4`; 4.x devDep is `chai@6`, which is **ESM-on
|
|
|
148
302
|
| `commentStep` | `import step from 'codeceptjs/steps'` then `step.section('name')` / `step.endSection()` |
|
|
149
303
|
| `fakerTransform` | Import `@faker-js/faker` directly in tests. |
|
|
150
304
|
| `enhancedRetryFailedStep` | Merged into `retryFailedStep`. Rename in config. |
|
|
151
|
-
| `allure` | Use [@testomatio/reporter](https://testomat.io)
|
|
152
|
-
| `htmlReporter` | Use
|
|
305
|
+
| `allure` | Use [@testomatio/reporter](https://testomat.io). |
|
|
306
|
+
| `htmlReporter` | Use the [Testomat.io Reporter](https://github.com/testomatio/reporter) HTML pipe — see [Reports](/reports). |
|
|
153
307
|
| `wdio` | Configure WebdriverIO services directly in `helpers.WebDriver`. |
|
|
154
308
|
| `selenoid` | Run Selenoid externally. |
|
|
155
309
|
| `standardActingHelpers` | No longer needed; the list lives in core. |
|
|
@@ -320,18 +474,50 @@ Use one of:
|
|
|
320
474
|
|
|
321
475
|
The `customLocators` strategy registration in Playwright config is removed. Use the `customLocator` plugin or built-in ARIA locators (`{ role: 'button', name: 'Submit' }`).
|
|
322
476
|
|
|
323
|
-
###
|
|
477
|
+
### React and Vue Locators removed
|
|
478
|
+
|
|
479
|
+
The `react` component locator and the bare-string `_react=`/`_vue=` Playwright selectors are removed from the Playwright, Puppeteer, and WebDriver helpers. The `resq` dependency is dropped.
|
|
480
|
+
|
|
481
|
+
```js
|
|
482
|
+
// 3.x (removed)
|
|
483
|
+
I.click({ react: 'SubmitButton' })
|
|
484
|
+
I.seeElement({ react: 'Alert' })
|
|
485
|
+
I.fillField({ react: 'EmailInput' }, 'a@b.com')
|
|
486
|
+
```
|
|
487
|
+
|
|
488
|
+
They relied on `resq`, which is unmaintained, supports only React 16, reads React's private internal tree, and breaks under production minification. There is no working path for React 17, 18, or 19.
|
|
489
|
+
|
|
490
|
+
Use [ARIA locators](/locators#aria-locators) instead — they match how a user perceives the page and survive refactoring and minification:
|
|
324
491
|
|
|
325
|
-
|
|
492
|
+
```js
|
|
493
|
+
// 4.x
|
|
494
|
+
I.click({ role: 'button', name: 'Submit' })
|
|
495
|
+
I.seeElement('[role=alert]')
|
|
496
|
+
I.fillField('Email', 'a@b.com')
|
|
497
|
+
```
|
|
498
|
+
|
|
499
|
+
For a component that renders no stable role, label, or text, add a `data-testid` in the JSX and locate by it: `I.click('[data-testid="submit"]')`.
|
|
500
|
+
|
|
501
|
+
### `I.retry()` and `I.limitTime()` removed
|
|
502
|
+
|
|
503
|
+
Both were deprecated in 3.x and are **removed in 4.x**. They configured the *next* step through a chained call; the replacement is the step options API — pass a `step.*` config as the **last argument** of the step itself.
|
|
326
504
|
|
|
327
505
|
```js
|
|
328
506
|
import step from 'codeceptjs/steps'
|
|
329
507
|
|
|
330
|
-
|
|
331
|
-
I.
|
|
508
|
+
// 3.x (removed) → 4.x
|
|
509
|
+
I.retry(3).click('Submit') // I.click('Submit', step.retry(3))
|
|
510
|
+
I.limitTime(10).fillField('Email', 'a@b.c') // I.fillField('Email', 'a@b.c', step.timeout(10))
|
|
511
|
+
```
|
|
512
|
+
|
|
513
|
+
`step.*` configs are also composable with the other step options:
|
|
514
|
+
|
|
515
|
+
```js
|
|
332
516
|
I.click('Add', step.opts({ elementIndex: 2 }))
|
|
333
517
|
```
|
|
334
518
|
|
|
519
|
+
The behavior is unchanged — the option applies only to the step it is attached to, not to subsequent steps (this also fixes the 3.x footgun where `I.retry()` could leak retry settings onto the following step). `recorder.retry()` is unaffected and remains available for custom helpers.
|
|
520
|
+
|
|
335
521
|
### `within` Is Now an Effect
|
|
336
522
|
|
|
337
523
|
In 3.x, `within(...)` was a global statement available everywhere. In 4.x it's an effect alongside `tryTo`, `retryTo`, and `hopeThat`. Under `noGlobals: true` you must import it:
|
|
@@ -527,6 +713,7 @@ If your project depends on these directly, check for breakage:
|
|
|
527
713
|
| `testcafe` | 3.7.2 | **removed** |
|
|
528
714
|
| `inquirer-test` | 2.0.1 | **removed** |
|
|
529
715
|
| `joi` | 18 | **removed** — use `zod` |
|
|
716
|
+
| `resq` | 1.11 | **removed** — `react`/`vue` locators dropped; use ARIA locators |
|
|
530
717
|
| `zod` | — | added (^4) — schema validation in `JSONResponse` |
|
|
531
718
|
| `tsx` | — | added as optional peer |
|
|
532
719
|
| `@modelcontextprotocol/sdk` | — | added |
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
---
|
|
2
|
+
permalink: /plugins/aiTrace
|
|
3
|
+
editLink: false
|
|
4
|
+
sidebar: auto
|
|
5
|
+
title: aiTrace
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
|
|
9
|
+
|
|
10
|
+
## aiTrace
|
|
11
|
+
|
|
12
|
+
Generates AI-friendly trace files for debugging with AI agents.
|
|
13
|
+
This plugin creates a markdown file with test execution logs and links to all artifacts
|
|
14
|
+
(screenshots, HTML, ARIA snapshots, browser logs, HTTP requests) for each step.
|
|
15
|
+
|
|
16
|
+
#### Configuration
|
|
17
|
+
|
|
18
|
+
```js
|
|
19
|
+
"plugins": {
|
|
20
|
+
"aiTrace": {
|
|
21
|
+
"enabled": true
|
|
22
|
+
}
|
|
23
|
+
}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Possible config options:
|
|
27
|
+
|
|
28
|
+
* `deleteSuccessful`: delete traces for successfully executed tests. Default: false.
|
|
29
|
+
* `fullPageScreenshots`: should full page screenshots be used. Default: false.
|
|
30
|
+
* `output`: a directory where traces should be stored. Default: `output`.
|
|
31
|
+
* `captureHTML`: capture HTML for each step. Default: true.
|
|
32
|
+
* `captureARIA`: capture ARIA snapshot for each step. Default: true.
|
|
33
|
+
* `captureBrowserLogs`: capture browser console logs. Default: true.
|
|
34
|
+
* `captureHTTP`: capture HTTP requests (requires `trace` or `recordHar` enabled in helper config). Default: true.
|
|
35
|
+
* `captureDebugOutput`: capture CodeceptJS debug output. Default: true.
|
|
36
|
+
* `ignoreSteps`: steps to ignore in trace. Array of RegExps is expected.
|
|
37
|
+
* `on`: trigger mode — `step` (default), `fail`, `test`, `file`, `url`.
|
|
38
|
+
|
|
39
|
+
#### `on=` modes
|
|
40
|
+
|
|
41
|
+
* **step** — persist every step (default)
|
|
42
|
+
* **fail** — persist only the failed step
|
|
43
|
+
* **test** — persist only the last step of each test
|
|
44
|
+
* **file** — persist steps from `path=...[;line=...]`
|
|
45
|
+
* **url** — persist when the current URL matches `pattern=...`
|
|
46
|
+
|
|
47
|
+
### Parameters
|
|
48
|
+
|
|
49
|
+
* `config` **any**
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
---
|
|
2
|
+
permalink: /plugins/analyze
|
|
3
|
+
editLink: false
|
|
4
|
+
sidebar: auto
|
|
5
|
+
title: analyze
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
|
|
9
|
+
|
|
10
|
+
## analyze
|
|
11
|
+
|
|
12
|
+
Uses AI to analyze test failures and provide insights
|
|
13
|
+
|
|
14
|
+
This plugin analyzes failed tests using AI to provide detailed explanations and group similar failures.
|
|
15
|
+
When enabled with --ai flag, it generates reports after test execution.
|
|
16
|
+
|
|
17
|
+
#### Usage
|
|
18
|
+
|
|
19
|
+
```js
|
|
20
|
+
// in codecept.conf.js
|
|
21
|
+
exports.config = {
|
|
22
|
+
plugins: {
|
|
23
|
+
analyze: {
|
|
24
|
+
enabled: true,
|
|
25
|
+
clusterize: 5,
|
|
26
|
+
analyze: 2,
|
|
27
|
+
vision: false
|
|
28
|
+
}
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
#### Configuration
|
|
34
|
+
|
|
35
|
+
* `clusterize` (number) - minimum number of failures to trigger clustering analysis. Default: 5
|
|
36
|
+
* `analyze` (number) - maximum number of individual test failures to analyze in detail. Default: 2
|
|
37
|
+
* `vision` (boolean) - enables visual analysis of test screenshots. Default: false
|
|
38
|
+
* `categories` (array) - list of failure categories for classification. Defaults to:
|
|
39
|
+
* Browser connection error / browser crash
|
|
40
|
+
* Network errors (server error, timeout, etc)
|
|
41
|
+
* HTML / page elements (not found, not visible, etc)
|
|
42
|
+
* Navigation errors (404, etc)
|
|
43
|
+
* Code errors (syntax error, JS errors, etc)
|
|
44
|
+
* Library & framework errors
|
|
45
|
+
* Data errors (password incorrect, invalid format, etc)
|
|
46
|
+
* Assertion failures
|
|
47
|
+
* Other errors
|
|
48
|
+
* `prompts` (object) - customize AI prompts for analysis
|
|
49
|
+
* `clusterize` - prompt for clustering analysis
|
|
50
|
+
* `analyze` - prompt for individual test analysis
|
|
51
|
+
|
|
52
|
+
#### Features
|
|
53
|
+
|
|
54
|
+
* Groups similar failures when number of failures >= clusterize value
|
|
55
|
+
* Provides detailed analysis of individual failures
|
|
56
|
+
* Analyzes screenshots if vision=true and screenshots are available
|
|
57
|
+
* Classifies failures into predefined categories
|
|
58
|
+
* Suggests possible causes and solutions
|
|
59
|
+
|
|
60
|
+
### Parameters
|
|
61
|
+
|
|
62
|
+
* `config` **[Object][1]** Plugin configuration
|
|
63
|
+
|
|
64
|
+
Returns **void** 
|
|
65
|
+
|
|
66
|
+
[1]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object
|
|
@@ -0,0 +1,241 @@
|
|
|
1
|
+
---
|
|
2
|
+
permalink: /plugins/auth
|
|
3
|
+
editLink: false
|
|
4
|
+
sidebar: auto
|
|
5
|
+
title: auth
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
|
|
9
|
+
|
|
10
|
+
## auth
|
|
11
|
+
|
|
12
|
+
Logs user in for the first test and reuses session for next tests.
|
|
13
|
+
Works by saving cookies into memory or file.
|
|
14
|
+
If a session expires automatically logs in again.
|
|
15
|
+
|
|
16
|
+
> For better development experience cookies can be saved into file, so a session can be reused while writing tests.
|
|
17
|
+
|
|
18
|
+
#### Usage
|
|
19
|
+
|
|
20
|
+
1. Enable this plugin and configure as described below
|
|
21
|
+
2. Define user session names (example: `user`, `editor`, `admin`, etc).
|
|
22
|
+
3. Define how users are logged in and how to check that user is logged in
|
|
23
|
+
4. Use `login` object inside your tests to log in:
|
|
24
|
+
|
|
25
|
+
```js
|
|
26
|
+
// inside a test file
|
|
27
|
+
// use login to inject auto-login function
|
|
28
|
+
Feature('Login');
|
|
29
|
+
|
|
30
|
+
Before(({ login }) => {
|
|
31
|
+
login('user'); // login using user session
|
|
32
|
+
});
|
|
33
|
+
|
|
34
|
+
// Alternatively log in for one scenario.
|
|
35
|
+
Scenario('log me in', ( { I, login } ) => {
|
|
36
|
+
login('admin');
|
|
37
|
+
I.see('I am logged in');
|
|
38
|
+
});
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
#### Configuration
|
|
42
|
+
|
|
43
|
+
* `saveToFile` (default: false) - save cookies to file. Allows to reuse session between execution.
|
|
44
|
+
* `inject` (default: `login`) - name of the login function to use
|
|
45
|
+
* `users` - an array containing different session names and functions to:
|
|
46
|
+
* `login` - sign in into the system
|
|
47
|
+
* `check` - check that user is logged in
|
|
48
|
+
* `fetch` - to get current cookies (by default `I.grabCookie()`)
|
|
49
|
+
* `restore` - to set cookies (by default `I.amOnPage('/'); I.setCookie(cookie)`)
|
|
50
|
+
|
|
51
|
+
#### How It Works
|
|
52
|
+
|
|
53
|
+
1. `restore` method is executed. It should open a page and set credentials.
|
|
54
|
+
2. `check` method is executed. It should reload a page (so cookies are applied) and check that this page belongs to logged-in user. When you pass the second args `session`, you could perform the validation using passed session.
|
|
55
|
+
3. If `restore` and `check` were not successful, `login` is executed
|
|
56
|
+
4. `login` should fill in login form
|
|
57
|
+
5. After successful login, `fetch` is executed to save cookies into memory or file.
|
|
58
|
+
|
|
59
|
+
#### Example: Simple login
|
|
60
|
+
|
|
61
|
+
```js
|
|
62
|
+
auth: {
|
|
63
|
+
enabled: true,
|
|
64
|
+
saveToFile: true,
|
|
65
|
+
inject: 'login',
|
|
66
|
+
users: {
|
|
67
|
+
admin: {
|
|
68
|
+
// loginAdmin function is defined in `steps_file.js`
|
|
69
|
+
login: (I) => I.loginAdmin(),
|
|
70
|
+
// if we see `Admin` on page, we assume we are logged in
|
|
71
|
+
check: (I) => {
|
|
72
|
+
I.amOnPage('/');
|
|
73
|
+
I.see('Admin');
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
}
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
#### Example: Multiple users
|
|
81
|
+
|
|
82
|
+
```js
|
|
83
|
+
auth: {
|
|
84
|
+
enabled: true,
|
|
85
|
+
saveToFile: true,
|
|
86
|
+
inject: 'loginAs', // use `loginAs` instead of login
|
|
87
|
+
users: {
|
|
88
|
+
user: {
|
|
89
|
+
login: (I) => {
|
|
90
|
+
I.amOnPage('/login');
|
|
91
|
+
I.fillField('email', 'user@site.com');
|
|
92
|
+
I.fillField('password', '123456');
|
|
93
|
+
I.click('Login');
|
|
94
|
+
},
|
|
95
|
+
check: (I) => {
|
|
96
|
+
I.amOnPage('/');
|
|
97
|
+
I.see('User', '.navbar');
|
|
98
|
+
},
|
|
99
|
+
},
|
|
100
|
+
admin: {
|
|
101
|
+
login: (I) => {
|
|
102
|
+
I.amOnPage('/login');
|
|
103
|
+
I.fillField('email', 'admin@site.com');
|
|
104
|
+
I.fillField('password', '123456');
|
|
105
|
+
I.click('Login');
|
|
106
|
+
},
|
|
107
|
+
check: (I) => {
|
|
108
|
+
I.amOnPage('/');
|
|
109
|
+
I.see('Admin', '.navbar');
|
|
110
|
+
},
|
|
111
|
+
},
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
#### Example: Keep cookies between tests
|
|
117
|
+
|
|
118
|
+
If you decide to keep cookies between tests you don't need to save/retrieve cookies between tests.
|
|
119
|
+
But you need to login once work until session expires.
|
|
120
|
+
For this case, disable `fetch` and `restore` methods.
|
|
121
|
+
|
|
122
|
+
```js
|
|
123
|
+
helpers: {
|
|
124
|
+
WebDriver: {
|
|
125
|
+
// config goes here
|
|
126
|
+
keepCookies: true; // keep cookies for all tests
|
|
127
|
+
}
|
|
128
|
+
},
|
|
129
|
+
plugins: {
|
|
130
|
+
auth: {
|
|
131
|
+
users: {
|
|
132
|
+
admin: {
|
|
133
|
+
login: (I) => {
|
|
134
|
+
I.amOnPage('/login');
|
|
135
|
+
I.fillField('email', 'admin@site.com');
|
|
136
|
+
I.fillField('password', '123456');
|
|
137
|
+
I.click('Login');
|
|
138
|
+
},
|
|
139
|
+
check: (I) => {
|
|
140
|
+
I.amOnPage('/dashboard');
|
|
141
|
+
I.see('Admin', '.navbar');
|
|
142
|
+
},
|
|
143
|
+
fetch: () => {}, // empty function
|
|
144
|
+
restore: () => {}, // empty funciton
|
|
145
|
+
}
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
#### Example: Getting sessions from local storage
|
|
152
|
+
|
|
153
|
+
If your session is stored in local storage instead of cookies you still can obtain sessions.
|
|
154
|
+
|
|
155
|
+
```js
|
|
156
|
+
plugins: {
|
|
157
|
+
auth: {
|
|
158
|
+
admin: {
|
|
159
|
+
login: (I) => I.loginAsAdmin(),
|
|
160
|
+
check: (I) => I.see('Admin', '.navbar'),
|
|
161
|
+
fetch: (I) => {
|
|
162
|
+
return I.executeScript(() => localStorage.getItem('session_id'));
|
|
163
|
+
},
|
|
164
|
+
restore: (I, session) => {
|
|
165
|
+
I.amOnPage('/');
|
|
166
|
+
I.executeScript((session) => localStorage.setItem('session_id', session), session);
|
|
167
|
+
},
|
|
168
|
+
}
|
|
169
|
+
}
|
|
170
|
+
}
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
#### Tips: Using async function in the auth
|
|
174
|
+
|
|
175
|
+
If you use async functions in the auth plugin, login function should be used with `await` keyword.
|
|
176
|
+
|
|
177
|
+
```js
|
|
178
|
+
auth: {
|
|
179
|
+
enabled: true,
|
|
180
|
+
saveToFile: true,
|
|
181
|
+
inject: 'login',
|
|
182
|
+
users: {
|
|
183
|
+
admin: {
|
|
184
|
+
login: async (I) => { // If you use async function in the auth plugin
|
|
185
|
+
const phrase = await I.grabTextFrom('#phrase')
|
|
186
|
+
I.fillField('username', 'admin'),
|
|
187
|
+
I.fillField('password', 'password')
|
|
188
|
+
I.fillField('phrase', phrase)
|
|
189
|
+
},
|
|
190
|
+
check: (I) => {
|
|
191
|
+
I.amOnPage('/');
|
|
192
|
+
I.see('Admin');
|
|
193
|
+
},
|
|
194
|
+
}
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
```js
|
|
200
|
+
Scenario('login', async ( {I, login} ) => {
|
|
201
|
+
await login('admin') // you should use `await`
|
|
202
|
+
})
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
#### Tips: Using session to validate user
|
|
206
|
+
|
|
207
|
+
Instead of asserting on page elements for the current user in `check`, you can use the `session` you saved in `fetch`
|
|
208
|
+
|
|
209
|
+
```js
|
|
210
|
+
auth: {
|
|
211
|
+
enabled: true,
|
|
212
|
+
saveToFile: true,
|
|
213
|
+
inject: 'login',
|
|
214
|
+
users: {
|
|
215
|
+
admin: {
|
|
216
|
+
login: async (I) => { // If you use async function in the auth plugin
|
|
217
|
+
const phrase = await I.grabTextFrom('#phrase')
|
|
218
|
+
I.fillField('username', 'admin'),
|
|
219
|
+
I.fillField('password', 'password')
|
|
220
|
+
I.fillField('phrase', phrase)
|
|
221
|
+
},
|
|
222
|
+
check: (I, session) => {
|
|
223
|
+
// Throwing an error in `check` will make CodeceptJS perform the login step for the user
|
|
224
|
+
if (session.profile.email !== the.email.you.expect@some-mail.com) {
|
|
225
|
+
throw new Error ('Wrong user signed in');
|
|
226
|
+
}
|
|
227
|
+
},
|
|
228
|
+
}
|
|
229
|
+
}
|
|
230
|
+
}
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
```js
|
|
234
|
+
Scenario('login', async ( {I, login} ) => {
|
|
235
|
+
await login('admin') // you should use `await`
|
|
236
|
+
})
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
### Parameters
|
|
240
|
+
|
|
241
|
+
* `config`  
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
---
|
|
2
|
+
permalink: /plugins/autoDelay
|
|
3
|
+
editLink: false
|
|
4
|
+
sidebar: auto
|
|
5
|
+
title: autoDelay
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
|
|
9
|
+
|
|
10
|
+
## autoDelay
|
|
11
|
+
|
|
12
|
+
Sometimes it takes some time for a page to respond to user's actions.
|
|
13
|
+
Depending on app's performance this can be either slow or fast.
|
|
14
|
+
|
|
15
|
+
For instance, if you click a button and nothing happens - probably JS event is not attached to this button yet
|
|
16
|
+
Also, if you fill field and input validation doesn't accept your input - maybe because you typed value too fast.
|
|
17
|
+
|
|
18
|
+
This plugin allows to slow down tests execution when a test running too fast.
|
|
19
|
+
It puts a tiny delay for before and after action commands.
|
|
20
|
+
|
|
21
|
+
Commands affected (by default):
|
|
22
|
+
|
|
23
|
+
* `click`
|
|
24
|
+
* `fillField`
|
|
25
|
+
* `checkOption`
|
|
26
|
+
* `pressKey`
|
|
27
|
+
* `doubleClick`
|
|
28
|
+
* `rightClick`
|
|
29
|
+
|
|
30
|
+
#### Configuration
|
|
31
|
+
|
|
32
|
+
```js
|
|
33
|
+
plugins: {
|
|
34
|
+
autoDelay: {
|
|
35
|
+
enabled: true
|
|
36
|
+
}
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Possible config options:
|
|
41
|
+
|
|
42
|
+
* `methods`: list of affected commands. Can be overridden
|
|
43
|
+
* `delayBefore`: put a delay before a command. 100ms by default
|
|
44
|
+
* `delayAfter`: put a delay after a command. 200ms by default
|
|
45
|
+
|
|
46
|
+
### Parameters
|
|
47
|
+
|
|
48
|
+
* `config`  
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
---
|
|
2
|
+
permalink: /plugins/browser
|
|
3
|
+
editLink: false
|
|
4
|
+
sidebar: auto
|
|
5
|
+
title: browser
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
|
|
9
|
+
|
|
10
|
+
## browser
|
|
11
|
+
|
|
12
|
+
Overrides browser helper config from the command line. Works for all browser helpers
|
|
13
|
+
(Playwright, Puppeteer, WebDriver, Appium) without touching `codecept.conf`.
|
|
14
|
+
|
|
15
|
+
Enable it via `-p` option with one or more colon-chained args:
|
|
16
|
+
|
|
17
|
+
npx codeceptjs run -p browser:show
|
|
18
|
+
npx codeceptjs run -p browser:hide
|
|
19
|
+
npx codeceptjs run -p browser:browser=firefox
|
|
20
|
+
npx codeceptjs run -p browser:windowSize=1024x768:video=false
|
|
21
|
+
npx codeceptjs run -p browser:hide:browser=webkit:windowSize=800x600
|
|
22
|
+
|
|
23
|
+
#### Args
|
|
24
|
+
|
|
25
|
+
* **show** — force visible browser
|
|
26
|
+
* **hide** — force headless (also injects `--headless` into WebDriver chrome/firefox capability args)
|
|
27
|
+
* **`<key>=<value>`** — set `helpers.<eachBrowserHelper>.<key> = <value>`. Three keys
|
|
28
|
+
get per-helper translation via `setBrowserConfig`:
|
|
29
|
+
* `browser=<name>` — Puppeteer receives `product`, Playwright receives `browser`
|
|
30
|
+
* `windowSize=WxH` — also adds `--window-size=W,H` chromium/chrome args
|
|
31
|
+
* `show=true|false` — toggles `show` on Playwright/Puppeteer; injects/strips
|
|
32
|
+
`--headless` in WebDriver chrome/firefox capability args
|
|
33
|
+
|
|
34
|
+
Values stay as strings. `true` / `false` are coerced to booleans.
|
|
35
|
+
|
|
36
|
+
Requires `@codeceptjs/configure` to be installed; if missing, the plugin
|
|
37
|
+
logs a hint and skips the override.
|
|
38
|
+
|
|
39
|
+
### Parameters
|
|
40
|
+
|
|
41
|
+
* `config`
|