codeceptjs 4.0.0-rc.20 → 4.0.0-rc.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/mcp-server.js +7 -5
- package/docs/advanced.md +1 -1
- package/docs/agents.md +32 -10
- package/docs/architecture.md +219 -0
- package/docs/configuration.md +82 -127
- package/docs/continuous-integration.md +113 -151
- package/docs/custom-helpers.md +1 -1
- package/docs/environment-variables.md +131 -0
- package/docs/hooks.md +76 -277
- package/docs/installation.md +95 -40
- package/docs/parallel.md +98 -496
- package/docs/plugins.md +43 -0
- package/docs/reports.md +102 -401
- package/docs/retry.md +44 -37
- package/docs/typescript.md +54 -269
- package/lib/codecept.js +1 -1
- package/lib/command/run-workers.js +0 -14
- package/lib/command/run.js +2 -16
- package/lib/command/utils.js +14 -0
- package/lib/command/workers/runTests.js +1 -5
- package/lib/heal.js +2 -0
- package/lib/helper/Playwright.js +15 -4
- package/lib/plugin/aiTrace.js +16 -13
- package/lib/plugin/analyze.js +5 -4
- package/lib/plugin/heal.js +3 -2
- package/lib/plugin/junitReporter.js +303 -0
- package/lib/plugin/pageInfo.js +5 -7
- package/lib/plugin/retryFailedStep.js +4 -3
- package/lib/plugin/screencast.js +6 -4
- package/lib/workers.js +11 -3
- package/package.json +1 -1
- package/docs/internal-api.md +0 -265
|
@@ -5,119 +5,63 @@ title: Continuous Integration
|
|
|
5
5
|
|
|
6
6
|
# Continuous Integration
|
|
7
7
|
|
|
8
|
-
CodeceptJS runs in any CI
|
|
8
|
+
CodeceptJS runs in any CI that can install Node.js. This page covers the setup, then provides ready-to-use configs for the major CI systems.
|
|
9
9
|
|
|
10
|
-
##
|
|
10
|
+
## Setup
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
- **Node.js** — install it on the runner (`actions/setup-node`, the `node:20` image, `NodeTool@0` on Azure). Examples below use Node 20.
|
|
13
|
+
- **Headless** — `codecept.conf.js` must contain `setHeadlessWhen(process.env.HEADLESS || process.env.CI)`. `codeceptjs init` adds it; since CI sets `CI=true`, the suite runs headless automatically.
|
|
13
14
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
- **Failure artifacts.** Keep `screenshotOnFail` enabled (it is on by default). For Playwright, also enable `trace` and `video` in the helper config — they make a remote failure diagnosable from a single artifact.
|
|
17
|
-
- **Self-healing for flaky tests.** Use the [`heal` plugin](/heal) to recover from broken locators. The `retryFailedStep` plugin is already enabled by default — you do not need to configure it.
|
|
15
|
+
```js
|
|
16
|
+
import { setHeadlessWhen } from '@codeceptjs/configure'
|
|
18
17
|
|
|
19
|
-
|
|
18
|
+
setHeadlessWhen(process.env.HEADLESS || process.env.CI)
|
|
19
|
+
```
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
Override the browser or viewport per run with the [`browser` plugin](/plugins#browser): `-p browser:browser=firefox:windowSize=1280x1024`.
|
|
22
|
+
- **Artifacts** — enable the on-failure capture plugins:
|
|
22
23
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
npm ci
|
|
29
|
-
npx playwright install --with-deps chromium
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
`--with-deps` pulls in `libnss`, fonts, and other OS packages. To install all engines, drop the `chromium` argument. Playwright explicitly recommends against caching browser binaries — restoring the cache takes about as long as a fresh download.
|
|
33
|
-
|
|
34
|
-
If you prefer the official Playwright Docker image, see the [Playwright Docker docs](https://playwright.dev/docs/docker). Pin the image tag to **the same version as your installed `playwright` package** — a mismatched image will fail to find browser executables. The examples below use `node:20` + `npx playwright install --with-deps` to avoid this version-pin problem entirely.
|
|
35
|
-
|
|
36
|
-
### WebDriver
|
|
37
|
-
|
|
38
|
-
CodeceptJS's WebDriver helper talks to any WebDriver-protocol endpoint. In CI, the simplest setup is a [Selenium Docker container](https://github.com/SeleniumHQ/docker-selenium):
|
|
39
|
-
|
|
40
|
-
```bash
|
|
41
|
-
docker run -d --net=host --shm-size=2g selenium/standalone-chrome
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
Point the helper at it:
|
|
45
|
-
|
|
46
|
-
```js
|
|
47
|
-
helpers: {
|
|
48
|
-
WebDriver: {
|
|
49
|
-
url: 'http://localhost:8000',
|
|
50
|
-
browser: 'chrome',
|
|
51
|
-
host: process.env.SELENIUM_HOST || 'localhost',
|
|
52
|
-
port: parseInt(process.env.SELENIUM_PORT || '4444', 10),
|
|
24
|
+
```js
|
|
25
|
+
plugins: {
|
|
26
|
+
screenshot: { enabled: true }, // screenshot on failure
|
|
27
|
+
pageInfo: { enabled: true }, // page URL, HTML errors, console logs
|
|
28
|
+
aiTrace: { enabled: true, on: 'fail' }, // AI-friendly trace.md
|
|
53
29
|
}
|
|
54
|
-
|
|
55
|
-
```
|
|
30
|
+
```
|
|
56
31
|
|
|
57
|
-
|
|
32
|
+
[`screencast`](/plugins#screencast) records video of failed tests (Playwright). Everything lands in `output/` — upload that directory as a build artifact. Every example below does.
|
|
58
33
|
|
|
59
|
-
|
|
34
|
+
## Browsers and drivers
|
|
60
35
|
|
|
61
|
-
|
|
36
|
+
- **Playwright** — `npx playwright install --with-deps`. Docs: [Playwright CI](https://playwright.dev/docs/ci), [Playwright Docker image](https://playwright.dev/docs/docker) (pin the tag to your installed `playwright` version).
|
|
37
|
+
- **WebDriver** — run a Selenium server: `selenium/standalone-chrome` on port `4444`. Docs: [WebDriver helper](/webdriver), [WebdriverIO Selenium Grid](https://webdriver.io/docs/seleniumgrid), [Selenium Docker images](https://github.com/SeleniumHQ/docker-selenium).
|
|
62
38
|
|
|
63
|
-
|
|
39
|
+
## Check before running
|
|
64
40
|
|
|
65
|
-
|
|
66
|
-
npx codeceptjs run
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
Parallel workers on one machine:
|
|
41
|
+
`npx codeceptjs check` loads the config, opens the browser (or connects to Selenium), and counts tests. Prepend it so a broken environment fails fast with a clear message:
|
|
70
42
|
|
|
71
43
|
```bash
|
|
72
|
-
npx codeceptjs
|
|
44
|
+
npx codeceptjs check
|
|
45
|
+
npx codeceptjs run
|
|
73
46
|
```
|
|
74
47
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
Sharded across multiple machines (CI matrix):
|
|
48
|
+
Every CI example below does this.
|
|
78
49
|
|
|
79
|
-
|
|
80
|
-
npx codeceptjs run --shard 1/4
|
|
81
|
-
npx codeceptjs run --shard 2/4
|
|
82
|
-
npx codeceptjs run --shard 3/4
|
|
83
|
-
npx codeceptjs run --shard 4/4
|
|
84
|
-
```
|
|
50
|
+
## Parallel execution
|
|
85
51
|
|
|
86
|
-
|
|
52
|
+
- `npx codeceptjs run` — one process.
|
|
53
|
+
- `npx codeceptjs run-workers 4` — parallel on one machine, tests handed to workers dynamically.
|
|
54
|
+
- `npx codeceptjs run --shard 1/4` — split the suite across CI matrix jobs (one shard per machine).
|
|
87
55
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
```bash
|
|
91
|
-
npx codeceptjs run --grep "@smoke"
|
|
92
|
-
npx codeceptjs run --grep "@slow" --invert
|
|
93
|
-
```
|
|
56
|
+
Shards and workers combine. Full reference: [Parallel Execution](/parallel).
|
|
94
57
|
|
|
95
58
|
## Reporting
|
|
96
59
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
| CI | Recommended pipes | Result |
|
|
100
|
-
|---|---|---|
|
|
101
|
-
| GitHub Actions | `github` + `html` | PR check annotations + a self-contained HTML report |
|
|
102
|
-
| GitLab CI | `gitlab` | Merge request widget with test results |
|
|
103
|
-
| Bitbucket Pipelines | `bitbucket` | Pipeline test report |
|
|
104
|
-
| Any | `html` | HTML report you can upload as an artifact |
|
|
105
|
-
|
|
106
|
-
Install:
|
|
107
|
-
|
|
108
|
-
```bash
|
|
109
|
-
npm i --save-dev @testomatio/reporter
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
See the [reporter README](https://github.com/testomatio/reporter) for the per-pipe environment variables.
|
|
113
|
-
|
|
114
|
-
Whatever reporter you use, also upload the `output/` directory as a build artifact. It contains failure screenshots and, with Playwright, traces and videos.
|
|
60
|
+
Use [`@testomatio/reporter`](https://github.com/testomatio/reporter). It ships pipes that publish results into GitHub PR checks, GitLab merge-request widgets, and Bitbucket pipeline reports. [See Reporting](/reports).
|
|
115
61
|
|
|
116
|
-
|
|
62
|
+
## CI examples
|
|
117
63
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
The examples below use Playwright by default. A WebDriver-with-Selenium variant follows where it differs.
|
|
64
|
+
Each example uses Playwright by default; a WebDriver-with-Selenium variant follows where it differs. A `node:20` base image plus `npx playwright install --with-deps` keeps these configs free of version pins.
|
|
121
65
|
|
|
122
66
|
### GitHub Actions — Playwright
|
|
123
67
|
|
|
@@ -145,6 +89,7 @@ jobs:
|
|
|
145
89
|
cache: npm
|
|
146
90
|
- run: npm ci
|
|
147
91
|
- run: npx playwright install --with-deps chromium
|
|
92
|
+
- run: npx codeceptjs check
|
|
148
93
|
- run: npx codeceptjs run-workers 4 --by pool
|
|
149
94
|
- uses: actions/upload-artifact@v4
|
|
150
95
|
if: failure()
|
|
@@ -178,6 +123,7 @@ jobs:
|
|
|
178
123
|
with:
|
|
179
124
|
node-version: 20
|
|
180
125
|
- run: npm ci
|
|
126
|
+
- run: npx codeceptjs check
|
|
181
127
|
- run: npx codeceptjs run-workers 2 --by pool
|
|
182
128
|
- uses: actions/upload-artifact@v4
|
|
183
129
|
if: failure()
|
|
@@ -186,7 +132,7 @@ jobs:
|
|
|
186
132
|
path: output/
|
|
187
133
|
```
|
|
188
134
|
|
|
189
|
-
### GitHub Actions —
|
|
135
|
+
### GitHub Actions — sharded matrix
|
|
190
136
|
|
|
191
137
|
Each shard runs on its own runner in parallel:
|
|
192
138
|
|
|
@@ -205,11 +151,43 @@ jobs:
|
|
|
205
151
|
node-version: 20
|
|
206
152
|
- run: npm ci
|
|
207
153
|
- run: npx playwright install --with-deps chromium
|
|
154
|
+
- run: npx codeceptjs check
|
|
208
155
|
- run: npx codeceptjs run --shard ${{ matrix.shard }}
|
|
209
156
|
- uses: actions/upload-artifact@v4
|
|
210
157
|
if: failure()
|
|
211
158
|
with:
|
|
212
|
-
name: output
|
|
159
|
+
name: output-${{ strategy.job-index }}
|
|
160
|
+
path: output/
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
### GitHub Actions — browser matrix
|
|
164
|
+
|
|
165
|
+
Run the same suite across browsers and viewports via the [`browser` plugin](/plugins#browser) (`npm i --save-dev @codeceptjs/configure`):
|
|
166
|
+
|
|
167
|
+
```yaml
|
|
168
|
+
jobs:
|
|
169
|
+
test:
|
|
170
|
+
runs-on: ubuntu-latest
|
|
171
|
+
strategy:
|
|
172
|
+
fail-fast: false
|
|
173
|
+
matrix:
|
|
174
|
+
include:
|
|
175
|
+
- { browser: chromium, size: 1920x1080 }
|
|
176
|
+
- { browser: firefox, size: 1366x768 }
|
|
177
|
+
- { browser: webkit, size: 414x896 }
|
|
178
|
+
steps:
|
|
179
|
+
- uses: actions/checkout@v4
|
|
180
|
+
- uses: actions/setup-node@v4
|
|
181
|
+
with:
|
|
182
|
+
node-version: 20
|
|
183
|
+
- run: npm ci
|
|
184
|
+
- run: npx playwright install --with-deps
|
|
185
|
+
- run: npx codeceptjs check
|
|
186
|
+
- run: npx codeceptjs run -p browser:browser=${{ matrix.browser }}:windowSize=${{ matrix.size }}
|
|
187
|
+
- uses: actions/upload-artifact@v4
|
|
188
|
+
if: failure()
|
|
189
|
+
with:
|
|
190
|
+
name: output-${{ matrix.browser }}-${{ matrix.size }}
|
|
213
191
|
path: output/
|
|
214
192
|
```
|
|
215
193
|
|
|
@@ -230,11 +208,11 @@ playwright:
|
|
|
230
208
|
- npm ci
|
|
231
209
|
- npx playwright install --with-deps chromium
|
|
232
210
|
script:
|
|
211
|
+
- npx codeceptjs check
|
|
233
212
|
- npx codeceptjs run --shard $CI_NODE_INDEX/$CI_NODE_TOTAL
|
|
234
213
|
artifacts:
|
|
235
214
|
when: on_failure
|
|
236
|
-
paths:
|
|
237
|
-
- output/
|
|
215
|
+
paths: [output/]
|
|
238
216
|
expire_in: 1 week
|
|
239
217
|
|
|
240
218
|
webdriver:
|
|
@@ -248,13 +226,14 @@ webdriver:
|
|
|
248
226
|
SELENIUM_PORT: "4444"
|
|
249
227
|
script:
|
|
250
228
|
- npm ci
|
|
229
|
+
- npx codeceptjs check
|
|
251
230
|
- npx codeceptjs run-workers 2 --by pool
|
|
252
231
|
artifacts:
|
|
253
232
|
when: on_failure
|
|
254
233
|
paths: [output/]
|
|
255
234
|
```
|
|
256
235
|
|
|
257
|
-
`$CI_NODE_INDEX` is 1-based
|
|
236
|
+
`$CI_NODE_INDEX` is 1-based — it maps directly to CodeceptJS's `--shard` index.
|
|
258
237
|
|
|
259
238
|
### Bitbucket Pipelines
|
|
260
239
|
|
|
@@ -264,6 +243,8 @@ webdriver:
|
|
|
264
243
|
image: node:20
|
|
265
244
|
|
|
266
245
|
definitions:
|
|
246
|
+
caches:
|
|
247
|
+
playwright: ~/.cache/ms-playwright
|
|
267
248
|
services:
|
|
268
249
|
selenium:
|
|
269
250
|
image: selenium/standalone-chrome
|
|
@@ -271,37 +252,25 @@ definitions:
|
|
|
271
252
|
|
|
272
253
|
pipelines:
|
|
273
254
|
default:
|
|
274
|
-
- step:
|
|
275
|
-
name: Install
|
|
276
|
-
caches: [node]
|
|
277
|
-
script:
|
|
278
|
-
- npm ci
|
|
279
|
-
- npx playwright install --with-deps chromium
|
|
280
255
|
- parallel:
|
|
281
256
|
- step:
|
|
282
|
-
name:
|
|
283
|
-
|
|
284
|
-
- npx codeceptjs run --shard 1/4
|
|
285
|
-
artifacts:
|
|
286
|
-
- output/**
|
|
287
|
-
- step:
|
|
288
|
-
name: Shard 2/4
|
|
289
|
-
script:
|
|
290
|
-
- npx codeceptjs run --shard 2/4
|
|
291
|
-
artifacts:
|
|
292
|
-
- output/**
|
|
293
|
-
- step:
|
|
294
|
-
name: Shard 3/4
|
|
257
|
+
name: Playwright — shard 1/2
|
|
258
|
+
caches: [node, playwright]
|
|
295
259
|
script:
|
|
296
|
-
-
|
|
297
|
-
|
|
298
|
-
-
|
|
260
|
+
- npm ci
|
|
261
|
+
- npx playwright install --with-deps chromium
|
|
262
|
+
- npx codeceptjs check
|
|
263
|
+
- npx codeceptjs run --shard 1/2
|
|
264
|
+
artifacts: [output/**]
|
|
299
265
|
- step:
|
|
300
|
-
name:
|
|
266
|
+
name: Playwright — shard 2/2
|
|
267
|
+
caches: [node, playwright]
|
|
301
268
|
script:
|
|
302
|
-
-
|
|
303
|
-
|
|
304
|
-
-
|
|
269
|
+
- npm ci
|
|
270
|
+
- npx playwright install --with-deps chromium
|
|
271
|
+
- npx codeceptjs check
|
|
272
|
+
- npx codeceptjs run --shard 2/2
|
|
273
|
+
artifacts: [output/**]
|
|
305
274
|
```
|
|
306
275
|
|
|
307
276
|
For WebDriver, attach the Selenium service to the step:
|
|
@@ -310,14 +279,13 @@ For WebDriver, attach the Selenium service to the step:
|
|
|
310
279
|
pipelines:
|
|
311
280
|
default:
|
|
312
281
|
- step:
|
|
313
|
-
image: node:20
|
|
314
282
|
services: [selenium]
|
|
315
283
|
script:
|
|
316
284
|
- npm ci
|
|
317
285
|
- export SELENIUM_HOST=localhost SELENIUM_PORT=4444
|
|
286
|
+
- npx codeceptjs check
|
|
318
287
|
- npx codeceptjs run-workers 2 --by pool
|
|
319
|
-
artifacts:
|
|
320
|
-
- output/**
|
|
288
|
+
artifacts: [output/**]
|
|
321
289
|
```
|
|
322
290
|
|
|
323
291
|
### Jenkins
|
|
@@ -344,10 +312,10 @@ pipeline {
|
|
|
344
312
|
}
|
|
345
313
|
stage('Test') {
|
|
346
314
|
parallel {
|
|
347
|
-
stage('Shard 1/4') { steps { sh 'npx codeceptjs run --shard 1/4' } }
|
|
348
|
-
stage('Shard 2/4') { steps { sh 'npx codeceptjs run --shard 2/4' } }
|
|
349
|
-
stage('Shard 3/4') { steps { sh 'npx codeceptjs run --shard 3/4' } }
|
|
350
|
-
stage('Shard 4/4') { steps { sh 'npx codeceptjs run --shard 4/4' } }
|
|
315
|
+
stage('Shard 1/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 1/4' } }
|
|
316
|
+
stage('Shard 2/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 2/4' } }
|
|
317
|
+
stage('Shard 3/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 3/4' } }
|
|
318
|
+
stage('Shard 4/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 4/4' } }
|
|
351
319
|
}
|
|
352
320
|
}
|
|
353
321
|
}
|
|
@@ -359,18 +327,17 @@ pipeline {
|
|
|
359
327
|
}
|
|
360
328
|
```
|
|
361
329
|
|
|
362
|
-
For WebDriver,
|
|
330
|
+
For WebDriver, run Selenium alongside the test container:
|
|
363
331
|
|
|
364
332
|
```groovy
|
|
365
333
|
stage('Test') {
|
|
366
334
|
steps {
|
|
367
335
|
script {
|
|
368
|
-
docker.image('selenium/standalone-chrome')
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
'''
|
|
336
|
+
docker.image('selenium/standalone-chrome').withRun('--shm-size=2g -p 4444:4444') { c ->
|
|
337
|
+
withEnv(['SELENIUM_HOST=localhost', 'SELENIUM_PORT=4444']) {
|
|
338
|
+
sh 'npx codeceptjs check'
|
|
339
|
+
sh 'npx codeceptjs run-workers 2 --by pool'
|
|
340
|
+
}
|
|
374
341
|
}
|
|
375
342
|
}
|
|
376
343
|
}
|
|
@@ -393,6 +360,7 @@ jobs:
|
|
|
393
360
|
- checkout
|
|
394
361
|
- run: npm ci
|
|
395
362
|
- run: npx playwright install --with-deps chromium
|
|
363
|
+
- run: npx codeceptjs check
|
|
396
364
|
- run:
|
|
397
365
|
name: Run shard
|
|
398
366
|
command: |
|
|
@@ -411,6 +379,7 @@ jobs:
|
|
|
411
379
|
steps:
|
|
412
380
|
- checkout
|
|
413
381
|
- run: npm ci
|
|
382
|
+
- run: npx codeceptjs check
|
|
414
383
|
- run: npx codeceptjs run-workers 2 --by pool
|
|
415
384
|
- store_artifacts:
|
|
416
385
|
path: output
|
|
@@ -444,9 +413,10 @@ steps:
|
|
|
444
413
|
- script: npm ci
|
|
445
414
|
displayName: Install dependencies
|
|
446
415
|
- script: npx playwright install --with-deps chromium
|
|
447
|
-
displayName: Install
|
|
448
|
-
- script:
|
|
449
|
-
|
|
416
|
+
displayName: Install browsers
|
|
417
|
+
- script: npx codeceptjs check
|
|
418
|
+
displayName: Check setup
|
|
419
|
+
- script: npx codeceptjs run --shard $(System.JobPositionInPhase)/$(System.TotalJobsInPhase)
|
|
450
420
|
displayName: Run shard $(System.JobPositionInPhase)/$(System.TotalJobsInPhase)
|
|
451
421
|
env:
|
|
452
422
|
FORCE_COLOR: 1
|
|
@@ -457,35 +427,27 @@ steps:
|
|
|
457
427
|
artifactName: codeceptjs-output-$(System.JobPositionInPhase)
|
|
458
428
|
```
|
|
459
429
|
|
|
460
|
-
For WebDriver, run Selenium as a sidecar before tests:
|
|
430
|
+
For WebDriver, run Selenium as a sidecar before the tests:
|
|
461
431
|
|
|
462
432
|
```yaml
|
|
463
433
|
- script: docker run -d --net=host --shm-size=2g selenium/standalone-chrome
|
|
464
434
|
displayName: Start Selenium
|
|
465
435
|
- script: |
|
|
466
436
|
export SELENIUM_HOST=localhost SELENIUM_PORT=4444
|
|
437
|
+
npx codeceptjs check
|
|
467
438
|
npx codeceptjs run-workers 2 --by pool
|
|
468
439
|
displayName: Run tests
|
|
469
440
|
```
|
|
470
441
|
|
|
471
442
|
## Docker
|
|
472
443
|
|
|
473
|
-
The official `codeceptjs/codeceptjs` image runs Playwright, Puppeteer, and WebDriver suites
|
|
474
|
-
|
|
475
|
-
## Tips
|
|
476
|
-
|
|
477
|
-
- **Raise per-test timeouts in CI.** CI machines are slower than your laptop. Bump `timeout` in `codecept.conf.js` when assertions race the page.
|
|
478
|
-
- **Diagnose from logs.** Re-run with `--debug` or `DEBUG=codeceptjs:*` when a job fails and you cannot reproduce locally.
|
|
479
|
-
- **Selenium Chrome: always `--shm-size=2g`.** The default 64 MB causes tab crashes on heavy pages.
|
|
480
|
-
- **Custom Playwright images: install OS deps.** When you cannot use `mcr.microsoft.com/playwright`, run `npx playwright install --with-deps` to pull in `libnss`, fonts, and other system libraries.
|
|
481
|
-
- **Upload `output/` only on failure.** Successful runs produce no useful artifacts.
|
|
444
|
+
The official `codeceptjs/codeceptjs` image runs Playwright, Puppeteer, and WebDriver suites with no extra setup. Pass runner flags through `CODECEPT_ARGS` and the worker count through `NO_OF_WORKERS`. See [Docker](/docker).
|
|
482
445
|
|
|
483
446
|
## See also
|
|
484
447
|
|
|
485
|
-
- [Playwright CI guide](https://playwright.dev/docs/ci)
|
|
486
|
-
- [
|
|
487
|
-
- [
|
|
488
|
-
- [Selenium Docker images](https://github.com/SeleniumHQ/docker-selenium) — image variants (`standalone-chrome`, `standalone-firefox`, debug images with VNC).
|
|
448
|
+
- [Playwright CI guide](https://playwright.dev/docs/ci) · [Playwright Docker image](https://playwright.dev/docs/docker)
|
|
449
|
+
- [WebdriverIO Selenium Grid](https://webdriver.io/docs/seleniumgrid) · [Selenium Docker images](https://github.com/SeleniumHQ/docker-selenium)
|
|
450
|
+
- [Parallel Execution](/parallel) · [Reports](/reports) · [Plugins](/plugins) · [Docker](/docker)
|
|
489
451
|
|
|
490
452
|
## Community recipes
|
|
491
453
|
|
package/docs/custom-helpers.md
CHANGED
|
@@ -197,7 +197,7 @@ Each implemented method should return a value as they will be added to global pr
|
|
|
197
197
|
It is possible to execute global conditional retries to handle unforseen errors.
|
|
198
198
|
Lost connections and network issues are good candidates to be retried whenever they appear.
|
|
199
199
|
|
|
200
|
-
This can be done inside a helper using the global [promise recorder](/
|
|
200
|
+
This can be done inside a helper using the global [promise recorder](/architecture#the-recorder):
|
|
201
201
|
|
|
202
202
|
Example: Retrying rendering errors in Puppeteer.
|
|
203
203
|
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
---
|
|
2
|
+
permalink: /environment-variables
|
|
3
|
+
title: Environment Variables
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Environment Variables
|
|
7
|
+
|
|
8
|
+
CodeceptJS reads several environment variables that change runtime behavior. They are useful for tuning CI runs, enabling extra logging, configuring the MCP server, or wiring values into the generated `codecept.conf.js`.
|
|
9
|
+
|
|
10
|
+
This page is a reference for every `process.env.*` switch CodeceptJS reads or sets internally.
|
|
11
|
+
|
|
12
|
+
## Quick Reference
|
|
13
|
+
|
|
14
|
+
| Variable | Category | Default | Purpose |
|
|
15
|
+
| :--- | :--- | :--- | :--- |
|
|
16
|
+
| `profile` | Test execution | _unset_ | Profile name forwarded from `--profile`; readable from config and helpers. |
|
|
17
|
+
| `CI` | Test execution | _unset_ | Auto-set by every CI provider; enables CI-aware behavior. |
|
|
18
|
+
| `DONT_FAIL_ON_EMPTY_RUN` | Test execution | _unset_ | When set, do not fail CI if zero tests were executed. |
|
|
19
|
+
| `RUNS_WITH_WORKERS` | Workers | _unset_ | Set to `true` while running in workers; read by helpers/plugins to branch behavior. |
|
|
20
|
+
| `CODECEPT_WORKER_TIMEOUT` | Workers | `5m` | Hung-worker watchdog timeout. Accepts [ms](https://github.com/vercel/ms) strings (`30s`, `2m`, `1h`). |
|
|
21
|
+
| `CODECEPT_AUTO_EXIT_TIMEOUT` | Workers | `2000` | Time in ms allowed for helper cleanup before forced `process.exit`. Set to `0` to disable. |
|
|
22
|
+
| `DEBUG` | Debug | _unset_ | `debug` package namespace filter, e.g. `codeceptjs:*`, `codeceptjs:heal`. |
|
|
23
|
+
| `HEADLESS` | Init template | _unset_ | Read by `setHeadlessWhen()` in the generated config to toggle headless mode. |
|
|
24
|
+
| `BASE_URL` | Init template | `http://localhost` | Default URL written into the generated Playwright config. |
|
|
25
|
+
|
|
26
|
+
## Test Execution
|
|
27
|
+
|
|
28
|
+
### `profile`
|
|
29
|
+
|
|
30
|
+
Set automatically when you pass `--profile <name>` to `run`, `run-workers`, `run-rerun`, `run-multiple`, or `interactive`. The value is available inside `codecept.conf.js`, helpers, and plugins via `process.env.profile`, which is the standard way to switch configuration based on environment:
|
|
31
|
+
|
|
32
|
+
```js
|
|
33
|
+
exports.config = {
|
|
34
|
+
helpers: {
|
|
35
|
+
Playwright: {
|
|
36
|
+
url: process.env.profile === 'staging' ? 'https://staging.example.com' : 'http://localhost:3000',
|
|
37
|
+
},
|
|
38
|
+
},
|
|
39
|
+
}
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
### `CI`
|
|
43
|
+
|
|
44
|
+
A de-facto standard variable exported by every major CI provider (GitHub Actions, GitLab CI, CircleCI, Jenkins, Travis, etc.). CodeceptJS uses it for two CI-aware behaviors:
|
|
45
|
+
|
|
46
|
+
- Extends the polling timeout for `submittedData()` to 60s (vs 2s locally).
|
|
47
|
+
- Fails the run with exit code `1` if no tests were executed (see `DONT_FAIL_ON_EMPTY_RUN`).
|
|
48
|
+
|
|
49
|
+
You should not need to set this manually.
|
|
50
|
+
|
|
51
|
+
### `DONT_FAIL_ON_EMPTY_RUN`
|
|
52
|
+
|
|
53
|
+
By default, when `CI` is set and no tests are executed (e.g. an over-restrictive `--grep`), CodeceptJS fails the run to surface false-positive green builds. Set `DONT_FAIL_ON_EMPTY_RUN=true` to keep the run green even when nothing ran.
|
|
54
|
+
|
|
55
|
+
```yaml
|
|
56
|
+
# .github/workflows/tests.yml
|
|
57
|
+
env:
|
|
58
|
+
DONT_FAIL_ON_EMPTY_RUN: 'true'
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Workers
|
|
62
|
+
|
|
63
|
+
### `RUNS_WITH_WORKERS`
|
|
64
|
+
|
|
65
|
+
Automatically set to `'true'` while a `run-workers` invocation is active and reset to `'false'` when it finishes. Read it from helpers or plugins when you need to behave differently in worker mode — for example, to disable an interactive feature or change reporter output:
|
|
66
|
+
|
|
67
|
+
```js
|
|
68
|
+
if (process.env.RUNS_WITH_WORKERS === 'true') {
|
|
69
|
+
// running in a worker thread
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Do not set this manually.
|
|
74
|
+
|
|
75
|
+
### `CODECEPT_WORKER_TIMEOUT`
|
|
76
|
+
|
|
77
|
+
Per-worker hung-process watchdog. If a worker produces no output for this duration, it is auto-terminated and reported as failed. Accepts any string parsed by the [`ms`](https://github.com/vercel/ms) package: `30s`, `2m`, `5m`, `1h`. Defaults to `5m`.
|
|
78
|
+
|
|
79
|
+
```sh
|
|
80
|
+
CODECEPT_WORKER_TIMEOUT=10m npx codeceptjs run-workers 4
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### `CODECEPT_AUTO_EXIT_TIMEOUT`
|
|
84
|
+
|
|
85
|
+
Time in **milliseconds** that CodeceptJS waits for helper `_cleanup()` hooks to complete before calling `process.exit()`. Defaults to `2000`. Set to `0` to disable the auto-exit and let the Node event loop drain naturally — useful when long-running async work (e.g. report uploads) must complete after the test run.
|
|
86
|
+
|
|
87
|
+
```sh
|
|
88
|
+
CODECEPT_AUTO_EXIT_TIMEOUT=0 npx codeceptjs run
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## Debug
|
|
92
|
+
|
|
93
|
+
### `DEBUG`
|
|
94
|
+
|
|
95
|
+
Standard [`debug`](https://www.npmjs.com/package/debug) package selector. CodeceptJS emits debug output under the `codeceptjs:*` namespace. Useful namespaces:
|
|
96
|
+
|
|
97
|
+
| Namespace | Source |
|
|
98
|
+
| :--- | :--- |
|
|
99
|
+
| `codeceptjs:*` | All internal logs |
|
|
100
|
+
| `codeceptjs:recorder` | Recorder/promise queue |
|
|
101
|
+
| `codeceptjs:event` | Event dispatcher |
|
|
102
|
+
| `codeceptjs:container` | DI container |
|
|
103
|
+
| `codeceptjs:steps` | Step lifecycle |
|
|
104
|
+
| `codeceptjs:exit` | Exit listener |
|
|
105
|
+
| `codeceptjs:timeout` | Global timeout listener |
|
|
106
|
+
| `codeceptjs:pause` | Interactive pause |
|
|
107
|
+
| `codeceptjs:ai` | AI features |
|
|
108
|
+
| `codeceptjs:heal` | Heal plugin |
|
|
109
|
+
| `codeceptjs:analyze` | Analyze plugin |
|
|
110
|
+
| `codeceptjs:retryFailedStep` | retryFailedStep plugin |
|
|
111
|
+
| `codeceptjs:bdd` | Gherkin/BDD |
|
|
112
|
+
|
|
113
|
+
```sh
|
|
114
|
+
DEBUG="codeceptjs:*" npx codeceptjs run --verbose
|
|
115
|
+
DEBUG="codeceptjs:heal" npx codeceptjs run
|
|
116
|
+
DEBUG="codeceptjs:retryFailedStep" npx codeceptjs run
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
When `DEBUG` includes a `codeceptjs:` selector, the `heal` plugin keeps healing enabled in `--debug` mode (which otherwise disables it).
|
|
120
|
+
|
|
121
|
+
## Init Template
|
|
122
|
+
|
|
123
|
+
These variables are read by the configuration scaffolded by `codeceptjs init`. They have no effect unless your project's generated `codecept.conf.js` references them.
|
|
124
|
+
|
|
125
|
+
### `HEADLESS`
|
|
126
|
+
|
|
127
|
+
Read by [`setHeadlessWhen()`](https://www.npmjs.com/package/@codeceptjs/configure) in the generated config. When truthy, the browser starts headless. The init template wires this in so you can run `HEADLESS=true npx codeceptjs run` on CI without changing the config.
|
|
128
|
+
|
|
129
|
+
### `BASE_URL`
|
|
130
|
+
|
|
131
|
+
The init template uses `BASE_URL` (falling back to `http://localhost`) as the Playwright `url` value. After scaffolding, edit the generated config to change this default.
|