codeceptjs 4.2.0-beta.4 → 4.2.1-beta.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -0
- package/bin/codecept.js +11 -0
- package/docs/agents.md +26 -1
- package/docs/ai.md +57 -0
- package/docs/alternative-browsers.md +175 -148
- package/docs/assertions.md +74 -0
- package/docs/bootstrap.md +2 -0
- package/docs/commands.md +45 -0
- package/docs/configuration.md +37 -0
- package/docs/debugging.md +1 -1
- package/docs/helpers/ApiDataFactory.md +20 -20
- package/docs/helpers/Appium.md +1615 -670
- package/docs/helpers/Decision.md +146 -0
- package/docs/helpers/GraphQL.md +31 -31
- package/docs/helpers/GraphQLDataFactory.md +20 -20
- package/docs/helpers/Kitesurf.md +1808 -25
- package/docs/helpers/Lightpanda.md +139 -0
- package/docs/helpers/Obscura.md +1799 -74
- package/docs/helpers/Playwright.md +490 -428
- package/docs/helpers/Puppeteer.md +317 -255
- package/docs/helpers/REST.md +43 -43
- package/docs/helpers/WebDriver.md +237 -175
- package/docs/installation.md +16 -0
- package/docs/mcp.md +7 -1
- package/docs/parallel.md +35 -1
- package/docs/plugins/pause.md +2 -1
- package/docs/plugins/sitemap.md +48 -0
- package/docs/plugins.md +4 -0
- package/lib/ai.js +81 -2
- package/lib/codecept.js +32 -2
- package/lib/command/init.js +22 -0
- package/lib/command/initAgent.js +78 -0
- package/lib/command/run-multiple.js +3 -0
- package/lib/command/run-workers.js +10 -3
- package/lib/command/run.js +3 -2
- package/lib/command/utils.js +15 -0
- package/lib/command/workers/runTests.js +10 -10
- package/lib/container.js +4 -2
- package/lib/effects.js +19 -1
- package/lib/helper/Appium.js +23 -0
- package/lib/helper/CDPBrowser.js +387 -1338
- package/lib/helper/Decision.js +256 -0
- package/lib/helper/Kitesurf.js +3 -21
- package/lib/helper/Lightpanda.js +137 -0
- package/lib/helper/Obscura.js +21 -238
- package/lib/helper/Playwright.js +50 -4
- package/lib/helper/Puppeteer.js +62 -8
- package/lib/helper/WebDriver.js +45 -1
- package/lib/helper/extras/clipboard.js +31 -0
- package/lib/listener/result.js +4 -0
- package/lib/mocha/bdd.js +19 -1
- package/lib/plugin/junitReporter.js +4 -1
- package/lib/plugin/pause.js +4 -1
- package/lib/plugin/sitemap.js +228 -0
- package/lib/recorder.js +19 -18
- package/lib/result.js +16 -0
- package/lib/step/base.js +12 -0
- package/lib/utils.js +2 -10
- package/lib/webServer.js +193 -0
- package/lib/workers.js +118 -31
- package/package.json +4 -2
- package/typings/index.d.ts +60 -7
- package/typings/promiseBasedTypes.d.ts +985 -814
- package/typings/types.d.ts +1084 -815
- package/docs/helpers/CDPBrowser.md +0 -2138
package/docs/commands.md
CHANGED
|
@@ -53,6 +53,14 @@ Run test files in shuffled order
|
|
|
53
53
|
npx codeceptjs run --shuffle
|
|
54
54
|
```
|
|
55
55
|
|
|
56
|
+
Rerun only tests that failed in the previous run
|
|
57
|
+
|
|
58
|
+
```sh
|
|
59
|
+
npx codeceptjs run --last-failed
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
See [Last Failed](#last-failed) for details.
|
|
63
|
+
|
|
56
64
|
Run single test in debug mode (see more in [debugging](#Debugging) section)
|
|
57
65
|
|
|
58
66
|
```sh
|
|
@@ -157,6 +165,12 @@ npx codeceptjs run-workers 3 --by suite
|
|
|
157
165
|
|
|
158
166
|
# Pool mode with filtering
|
|
159
167
|
npx codeceptjs run-workers 4 --by pool --grep "@smoke"
|
|
168
|
+
|
|
169
|
+
# Rerun only tests that failed in the previous run
|
|
170
|
+
npx codeceptjs run-workers 3 --last-failed
|
|
171
|
+
|
|
172
|
+
# Run tests tagged @serial one by one after parallel workers finish
|
|
173
|
+
npx codeceptjs run-workers 3 --serial "@serial"
|
|
160
174
|
```
|
|
161
175
|
|
|
162
176
|
**Test Distribution Strategies:**
|
|
@@ -165,10 +179,31 @@ npx codeceptjs run-workers 4 --by pool --grep "@smoke"
|
|
|
165
179
|
- `--by suite`: Pre-assigns entire test suites to workers
|
|
166
180
|
- `--by pool`: Dynamic distribution for optimal load balancing (recommended for best performance)
|
|
167
181
|
|
|
182
|
+
`--serial <pattern>` keeps tests matching the pattern out of parallel workers and runs them one by one in an extra worker once the others finish, within the same run. It overrides the `serial` config option. See [Serial tests](/parallel#serial-tests).
|
|
183
|
+
|
|
168
184
|
The pool mode provides the best load balancing by maintaining tests in a shared pool and distributing them dynamically as workers become available. This prevents workers from sitting idle and ensures optimal CPU utilization, especially when tests have varying execution times.
|
|
169
185
|
|
|
170
186
|
See [Parallel Execution](/parallel) documentation for more details.
|
|
171
187
|
|
|
188
|
+
## Last Failed
|
|
189
|
+
|
|
190
|
+
Rerun only the tests that failed in the previous run. Works with `run` and `run-workers`, including all `--by` strategies:
|
|
191
|
+
|
|
192
|
+
```sh
|
|
193
|
+
npx codeceptjs run --last-failed
|
|
194
|
+
npx codeceptjs run-workers 3 --last-failed
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
Every `run` and `run-workers` writes a report to `result.json` in the output directory (`output` in config). It lists each executed test with its state. `--last-failed` reads this report and runs only the tests whose state is `failed`:
|
|
198
|
+
|
|
199
|
+
- Skipped and pending tests are not considered failed.
|
|
200
|
+
- Tests are matched by their title, including the Feature title. A renamed test or Feature is not matched.
|
|
201
|
+
- Combined with `--grep`, only failed tests that also match the pattern are run.
|
|
202
|
+
- The report always reflects the most recent run, including a `--last-failed` run. Tests that pass in a rerun drop out of the next `--last-failed` set; tests that did not run (filtered out by `--grep`, sharding, and so on) are not included.
|
|
203
|
+
- If the report does not exist, the command fails with "No previous run found". If the last run had no failures, it prints "No failed tests in the last run" and exits with code 0.
|
|
204
|
+
|
|
205
|
+
`--last-failed` is different from [run-rerun](#run-rerun), which repeats whole runs to detect flaky tests.
|
|
206
|
+
|
|
172
207
|
## Run Rerun <Badge text="Since 3.3.6" type="warning"/>
|
|
173
208
|
|
|
174
209
|
Run tests multiple times to detect and fix flaky tests.
|
|
@@ -262,6 +297,16 @@ Or in provided path
|
|
|
262
297
|
npx codecept init test
|
|
263
298
|
```
|
|
264
299
|
|
|
300
|
+
## Init Agent
|
|
301
|
+
|
|
302
|
+
Registers the CodeceptJS MCP server and installs CodeceptJS skills for a coding agent (`claude`, `codex`, `cursor`, `opencode`):
|
|
303
|
+
|
|
304
|
+
```sh
|
|
305
|
+
npx codeceptjs init:agent claude
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
Omit the agent to pick it from a list. See [Agentic Testing](/agents).
|
|
309
|
+
|
|
265
310
|
## Migrate
|
|
266
311
|
|
|
267
312
|
Migrate your current `codecept.json` to `codecept.conf.js`
|
package/docs/configuration.md
CHANGED
|
@@ -39,11 +39,13 @@ export const config = {
|
|
|
39
39
|
|
|
40
40
|
- `bootstrap` / `teardown` — run code before / after the whole run; an async function or a path to a JS module. See [Bootstrap](/bootstrap).
|
|
41
41
|
- `bootstrapAll` / `teardownAll` — run once around a parallel run (before any worker starts / after all finish). See [bootstrapAll / teardownAll](/bootstrap#bootstrapall-teardownall).
|
|
42
|
+
- `webServer` — start the application under test before the run and stop it after. See [Web Server](#web-server).
|
|
42
43
|
|
|
43
44
|
**Test runner**
|
|
44
45
|
|
|
45
46
|
- `timeout` — default per-test timeout in seconds; a test is killed if it stops responding.
|
|
46
47
|
- `mocha` — [Mocha options](https://mochajs.org/#configuring-mocha-nodejs), including extra reporters. See [Reporters](/reports).
|
|
48
|
+
- `serial` — pattern for tests that must not run in parallel, e.g. `serial: '@serial'`. With `run-workers` they run one by one after all parallel workers finish. `--serial` overrides it. See [Serial tests](/parallel#serial-tests).
|
|
47
49
|
- `workerInitializationDelay` — delay in milliseconds between spinning up parallel workers to prevent CPU spikes and stagger browser startup. Defaults to `200`. Set to `0` to disable.
|
|
48
50
|
- `workerInitializationMaxDelay` — maximum total delay (in milliseconds) for worker initialization staggering. Defaults to `10000` (10 s). Set to `0` to disable capping.
|
|
49
51
|
|
|
@@ -89,6 +91,41 @@ require: ['tsx/esm', 'should', './lib/testSetup']
|
|
|
89
91
|
|
|
90
92
|
The config file itself (`codecept.conf.ts`) and helpers are transpiled automatically — only test files need the loader. See [TypeScript](/typescript) for the full setup.
|
|
91
93
|
|
|
94
|
+
## Web Server
|
|
95
|
+
|
|
96
|
+
`webServer` starts your application before tests run and stops it when they finish, so you don't need a `bootstrap` script for it:
|
|
97
|
+
|
|
98
|
+
```js
|
|
99
|
+
export const config = {
|
|
100
|
+
webServer: {
|
|
101
|
+
command: 'npm run start',
|
|
102
|
+
url: 'http://localhost:3000',
|
|
103
|
+
enabled: !process.env.CI,
|
|
104
|
+
},
|
|
105
|
+
// ...
|
|
106
|
+
}
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
- `command` — shell command that starts the app.
|
|
110
|
+
- `url` — CodeceptJS sends GET requests here until it gets any HTTP response, then starts the tests. If `url` already responds before `command` is run, the running server is reused.
|
|
111
|
+
- `enabled` — set to `false` to skip starting the server, e.g. `enabled: !process.env.CI` when CI provides its own. Default `true`.
|
|
112
|
+
- `timeout` — milliseconds to wait for `url`. Default `60000`.
|
|
113
|
+
- `cwd` — working directory for `command`, relative to the config file. Default is the config directory.
|
|
114
|
+
- `env` — extra environment variables for `command`.
|
|
115
|
+
|
|
116
|
+
If the server exits early or doesn't respond within `timeout`, the run fails and prints the last 20 lines of its output. Run with `--debug` to see all of its output.
|
|
117
|
+
|
|
118
|
+
To start several services, pass an array. They start in order and stop in reverse order:
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
webServer: [
|
|
122
|
+
{ command: 'npm run api', url: 'http://localhost:4000/health' },
|
|
123
|
+
{ command: 'npm run start', url: 'http://localhost:3000' },
|
|
124
|
+
],
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
The server is started as part of bootstrap, right before the `bootstrap` hook (or `bootstrapAll` for `run-workers` and `run-multiple`), and stopped after `teardown` (`teardownAll`). It starts once in the main process; workers and child processes reuse it. The interactive `shell` starts it too, `dry-run` and `check` only with `--bootstrap`. CodeceptJS stops the command and every process it spawned, including when the run is interrupted with Ctrl+C.
|
|
128
|
+
|
|
92
129
|
## Dynamic configuration
|
|
93
130
|
|
|
94
131
|
A JS/TS config file is plain code, so you can read environment variables and build the config at runtime:
|
package/docs/debugging.md
CHANGED
|
@@ -120,7 +120,7 @@ For automated debugging without modifying test code, use the `pause` plugin. It
|
|
|
120
120
|
|
|
121
121
|
### Pause on Failure
|
|
122
122
|
|
|
123
|
-
Automatically enters interactive pause when a step fails:
|
|
123
|
+
Automatically enters interactive pause when a step fails. Failures inside `tryTo` and `retryTo` attempts that will be retried are ignored:
|
|
124
124
|
|
|
125
125
|
```bash
|
|
126
126
|
npx codeceptjs run -p pause
|
|
@@ -192,26 +192,6 @@ By default `id` property of response is taken. This behavior can be changed by s
|
|
|
192
192
|
|
|
193
193
|
* `config`  
|
|
194
194
|
|
|
195
|
-
### _requestCreate
|
|
196
|
-
|
|
197
|
-
Executes request to create a record in API.
|
|
198
|
-
Can be replaced from a in custom helper.
|
|
199
|
-
|
|
200
|
-
#### Parameters
|
|
201
|
-
|
|
202
|
-
* `factory` **any** 
|
|
203
|
-
* `data` **any** 
|
|
204
|
-
|
|
205
|
-
### _requestDelete
|
|
206
|
-
|
|
207
|
-
Executes request to delete a record in API
|
|
208
|
-
Can be replaced from a custom helper.
|
|
209
|
-
|
|
210
|
-
#### Parameters
|
|
211
|
-
|
|
212
|
-
* `factory` **any** 
|
|
213
|
-
* `id` **any** 
|
|
214
|
-
|
|
215
195
|
### have
|
|
216
196
|
|
|
217
197
|
Generates a new record using factory and saves API request to store it.
|
|
@@ -256,6 +236,26 @@ I.haveMultiple('post', 3, { author: 'davert' }, { publish_date: '01.01.1997' });
|
|
|
256
236
|
* `params` **any?** 
|
|
257
237
|
* `options` **any?** 
|
|
258
238
|
|
|
239
|
+
### _requestCreate
|
|
240
|
+
|
|
241
|
+
Executes request to create a record in API.
|
|
242
|
+
Can be replaced from a in custom helper.
|
|
243
|
+
|
|
244
|
+
#### Parameters
|
|
245
|
+
|
|
246
|
+
* `factory` **any** 
|
|
247
|
+
* `data` **any** 
|
|
248
|
+
|
|
249
|
+
### _requestDelete
|
|
250
|
+
|
|
251
|
+
Executes request to delete a record in API
|
|
252
|
+
Can be replaced from a custom helper.
|
|
253
|
+
|
|
254
|
+
#### Parameters
|
|
255
|
+
|
|
256
|
+
* `factory` **any** 
|
|
257
|
+
* `id` **any** 
|
|
258
|
+
|
|
259
259
|
[1]: https://github.com/rosiejs/rosie
|
|
260
260
|
|
|
261
261
|
[2]: https://www.npmjs.com/package/faker
|