codeceptjs 4.2.0-beta.4 → 4.2.1-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (65) hide show
  1. package/README.md +2 -0
  2. package/bin/codecept.js +11 -0
  3. package/docs/agents.md +26 -1
  4. package/docs/ai.md +57 -0
  5. package/docs/alternative-browsers.md +175 -148
  6. package/docs/assertions.md +74 -0
  7. package/docs/bootstrap.md +2 -0
  8. package/docs/commands.md +45 -0
  9. package/docs/configuration.md +37 -0
  10. package/docs/debugging.md +1 -1
  11. package/docs/helpers/ApiDataFactory.md +20 -20
  12. package/docs/helpers/Appium.md +1615 -670
  13. package/docs/helpers/Decision.md +146 -0
  14. package/docs/helpers/GraphQL.md +31 -31
  15. package/docs/helpers/GraphQLDataFactory.md +20 -20
  16. package/docs/helpers/Kitesurf.md +1808 -25
  17. package/docs/helpers/Lightpanda.md +139 -0
  18. package/docs/helpers/Obscura.md +1799 -74
  19. package/docs/helpers/Playwright.md +490 -428
  20. package/docs/helpers/Puppeteer.md +317 -255
  21. package/docs/helpers/REST.md +43 -43
  22. package/docs/helpers/WebDriver.md +237 -175
  23. package/docs/installation.md +16 -0
  24. package/docs/mcp.md +7 -1
  25. package/docs/parallel.md +35 -1
  26. package/docs/plugins/pause.md +2 -1
  27. package/docs/plugins/sitemap.md +48 -0
  28. package/docs/plugins.md +4 -0
  29. package/lib/ai.js +81 -2
  30. package/lib/codecept.js +32 -2
  31. package/lib/command/init.js +22 -0
  32. package/lib/command/initAgent.js +78 -0
  33. package/lib/command/run-multiple.js +3 -0
  34. package/lib/command/run-workers.js +10 -3
  35. package/lib/command/run.js +3 -2
  36. package/lib/command/utils.js +15 -0
  37. package/lib/command/workers/runTests.js +10 -10
  38. package/lib/container.js +4 -2
  39. package/lib/effects.js +19 -1
  40. package/lib/helper/Appium.js +23 -0
  41. package/lib/helper/CDPBrowser.js +387 -1338
  42. package/lib/helper/Decision.js +256 -0
  43. package/lib/helper/Kitesurf.js +3 -21
  44. package/lib/helper/Lightpanda.js +137 -0
  45. package/lib/helper/Obscura.js +21 -238
  46. package/lib/helper/Playwright.js +50 -4
  47. package/lib/helper/Puppeteer.js +62 -8
  48. package/lib/helper/WebDriver.js +45 -1
  49. package/lib/helper/extras/clipboard.js +31 -0
  50. package/lib/listener/result.js +4 -0
  51. package/lib/mocha/bdd.js +19 -1
  52. package/lib/plugin/junitReporter.js +4 -1
  53. package/lib/plugin/pause.js +4 -1
  54. package/lib/plugin/sitemap.js +228 -0
  55. package/lib/recorder.js +19 -18
  56. package/lib/result.js +16 -0
  57. package/lib/step/base.js +12 -0
  58. package/lib/utils.js +2 -10
  59. package/lib/webServer.js +193 -0
  60. package/lib/workers.js +118 -31
  61. package/package.json +4 -2
  62. package/typings/index.d.ts +60 -7
  63. package/typings/promiseBasedTypes.d.ts +985 -814
  64. package/typings/types.d.ts +1084 -815
  65. package/docs/helpers/CDPBrowser.md +0 -2138
package/docs/commands.md CHANGED
@@ -53,6 +53,14 @@ Run test files in shuffled order
53
53
  npx codeceptjs run --shuffle
54
54
  ```
55
55
 
56
+ Rerun only tests that failed in the previous run
57
+
58
+ ```sh
59
+ npx codeceptjs run --last-failed
60
+ ```
61
+
62
+ See [Last Failed](#last-failed) for details.
63
+
56
64
  Run single test in debug mode (see more in [debugging](#Debugging) section)
57
65
 
58
66
  ```sh
@@ -157,6 +165,12 @@ npx codeceptjs run-workers 3 --by suite
157
165
 
158
166
  # Pool mode with filtering
159
167
  npx codeceptjs run-workers 4 --by pool --grep "@smoke"
168
+
169
+ # Rerun only tests that failed in the previous run
170
+ npx codeceptjs run-workers 3 --last-failed
171
+
172
+ # Run tests tagged @serial one by one after parallel workers finish
173
+ npx codeceptjs run-workers 3 --serial "@serial"
160
174
  ```
161
175
 
162
176
  **Test Distribution Strategies:**
@@ -165,10 +179,31 @@ npx codeceptjs run-workers 4 --by pool --grep "@smoke"
165
179
  - `--by suite`: Pre-assigns entire test suites to workers
166
180
  - `--by pool`: Dynamic distribution for optimal load balancing (recommended for best performance)
167
181
 
182
+ `--serial <pattern>` keeps tests matching the pattern out of parallel workers and runs them one by one in an extra worker once the others finish, within the same run. It overrides the `serial` config option. See [Serial tests](/parallel#serial-tests).
183
+
168
184
  The pool mode provides the best load balancing by maintaining tests in a shared pool and distributing them dynamically as workers become available. This prevents workers from sitting idle and ensures optimal CPU utilization, especially when tests have varying execution times.
169
185
 
170
186
  See [Parallel Execution](/parallel) documentation for more details.
171
187
 
188
+ ## Last Failed
189
+
190
+ Rerun only the tests that failed in the previous run. Works with `run` and `run-workers`, including all `--by` strategies:
191
+
192
+ ```sh
193
+ npx codeceptjs run --last-failed
194
+ npx codeceptjs run-workers 3 --last-failed
195
+ ```
196
+
197
+ Every `run` and `run-workers` writes a report to `result.json` in the output directory (`output` in config). It lists each executed test with its state. `--last-failed` reads this report and runs only the tests whose state is `failed`:
198
+
199
+ - Skipped and pending tests are not considered failed.
200
+ - Tests are matched by their title, including the Feature title. A renamed test or Feature is not matched.
201
+ - Combined with `--grep`, only failed tests that also match the pattern are run.
202
+ - The report always reflects the most recent run, including a `--last-failed` run. Tests that pass in a rerun drop out of the next `--last-failed` set; tests that did not run (filtered out by `--grep`, sharding, and so on) are not included.
203
+ - If the report does not exist, the command fails with "No previous run found". If the last run had no failures, it prints "No failed tests in the last run" and exits with code 0.
204
+
205
+ `--last-failed` is different from [run-rerun](#run-rerun), which repeats whole runs to detect flaky tests.
206
+
172
207
  ## Run Rerun <Badge text="Since 3.3.6" type="warning"/>
173
208
 
174
209
  Run tests multiple times to detect and fix flaky tests.
@@ -262,6 +297,16 @@ Or in provided path
262
297
  npx codecept init test
263
298
  ```
264
299
 
300
+ ## Init Agent
301
+
302
+ Registers the CodeceptJS MCP server and installs CodeceptJS skills for a coding agent (`claude`, `codex`, `cursor`, `opencode`):
303
+
304
+ ```sh
305
+ npx codeceptjs init:agent claude
306
+ ```
307
+
308
+ Omit the agent to pick it from a list. See [Agentic Testing](/agents).
309
+
265
310
  ## Migrate
266
311
 
267
312
  Migrate your current `codecept.json` to `codecept.conf.js`
@@ -39,11 +39,13 @@ export const config = {
39
39
 
40
40
  - `bootstrap` / `teardown` — run code before / after the whole run; an async function or a path to a JS module. See [Bootstrap](/bootstrap).
41
41
  - `bootstrapAll` / `teardownAll` — run once around a parallel run (before any worker starts / after all finish). See [bootstrapAll / teardownAll](/bootstrap#bootstrapall-teardownall).
42
+ - `webServer` — start the application under test before the run and stop it after. See [Web Server](#web-server).
42
43
 
43
44
  **Test runner**
44
45
 
45
46
  - `timeout` — default per-test timeout in seconds; a test is killed if it stops responding.
46
47
  - `mocha` — [Mocha options](https://mochajs.org/#configuring-mocha-nodejs), including extra reporters. See [Reporters](/reports).
48
+ - `serial` — pattern for tests that must not run in parallel, e.g. `serial: '@serial'`. With `run-workers` they run one by one after all parallel workers finish. `--serial` overrides it. See [Serial tests](/parallel#serial-tests).
47
49
  - `workerInitializationDelay` — delay in milliseconds between spinning up parallel workers to prevent CPU spikes and stagger browser startup. Defaults to `200`. Set to `0` to disable.
48
50
  - `workerInitializationMaxDelay` — maximum total delay (in milliseconds) for worker initialization staggering. Defaults to `10000` (10 s). Set to `0` to disable capping.
49
51
 
@@ -89,6 +91,41 @@ require: ['tsx/esm', 'should', './lib/testSetup']
89
91
 
90
92
  The config file itself (`codecept.conf.ts`) and helpers are transpiled automatically — only test files need the loader. See [TypeScript](/typescript) for the full setup.
91
93
 
94
+ ## Web Server
95
+
96
+ `webServer` starts your application before tests run and stops it when they finish, so you don't need a `bootstrap` script for it:
97
+
98
+ ```js
99
+ export const config = {
100
+ webServer: {
101
+ command: 'npm run start',
102
+ url: 'http://localhost:3000',
103
+ enabled: !process.env.CI,
104
+ },
105
+ // ...
106
+ }
107
+ ```
108
+
109
+ - `command` — shell command that starts the app.
110
+ - `url` — CodeceptJS sends GET requests here until it gets any HTTP response, then starts the tests. If `url` already responds before `command` is run, the running server is reused.
111
+ - `enabled` — set to `false` to skip starting the server, e.g. `enabled: !process.env.CI` when CI provides its own. Default `true`.
112
+ - `timeout` — milliseconds to wait for `url`. Default `60000`.
113
+ - `cwd` — working directory for `command`, relative to the config file. Default is the config directory.
114
+ - `env` — extra environment variables for `command`.
115
+
116
+ If the server exits early or doesn't respond within `timeout`, the run fails and prints the last 20 lines of its output. Run with `--debug` to see all of its output.
117
+
118
+ To start several services, pass an array. They start in order and stop in reverse order:
119
+
120
+ ```js
121
+ webServer: [
122
+ { command: 'npm run api', url: 'http://localhost:4000/health' },
123
+ { command: 'npm run start', url: 'http://localhost:3000' },
124
+ ],
125
+ ```
126
+
127
+ The server is started as part of bootstrap, right before the `bootstrap` hook (or `bootstrapAll` for `run-workers` and `run-multiple`), and stopped after `teardown` (`teardownAll`). It starts once in the main process; workers and child processes reuse it. The interactive `shell` starts it too, `dry-run` and `check` only with `--bootstrap`. CodeceptJS stops the command and every process it spawned, including when the run is interrupted with Ctrl+C.
128
+
92
129
  ## Dynamic configuration
93
130
 
94
131
  A JS/TS config file is plain code, so you can read environment variables and build the config at runtime:
package/docs/debugging.md CHANGED
@@ -120,7 +120,7 @@ For automated debugging without modifying test code, use the `pause` plugin. It
120
120
 
121
121
  ### Pause on Failure
122
122
 
123
- Automatically enters interactive pause when a step fails:
123
+ Automatically enters interactive pause when a step fails. Failures inside `tryTo` and `retryTo` attempts that will be retried are ignored:
124
124
 
125
125
  ```bash
126
126
  npx codeceptjs run -p pause
@@ -192,26 +192,6 @@ By default `id` property of response is taken. This behavior can be changed by s
192
192
 
193
193
  * `config` &#x20;
194
194
 
195
- ### _requestCreate
196
-
197
- Executes request to create a record in API.
198
- Can be replaced from a in custom helper.
199
-
200
- #### Parameters
201
-
202
- * `factory` **any**&#x20;
203
- * `data` **any**&#x20;
204
-
205
- ### _requestDelete
206
-
207
- Executes request to delete a record in API
208
- Can be replaced from a custom helper.
209
-
210
- #### Parameters
211
-
212
- * `factory` **any**&#x20;
213
- * `id` **any**&#x20;
214
-
215
195
  ### have
216
196
 
217
197
  Generates a new record using factory and saves API request to store it.
@@ -256,6 +236,26 @@ I.haveMultiple('post', 3, { author: 'davert' }, { publish_date: '01.01.1997' });
256
236
  * `params` **any?**&#x20;
257
237
  * `options` **any?**&#x20;
258
238
 
239
+ ### _requestCreate
240
+
241
+ Executes request to create a record in API.
242
+ Can be replaced from a in custom helper.
243
+
244
+ #### Parameters
245
+
246
+ * `factory` **any**&#x20;
247
+ * `data` **any**&#x20;
248
+
249
+ ### _requestDelete
250
+
251
+ Executes request to delete a record in API
252
+ Can be replaced from a custom helper.
253
+
254
+ #### Parameters
255
+
256
+ * `factory` **any**&#x20;
257
+ * `id` **any**&#x20;
258
+
259
259
  [1]: https://github.com/rosiejs/rosie
260
260
 
261
261
  [2]: https://www.npmjs.com/package/faker