webship-js 2.0.10 → 2.0.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.circleci/config.yml +10 -1
- package/.drone.yml +7 -7
- package/.github/ISSUE_TEMPLATE/issue.md +37 -0
- package/.github/PULL_REQUEST_TEMPLATE.md +24 -0
- package/.github/workflows/github-actions.yml +101 -4
- package/.gitlab-ci.yml +9 -4
- package/.harness/webship-js-pipeline.yml +5 -6
- package/.semaphore/semaphore.yml +6 -9
- package/.teamcity/settings.kts +7 -6
- package/.travis.yml +6 -1
- package/AGENTS.md +1 -1
- package/CLAUDE.md +69 -24
- package/Jenkinsfile +5 -3
- package/README.md +13 -1
- package/azure-pipelines.yml +17 -8
- package/bamboo-specs/bamboo.yml +9 -15
- package/bin/generate-reports.js +51 -7
- package/bin/init-webship.js +127 -23
- package/bitbucket-pipelines.yml +5 -1
- package/buildspec.yml +6 -7
- package/cloudbuild.yaml +28 -12
- package/codefresh.yml +6 -6
- package/cucumber.js +46 -11
- package/docs/00-quick-start.md +38 -7
- package/docs/02-bbr-smart-waits.md +21 -10
- package/docs/03-selector-registry.md +24 -0
- package/docs/04-step-reference.md +53 -27
- package/docs/05-web-first-assertions.md +6 -0
- package/docs/07-auth-state.md +1 -1
- package/docs/08-clock-mocking.md +21 -2
- package/docs/09-api-testing.md +40 -2
- package/docs/10-accessibility.md +2 -0
- package/docs/11-debugging.md +24 -1
- package/docs/12-ai-agent-guide.md +8 -1
- package/docs/13-faq.md +75 -1
- package/docs/14-recipes-cookbook.md +2 -0
- package/docs/15-tag-conventions.md +5 -1
- package/docs/16-ci-cd.md +161 -124
- package/docs/README.md +5 -5
- package/docs/advanced-screenshots.md +3 -1
- package/docs/api-step-definitions.md +16 -2
- package/docs/global-settings.md +70 -18
- package/docs/install-webship-js/ddev-webship-js.md +3 -0
- package/docs/install-webship-js.md +16 -2
- package/docs/step-definitions/then-i-should-see.md +14 -1
- package/docs/step-definitions/then-item-checkbox.md +2 -0
- package/docs/step-definitions/when-i-fill.md +4 -0
- package/docs/step-definitions/when-i-select-option-select-list.md +2 -0
- package/examples/a11y-legacy-widget.html +34 -0
- package/examples/auth-state.html +42 -0
- package/examples/choice-controls.html +29 -0
- package/examples/clock.html +33 -0
- package/examples/datetime-fields.html +52 -0
- package/examples/downloads/webship-archive.zip +0 -0
- package/examples/downloads/webship-notes.txt +4 -0
- package/examples/downloads/webship-report.csv +4 -0
- package/examples/downloads.html +16 -0
- package/examples/element-selector-hint.html +16 -0
- package/examples/element-viewport.html +42 -0
- package/examples/field-states.html +27 -0
- package/examples/field-types.html +98 -0
- package/examples/fill-in-empty.html +26 -0
- package/examples/fill-unreachable.html +27 -0
- package/examples/iframe-inner.html +20 -0
- package/examples/iframe-nested.html +16 -0
- package/examples/iframe-newsletter.html +29 -0
- package/examples/iframe-order.html +26 -0
- package/examples/iframe-outer.html +15 -0
- package/examples/iframe.html +21 -0
- package/examples/javascript-errors.html +35 -0
- package/examples/links.html +1 -0
- package/examples/metatags.html +18 -0
- package/examples/mouse-and-pointer.html +83 -0
- package/examples/multi-value-fields.html +54 -0
- package/examples/native-dialogs.html +8 -0
- package/examples/network.html +83 -0
- package/examples/registered-names.html +15 -0
- package/examples/scroll-horizontal.html +50 -0
- package/examples/select-by-label.html +33 -0
- package/examples/selectors-position.html +29 -0
- package/examples/settle-budget.html +21 -0
- package/examples/sorted-tables.html +38 -0
- package/examples/storage.html +57 -0
- package/examples/text-matching.html +48 -0
- package/examples/text-order.html +25 -0
- package/examples/touch.html +22 -0
- package/examples/viewport-size.html +24 -0
- package/examples/viewport-top.html +36 -0
- package/examples/wait-conditions.html +69 -0
- package/examples/web-first.html +99 -0
- package/package.json +24 -2
- package/playwright.config.ts +3 -1
- package/tests/assets/catalog.xml +19 -0
- package/tests/assets/feed-namespaced.xml +13 -0
- package/tests/assets/sitemap.xml +14 -0
- package/tests/assets/webship-site-expected.yaml +21 -0
- package/tests/assets/webship-site.schema.json +46 -0
- package/tests/assets/webship-site.yaml +21 -0
- package/tests/check-api-launch-url.js +106 -0
- package/tests/check-assets.js +103 -0
- package/tests/check-attachments.js +105 -0
- package/tests/check-breakpoints.js +117 -0
- package/tests/check-clock.js +143 -0
- package/tests/check-config.js +90 -0
- package/tests/check-context.js +215 -0
- package/tests/check-dry-run.js +104 -0
- package/tests/check-examples.js +241 -0
- package/tests/check-exit-report.js +141 -0
- package/tests/check-javascript-errors.js +105 -0
- package/tests/check-must-fail.js +85 -0
- package/tests/check-paths.js +82 -0
- package/tests/check-report.js +119 -0
- package/tests/check-scaffold.js +246 -0
- package/tests/check-screenshots.js +117 -0
- package/tests/check-settle-verbs.js +137 -0
- package/tests/check-time-limit.js +104 -0
- package/tests/check-video.js +156 -0
- package/tests/check-world.js +167 -0
- package/tests/features/a11y.feature +18 -0
- package/tests/features/api-base-url.feature +25 -0
- package/tests/features/api-form-data.feature +18 -0
- package/tests/features/api-json-paths.feature +24 -0
- package/tests/features/api-response-contains.feature +21 -0
- package/tests/features/api-response-header-presence.feature +17 -0
- package/tests/features/api-response-header.feature +17 -0
- package/tests/features/api-saved-placeholders.feature +37 -0
- package/tests/features/api-step-definitions-individual.feature +1 -1
- package/tests/features/api-tables.feature +40 -0
- package/tests/features/auth-state.feature +32 -0
- package/tests/features/breakpoints.feature +25 -0
- package/tests/features/choice-controls.feature +35 -0
- package/tests/features/clock.feature +27 -0
- package/tests/features/complete-api-testing-examples.feature +1 -1
- package/tests/features/cookie.feature +9 -0
- package/tests/features/datetime-field-lookup.feature +29 -0
- package/tests/features/dialog.feature +27 -0
- package/tests/features/element-selector-hint.feature +14 -0
- package/tests/features/element.feature +47 -0
- package/tests/features/field-state.feature +21 -0
- package/tests/features/field.feature +104 -0
- package/tests/features/file-download.feature +64 -0
- package/tests/features/fill-in-empty-value.feature +41 -0
- package/tests/features/fill-reachable.feature +13 -0
- package/tests/features/iframe-nested.feature +43 -0
- package/tests/features/iframe.feature +83 -0
- package/tests/features/javascript-tolerated-errors.feature +20 -0
- package/tests/features/javascript.feature +47 -0
- package/tests/features/link.feature +11 -0
- package/tests/features/metatag.feature +38 -0
- package/tests/features/modal-open.feature +13 -0
- package/tests/features/mouse-and-pointer.feature +86 -0
- package/tests/features/multi-value-field-lookup.feature +30 -0
- package/tests/features/network-mocking.feature +80 -0
- package/tests/features/registered-names.feature +14 -0
- package/tests/features/responsive.feature +27 -0
- package/tests/features/scroll-horizontal.feature +55 -0
- package/tests/features/select-by-label.feature +31 -0
- package/tests/features/selector-files.feature +14 -0
- package/tests/features/selectors-position.feature +41 -0
- package/tests/features/selectors-table-header.feature +35 -0
- package/tests/features/settle-budget.feature +11 -0
- package/tests/features/sorted-tables.feature +19 -0
- package/tests/features/storage-quoted-values.feature +26 -0
- package/tests/features/storage.feature +98 -0
- package/tests/features/test--then--i-see-locator.feature +13 -0
- package/tests/features/test--then--i-should--see-text-in-element.feature +9 -1
- package/tests/features/test--then--the-field-should-contain-value.feature +8 -0
- package/tests/features/text-matching.feature +38 -0
- package/tests/features/text-order.feature +14 -0
- package/tests/features/touch.feature +21 -0
- package/tests/features/video.feature +1 -0
- package/tests/features/viewport-top.feature +13 -0
- package/tests/features/wait.feature +89 -0
- package/tests/features/web-first.feature +100 -0
- package/tests/features/xml-namespaces.feature +34 -0
- package/tests/features/xml.feature +96 -0
- package/tests/features/yaml.feature +75 -0
- package/tests/must-fail/api-json-paths.feature +21 -0
- package/tests/must-fail/api-response-contains.feature +23 -0
- package/tests/must-fail/api-response-header-presence.feature +17 -0
- package/tests/must-fail/api-response-header.feature +21 -0
- package/tests/must-fail/api-without-base-url.feature +11 -0
- package/tests/must-fail/assertions.feature +96 -0
- package/tests/must-fail/breakpoints.feature +15 -0
- package/tests/must-fail/choice-controls.feature +25 -0
- package/tests/must-fail/clock.feature +11 -0
- package/tests/must-fail/datetime-field.feature +10 -0
- package/tests/must-fail/element-selector-hint.feature +17 -0
- package/tests/must-fail/field-state.feature +20 -0
- package/tests/must-fail/fill-field.feature +30 -0
- package/tests/must-fail/iframe-text.feature +17 -0
- package/tests/must-fail/modal-open.feature +26 -0
- package/tests/must-fail/multi-value-field.feature +18 -0
- package/tests/must-fail/registered-names.feature +25 -0
- package/tests/must-fail/select-list.feature +15 -0
- package/tests/must-fail/selector-files.feature +10 -0
- package/tests/must-fail/sorted-tables.feature +20 -0
- package/tests/must-fail/storage-before-a-page.feature +13 -0
- package/tests/must-fail/text-matching.feature +16 -0
- package/tests/must-fail/text-order.feature +15 -0
- package/tests/must-fail/viewport-top.feature +20 -0
- package/tests/must-fail/waits.feature +51 -0
- package/tests/must-fail/yaml-numbers.feature +48 -0
- package/tests/reports/README.md +2 -2
- package/tests/step-definitions/a11y.steps.js +36 -23
- package/tests/step-definitions/action.steps.js +22 -8
- package/tests/step-definitions/api.steps.js +349 -67
- package/tests/step-definitions/assertion.steps.js +327 -63
- package/tests/step-definitions/auth.steps.js +15 -18
- package/tests/step-definitions/clock.steps.js +22 -0
- package/tests/step-definitions/debug.steps.js +8 -0
- package/tests/step-definitions/dialog.steps.js +9 -2
- package/tests/step-definitions/element.steps.js +135 -18
- package/tests/step-definitions/field.steps.js +523 -80
- package/tests/step-definitions/file-download.steps.js +74 -10
- package/tests/step-definitions/form.steps.js +76 -29
- package/tests/step-definitions/iframe.steps.js +89 -29
- package/tests/step-definitions/input.steps.js +26 -7
- package/tests/step-definitions/javascript.steps.js +64 -22
- package/tests/step-definitions/modal.steps.js +79 -13
- package/tests/step-definitions/navigation.steps.js +14 -0
- package/tests/step-definitions/network.steps.js +15 -5
- package/tests/step-definitions/response.steps.js +6 -4
- package/tests/step-definitions/responsive.steps.js +26 -33
- package/tests/step-definitions/screenshot.steps.js +26 -9
- package/tests/step-definitions/scroll.steps.js +20 -0
- package/tests/step-definitions/selectors.steps.js +104 -51
- package/tests/step-definitions/storage.steps.js +41 -16
- package/tests/step-definitions/table.steps.js +81 -6
- package/tests/step-definitions/video.steps.js +52 -55
- package/tests/step-definitions/wait.steps.js +209 -54
- package/tests/step-definitions/web-first.steps.js +69 -35
- package/tests/step-definitions/webship.js +807 -128
- package/tests/step-definitions/xml.steps.js +124 -28
- package/tests/step-definitions/yaml.steps.js +90 -46
- package/.scratch/commit.txt +0 -14
- package/.scratch/docs.py +0 -42
- package/.scratch/echo-server.js +0 -8
- package/.scratch/echo.js +0 -35
- package/.scratch/echo.pid +0 -1
- package/.scratch/features/cookies.feature +0 -22
- package/.scratch/http.pid +0 -1
- package/.scratch/https.pid +0 -1
- package/.scratch/issue.md +0 -49
- package/.scratch/one.js +0 -2
- package/.scratch/one.json +0 -280
- package/.scratch/patch.py +0 -163
- package/.scratch/pr.md +0 -30
package/docs/16-ci-cd.md
CHANGED
|
@@ -1,6 +1,10 @@
|
|
|
1
1
|
# CI / CD setup
|
|
2
2
|
|
|
3
|
-
Webship-js ships ready-to-use config files for every major CI/CD provider. The pattern is the same everywhere:
|
|
3
|
+
Webship-js ships ready-to-use config files for every major CI/CD provider. The pattern is the same everywhere: Node 24 (or the Node of the official Playwright image), install Playwright's chromium with its system deps, start the fixture server on port 8080, run `npm test`, and keep `tests/reports/`, `screenshots/` and `videos/` as artifacts of the run, passed or failed, wherever the provider keeps artifacts. No build step — `tsx` transpiles `.ts` step files on the fly.
|
|
4
|
+
|
|
5
|
+
The configs that run in the official Playwright image (Jenkins, Google Cloud Build, TeamCity, Drone, Bamboo, Codefresh) use the image of the Playwright version a fresh `npm install` brings, `mcr.microsoft.com/playwright:v1.65.1-noble` today, and still run `npx playwright install --with-deps chromium`: when a newer Playwright gets installed, its browser is downloaded, and moving the image tag to that version skips the download again.
|
|
6
|
+
|
|
7
|
+
Prices, free minutes, credits and plan names change without notice, so this page and the config comments do not quote them: each provider's own pricing page has the current ones.
|
|
4
8
|
|
|
5
9
|
## Supported providers
|
|
6
10
|
|
|
@@ -24,7 +28,7 @@ Webship-js ships ready-to-use config files for every major CI/CD provider. The p
|
|
|
24
28
|
| Bamboo Data Center | `bamboo-specs/bamboo.yml` | self-hosted (Atlassian) | yes |
|
|
25
29
|
| Codefresh | `codefresh.yml` | codefresh.io | yes |
|
|
26
30
|
| Octopus Deploy | (no test runner — CD only, see notes) | octopus.com | n/a |
|
|
27
|
-
| ~~Codeship~~ | retired
|
|
31
|
+
| ~~Codeship~~ | retired — do not use | — | — |
|
|
28
32
|
|
|
29
33
|
Each provider's setup notes live in its own section below.
|
|
30
34
|
|
|
@@ -34,7 +38,7 @@ Each provider's setup notes live in its own section below.
|
|
|
34
38
|
|
|
35
39
|
**File**: `.github/workflows/github-actions.yml`.
|
|
36
40
|
|
|
37
|
-
GitHub's native CI/CD.
|
|
41
|
+
GitHub's native CI/CD.
|
|
38
42
|
|
|
39
43
|
### Setup steps — open account + connect repo
|
|
40
44
|
|
|
@@ -45,15 +49,38 @@ GitHub's native CI/CD. #1 by adoption (~33% of all open-source repos according t
|
|
|
45
49
|
|
|
46
50
|
### Workflow contents
|
|
47
51
|
|
|
48
|
-
|
|
52
|
+
Runs on a push to `2.0.x`, on a pull request to `2.0.x` and by hand (`Actions → Github Actions → Run workflow`). The token is read-only (`permissions: contents: read`). A newer push to the same pull request cancels the run in progress. A run on `2.0.x` and a run started by hand have a concurrency group of their own: they are never queued behind another run and never cancelled.
|
|
53
|
+
|
|
54
|
+
Single job `build` on `ubuntu-latest`, 45-minute limit:
|
|
49
55
|
|
|
50
|
-
1. `actions/checkout@
|
|
51
|
-
2. `actions/setup-node@
|
|
56
|
+
1. `actions/checkout@v7` — pulls the repo.
|
|
57
|
+
2. `actions/setup-node@v7` — Node 20.x, with the npm cache keyed on `package.json`.
|
|
52
58
|
3. `npm install`.
|
|
53
|
-
4. `
|
|
54
|
-
5. `
|
|
55
|
-
6. `
|
|
56
|
-
7. `npm test
|
|
59
|
+
4. `npm run test:report` — generates an HTML report from a small JSON and checks it: no third-party icon, the project's own version, the place of execution.
|
|
60
|
+
5. `npx playwright install --with-deps chromium`.
|
|
61
|
+
6. `npm run test:dry-run` — no server: a dry run with the pretty formatter prints the steps, `? undefined` and the summary with no `- skipped` line for each hook or step; a real run still shows a skipped step.
|
|
62
|
+
7. `npm run test:api-launch-url` — no server: a relative API base URL follows `worldParameters.launchUrl`, and a scenario without a base URL fails plainly instead of reusing the previous one.
|
|
63
|
+
8. `npm run test:examples` — no browser: every step definition is documented with at least five examples that use it, every example step and every feature step resolves to exactly one definition.
|
|
64
|
+
9. `npm run test:time-limit` — no server: under a short `WEBSHIP_STEP_TIMEOUT`, the waits that take a length and the web-first `within N seconds` matchers run past the step timeout and pass, or fail with their own message.
|
|
65
|
+
10. `npm run test:paths` — no browser: a feature or a scenario (`file:line`) named on the command line runs alone, and naming none runs the whole suite (dry runs).
|
|
66
|
+
11. `npm run test:attachments` — the print steps (accessibility violations, …) attach what they print to the step in the JSON report.
|
|
67
|
+
12. `npm run test:js-errors` — no server: with warnings captured, the errors check counts errors only and the warnings check warnings only; what a check did not read is reported at the end of the scenario, in warn and in fail mode.
|
|
68
|
+
13. `npm run test:screenshots` — runs a skipped, a pending and a failing scenario and checks that only the failing one leaves a failure screenshot.
|
|
69
|
+
14. `npm run test:assets` — no server: in a project of its own, the attach, XML-file and YAML-file steps read its `tests/assets/` and paths from its root, and a missing file fails plainly.
|
|
70
|
+
15. `npm run test:config` — runs a scenario with a `cucumber.js` whose `worldParameters` carry only the launch URL, and checks that it passes and that the run ends by itself.
|
|
71
|
+
16. `npm run test:exit-report` — no server: the HTML report built at the end of a run is built once with `--parallel`, from a JSON report written elsewhere, and says so in one line when there is none.
|
|
72
|
+
17. `npm run test:breakpoints` — the `@mobile` / `@desktop` tags and the two named-viewport steps open at the sizes of one registry: the config, `WEBSHIP_SELECTORS_BREAKPOINTS`, the breakpoint marked `default: true` and the names a scenario adds.
|
|
73
|
+
18. `npm run test:context` — runs scenarios that restore an auth state and start and stop a video recording, and checks that the new page keeps the viewport of the `@mobile` tag, the smart waits, the cookies, JavaScript-error capture and the request log (it serves its own page).
|
|
74
|
+
19. `npm run test:video` — runs recorded scenarios in the `on`, `on-failure` and `off` modes and checks the video folder: final names with the status in lower case, no temporary `page@….webm` left, `print video path` printing the saved path, a reserved name kept on failure, a stopped recording named by the pattern, `WEBSHIP_VIDEO_SIZE` read.
|
|
75
|
+
20. `npm run test:world` — runs a project World that extends `PlaywrightWorld` with a project step ending in `this.smartSettle()`, then a World that replaces it, and checks the first keeps the `@mobile` viewport and the settle and the second is reported (it serves its own page).
|
|
76
|
+
21. `npm run test:clock` — installs a fake clock and checks that a navigation and a press still settle as soon as the page is quiet, and that advancing a clock nobody installed prints a note (it serves its own page).
|
|
77
|
+
22. `npm run test:settle-verbs` — runs a project's own steps whose words carry a page-changing verb (add, log out, enter, open, delete, save) and checks that the settle after each one waited for the page; `wait for ajax to finish` in lower case is a step (it serves its own page).
|
|
78
|
+
23. `npm start &` — backgrounds the fixture server.
|
|
79
|
+
24. Waits until the server answers on port 8080 (30 seconds at most).
|
|
80
|
+
25. `npm run test:must-fail` — runs the scenarios of `tests/must-fail/`, which must fail, and checks that each fails at its last step with the message it announces, the address of the page and a hint.
|
|
81
|
+
26. `npm test`.
|
|
82
|
+
27. `npm run test:scaffold` — scaffolds a new project, runs its first `npm test` and one feature named on the command line (see the notes below).
|
|
83
|
+
28. `actions/upload-artifact@v7` — uploads the report, screenshots and videos, on success and on failure.
|
|
57
84
|
|
|
58
85
|
`FORCE_COLOR=1` is set on the job so cucumber-js v10 emits ANSI colours.
|
|
59
86
|
|
|
@@ -69,24 +96,32 @@ The badge follows the workflow file name and branch — no extra setup.
|
|
|
69
96
|
|
|
70
97
|
### Reports
|
|
71
98
|
|
|
72
|
-
|
|
99
|
+
Every run, passed or failed, uploads one artifact named `cucumber-report`, kept 14 days (`run page → Artifacts`):
|
|
73
100
|
|
|
74
101
|
```yaml
|
|
75
|
-
- uses: actions/upload-artifact@
|
|
102
|
+
- uses: actions/upload-artifact@v7
|
|
76
103
|
if: always()
|
|
77
104
|
with:
|
|
78
105
|
name: cucumber-report
|
|
79
106
|
path: |
|
|
80
|
-
tests/reports/
|
|
81
|
-
tests/reports/cucumber_report.pdf
|
|
82
|
-
tests/reports/cucumber_report.json
|
|
107
|
+
tests/reports/
|
|
83
108
|
screenshots/
|
|
109
|
+
videos/
|
|
110
|
+
retention-days: 14
|
|
111
|
+
if-no-files-found: ignore
|
|
84
112
|
```
|
|
85
113
|
|
|
114
|
+
- `tests/reports/` — `cucumber_report.json` and the HTML report `cucumber_report.html`.
|
|
115
|
+
- `screenshots/` — the screenshot of each failed step (`failed_*.png`) and the ones the screenshot steps save.
|
|
116
|
+
- `videos/` — the recordings, when `WEBSHIP_VIDEO` or a `@video` tag turns recording on.
|
|
117
|
+
|
|
118
|
+
For a PDF next to the HTML report, add `npx generate-reports --format pdf` as a step with `if: always()` before the upload.
|
|
119
|
+
|
|
86
120
|
### Notes
|
|
87
121
|
|
|
88
|
-
- A deprecation warning fires on every run: `actions/checkout@v3` and `actions/setup-node@v3` use Node 20 internally; GitHub will force Node 24 on 2026-06-02. Bump to `@v4` (which targets Node 24) before then. We left them at `@v3` for now so existing forks do not need to rewrite YAML.
|
|
89
122
|
- The fixture server is a tiny `http-server` static site on port 8080 — no Docker daemon required on the runner.
|
|
123
|
+
- After the suite, `npm run test:scaffold` scaffolds a new project in a temporary folder the way the post-install does after `npm init -y`, checks its files and scripts, and runs its first `npm test` against the `examples/` pages.
|
|
124
|
+
- The repository tracks no `package-lock.json`, so the npm cache is keyed on `package.json` (`cache-dependency-path`).
|
|
90
125
|
- For per-browser matrix: add `strategy.matrix.browser: [chromium, firefox, webkit]` + `BROWSER: ${{ matrix.browser }}` to the env block, and replace `chromium` in the playwright install line with `${{ matrix.browser }}`.
|
|
91
126
|
|
|
92
127
|
---
|
|
@@ -95,7 +130,7 @@ The current workflow does not upload artefacts. To add them, append:
|
|
|
95
130
|
|
|
96
131
|
**File**: `.gitlab-ci.yml`.
|
|
97
132
|
|
|
98
|
-
GitLab's first-party CI/CD
|
|
133
|
+
GitLab's first-party CI/CD, on GitLab.com's shared runners or on runners of your own.
|
|
99
134
|
|
|
100
135
|
### Setup steps — open account + connect repo
|
|
101
136
|
|
|
@@ -107,12 +142,13 @@ GitLab's first-party CI/CD. #3 by adoption (~19%). Free tier on GitLab.com: 400
|
|
|
107
142
|
|
|
108
143
|
### Pipeline contents
|
|
109
144
|
|
|
110
|
-
Single job `test` running on the `node:
|
|
145
|
+
Single job `webship-js-test` running on the `node:24` image:
|
|
111
146
|
|
|
112
147
|
1. `npm install`.
|
|
113
|
-
2. `npx playwright install --with-deps chromium
|
|
148
|
+
2. `npx playwright install --with-deps chromium` (installs the system libraries Chromium needs, nothing else).
|
|
114
149
|
3. `npm start &` + `sleep 3`.
|
|
115
150
|
4. `npm test`.
|
|
151
|
+
5. Keeps `tests/reports/`, `screenshots/` and `videos/` as the `cucumber-report` artifact for 14 days, passed or failed.
|
|
116
152
|
|
|
117
153
|
`FORCE_COLOR: "1"` is set in the job's `variables:` block.
|
|
118
154
|
|
|
@@ -128,27 +164,24 @@ The `job=` query param targets a specific job's status; drop it to show the whol
|
|
|
128
164
|
|
|
129
165
|
### Reports
|
|
130
166
|
|
|
131
|
-
|
|
167
|
+
The job keeps its report, passed or failed:
|
|
132
168
|
|
|
133
169
|
```yaml
|
|
134
|
-
test:
|
|
135
170
|
artifacts:
|
|
171
|
+
name: cucumber-report
|
|
136
172
|
when: always
|
|
137
|
-
expire_in:
|
|
173
|
+
expire_in: 14 days
|
|
138
174
|
paths:
|
|
139
|
-
- tests/reports/
|
|
140
|
-
- tests/reports/cucumber_report.pdf
|
|
141
|
-
- tests/reports/cucumber_report.json
|
|
175
|
+
- tests/reports/
|
|
142
176
|
- screenshots/
|
|
143
|
-
|
|
144
|
-
junit: tests/reports/junit.xml # if you add --format junit:
|
|
177
|
+
- videos/
|
|
145
178
|
```
|
|
146
179
|
|
|
147
|
-
|
|
180
|
+
Download it from the job page. For test results on the merge-request page, add `reports: junit: tests/reports/junit.xml` to the block and a `--format junit:tests/reports/junit.xml` formatter to the run.
|
|
148
181
|
|
|
149
182
|
### Notes
|
|
150
183
|
|
|
151
|
-
- The `node:
|
|
184
|
+
- The `node:24` Docker image carries no browser: `--with-deps` installs Chromium and its system libraries on each run. Use `cache:` keyed on a lockfile to skip the npm part on reruns.
|
|
152
185
|
- Merge-request pipelines (`workflow.rules`) trigger by default; add `rules:` to scope to specific branches.
|
|
153
186
|
- For self-hosted runners on macOS / Windows, set `tags:` on the job and on the runner registration so the right runner picks up the work.
|
|
154
187
|
- Per-browser matrix: use `parallel.matrix` with `BROWSER: [chromium, firefox, webkit]`.
|
|
@@ -159,7 +192,7 @@ GitLab renders JUnit XML inline on the merge-request page.
|
|
|
159
192
|
|
|
160
193
|
**File**: `bitbucket-pipelines.yml`.
|
|
161
194
|
|
|
162
|
-
Atlassian Bitbucket's native CI/CD
|
|
195
|
+
Atlassian Bitbucket's native CI/CD, linked tightly to Jira / Confluence / Trello, so handy in Atlassian-stack shops.
|
|
163
196
|
|
|
164
197
|
### Setup steps — open account + connect repo
|
|
165
198
|
|
|
@@ -167,27 +200,34 @@ Atlassian Bitbucket's native CI/CD. Free tier: 50 build-minutes per month for pr
|
|
|
167
200
|
2. **Create a workspace** (e.g. `webshipco`). Required — every repo lives inside a workspace.
|
|
168
201
|
3. **Create or import the repo**. Bitbucket can import from GitHub directly.
|
|
169
202
|
4. **Enable Pipelines**: `repo → Repository settings → Pipelines → Settings → Enable Pipelines`. Toggle is off by default until you read & accept the runner terms.
|
|
170
|
-
5. **(Optional) Self-hosted runners**: `Repository settings → Pipelines → Runners → Add runner`.
|
|
203
|
+
5. **(Optional) Self-hosted runners**: `Repository settings → Pipelines → Runners → Add runner`.
|
|
171
204
|
|
|
172
205
|
### Pipeline contents
|
|
173
206
|
|
|
174
207
|
`bitbucket-pipelines.yml` defines a single default pipeline:
|
|
175
208
|
|
|
176
209
|
```yaml
|
|
177
|
-
image: node:
|
|
210
|
+
image: node:24
|
|
211
|
+
|
|
178
212
|
pipelines:
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
213
|
+
default:
|
|
214
|
+
- step:
|
|
215
|
+
name: Install and Test
|
|
216
|
+
caches:
|
|
217
|
+
- node
|
|
218
|
+
script:
|
|
219
|
+
- npm install
|
|
220
|
+
- npx playwright install --with-deps chromium
|
|
221
|
+
- npm start &
|
|
222
|
+
- sleep 3
|
|
223
|
+
- FORCE_COLOR=1 npm test
|
|
224
|
+
artifacts:
|
|
225
|
+
- tests/reports/**
|
|
226
|
+
- screenshots/**
|
|
227
|
+
- videos/**
|
|
188
228
|
```
|
|
189
229
|
|
|
190
|
-
`FORCE_COLOR=1` is set
|
|
230
|
+
`FORCE_COLOR=1` is set on the `npm test` line.
|
|
191
231
|
|
|
192
232
|
### Badge
|
|
193
233
|
|
|
@@ -201,25 +241,12 @@ Uses shields.io as a proxy because Bitbucket does not expose a native SVG endpoi
|
|
|
201
241
|
|
|
202
242
|
### Reports
|
|
203
243
|
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
```yaml
|
|
207
|
-
- step:
|
|
208
|
-
name: test
|
|
209
|
-
script: [ npm test ]
|
|
210
|
-
artifacts:
|
|
211
|
-
- tests/reports/cucumber_report.html
|
|
212
|
-
- tests/reports/cucumber_report.pdf
|
|
213
|
-
- tests/reports/cucumber_report.json
|
|
214
|
-
- screenshots/**
|
|
215
|
-
```
|
|
216
|
-
|
|
217
|
-
Artefacts live for 14 days on the free tier, 30 days on paid plans. Download from the build page.
|
|
244
|
+
The step's `artifacts:` keep `tests/reports/`, `screenshots/` and `videos/`; Bitbucket captures artifacts whether the step passed or failed. Download them from the build page; how long they are kept depends on the plan (see the Bitbucket docs).
|
|
218
245
|
|
|
219
246
|
### Notes
|
|
220
247
|
|
|
221
|
-
-
|
|
222
|
-
- `pipelines.
|
|
248
|
+
- `size: 2x` on the step gives it twice the memory and CPU, for a slower site or a bigger suite.
|
|
249
|
+
- `pipelines.default` runs on every branch; use `pipelines.branches.<name>` to scope it, and `pull-requests:` for PR builds.
|
|
223
250
|
- Atlassian's Pipes (`pipe:`) catalogue covers Slack notifications, AWS deploys, Jira issue transitions, etc. — drop them into the `script:` section as named steps.
|
|
224
251
|
- Per-browser matrix: define three steps under `parallel:` with `BROWSER` env variations.
|
|
225
252
|
|
|
@@ -229,24 +256,25 @@ Artefacts live for 14 days on the free tier, 30 days on paid plans. Download fro
|
|
|
229
256
|
|
|
230
257
|
**File**: `.circleci/config.yml`.
|
|
231
258
|
|
|
232
|
-
|
|
259
|
+
Hosted CI with Docker executors and a long-standing open-source following.
|
|
233
260
|
|
|
234
261
|
### Setup steps — open account + connect repo
|
|
235
262
|
|
|
236
263
|
1. **Sign up**: <https://circleci.com/signup/> → "Sign Up with GitHub" → authorise the CircleCI GitHub App on `webship/webship-js`.
|
|
237
264
|
2. **Choose an organisation** (your GitHub org / user). CircleCI mirrors the GitHub permissions model.
|
|
238
265
|
3. **Set up the project**: dashboard → `Projects → Set up project` → pick `webship-js` → "Use the .circleci/config.yml in my repo" → branch `2.0.x`.
|
|
239
|
-
4. **Pick a plan**:
|
|
266
|
+
4. **Pick a plan**: see CircleCI's pricing page for what the free plan covers today.
|
|
240
267
|
|
|
241
268
|
### Pipeline contents
|
|
242
269
|
|
|
243
|
-
`.circleci/config.yml` uses the `cimg/node:
|
|
270
|
+
`.circleci/config.yml` uses the `cimg/node:24` Docker executor (Node 24 + npm preinstalled, the newest 24.x). Steps:
|
|
244
271
|
|
|
245
272
|
1. `checkout`.
|
|
246
273
|
2. `npm install`.
|
|
247
274
|
3. `npx playwright install --with-deps chromium`.
|
|
248
275
|
4. `npm start &` + `sleep 3`.
|
|
249
276
|
5. `npm test`.
|
|
277
|
+
6. `store_artifacts` for `tests/reports`, `screenshots` and `videos`.
|
|
250
278
|
|
|
251
279
|
`FORCE_COLOR: "1"` is exported in the job environment. Filter `branches.only: /^2.0.x/` keeps feature-branch pushes off the queue.
|
|
252
280
|
|
|
@@ -262,7 +290,7 @@ Already shipped in `README.md`:
|
|
|
262
290
|
|
|
263
291
|
### Reports + screenshots
|
|
264
292
|
|
|
265
|
-
|
|
293
|
+
The job ends with three `store_artifacts` steps:
|
|
266
294
|
|
|
267
295
|
```yaml
|
|
268
296
|
- store_artifacts:
|
|
@@ -270,26 +298,40 @@ Add `store_artifacts` to the test step:
|
|
|
270
298
|
destination: cucumber-report
|
|
271
299
|
- store_artifacts:
|
|
272
300
|
path: screenshots
|
|
273
|
-
destination:
|
|
274
|
-
|
|
301
|
+
destination: screenshots
|
|
302
|
+
- store_artifacts:
|
|
303
|
+
path: videos
|
|
304
|
+
destination: videos
|
|
275
305
|
```
|
|
276
306
|
|
|
277
|
-
|
|
307
|
+
`store_artifacts` runs even when an earlier step failed, so a failed run keeps its report too. The files are links on the job's Artifacts tab.
|
|
278
308
|
|
|
279
309
|
### Notes
|
|
280
310
|
|
|
281
|
-
- `cimg/node:
|
|
311
|
+
- `cimg/node:24` is CircleCI's "convenience image" for the newest Node 24.x; pin a minor (`cimg/node:24.22`) for repeatable runs.
|
|
282
312
|
- The earlier config used `cimg/base:stable-20.04` with manual `apt upgrade` and a NodeSource curl-pipe. That path hit 200-package apt upgrades and PPA 503s (issue #280) and was replaced with the slim form documented above.
|
|
283
|
-
- Docker layer caching is
|
|
313
|
+
- Docker layer caching is a paid feature — toggle it in the executor block with `docker_layer_caching: true` on a plan that has it.
|
|
284
314
|
- Per-browser matrix: use a `matrix` block under `jobs:` or duplicate the job per browser.
|
|
285
315
|
|
|
286
316
|
---
|
|
287
317
|
|
|
318
|
+
## Travis CI
|
|
319
|
+
|
|
320
|
+
**File**: `.travis.yml`.
|
|
321
|
+
|
|
322
|
+
Hosted CI on travis-ci.com. The shipped file runs Node 24 on Ubuntu 22.04 (`dist: jammy`) for the `2.0.x` branch and its tags: it caches npm and `~/.cache/ms-playwright`, installs Chromium with `--with-deps`, starts the fixture server, checks that it answers and runs `npm test`.
|
|
323
|
+
|
|
324
|
+
### Reports
|
|
325
|
+
|
|
326
|
+
Travis CI keeps no build artifacts of its own: the report, the screenshots and the videos (`tests/reports/`, `screenshots/`, `videos/`) stay on the build machine and are lost with it. To keep them, upload them to storage of your own in an `after_script` step (an S3 bucket, for example), or run the suite on another provider of this page.
|
|
327
|
+
|
|
328
|
+
---
|
|
329
|
+
|
|
288
330
|
## Jenkins
|
|
289
331
|
|
|
290
332
|
**File**: `Jenkinsfile` (declarative pipeline, at repo root).
|
|
291
333
|
|
|
292
|
-
Jenkins has no SaaS — you run a controller yourself. The shipped `Jenkinsfile` uses the official `mcr.microsoft.com/playwright:v1.
|
|
334
|
+
Jenkins has no SaaS — you run a controller yourself. The shipped `Jenkinsfile` uses the official `mcr.microsoft.com/playwright:v1.65.1-noble` Docker image, of the Playwright version the install brings, so chromium and every apt dependency are already inside.
|
|
293
335
|
|
|
294
336
|
### Setup steps
|
|
295
337
|
|
|
@@ -313,7 +355,7 @@ Jenkins has no SaaS — you run a controller yourself. The shipped `Jenkinsfile`
|
|
|
313
355
|
|
|
314
356
|
### Reports + screenshots
|
|
315
357
|
|
|
316
|
-
The `post { always }` block archives `tests/reports
|
|
358
|
+
The `post { always }` block archives `tests/reports/**`, `screenshots/**` and `videos/**`, passed or failed. Browse them per build in the Jenkins UI under "Build Artifacts".
|
|
317
359
|
|
|
318
360
|
### Badge
|
|
319
361
|
|
|
@@ -322,7 +364,7 @@ Jenkins has no public badge endpoint by default — the controller is usually pr
|
|
|
322
364
|
### Notes
|
|
323
365
|
|
|
324
366
|
- `args '-u root:root'` runs the container as root so `npm install` and `npx playwright install --with-deps` can touch system paths. Switch to a non-root image if you tighten security.
|
|
325
|
-
- The Playwright
|
|
367
|
+
- The Playwright image already has the browser of its own Playwright version and every browser dep: `npx playwright install --with-deps chromium` is a no-op while the installed Playwright matches the image, and downloads the newer browser when it does not.
|
|
326
368
|
- Need parallelism? Wrap the `Test` stage in `parallel { stage('chromium') {...}; stage('firefox') {...} }` and switch the image to `mcr.microsoft.com/playwright` (latest) which includes all three browsers.
|
|
327
369
|
|
|
328
370
|
---
|
|
@@ -331,19 +373,16 @@ Jenkins has no public badge endpoint by default — the controller is usually pr
|
|
|
331
373
|
|
|
332
374
|
**File**: `azure-pipelines.yml` at repo root.
|
|
333
375
|
|
|
334
|
-
Microsoft's hosted CI
|
|
376
|
+
Microsoft's hosted CI. Microsoft-hosted `ubuntu-latest` agents already have Node + most browser deps; the pipeline installs Node 24 (`NodeTool@0`) and still calls `--with-deps` for safety.
|
|
335
377
|
|
|
336
378
|
### Setup steps — open account + connect repo
|
|
337
379
|
|
|
338
380
|
1. **Microsoft account**: if you do not already have one, create at <https://account.microsoft.com/>. Free.
|
|
339
381
|
2. **Sign in to Azure DevOps**: <https://dev.azure.com> → "Start free with GitHub" *or* sign in with the Microsoft account.
|
|
340
382
|
3. **Create an Organization**. Pick a unique slug (used in the URL `https://dev.azure.com/<org>/`). Example: `webship`.
|
|
341
|
-
4. **Create a Project**. Name `webship-js`.
|
|
342
|
-
- **Public** for open source — unlocks the free OSS tier (10 parallel jobs, unlimited minutes).
|
|
343
|
-
- **Private** otherwise (1 free parallel job, 1,800 min/mo).
|
|
383
|
+
4. **Create a Project**. Name `webship-js`. The free parallel jobs and minutes depend on the visibility and on Microsoft's current terms: see the Azure DevOps "parallel jobs" page.
|
|
344
384
|
5. **Pipelines → Create Pipeline → GitHub**. Authorise the **Azure Pipelines** GitHub app, scope it to `webship/webship-js`.
|
|
345
385
|
6. **Configure your pipeline**: choose **"Existing Azure Pipelines YAML file"** → branch `2.0.x` → path `/azure-pipelines.yml` → **Continue → Run**.
|
|
346
|
-
7. **Request OSS free parallelism** (only for public projects): `Organization settings → Billing → Public project parallelism request`. Microsoft enables it manually within a day or two.
|
|
347
386
|
|
|
348
387
|
### Badge
|
|
349
388
|
|
|
@@ -357,10 +396,11 @@ Replace `<org>`, `<project>`, `<id>`. Paste into `README.md` next to the other b
|
|
|
357
396
|
|
|
358
397
|
### Reports
|
|
359
398
|
|
|
360
|
-
The pipeline publishes
|
|
399
|
+
The pipeline publishes three build artefacts, passed or failed:
|
|
361
400
|
|
|
362
|
-
- `cucumber-report` — `tests/reports
|
|
363
|
-
- `
|
|
401
|
+
- `cucumber-report` — `tests/reports/` (the HTML and JSON report, and the PDF when it is generated)
|
|
402
|
+
- `screenshots` — every `screenshots/**/*` capture
|
|
403
|
+
- `videos` — the recordings, when recording is on (`continueOnError`: a run without videos is not a failure)
|
|
364
404
|
|
|
365
405
|
Browse via **Pipelines → Run → Artifacts → 1 published**.
|
|
366
406
|
|
|
@@ -386,11 +426,11 @@ Two `Cache@2` tasks are configured:
|
|
|
386
426
|
|
|
387
427
|
**File**: `buildspec.yml` at repo root.
|
|
388
428
|
|
|
389
|
-
AWS-managed build service.
|
|
429
|
+
AWS-managed build service. For the free tier and the price of a build minute, see the CodeBuild pricing page.
|
|
390
430
|
|
|
391
431
|
### Setup steps — open account + create project
|
|
392
432
|
|
|
393
|
-
1. **Create an AWS account** at <https://aws.amazon.com/>. Requires a credit card, phone verification, and an email confirmation.
|
|
433
|
+
1. **Create an AWS account** at <https://aws.amazon.com/>. Requires a credit card, phone verification, and an email confirmation.
|
|
394
434
|
2. **Sign in to the AWS Console** → switch to a region close to you (e.g. `us-east-1`, `eu-west-1`).
|
|
395
435
|
3. **IAM → Roles → Create role**:
|
|
396
436
|
- Trusted entity: **AWS service** → **CodeBuild**.
|
|
@@ -401,7 +441,7 @@ AWS-managed build service. Free tier: 100 build-minutes per month on the `genera
|
|
|
401
441
|
- Project name: `webship-js`.
|
|
402
442
|
- Source provider: **GitHub** → "Connect using OAuth" → authorise the AWS CodeBuild GitHub app → pick `webship/webship-js` → branch `2.0.x`.
|
|
403
443
|
- Webhook: tick "Rebuild every time a code change is pushed".
|
|
404
|
-
- Environment: **Managed image** → operating system **Ubuntu** → runtime **Standard** → image `aws/codebuild/standard:
|
|
444
|
+
- Environment: **Managed image** → operating system **Ubuntu** → runtime **Standard** → image `aws/codebuild/standard:8.0` (its Node runtimes include the `nodejs: 24` of `buildspec.yml`) → privileged unchecked.
|
|
405
445
|
- Service role: pick the existing `webship-js-codebuild-role` you just created.
|
|
406
446
|
- Build spec: **Use a buildspec file** → buildspec name `buildspec.yml` (default).
|
|
407
447
|
- Artifacts: **Amazon S3** → bucket `webship-js-reports` → name `webship-js-report-$(date)`.
|
|
@@ -420,13 +460,12 @@ CodeBuild has no native badge endpoint. Two common patterns:
|
|
|
420
460
|
`buildspec.yml` declares artefacts:
|
|
421
461
|
|
|
422
462
|
```
|
|
423
|
-
tests/reports
|
|
424
|
-
tests/reports/cucumber_report.pdf
|
|
425
|
-
tests/reports/cucumber_report.json
|
|
463
|
+
tests/reports/**/*
|
|
426
464
|
screenshots/**/*
|
|
465
|
+
videos/**/*
|
|
427
466
|
```
|
|
428
467
|
|
|
429
|
-
They land in the configured S3 bucket. Grant pre-signed URLs or make the bucket public if you want links you can share.
|
|
468
|
+
CodeBuild uploads them after the build phase, passed or failed. They land in the configured S3 bucket. Grant pre-signed URLs or make the bucket public if you want links you can share.
|
|
430
469
|
|
|
431
470
|
### Caching
|
|
432
471
|
|
|
@@ -434,7 +473,6 @@ They land in the configured S3 bucket. Grant pre-signed URLs or make the bucket
|
|
|
434
473
|
|
|
435
474
|
### Notes
|
|
436
475
|
|
|
437
|
-
- 9-min suite × $0.005/min ≈ $0.045 per build past the free tier — cheap unless you push hundreds of builds.
|
|
438
476
|
- For multi-AZ or multi-region resilience, place the buildspec in `aws-codebuild/buildspec.yml` and point the CodeBuild project at that path — the file is just a relative source path.
|
|
439
477
|
- CodeBuild can run inside a VPC if your dev server needs private network access — toggle in **VPC configuration** before starting the first build.
|
|
440
478
|
|
|
@@ -444,11 +482,11 @@ They land in the configured S3 bucket. Grant pre-signed URLs or make the bucket
|
|
|
444
482
|
|
|
445
483
|
**File**: `cloudbuild.yaml` at repo root.
|
|
446
484
|
|
|
447
|
-
Google's managed CI inside GCP.
|
|
485
|
+
Google's managed CI inside GCP. For the free tier and the prices, see the Cloud Build pricing page.
|
|
448
486
|
|
|
449
487
|
### Setup steps — open account + connect repo
|
|
450
488
|
|
|
451
|
-
1. **Create a Google Cloud account** at <https://console.cloud.google.com/>. Sign in with a Google account
|
|
489
|
+
1. **Create a Google Cloud account** at <https://console.cloud.google.com/>. Sign in with a Google account and accept the terms (a card is required for verification).
|
|
452
490
|
2. **Create a Project**: top bar → "Select a project" → "New project" → name `webship-js`. Note the project ID — Cloud Build references it implicitly.
|
|
453
491
|
3. **Enable billing** on the project: `Billing → Link a billing account`. Required even for free-tier-only usage.
|
|
454
492
|
4. **Enable the Cloud Build API**: `APIs & Services → Library → Cloud Build API → Enable`. Wait ~30 s.
|
|
@@ -474,19 +512,19 @@ Cloud Build has no native badge. Two workarounds:
|
|
|
474
512
|
|
|
475
513
|
### Reports
|
|
476
514
|
|
|
477
|
-
|
|
515
|
+
Cloud Build uploads the files of an `artifacts:` block only when every step succeeds, so the pipeline keeps its report in steps of its own: the test step records a failure instead of stopping the build, `keep-report` copies `tests/reports/`, `screenshots/` and `videos/` to `gs://webship-js-reports/<BUILD_ID>/`, and `test-result` then fails the build when the suite failed. Browse via the Cloud Storage UI or share pre-signed URLs.
|
|
478
516
|
|
|
479
517
|
### Caching
|
|
480
518
|
|
|
481
519
|
Cloud Build does not have a first-class cache step. Common patterns:
|
|
482
520
|
|
|
483
521
|
1. **Kaniko image cache** for Docker-based builds (not what this pipeline does).
|
|
484
|
-
2. **Volume cache via `dir` + GCS sync**: copy `node_modules` and `~/.cache/ms-playwright` to GCS at the end of each build, restore at the start. Adds two extra steps
|
|
522
|
+
2. **Volume cache via `dir` + GCS sync**: copy `node_modules` and `~/.cache/ms-playwright` to GCS at the end of each build, restore at the start. Adds two extra steps and shortens a rerun.
|
|
485
523
|
|
|
486
524
|
### Notes
|
|
487
525
|
|
|
488
|
-
- `options.logging: CLOUD_LOGGING_ONLY` skips the legacy log bucket
|
|
489
|
-
- `timeout: 1800s`
|
|
526
|
+
- `options.logging: CLOUD_LOGGING_ONLY` skips the legacy log bucket and avoids the IAM warning Cloud Build prints by default.
|
|
527
|
+
- `timeout: 1800s` is a 30-minute cap on the whole build.
|
|
490
528
|
- `_BRANCH_NAME` and `_ARTIFACT_BUCKET` are substitution variables — override per-trigger to point at a different bucket without forking the YAML.
|
|
491
529
|
- For per-browser parallelism, replace the single `step` with three steps, each setting `BROWSER=chromium/firefox/webkit` — `--with-deps` will fetch the matching browser deps inside the container.
|
|
492
530
|
|
|
@@ -496,12 +534,12 @@ Cloud Build does not have a first-class cache step. Common patterns:
|
|
|
496
534
|
|
|
497
535
|
**Files**: `.teamcity/settings.kts` + `.teamcity/pom.xml` at repo root.
|
|
498
536
|
|
|
499
|
-
JetBrains' CI server, available as Cloud (SaaS, no install) or self-hosted.
|
|
537
|
+
JetBrains' CI server, available as Cloud (SaaS, no install) or self-hosted. For the plans of TeamCity Cloud, see the TeamCity pricing page.
|
|
500
538
|
|
|
501
539
|
### Setup steps — open account + connect repo
|
|
502
540
|
|
|
503
541
|
1. **JetBrains account**: <https://account.jetbrains.com/> → sign up (free, OAuth via GitHub / Google / Microsoft also works).
|
|
504
|
-
2. **Start TeamCity Cloud
|
|
542
|
+
2. **Start TeamCity Cloud**: <https://www.jetbrains.com/teamcity/cloud/> → confirm the JetBrains-issued instance URL (e.g. `https://webship.teamcity.com`).
|
|
505
543
|
3. **Create a Project**: top bar `Administration → Projects → Create project`. Pick "From a repository URL" → paste the GitHub HTTPS URL → "Proceed". TeamCity creates a VCS root.
|
|
506
544
|
4. **Authorise GitHub OAuth**: TeamCity asks for GitHub credentials the first time it touches the repo. Use a Personal Access Token with `repo` scope or the OAuth flow.
|
|
507
545
|
5. **Enable Versioned Settings**: open the project → `Versioned Settings → Synchronization enabled → Kotlin DSL`. TeamCity will offer to scan the repo for `.teamcity/settings.kts` — point it at the file we shipped. Commit any generated config UUIDs back to the repo so the next clone is identical.
|
|
@@ -517,11 +555,11 @@ TeamCity has no native public badge. Common patterns:
|
|
|
517
555
|
|
|
518
556
|
### Reports
|
|
519
557
|
|
|
520
|
-
`artifactRules` archives `tests/reports
|
|
558
|
+
`artifactRules` archives `tests/reports/**`, `screenshots/**` and `videos/**`, and `publishArtifacts = PublishMode.ALWAYS` keeps them for a failed build too. Browse per build under `Artifacts` in the TeamCity UI.
|
|
521
559
|
|
|
522
560
|
### Notes
|
|
523
561
|
|
|
524
|
-
- `dockerImage = "mcr.microsoft.com/playwright:v1.
|
|
562
|
+
- `dockerImage = "mcr.microsoft.com/playwright:v1.65.1-noble"` runs the script inside the official Playwright image of the version the install brings — chromium and every apt dep are already there.
|
|
525
563
|
- `dockerRunParameters = "--user root:root --network host"` keeps the steps simple (no `sudo`, the local fixture server on `localhost:8080` is reachable without port mapping).
|
|
526
564
|
- `pom.xml` exists only so IntelliJ can resolve `configs-dsl-kotlin` types and give completion / inspections in the IDE. TeamCity Cloud ignores it at build time.
|
|
527
565
|
- Kotlin DSL version `2024.03` matches TeamCity Cloud at time of writing — TeamCity warns and offers an auto-upgrade when the server version moves past it.
|
|
@@ -533,12 +571,12 @@ TeamCity has no native public badge. Common patterns:
|
|
|
533
571
|
|
|
534
572
|
**File**: `.semaphore/semaphore.yml`.
|
|
535
573
|
|
|
536
|
-
Hosted CI with a developer-friendly UI and
|
|
574
|
+
Hosted CI with a developer-friendly UI and fast cold-build queues. For the plans and their minutes, see the Semaphore pricing page.
|
|
537
575
|
|
|
538
576
|
### Setup steps — open account + connect repo
|
|
539
577
|
|
|
540
578
|
1. **Sign up**: <https://semaphoreci.com/> → "Sign up with GitHub" → authorise the Semaphore GitHub App on `webship/webship-js`.
|
|
541
|
-
2. **Pick a plan
|
|
579
|
+
2. **Pick a plan** (see the Semaphore pricing page for what each one covers).
|
|
542
580
|
3. **Create project**: dashboard → "Create new project" → pick `webship-js` → branch `2.0.x` (Semaphore can auto-detect other branches later).
|
|
543
581
|
4. **Choose pipeline source**: select "Customise the workflow" → "I have a configuration file in my repository" → point at `.semaphore/semaphore.yml`. Semaphore validates the YAML and shows a graph preview.
|
|
544
582
|
5. **Add the project** → Semaphore queues the first run on a fresh worker. The `--with-deps` step pulls chromium apt deps; the cache stores `node_modules` + `~/.cache/ms-playwright` so reruns are fast.
|
|
@@ -555,7 +593,7 @@ Replace `<org>` with the organisation slug you picked at signup. Paste in `READM
|
|
|
555
593
|
|
|
556
594
|
### Reports
|
|
557
595
|
|
|
558
|
-
`epilogue.always` uploads `
|
|
596
|
+
`epilogue.always` uploads `tests/reports/`, `screenshots/` and `videos/` to the per-job artefact store (`Artifacts` tab in the UI), passed or failed. `--expire-in 30d` purges them after a month — raise it for longer retention.
|
|
559
597
|
|
|
560
598
|
### Caching
|
|
561
599
|
|
|
@@ -570,11 +608,11 @@ Two cache slots are populated in the `Install` block and restored in `prologue`
|
|
|
570
608
|
|
|
571
609
|
### Notes
|
|
572
610
|
|
|
573
|
-
- `sem-version node
|
|
574
|
-
- `agent.machine.type: e1-standard-2` is
|
|
611
|
+
- `sem-version node 24` is Semaphore's runtime switcher — preinstalled on every `os_image: ubuntu2204` agent.
|
|
612
|
+
- `agent.machine.type: e1-standard-2` is a small machine; a bigger one (`e2-standard-4`) shortens the run.
|
|
575
613
|
- `nohup npm start > /tmp/srv.log 2>&1 &` keeps the fixture server alive after the `commands` block exits; `curl -sf` is the same ready-probe as the other lanes.
|
|
576
614
|
- Add a `promotions` block (e.g. tag release, deploy to staging) once the test pipeline is stable — see Semaphore docs for the `auto_promote` shape.
|
|
577
|
-
- For per-browser matrix, add three jobs under the `Test` task and parameterise via env (`BROWSER=chromium` etc.) — Semaphore runs them in parallel within the
|
|
615
|
+
- For per-browser matrix, add three jobs under the `Test` task and parameterise via env (`BROWSER=chromium` etc.) — Semaphore runs them in parallel within the concurrency of your plan.
|
|
578
616
|
|
|
579
617
|
---
|
|
580
618
|
|
|
@@ -582,7 +620,7 @@ Two cache slots are populated in the `Install` block and restored in `prologue`
|
|
|
582
620
|
|
|
583
621
|
**File**: `.drone.yml` at repo root.
|
|
584
622
|
|
|
585
|
-
Open-source CI/CD that pipelines via simple YAML and runs steps in Docker containers.
|
|
623
|
+
Open-source CI/CD that pipelines via simple YAML and runs steps in Docker containers. The open-source path is **self-host**. The shipped `.drone.yml` is provider-agnostic — anyone running a Drone server can enable the repo and go.
|
|
586
624
|
|
|
587
625
|
### Setup steps — open account + bring up a server
|
|
588
626
|
|
|
@@ -620,9 +658,9 @@ There is **no SaaS sign-up** for the free version. You have two routes:
|
|
|
620
658
|
```
|
|
621
659
|
6. Visit `https://ci.example.com` → sign in with GitHub → **Activate** `webship/webship-js`. Drone reads `.drone.yml` on the next push.
|
|
622
660
|
|
|
623
|
-
#### Route B — Drone
|
|
661
|
+
#### Route B — hosted Drone
|
|
624
662
|
|
|
625
|
-
<https://drone.io>
|
|
663
|
+
<https://drone.io> lists the hosted offers. Point it at the same `.drone.yml`; no further config differences.
|
|
626
664
|
|
|
627
665
|
### Badge
|
|
628
666
|
|
|
@@ -634,7 +672,7 @@ Replace `ci.example.com` with your server hostname.
|
|
|
634
672
|
|
|
635
673
|
### Reports
|
|
636
674
|
|
|
637
|
-
The `archive-reports` step tars `tests/reports/`
|
|
675
|
+
The `archive-reports` step, passed or failed, tars `tests/reports/`, `screenshots/` and `videos/` into `/var/lib/drone/artifacts/cucumber-report-<build>.tar.gz` on the host. Serve the directory via a static file server (e.g. `python3 -m http.server` behind nginx) or copy to S3/GCS via an extra step.
|
|
638
676
|
|
|
639
677
|
### Notes
|
|
640
678
|
|
|
@@ -704,7 +742,7 @@ Adjust to your Forgejo host.
|
|
|
704
742
|
|
|
705
743
|
**File**: `.harness/webship-js-pipeline.yml`.
|
|
706
744
|
|
|
707
|
-
Harness ships an enterprise Software Delivery Platform; Harness CI is the build/test module.
|
|
745
|
+
Harness ships an enterprise Software Delivery Platform; Harness CI is the build/test module. For the free plan and its build credits, see the Harness pricing page.
|
|
708
746
|
|
|
709
747
|
### Setup steps — open account + connect repo
|
|
710
748
|
|
|
@@ -725,11 +763,11 @@ Harness ships an enterprise Software Delivery Platform; Harness CI is the build/
|
|
|
725
763
|
|
|
726
764
|
### Badge
|
|
727
765
|
|
|
728
|
-
|
|
766
|
+
Expose the status via the GitHub commit status that the Harness GitHub App posts automatically, or write a tiny shields.io endpoint backed by the Harness REST API (`GET /pipeline/api/pipelines/execution`).
|
|
729
767
|
|
|
730
768
|
### Reports
|
|
731
769
|
|
|
732
|
-
The shipped pipeline includes an optional `Upload report artefacts` step that uses the `plugins/s3` connector
|
|
770
|
+
The shipped pipeline includes an optional `Upload report artefacts` step (run whatever the stage status) that uses the `plugins/s3` connector to upload `tests/reports/`, `screenshots/` and `videos/`. Replace the secret IDs and bucket name; or rip the step out entirely and pull reports from Harness's built-in **CI Insights** tab (test logs are retained per-build).
|
|
733
771
|
|
|
734
772
|
### Notes
|
|
735
773
|
|
|
@@ -744,13 +782,13 @@ The shipped pipeline includes an optional `Upload report artefacts` step that us
|
|
|
744
782
|
|
|
745
783
|
**File**: `bamboo-specs/bamboo.yml`.
|
|
746
784
|
|
|
747
|
-
Atlassian's on-prem CI/CD
|
|
785
|
+
Atlassian's on-prem CI/CD: Atlassian offers Bamboo as **Data Center** only, self-hosted. The shipped YAML targets the official YAML Specs format introduced in Bamboo 7+.
|
|
748
786
|
|
|
749
787
|
### Setup steps — open account + connect repo
|
|
750
788
|
|
|
751
789
|
There is no Atlassian-hosted Bamboo any more. You bring your own server.
|
|
752
790
|
|
|
753
|
-
1. **
|
|
791
|
+
1. **Get a Bamboo Data Center licence** (or an evaluation licence) from Atlassian: <https://www.atlassian.com/software/bamboo>.
|
|
754
792
|
2. **Install Bamboo Data Center**:
|
|
755
793
|
- Recommended path: official Docker image `atlassian/bamboo` on a host with at least 4 GB RAM and a persistent volume.
|
|
756
794
|
- Database: PostgreSQL 13+ or MySQL 8+ is supported; SQLite for tiny installs.
|
|
@@ -758,7 +796,7 @@ There is no Atlassian-hosted Bamboo any more. You bring your own server.
|
|
|
758
796
|
3. **Sign in as admin** at `https://bamboo.example.com` → run the setup wizard → activate the licence.
|
|
759
797
|
4. **Link the GitHub repository**: `Bamboo administration → Linked repositories → Add → Git`. Use HTTPS + a Personal Access Token, or SSH with a deploy key. Name the linked repo `webship-js`.
|
|
760
798
|
5. **Enable Bamboo Specs scanning** on that repo: `Bamboo administration → Specs → Repositories → Add`. Bamboo reads `bamboo-specs/bamboo.yml` on every push, recreating plan + permissions from the file.
|
|
761
|
-
6. **Build agent**: at least one Docker-capable agent must be online. The plan runs inside `mcr.microsoft.com/playwright:v1.
|
|
799
|
+
6. **Build agent**: at least one Docker-capable agent must be online. The plan runs inside `mcr.microsoft.com/playwright:v1.65.1-noble`, so the host needs Docker installed and the Bamboo agent needs the Docker plugin enabled.
|
|
762
800
|
|
|
763
801
|
### Badge
|
|
764
802
|
|
|
@@ -769,12 +807,11 @@ Bamboo emits chat-style status webhooks (Slack, MS Teams) but no native badge im
|
|
|
769
807
|
|
|
770
808
|
### Reports
|
|
771
809
|
|
|
772
|
-
`Cucumber.artifacts` declares
|
|
810
|
+
`Cucumber.artifacts` declares three shared artefacts:
|
|
773
811
|
|
|
774
|
-
- `cucumber-report
|
|
775
|
-
- `
|
|
776
|
-
- `
|
|
777
|
-
- `screenshots` → `screenshots/**` (failure captures)
|
|
812
|
+
- `cucumber-report` → `tests/reports/**`
|
|
813
|
+
- `screenshots` → `screenshots/**`
|
|
814
|
+
- `videos` → `videos/**`
|
|
778
815
|
|
|
779
816
|
Browse per build under `Plan Result → Artifacts`. The `shared: true` flag makes them available to downstream plans and to the Bamboo REST API.
|
|
780
817
|
|
|
@@ -792,12 +829,12 @@ Browse per build under `Plan Result → Artifacts`. The `shared: true` flag make
|
|
|
792
829
|
|
|
793
830
|
**File**: `codefresh.yml` at repo root.
|
|
794
831
|
|
|
795
|
-
Kubernetes-native CI/CD. The Classic pipeline format reads `codefresh.yml` from the repo.
|
|
832
|
+
Kubernetes-native CI/CD. The Classic pipeline format reads `codefresh.yml` from the repo. For the plans and their build capacity, see the Codefresh pricing page.
|
|
796
833
|
|
|
797
834
|
### Setup steps — open account + connect repo
|
|
798
835
|
|
|
799
836
|
1. **Sign up**: <https://g.codefresh.io/signup> → "Sign up with GitHub" → authorise the Codefresh GitHub App on `webship/webship-js`.
|
|
800
|
-
2. **Pick a runtime**: dashboard prompts you to "Add Runtime" → pick **Codefresh Hosted** for the managed
|
|
837
|
+
2. **Pick a runtime**: dashboard prompts you to "Add Runtime" → pick **Codefresh Hosted** for the managed one. (Self-hosted runtimes via `cf-runtime install` are also available but out of scope here.)
|
|
801
838
|
3. **Create a pipeline**:
|
|
802
839
|
- `Pipelines → New Pipeline → From Git`.
|
|
803
840
|
- Repository `webship/webship-js`, branch `2.0.x`.
|
|
@@ -816,10 +853,10 @@ Replace `<account>` with the Codefresh account slug picked at signup and `<pipel
|
|
|
816
853
|
|
|
817
854
|
### Reports
|
|
818
855
|
|
|
819
|
-
The `upload_reports` step copies `
|
|
856
|
+
The `upload_reports` step, passed or failed, copies `tests/reports/`, `screenshots/` and `videos/` to `/codefresh/volume/cucumber-report/` — a Codefresh-managed shared volume that persists across pipeline runs. From there you can:
|
|
820
857
|
|
|
821
858
|
1. Mount the same volume in a follow-up step that uploads to S3/GCS.
|
|
822
|
-
2. Browse via the **Codefresh storage browser
|
|
859
|
+
2. Browse via the **Codefresh storage browser**, on the plans that have it.
|
|
823
860
|
3. Pipe to the built-in **Test Reports** feature by emitting JUnit XML — add a `--format junit:tests/reports/junit.xml` argument to the cucumber-js call.
|
|
824
861
|
|
|
825
862
|
### Notes
|
|
@@ -846,8 +883,8 @@ Octopus Deploy is a release-orchestration tool. It does not run unit / integrati
|
|
|
846
883
|
|
|
847
884
|
### Setup steps — open account
|
|
848
885
|
|
|
849
|
-
1. **Octopus Cloud** (SaaS): <https://octopus.com/start> → sign in with email or Google
|
|
850
|
-
2. **Octopus Server** (self-hosted): download the Windows / Linux installer from <https://octopus.com/downloads>.
|
|
886
|
+
1. **Octopus Cloud** (SaaS): <https://octopus.com/start> → sign in with email or Google. See the Octopus pricing page for the free and paid plans.
|
|
887
|
+
2. **Octopus Server** (self-hosted): download the Windows / Linux installer from <https://octopus.com/downloads>.
|
|
851
888
|
|
|
852
889
|
### Wire any CI lane into Octopus
|
|
853
890
|
|
|
@@ -876,4 +913,4 @@ Plug that snippet into any of the YAML files we ship — Octopus does not care w
|
|
|
876
913
|
|
|
877
914
|
## Codeship
|
|
878
915
|
|
|
879
|
-
**Service retired by CloudBees
|
|
916
|
+
**Service retired by CloudBees.** Do not add a Codeship config. Existing `codeship-steps.yml` / `codeship-services.yml` files on the internet are dead. If a tutorial references Codeship, treat it as Tier-1 advice for **Travis** or **CircleCI** instead — both ship the same `npm test` pattern.
|