@vardot/varbase-e2e 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +148 -0
- package/CLAUDE.md +544 -0
- package/README.md +22 -0
- package/bin/generate-reports.js +363 -0
- package/bin/init-varbase-e2e.js +330 -0
- package/bin/postinstall.js +26 -0
- package/cucumber.js +125 -0
- package/docs/00-quick-start.md +102 -0
- package/docs/01-getting-started.md +102 -0
- package/docs/02-bbr-smart-waits.md +77 -0
- package/docs/03-selector-registry.md +95 -0
- package/docs/04-step-reference.md +801 -0
- package/docs/05-web-first-assertions.md +91 -0
- package/docs/06-network-and-dialogs.md +83 -0
- package/docs/07-auth-state.md +75 -0
- package/docs/08-clock-mocking.md +59 -0
- package/docs/09-api-testing.md +114 -0
- package/docs/10-accessibility.md +167 -0
- package/docs/11-debugging.md +129 -0
- package/docs/12-ai-agent-guide.md +338 -0
- package/docs/13-faq.md +236 -0
- package/docs/14-recipes-cookbook.md +255 -0
- package/docs/15-tag-conventions.md +76 -0
- package/docs/16-ci-cd.md +90 -0
- package/docs/README.md +125 -0
- package/docs/advanced-screenshots/then-i-save-fullscreen-screenshot.md +36 -0
- package/docs/advanced-screenshots/then-i-save-width-x-height-screenshot.md +34 -0
- package/docs/advanced-screenshots/when-i-save-screenshot-name-filename.md +45 -0
- package/docs/advanced-screenshots.md +52 -0
- package/docs/advanced-selectors.md +18 -0
- package/docs/api-step-definitions.md +45 -0
- package/docs/global-settings.md +267 -0
- package/docs/install-varbase-e2e/ddev-varbase-e2e.md +52 -0
- package/docs/install-varbase-e2e.md +34 -0
- package/docs/overview.md +40 -0
- package/docs/step-definitions/given-i-am-anonymous-user.md +67 -0
- package/docs/step-definitions/given-i-am-homepage.md +68 -0
- package/docs/step-definitions/given-i-am-specific-page.md +72 -0
- package/docs/step-definitions/then-i-should-be.md +56 -0
- package/docs/step-definitions/then-i-should-see.md +62 -0
- package/docs/step-definitions/then-item-checkbox.md +54 -0
- package/docs/step-definitions/then-item-link-should-contain-url.md +84 -0
- package/docs/step-definitions/then-item-should-contain.md +67 -0
- package/docs/step-definitions/then-radio-button-value-should-be-selected.md +68 -0
- package/docs/step-definitions/then-response-status-code-should-be-number.md +69 -0
- package/docs/step-definitions/then-value-radio-button-selected.md +62 -0
- package/docs/step-definitions/when-i-attach-file-file-name-element.md +71 -0
- package/docs/step-definitions/when-i-check-checkbox.md +60 -0
- package/docs/step-definitions/when-i-click-link.md +51 -0
- package/docs/step-definitions/when-i-click-operation-text-row.md +71 -0
- package/docs/step-definitions/when-i-fill.md +59 -0
- package/docs/step-definitions/when-i-go-homepage.md +60 -0
- package/docs/step-definitions/when-i-go-specific-page.md +67 -0
- package/docs/step-definitions/when-i-move-backward-one-page.md +49 -0
- package/docs/step-definitions/when-i-move-forward-one-page.md +58 -0
- package/docs/step-definitions/when-i-press-button.md +58 -0
- package/docs/step-definitions/when-i-reload-page.md +51 -0
- package/docs/step-definitions/when-i-scroll.md +73 -0
- package/docs/step-definitions/when-i-select-option-select-list.md +53 -0
- package/docs/step-definitions/when-i-select-radio-button-value.md +62 -0
- package/docs/step-definitions/when-i-uncheck-checkbox.md +57 -0
- package/docs/step-definitions/when-i-wait-ajax-finish.md +64 -0
- package/docs/step-definitions/when-i-wait.md +62 -0
- package/docs/step-definitions.md +82 -0
- package/docs/steps/README.md +137 -0
- package/docs/steps/a11y.md +731 -0
- package/docs/steps/action.md +181 -0
- package/docs/steps/api.md +679 -0
- package/docs/steps/assertion.md +328 -0
- package/docs/steps/auth.md +108 -0
- package/docs/steps/clock.md +171 -0
- package/docs/steps/cookie.md +293 -0
- package/docs/steps/debug.md +52 -0
- package/docs/steps/dialog.md +226 -0
- package/docs/steps/drupal-canvas.md +414 -0
- package/docs/steps/drupal-ckeditor.md +118 -0
- package/docs/steps/drupal-core.md +688 -0
- package/docs/steps/drupal-layout-builder.md +437 -0
- package/docs/steps/drupal-media.md +107 -0
- package/docs/steps/drupal-moderation.md +81 -0
- package/docs/steps/drupal-paragraphs.md +36 -0
- package/docs/steps/element.md +455 -0
- package/docs/steps/field.md +648 -0
- package/docs/steps/file-download.md +235 -0
- package/docs/steps/form.md +344 -0
- package/docs/steps/iframe.md +248 -0
- package/docs/steps/input.md +220 -0
- package/docs/steps/javascript.md +111 -0
- package/docs/steps/keyboard.md +107 -0
- package/docs/steps/link.md +217 -0
- package/docs/steps/metatag.md +99 -0
- package/docs/steps/modal.md +232 -0
- package/docs/steps/navigation.md +275 -0
- package/docs/steps/network.md +284 -0
- package/docs/steps/path.md +203 -0
- package/docs/steps/response.md +102 -0
- package/docs/steps/responsive.md +139 -0
- package/docs/steps/rest.md +140 -0
- package/docs/steps/screenshot.md +211 -0
- package/docs/steps/scroll.md +266 -0
- package/docs/steps/selectors.md +848 -0
- package/docs/steps/storage.md +254 -0
- package/docs/steps/table.md +216 -0
- package/docs/steps/varbase.md +391 -0
- package/docs/steps/video.md +111 -0
- package/docs/steps/wait.md +553 -0
- package/docs/steps/web-first.md +289 -0
- package/docs/steps/xml.md +505 -0
- package/docs/steps/yaml.md +951 -0
- package/examples/a11y.html +31 -0
- package/examples/about-us.html +11 -0
- package/examples/account.html +32 -0
- package/examples/ajax-wait-examples.html +147 -0
- package/examples/contact-us.html +10 -0
- package/examples/cookies.html +10 -0
- package/examples/date.html +12 -0
- package/examples/dynamic-examples.html +498 -0
- package/examples/element.html +11 -0
- package/examples/example-api.json +28 -0
- package/examples/existing-page.html +11 -0
- package/examples/field.html +24 -0
- package/examples/index.html +37 -0
- package/examples/keyboard.html +11 -0
- package/examples/links.html +9 -0
- package/examples/modal.html +14 -0
- package/examples/path.html +3 -0
- package/examples/table.html +13 -0
- package/examples/test--modal-dialogs.html +252 -0
- package/examples/test--then--i-should--not--see-text-in-element.html +48 -0
- package/examples/test--then--i-should--see-text-in-element.html +48 -0
- package/examples/test--then--i-should--see-text-matching-pattern.html +31 -0
- package/examples/test--then--i-should-see-text-in-table-row.html +232 -0
- package/examples/test--then--the-checkbox-checked.html +68 -0
- package/examples/test--then--the-element-should_not-contain-cssproperty.html +47 -0
- package/examples/test--then--the-link-should-contain.html +21 -0
- package/examples/test--when--i-additionally-select.html +27 -0
- package/examples/test--when--i-attach-file-to-feild.html +50 -0
- package/examples/test--when--i-check-uncheck-checkbox.html +47 -0
- package/examples/test--when--i-click-link.html +18 -0
- package/examples/test--when--i-click-text-in-table-row.html +178 -0
- package/examples/test--when--i-fill-in.html +48 -0
- package/examples/test--when--i-press-button.html +31 -0
- package/examples/test--when--i-reload-page.html +27 -0
- package/examples/test--when--i-scroll-functionality.html +177 -0
- package/examples/test--when--i-scroll-left-right-functionality.html +149 -0
- package/examples/test--when--i-select-option-from-dropdownlist.html +35 -0
- package/examples/test--when--i-select-radio-button.html +95 -0
- package/examples/test--when-i-wait-max-of-time.html +39 -0
- package/examples/test--when-i-wait-page-until-is-loaded.html +47 -0
- package/examples/test--when-i-wait-specific-time.html +39 -0
- package/examples/test-acceptable-url-path.html +11 -0
- package/examples/video-demo.html +43 -0
- package/package.json +56 -0
- package/playwright.config.ts +39 -0
- package/tests/assets/test-document.pdf +14 -0
- package/tests/assets/vardot.png +0 -0
- package/tests/features/a11y.feature +39 -0
- package/tests/features/ajax-wait-examples.feature +24 -0
- package/tests/features/api-step-definitions-individual.feature +185 -0
- package/tests/features/complete-api-testing-examples.feature +205 -0
- package/tests/features/cookie.feature +13 -0
- package/tests/features/date.feature +11 -0
- package/tests/features/dynamic-content-examples.feature +156 -0
- package/tests/features/element.feature +16 -0
- package/tests/features/field.feature +25 -0
- package/tests/features/keyboard.feature +10 -0
- package/tests/features/link.feature +11 -0
- package/tests/features/modal-advanced-example.feature +54 -0
- package/tests/features/modal-basic-example.feature +42 -0
- package/tests/features/modal-error-handling-example.feature +37 -0
- package/tests/features/modal.feature +11 -0
- package/tests/features/path.feature +16 -0
- package/tests/features/response.feature +8 -0
- package/tests/features/responsive.feature +10 -0
- package/tests/features/table.feature +18 -0
- package/tests/features/test--given--i-am-an-anonymous-user.feature +10 -0
- package/tests/features/test--given--i-am-on-homepage.feature +16 -0
- package/tests/features/test--then--i-save-screenshot.feature +33 -0
- package/tests/features/test--then--i-see-locator.feature +320 -0
- package/tests/features/test--then--i-should--not--see-text-in-element.feature +13 -0
- package/tests/features/test--then--i-should--see-text-in-element-by-attr.feature +10 -0
- package/tests/features/test--then--i-should--see-text-in-element.feature +10 -0
- package/tests/features/test--then--i-should--see-text-matching-pattern-in-element.feature +13 -0
- package/tests/features/test--then--i-should--see-text-matching-pattern.feature +18 -0
- package/tests/features/test--then--i-should-be-on-page.feature +10 -0
- package/tests/features/test--then--i-should-be-on-the-homepage.feature +10 -0
- package/tests/features/test--then--i-should-not-see-an-element-by-attr.feature +8 -0
- package/tests/features/test--then--i-should-not-see-an-element.feature +8 -0
- package/tests/features/test--then--i-should-see-an-element-by-attr.feature +8 -0
- package/tests/features/test--then--i-should-see-an-element.feature +8 -0
- package/tests/features/test--then--i-should-see-n-elements.feature +24 -0
- package/tests/features/test--then--i-should-see-text-in-table-row.feature +96 -0
- package/tests/features/test--then--print-current-url-and-response.feature +21 -0
- package/tests/features/test--then--the-checkbox-is-checked.feature +18 -0
- package/tests/features/test--then--the-checkbox-should-be-checked.feature +16 -0
- package/tests/features/test--then--the-element-should-contain-cssproperty.feature +9 -0
- package/tests/features/test--then--the-element-should-not-contain-cssproperty.feature +10 -0
- package/tests/features/test--then--the-field-should-contain-value.feature +11 -0
- package/tests/features/test--then--the-response-should-contain.feature +11 -0
- package/tests/features/test--then--the-response-status-code-should-be.feature +31 -0
- package/tests/features/test--then--the-url-should-match.feature +7 -0
- package/tests/features/test--then-the-link-should-contain-by-attribute.feature +18 -0
- package/tests/features/test--then-the-link-should-contain.feature +7 -0
- package/tests/features/test--when--i-additionally-select.feature +30 -0
- package/tests/features/test--when--i-attach-file-to-feild.feature +9 -0
- package/tests/features/test--when--i-check-checkbox.feature +21 -0
- package/tests/features/test--when--i-click-link-by-attribute.feature +21 -0
- package/tests/features/test--when--i-click-link.feature +9 -0
- package/tests/features/test--when--i-click-text-in-table-row.feature +37 -0
- package/tests/features/test--when--i-fill-in-by-attr.feature +31 -0
- package/tests/features/test--when--i-fill-in-table-by-attr.feature +19 -0
- package/tests/features/test--when--i-fill-in.feature +32 -0
- package/tests/features/test--when--i-follow.feature +22 -0
- package/tests/features/test--when--i-go-to-homepage.feature +9 -0
- package/tests/features/test--when--i-move-forward-backward-one-page.feature +14 -0
- package/tests/features/test--when--i-press-button-by-attribute.feature +18 -0
- package/tests/features/test--when--i-press-button.feature +13 -0
- package/tests/features/test--when--i-reload-page.feature +11 -0
- package/tests/features/test--when--i-scroll-left-right.feature +48 -0
- package/tests/features/test--when--i-scroll.feature +48 -0
- package/tests/features/test--when--i-select-option-from-dropdownlist.feature +21 -0
- package/tests/features/test--when--i-select-radio-button.feature +37 -0
- package/tests/features/test--when--i-wait-page-is-loaded.feature +8 -0
- package/tests/features/test--when--i-wait-time.feature +21 -0
- package/tests/features/video.feature +18 -0
- package/tests/features/wait.feature +7 -0
- package/tests/features/yaml.feature +102 -0
- package/tests/selectors/_canonical-keys.json +113 -0
- package/tests/selectors/back-end-selectors.json +19 -0
- package/tests/selectors/cms-contentful-admin.json +35 -0
- package/tests/selectors/cms-craft-admin.json +39 -0
- package/tests/selectors/cms-drupal-cms-gin.json +56 -0
- package/tests/selectors/cms-drupal-core-claro.json +50 -0
- package/tests/selectors/cms-generic-admin.json +34 -0
- package/tests/selectors/cms-ghost-admin.json +34 -0
- package/tests/selectors/cms-joomla-admin.json +41 -0
- package/tests/selectors/cms-magento2-admin.json +32 -0
- package/tests/selectors/cms-prestashop-admin.json +38 -0
- package/tests/selectors/cms-shopify-admin.json +25 -0
- package/tests/selectors/cms-strapi-admin.json +42 -0
- package/tests/selectors/cms-typo3-admin.json +41 -0
- package/tests/selectors/cms-woocommerce-front.json +50 -0
- package/tests/selectors/cms-wordpress-admin.json +31 -0
- package/tests/selectors/framework-ant-design.json +80 -0
- package/tests/selectors/framework-bootstrap.json +86 -0
- package/tests/selectors/framework-bulma.json +74 -0
- package/tests/selectors/framework-chakra.json +58 -0
- package/tests/selectors/framework-foundation.json +51 -0
- package/tests/selectors/framework-material-ui.json +72 -0
- package/tests/selectors/framework-shadcn.json +51 -0
- package/tests/selectors/framework-tailwind.json +40 -0
- package/tests/selectors/framework-vuetify.json +61 -0
- package/tests/selectors/front-end-selectors.json +16 -0
- package/tests/selectors/homepage-selectors.json +13 -0
- package/tests/step-definitions/a11y.steps.js +800 -0
- package/tests/step-definitions/action.steps.js +167 -0
- package/tests/step-definitions/api.steps.js +732 -0
- package/tests/step-definitions/assertion.steps.js +352 -0
- package/tests/step-definitions/auth.steps.js +126 -0
- package/tests/step-definitions/clock.steps.js +128 -0
- package/tests/step-definitions/cookie.steps.js +216 -0
- package/tests/step-definitions/debug.steps.js +39 -0
- package/tests/step-definitions/dialog.steps.js +193 -0
- package/tests/step-definitions/drupal-canvas.steps.js +764 -0
- package/tests/step-definitions/drupal-ckeditor.steps.js +108 -0
- package/tests/step-definitions/drupal-core.steps.js +665 -0
- package/tests/step-definitions/drupal-helpers.js +371 -0
- package/tests/step-definitions/drupal-layout-builder.steps.js +466 -0
- package/tests/step-definitions/drupal-media.steps.js +99 -0
- package/tests/step-definitions/drupal-moderation.steps.js +71 -0
- package/tests/step-definitions/drupal-paragraphs.steps.js +50 -0
- package/tests/step-definitions/element.steps.js +356 -0
- package/tests/step-definitions/field.steps.js +653 -0
- package/tests/step-definitions/file-download.steps.js +211 -0
- package/tests/step-definitions/form.steps.js +333 -0
- package/tests/step-definitions/iframe.steps.js +228 -0
- package/tests/step-definitions/input.steps.js +202 -0
- package/tests/step-definitions/javascript.steps.js +232 -0
- package/tests/step-definitions/keyboard.steps.js +112 -0
- package/tests/step-definitions/link.steps.js +142 -0
- package/tests/step-definitions/metatag.steps.js +91 -0
- package/tests/step-definitions/modal.steps.js +244 -0
- package/tests/step-definitions/navigation.steps.js +218 -0
- package/tests/step-definitions/network.steps.js +280 -0
- package/tests/step-definitions/path.steps.js +141 -0
- package/tests/step-definitions/response.steps.js +98 -0
- package/tests/step-definitions/responsive.steps.js +125 -0
- package/tests/step-definitions/rest.steps.js +111 -0
- package/tests/step-definitions/screenshot.steps.js +423 -0
- package/tests/step-definitions/scroll.steps.js +189 -0
- package/tests/step-definitions/selectors.steps.js +1003 -0
- package/tests/step-definitions/storage.steps.js +194 -0
- package/tests/step-definitions/table.steps.js +181 -0
- package/tests/step-definitions/varbase-e2e.js +897 -0
- package/tests/step-definitions/varbase.steps.js +458 -0
- package/tests/step-definitions/video.steps.js +178 -0
- package/tests/step-definitions/wait.steps.js +400 -0
- package/tests/step-definitions/web-first.steps.js +298 -0
- package/tests/step-definitions/xml.steps.js +411 -0
- package/tests/step-definitions/yaml.steps.js +913 -0
- package/tsconfig.json +26 -0
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
# Debugging
|
|
2
|
+
|
|
3
|
+
When a scenario fails, you have several layers of evidence to inspect.
|
|
4
|
+
|
|
5
|
+
## Auto-screenshot on failure
|
|
6
|
+
|
|
7
|
+
Every failed step automatically writes a screenshot under `screenshots/` with a timestamped filename. Configure in `cucumber.js`:
|
|
8
|
+
|
|
9
|
+
```js
|
|
10
|
+
worldParameters: {
|
|
11
|
+
screenshot: {
|
|
12
|
+
dir: './screenshots',
|
|
13
|
+
onFailed: true,
|
|
14
|
+
failedPrefix: 'failed_',
|
|
15
|
+
filenamePattern: '{datetime}.{feature_file}.feature_{step_line}.{ext}',
|
|
16
|
+
filenamePatternFailed: '{failed_prefix}{datetime}.{feature_file}.feature_{step_line}.{ext}',
|
|
17
|
+
}
|
|
18
|
+
}
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Override per-run via env vars:
|
|
22
|
+
|
|
23
|
+
| Variable | Effect |
|
|
24
|
+
| --- | --- |
|
|
25
|
+
| `VARBASE_E2E_SCREENSHOT_DIR` | Output directory |
|
|
26
|
+
| `VARBASE_E2E_SCREENSHOT_PURGE` | `1` to wipe `dir` at the start of the run |
|
|
27
|
+
| `VARBASE_E2E_SCREENSHOT_ON_FAILED` | `0` to disable failure screenshots |
|
|
28
|
+
| `VARBASE_E2E_SCREENSHOT_ON_EVERY_STEP` | `1` to screenshot every step |
|
|
29
|
+
| `VARBASE_E2E_SCREENSHOT_FULLSCREEN` | `1` to always full-page |
|
|
30
|
+
|
|
31
|
+
## Headed mode
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
HEADLESS=false SLOW_MO=800 npm test
|
|
35
|
+
# or
|
|
36
|
+
npm run test:headed
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Browser opens, every action is delayed by `SLOW_MO` ms so you can watch.
|
|
40
|
+
|
|
41
|
+
## Console + page errors
|
|
42
|
+
|
|
43
|
+
JS errors surface in two ways:
|
|
44
|
+
|
|
45
|
+
1. **Always tracked** — every scenario captures `pageerror` and `console.error`. Inspect inside a custom step via `this._jsErrors`.
|
|
46
|
+
2. **Auto-fail** — tag a scenario `@javascript` to assert "no errors" at scenario end. Suppress with `@js-errors`.
|
|
47
|
+
|
|
48
|
+
Explicit assertion at any point:
|
|
49
|
+
|
|
50
|
+
```gherkin
|
|
51
|
+
Then there should be no JavaScript errors
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## HTML / PDF report
|
|
55
|
+
|
|
56
|
+
After every run, `tests/reports/cucumber_report.html` is regenerated. Open it for a clickable summary of features, scenarios, durations, screenshots, and per-step output.
|
|
57
|
+
|
|
58
|
+
Disable: `VARBASE_E2E_REPORT_DISABLE=1`.
|
|
59
|
+
|
|
60
|
+
### Export to PDF
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
# HTML + PDF in one shot
|
|
64
|
+
npx generate-reports --format all
|
|
65
|
+
|
|
66
|
+
# PDF only — Letter, landscape, slim margin
|
|
67
|
+
npx generate-reports --format pdf --pdf-format Letter --pdf-landscape --pdf-margin 10mm
|
|
68
|
+
|
|
69
|
+
# Branded PDF with header / footer
|
|
70
|
+
npx generate-reports --format pdf \
|
|
71
|
+
--pdf-header '<div style="font-size:10px;width:100%;text-align:center;">Acme Q3 Regression</div>' \
|
|
72
|
+
--pdf-footer '<div style="font-size:10px;width:100%;text-align:center;"><span class="pageNumber"></span>/<span class="totalPages"></span></div>'
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
| Flag | Purpose |
|
|
76
|
+
| --- | --- |
|
|
77
|
+
| `--format html\|pdf\|all` | Repeatable. Default `html`. |
|
|
78
|
+
| `--pdf-format` | `Letter`, `Legal`, `A3`, `A4`, `A5`, … (default `A4`) |
|
|
79
|
+
| `--pdf-landscape` | Landscape orientation |
|
|
80
|
+
| `--pdf-margin` | CSS margin for all sides (default `20mm`) |
|
|
81
|
+
| `--pdf-header` / `--pdf-footer` | HTML templates (use `<span class="pageNumber">` / `<span class="totalPages">`) |
|
|
82
|
+
| `--pdf-no-background` | Disable print backgrounds |
|
|
83
|
+
| `--pdf-scale` | Scale factor 0.1–2.0 |
|
|
84
|
+
| `--pdf-out` | Override PDF output path |
|
|
85
|
+
|
|
86
|
+
PDF generation uses Playwright (already installed) — no extra setup. Every panel auto-expanded and colors preserved (`print-color-adjust: exact`).
|
|
87
|
+
|
|
88
|
+
## Scenario timing
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
time npx cucumber-js
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
For per-scenario breakdown, parse the JSON report:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
node -e "
|
|
98
|
+
const r = require('./tests/reports/cucumber_report.json');
|
|
99
|
+
const s = [];
|
|
100
|
+
r.forEach(f => f.elements.forEach(sc => {
|
|
101
|
+
const ms = sc.steps.reduce((a,b)=>a+(b.result?.duration||0),0)/1e6;
|
|
102
|
+
s.push({n: sc.name, ms});
|
|
103
|
+
}));
|
|
104
|
+
s.sort((a,b)=>b.ms-a.ms).slice(0,10).forEach(x =>
|
|
105
|
+
console.log(x.ms.toFixed(0).padStart(6), 'ms', x.n));
|
|
106
|
+
"
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Trace viewer
|
|
110
|
+
|
|
111
|
+
Varbase E2E does not enable Playwright traces by default (they are heavy). To opt in for a single run, edit `playwright.config.ts`:
|
|
112
|
+
|
|
113
|
+
```ts
|
|
114
|
+
contextOptions: {
|
|
115
|
+
...,
|
|
116
|
+
recordVideo: { dir: './videos' },
|
|
117
|
+
// tracing requires manual start/stop in a hook — see Playwright docs.
|
|
118
|
+
}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
## Common diagnoses
|
|
122
|
+
|
|
123
|
+
| Symptom | Likely cause | Fix |
|
|
124
|
+
| --- | --- | --- |
|
|
125
|
+
| Passes locally, fails in CI | Timing | Use web-first assertion or edge wait |
|
|
126
|
+
| Flickering — sometimes passes | Race condition | `wait for "selector" to appear` instead of `wait Ns` |
|
|
127
|
+
| `Modal dialog is visible, but it should not be` | Stale modal selector | Register a `modal` selector in your CMS preset |
|
|
128
|
+
| `page.goto: net::ERR_CONNECTION_REFUSED` | Dev server not started | `npm start` in another terminal |
|
|
129
|
+
| `function has 0 arguments, should have 3` | Step regex captures missing in callback | Add the captured group params in the callback signature |
|
|
@@ -0,0 +1,338 @@
|
|
|
1
|
+
# AI Agent Guide — Wisdom from the Recipes Book
|
|
2
|
+
|
|
3
|
+
This page distills the canonical guidance from
|
|
4
|
+
**Varbase-E2E-Recipes v1.0.30** (583 pages, 35 chapters, 340+ recipes) into
|
|
5
|
+
a compact reference for AI coding assistants working on any varbase-e2e
|
|
6
|
+
project.
|
|
7
|
+
|
|
8
|
+
The wisdom here is the product of years of BDD practice — Kent Beck's
|
|
9
|
+
TDD revolution, Dan North's BDD reframing, Martin Fowler / Thoughtworks
|
|
10
|
+
SPDD, and the daily reality of automated browser testing. Read it once.
|
|
11
|
+
Apply it on every change.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## The First Principle
|
|
16
|
+
|
|
17
|
+
> **AI generates. Humans validate. Tests verify.**
|
|
18
|
+
|
|
19
|
+
AI does not know what "correct" means for the business. It can write
|
|
20
|
+
code that compiles, runs, and looks reasonable — but only the test
|
|
21
|
+
suite can prove the code does what the business actually needs. That is
|
|
22
|
+
why varbase-e2e scenarios exist: they are the executable contract.
|
|
23
|
+
|
|
24
|
+
Three corollaries:
|
|
25
|
+
|
|
26
|
+
1. **The person who writes the tests controls the quality.**
|
|
27
|
+
2. **The person who controls the quality controls the product.**
|
|
28
|
+
3. **Code is the artifact. The prompt and the tests are the source of truth.**
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Test-Drive-Develop (TDD's evolution for the AI age)
|
|
33
|
+
|
|
34
|
+
Traditional TDD: write a test, make it pass, refactor.
|
|
35
|
+
|
|
36
|
+
Test-Drive-Develop (TDD AI): a closed loop where humans specify, AI
|
|
37
|
+
implements, tests verify.
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
TEST → human writes the feature file (Gherkin = the contract).
|
|
41
|
+
DRIVE → human prompts AI: "implement what passes these scenarios".
|
|
42
|
+
DEVELOP → AI writes code. Tests pass → ship. Tests fail → iterate.
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
The varbase-e2e suite is the verification step. If green, AI produced
|
|
46
|
+
code matching the structured prompt. If red, AI iterates against the
|
|
47
|
+
failing scenario name + step + expected/actual — no human bottleneck.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## SPDD — Structured Prompt-Driven Development
|
|
52
|
+
|
|
53
|
+
Coined by Thoughtworks. Treats prompts as first-class delivery
|
|
54
|
+
artifacts: version-controlled, reviewed, reused.
|
|
55
|
+
|
|
56
|
+
### The REASONS canvas
|
|
57
|
+
|
|
58
|
+
Every prompt should pass through these seven sections:
|
|
59
|
+
|
|
60
|
+
| Letter | Section | Purpose |
|
|
61
|
+
| --- | --- | --- |
|
|
62
|
+
| **R** | Requirements | Problem statement + definition of done |
|
|
63
|
+
| **E** | Entities | Domain nouns + relationships |
|
|
64
|
+
| **A** | Approach | Strategy to meet the requirements |
|
|
65
|
+
| **S** | Structure | Components, pages, routes, dependencies |
|
|
66
|
+
| **O** | Operations | Concrete testable steps with signatures |
|
|
67
|
+
| **N** | Norms | Cross-cutting engineering standards (i18n, a11y, perf, logging) |
|
|
68
|
+
| **S** | Safeguards | Non-negotiable boundaries (security, privacy, rate limits, failure modes) |
|
|
69
|
+
|
|
70
|
+
Varbase E2E feature files are SPDD prompts in disguise. Map:
|
|
71
|
+
|
|
72
|
+
| REASONS | Varbase E2E artifact |
|
|
73
|
+
| --- | --- |
|
|
74
|
+
| Requirements | `Feature:` + `As / I want / So that` |
|
|
75
|
+
| Entities | Domain nouns in step text |
|
|
76
|
+
| Approach | `Background` + scenario outlines |
|
|
77
|
+
| Structure | `tests/` layout + `cucumber.js` |
|
|
78
|
+
| Operations | Individual `Given / When / Then` steps |
|
|
79
|
+
| Norms | `worldParameters` + locale / a11y scenarios |
|
|
80
|
+
| Safeguards | Tags (`@critical`, `@security`), assertion scenarios |
|
|
81
|
+
|
|
82
|
+
### The cardinal SPDD rule
|
|
83
|
+
|
|
84
|
+
> **When reality diverges from the prompt, fix the prompt first. Then
|
|
85
|
+
> update the code.**
|
|
86
|
+
|
|
87
|
+
A code-first fix forgets to update the spec. Six months later, a
|
|
88
|
+
refactor removes the fix, the spec says nothing about it, and the bug
|
|
89
|
+
returns. Prompt-first fixes prevent decay.
|
|
90
|
+
|
|
91
|
+
### The three SPDD skills
|
|
92
|
+
|
|
93
|
+
1. **Alignment** — lock intent before writing code. Show the `Feature:` narrative to product / design / security. Get nods. Then write scenarios.
|
|
94
|
+
2. **Abstraction-First** — design before generating. Define Background + named selectors before per-scenario steps.
|
|
95
|
+
3. **Iterative Review** — never let AI run unattended for more than one feature at a time. Review after every generation.
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## What BDD Actually Tests
|
|
100
|
+
|
|
101
|
+
| Question | TDD answer | BDD answer |
|
|
102
|
+
| --- | --- | --- |
|
|
103
|
+
| Tests what? | Components (classes / functions) | Features (user actions / business behaviour) |
|
|
104
|
+
| Written by? | Developers | Team (Three Amigos) |
|
|
105
|
+
| Language? | Programming language | Gherkin (plain text) |
|
|
106
|
+
| Audience? | Developers | Everyone |
|
|
107
|
+
| Granularity? | Function / class | User story / feature |
|
|
108
|
+
| On failure? | Knows which component | Knows which feature |
|
|
109
|
+
| Documentation? | Code comments | Living specification |
|
|
110
|
+
|
|
111
|
+
Use **both**. BDD at the feature level, TDD at the component level.
|
|
112
|
+
They complement each other.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## The Three Amigos
|
|
117
|
+
|
|
118
|
+
A 15–30 minute conversation format BEFORE coding starts:
|
|
119
|
+
|
|
120
|
+
```
|
|
121
|
+
Product Owner ─► describes intent
|
|
122
|
+
QA / Tester ─► writes scenarios
|
|
123
|
+
Developer ─► implements steps
|
|
124
|
+
│
|
|
125
|
+
▼
|
|
126
|
+
Executable specification
|
|
127
|
+
(.feature files)
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
When you (an AI agent) are asked to add a new feature, simulate the
|
|
131
|
+
Three Amigos: list the Product / QA / Dev questions a human team would
|
|
132
|
+
ask. Answer them in the feature description before writing any
|
|
133
|
+
scenarios.
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## DAMP, KISS, YAGNI
|
|
138
|
+
|
|
139
|
+
* **DAMP** — Descriptive And Meaningful Phrases. Every scenario must be
|
|
140
|
+
understandable without context. No abbreviations. No undefined nouns.
|
|
141
|
+
* **KISS** — Keep It Simple, Stupid. Write the simplest test that could
|
|
142
|
+
fail. Write the simplest code that makes it pass.
|
|
143
|
+
* **YAGNI** — You Ain't Gonna Need It. Do NOT write tests for features
|
|
144
|
+
nobody asked for. Every scenario must trace to a real business need.
|
|
145
|
+
|
|
146
|
+
The MMF principle (Minimum Marketable Feature): ship the smallest
|
|
147
|
+
piece of behaviour that delivers value. Add the next one only when the
|
|
148
|
+
first one is verified green.
|
|
149
|
+
|
|
150
|
+
---
|
|
151
|
+
|
|
152
|
+
## The Anti-Patterns Hall of Fame
|
|
153
|
+
|
|
154
|
+
| Anti-pattern | Symptom | Fix |
|
|
155
|
+
| --- | --- | --- |
|
|
156
|
+
| **Sleep-driven testing** | `wait 3s` after every action | BBR smart waits — wait for the EVENT, not the clock |
|
|
157
|
+
| **God step** | `Given the system is fully set up with admin user and products and orders` | Decompose into specific composable steps |
|
|
158
|
+
| **Coupled scenarios** | Scenario 2 assumes scenario 1's data exists | Each scenario creates its own test data |
|
|
159
|
+
| **Implementation testing** | `Then the database should have a record in users table` | Test what the user experiences, not how the code works |
|
|
160
|
+
| **Brittle selectors** | Tests break when CSS changes | Named selectors + accessible role locators |
|
|
161
|
+
| **Over-trusting AI output** | AI generates 500 tests, none cover the business edge case | AI generates, human validates against requirements |
|
|
162
|
+
| **God scenario** | One scenario tests login + search + checkout | One behaviour per scenario |
|
|
163
|
+
| **Sleep-then-check** | `wait 5s` then assertion | Web-first matcher with `within N seconds` |
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## The Golden Rules of varbase-e2e Testing
|
|
168
|
+
|
|
169
|
+
These ten rules emerged from years of BDD practice. Internalise them.
|
|
170
|
+
|
|
171
|
+
1. Write tests BEFORE code (or at least before you ship).
|
|
172
|
+
2. One scenario = one behaviour.
|
|
173
|
+
3. Each scenario creates its own test data.
|
|
174
|
+
4. **Wait for events, not time.** (BBR — see `docs/02-bbr-smart-waits.md`)
|
|
175
|
+
5. Test behaviour, not implementation.
|
|
176
|
+
6. Use business language, not developer jargon.
|
|
177
|
+
7. Decouple from CSS / HTML structure (named selectors + roles).
|
|
178
|
+
8. Make tests deterministic. No flaky `@flaky` allowed long-term.
|
|
179
|
+
9. Tag scenarios. `@critical` for the smoke set; `@security` / `@a11y` for boundary scenarios.
|
|
180
|
+
10. **Fix the prompt before the code** when reality diverges.
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## Recipes for AI Agents
|
|
185
|
+
|
|
186
|
+
### Recipe AI-1: Generate a feature file from a user story
|
|
187
|
+
|
|
188
|
+
When asked to author a varbase-e2e `.feature` file:
|
|
189
|
+
|
|
190
|
+
1. Read `templates/spdd-feature.md` if it exists, or use the REASONS
|
|
191
|
+
canvas from this guide.
|
|
192
|
+
2. Fill **every** section before writing any Gherkin.
|
|
193
|
+
3. Place the filled-in REASONS canvas as `#` comments at the top of the
|
|
194
|
+
`.feature` file.
|
|
195
|
+
4. Generate one scenario per Operations item.
|
|
196
|
+
5. Tag every scenario with the relevant Norm / Safeguard category
|
|
197
|
+
(`@critical`, `@auth`, `@a11y`, `@security`, `@i18n`).
|
|
198
|
+
6. Use `Background:` for setup steps shared across scenarios.
|
|
199
|
+
|
|
200
|
+
### Recipe AI-2: Generate a step definition
|
|
201
|
+
|
|
202
|
+
When asked to add a custom step:
|
|
203
|
+
|
|
204
|
+
1. Search `tests/step-definitions/*.steps.js` for an existing matching
|
|
205
|
+
step. If one exists, do NOT add a duplicate — point the user at it.
|
|
206
|
+
2. Pick the file whose topic matches (see `CLAUDE.md` §3 source map).
|
|
207
|
+
3. Use a regex with `(I |we )*` — never `'I ...'` Cucumber Expressions
|
|
208
|
+
unless the step genuinely cannot start with a pronoun.
|
|
209
|
+
4. Add a JSDoc block with at least 5 `Example #N:` Gherkin lines that
|
|
210
|
+
match the step pattern.
|
|
211
|
+
5. Use plain English in the step text. No camelCase identifiers.
|
|
212
|
+
6. Run the audit: every example must match its step pattern.
|
|
213
|
+
|
|
214
|
+
### Recipe AI-3: Maintain tests after a UI change
|
|
215
|
+
|
|
216
|
+
The UI changed. 30 tests are red.
|
|
217
|
+
|
|
218
|
+
1. Run the suite. Capture every failure: scenario name + failing step +
|
|
219
|
+
expected / actual.
|
|
220
|
+
2. Group failures by root cause. Usually 2–3 causes drive 90% of the red.
|
|
221
|
+
3. Find-and-replace step text where the change is purely cosmetic
|
|
222
|
+
("Sign in" → "Log in").
|
|
223
|
+
4. For structural changes, update the named selector preset, NOT every
|
|
224
|
+
feature file.
|
|
225
|
+
5. Re-run the suite. Iterate until green.
|
|
226
|
+
6. Commit prompt + code + selector changes together.
|
|
227
|
+
|
|
228
|
+
### Recipe AI-4: Debug a flaky test
|
|
229
|
+
|
|
230
|
+
A test passes locally but fails in CI.
|
|
231
|
+
|
|
232
|
+
1. Check the failing step. Does it run an action? Is the next step a
|
|
233
|
+
read assertion? If yes, suspect timing.
|
|
234
|
+
2. Replace `wait Ns` with an edge-driven wait — `wait until the URL
|
|
235
|
+
contains "..."`, `wait for ".success" to appear`, web-first matchers
|
|
236
|
+
with `within N seconds`.
|
|
237
|
+
3. Verify no shared state — does the test depend on a previous
|
|
238
|
+
scenario's side-effects? Make it self-contained.
|
|
239
|
+
4. Run with `HEADLESS=false SLOW_MO=800` to watch what really happens.
|
|
240
|
+
5. Inspect screenshots in `screenshots/failed_*.png`.
|
|
241
|
+
|
|
242
|
+
### Recipe AI-5: When tests go bad
|
|
243
|
+
|
|
244
|
+
| Symptom | Probable cause | Fix |
|
|
245
|
+
| --- | --- | --- |
|
|
246
|
+
| Passes sometimes, fails other times | Timing | Smart waits / BBR |
|
|
247
|
+
| Breaking when unrelated code changes | Coupling to CSS structure | Named selectors / roles |
|
|
248
|
+
| Taking too long | Too many browser tests for API-testable logic | Move to API level |
|
|
249
|
+
| Hard to understand | Imperative style | Declarative language |
|
|
250
|
+
| Depending on other tests | Shared state | Each scenario creates own data |
|
|
251
|
+
| Testing CSS classes / DOM | Implementation testing | Test visible behaviour |
|
|
252
|
+
| Hardcoded waits everywhere | Sleep-driven testing | `wait for AJAX to finish`, web-first |
|
|
253
|
+
| Giant `Background:` section | Setup overload | Move setup to dedicated steps |
|
|
254
|
+
| Asserting too many things per scenario | Multi-behaviour scenario | One behaviour per scenario |
|
|
255
|
+
| Checking database state directly | Implementation coupling | Check UI / API instead |
|
|
256
|
+
|
|
257
|
+
---
|
|
258
|
+
|
|
259
|
+
## What changes in the AI age
|
|
260
|
+
|
|
261
|
+
| Activity | Before AI | With AI |
|
|
262
|
+
| --- | --- | --- |
|
|
263
|
+
| Code generation | Skill | Commodity |
|
|
264
|
+
| Test writing | Skill | Amplified — humans guide, AI assists |
|
|
265
|
+
| Business understanding | Premium | Premium-er |
|
|
266
|
+
| Quality assurance | Critical | Critical (AI output needs validation) |
|
|
267
|
+
|
|
268
|
+
The economic shift: code is cheaper to produce, but specification and
|
|
269
|
+
verification are MORE valuable. Varbase E2E positions you on the
|
|
270
|
+
valuable side — Gherkin is the spec, step definitions are the
|
|
271
|
+
verification.
|
|
272
|
+
|
|
273
|
+
---
|
|
274
|
+
|
|
275
|
+
## Pitfalls of AI-Generated Tests
|
|
276
|
+
|
|
277
|
+
1. **Over-trusting output.** AI can generate tests that look correct
|
|
278
|
+
but assert the wrong thing. Always review against business
|
|
279
|
+
requirements.
|
|
280
|
+
2. **Testing implementation, not behaviour.** AI tends to test what the
|
|
281
|
+
code does. Human oversight keeps tests behaviour-focused.
|
|
282
|
+
3. **Missing edge cases.** AI generates the happy path well. Business
|
|
283
|
+
edge cases need a domain expert.
|
|
284
|
+
4. **False confidence.** "We have 500 AI-generated tests!" means
|
|
285
|
+
nothing if they don't cover what matters.
|
|
286
|
+
|
|
287
|
+
> **The rule:** AI generates. Humans validate.
|
|
288
|
+
|
|
289
|
+
---
|
|
290
|
+
|
|
291
|
+
## When to use SPDD vs vibe-code
|
|
292
|
+
|
|
293
|
+
**Use SPDD (REASONS canvas + structured prompts):**
|
|
294
|
+
|
|
295
|
+
* Regulated systems (finance, healthcare, government).
|
|
296
|
+
* Enterprise CMS where business logic accumulates over years.
|
|
297
|
+
* Customer-facing flows with security or accessibility requirements.
|
|
298
|
+
* Any feature whose behaviour must remain stable across releases.
|
|
299
|
+
|
|
300
|
+
**Skip SPDD (vibe coding is fine):**
|
|
301
|
+
|
|
302
|
+
* Throwaway prototypes.
|
|
303
|
+
* One-off internal scripts.
|
|
304
|
+
* UI explorations where you have not decided what the feature does.
|
|
305
|
+
|
|
306
|
+
The cost of SPDD is the time to fill the canvas. The benefit is years
|
|
307
|
+
of preserved intent. Pick the projects where the benefit clears the cost.
|
|
308
|
+
|
|
309
|
+
---
|
|
310
|
+
|
|
311
|
+
## The Future-Proof Mindset
|
|
312
|
+
|
|
313
|
+
Before SPDD: **code is primary, specs are documentation.**
|
|
314
|
+
After SPDD: **prompts are primary, code is the artifact the prompt produces.**
|
|
315
|
+
|
|
316
|
+
This is not academic. It changes how you debug. It changes how you
|
|
317
|
+
onboard. It changes how AI-driven refactors stay safe over years.
|
|
318
|
+
|
|
319
|
+
Varbase E2E was built for this world. Every step definition is a verb.
|
|
320
|
+
Every feature file is a contract. Stack SPDD on top, and you have a
|
|
321
|
+
system where:
|
|
322
|
+
|
|
323
|
+
* Prompts are committed.
|
|
324
|
+
* Tests are executable.
|
|
325
|
+
* AI generates code.
|
|
326
|
+
* Tests verify.
|
|
327
|
+
* Failures point at the prompt, not at the engineer's coffee intake.
|
|
328
|
+
|
|
329
|
+
Master the loop.
|
|
330
|
+
|
|
331
|
+
---
|
|
332
|
+
|
|
333
|
+
## Source
|
|
334
|
+
|
|
335
|
+
This page summarises *Varbase-E2E-Recipes v1.0.30* — chapters 1, 7, 21,
|
|
336
|
+
24, 26, 29, 30, 31, 33, 34, 35 plus appendices. The full PDF lives at
|
|
337
|
+
`/home/rajab/Desktop/w-book/Varbase-E2E-Recipes-v1.0.30.pdf`. The
|
|
338
|
+
chapter Markdown sources live at `/home/rajab/Desktop/w-book/chapters/`.
|
package/docs/13-faq.md
ADDED
|
@@ -0,0 +1,236 @@
|
|
|
1
|
+
# FAQ
|
|
2
|
+
|
|
3
|
+
Common questions for newcomers.
|
|
4
|
+
|
|
5
|
+
## How is varbase-e2e different from raw Playwright?
|
|
6
|
+
|
|
7
|
+
Playwright is a browser-automation library. Varbase E2E is a **BDD layer on
|
|
8
|
+
top of Playwright + Cucumber-js** with:
|
|
9
|
+
|
|
10
|
+
* 280+ pre-built Gherkin steps (so you write English, not JavaScript).
|
|
11
|
+
* BBR-style smart waits (no `sleep`).
|
|
12
|
+
* Named selector registry (`primary button` instead of
|
|
13
|
+
`button.btn-primary[type=submit]`).
|
|
14
|
+
* CMS / framework selector presets (Drupal, WordPress, Bootstrap, …).
|
|
15
|
+
* Auto-screenshot on failure, auto-HTML-report after every run.
|
|
16
|
+
|
|
17
|
+
You still get every Playwright primitive when you need it — `this.page`
|
|
18
|
+
inside any custom step is a real Playwright Page.
|
|
19
|
+
|
|
20
|
+
## Do I need to write JavaScript?
|
|
21
|
+
|
|
22
|
+
For most tests, no. Use the built-in steps. Write JavaScript only when
|
|
23
|
+
your business has a step that no preset covers — and even then, place
|
|
24
|
+
it in `tests/step-definitions/<topic>.steps.js`, not in feature files.
|
|
25
|
+
|
|
26
|
+
## Why no `wait 5 seconds` in the example tests?
|
|
27
|
+
|
|
28
|
+
Varbase E2E installs a smart-settle hook that runs after every
|
|
29
|
+
state-changing step. The hook waits for: DOM ready, no in-flight
|
|
30
|
+
fetch/XHR, no pending `setTimeout`, and a 250 ms DOM-quiet window. That
|
|
31
|
+
covers the cases that historically needed manual sleeps. Read
|
|
32
|
+
[BBR smart waits](02-bbr-smart-waits.md) for the full story.
|
|
33
|
+
|
|
34
|
+
## My test passes locally but fails in CI. Help.
|
|
35
|
+
|
|
36
|
+
Common causes, in order of likelihood:
|
|
37
|
+
|
|
38
|
+
1. **Race condition** — replace `wait Ns` with an edge wait
|
|
39
|
+
(`wait until the URL contains "..."`,
|
|
40
|
+
`wait for ".success" to appear`, web-first matcher with
|
|
41
|
+
`within N seconds`).
|
|
42
|
+
2. **Different viewport** — CI defaults differ. Pin a breakpoint:
|
|
43
|
+
`Given I set the viewport to the "desktop" breakpoint`.
|
|
44
|
+
3. **Different browser** — set `BROWSER=firefox npm test` locally to
|
|
45
|
+
reproduce.
|
|
46
|
+
4. **CI is slower** — bump the budget on smart waits explicitly,
|
|
47
|
+
`Then "<#dashboard>" should be visible within 15 seconds`.
|
|
48
|
+
5. **Auth state expired** — re-run the auth-setup feature.
|
|
49
|
+
|
|
50
|
+
## How do I see what the browser is doing?
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
npm run test:headed # opens a real browser, slow-mo 800ms
|
|
54
|
+
HEADLESS=false npm test # same, default slow-mo
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Failed scenarios save a PNG under `screenshots/failed_*.png`. The
|
|
58
|
+
filename embeds the feature, scenario line, and timestamp.
|
|
59
|
+
|
|
60
|
+
## How do I run a single feature / scenario?
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
# One feature.
|
|
64
|
+
npx cucumber-js tests/features/login.feature
|
|
65
|
+
|
|
66
|
+
# One scenario by name match.
|
|
67
|
+
npx cucumber-js --name "Successful login"
|
|
68
|
+
|
|
69
|
+
# Tagged scenarios.
|
|
70
|
+
npx cucumber-js --tags "@critical"
|
|
71
|
+
npx cucumber-js --tags "@critical and not @flaky"
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
## How do I skip a scenario without deleting it?
|
|
75
|
+
|
|
76
|
+
Tag it `@wip` and run with `--tags "not @wip"`. Or use a `Cucumber`
|
|
77
|
+
disabled step prefix (`Skip:` is not standard; tags are).
|
|
78
|
+
|
|
79
|
+
## How do I share setup between scenarios?
|
|
80
|
+
|
|
81
|
+
```gherkin
|
|
82
|
+
Feature: Admin dashboard
|
|
83
|
+
|
|
84
|
+
Background:
|
|
85
|
+
Given I restore the auth state from "tests/auth/admin.json"
|
|
86
|
+
And I am on "/admin"
|
|
87
|
+
|
|
88
|
+
Scenario: Lists users
|
|
89
|
+
Then I should see "Users"
|
|
90
|
+
|
|
91
|
+
Scenario: Lists products
|
|
92
|
+
When I follow "Products"
|
|
93
|
+
Then I should see "Products"
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The `Background:` block runs before every scenario in the file. Keep
|
|
97
|
+
it short — 3-5 lines.
|
|
98
|
+
|
|
99
|
+
## How do I parameterise a scenario?
|
|
100
|
+
|
|
101
|
+
```gherkin
|
|
102
|
+
Scenario Outline: Login fails with bad input
|
|
103
|
+
Given I am on "/login"
|
|
104
|
+
When I fill in "Email" with "<email>"
|
|
105
|
+
And I fill in "Password" with "<password>"
|
|
106
|
+
And I press "Sign in"
|
|
107
|
+
Then I should see "<error>"
|
|
108
|
+
|
|
109
|
+
Examples:
|
|
110
|
+
| email | password | error |
|
|
111
|
+
| not-an-email | secret | Email is not valid |
|
|
112
|
+
| | secret | Email is required |
|
|
113
|
+
| a@b.c | | Password is required |
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Each row becomes a separate scenario.
|
|
117
|
+
|
|
118
|
+
## How do I assert what the user sees, not the implementation?
|
|
119
|
+
|
|
120
|
+
Avoid:
|
|
121
|
+
|
|
122
|
+
```gherkin
|
|
123
|
+
Then the database should have a row in users # ❌ implementation
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Prefer:
|
|
127
|
+
|
|
128
|
+
```gherkin
|
|
129
|
+
Then I should see "Welcome, Alice" # ✅ user behaviour
|
|
130
|
+
And the URL should be "/dashboard"
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Behaviour-tested code is refactor-friendly. Implementation-tested code
|
|
134
|
+
breaks every time the schema changes.
|
|
135
|
+
|
|
136
|
+
## How do I name selectors I'll reuse?
|
|
137
|
+
|
|
138
|
+
In `tests/selectors/my-app.json`:
|
|
139
|
+
|
|
140
|
+
```json
|
|
141
|
+
{
|
|
142
|
+
"css": {
|
|
143
|
+
"primary cta": "main .hero button.cta-primary"
|
|
144
|
+
}
|
|
145
|
+
}
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Wire it in `cucumber.js`:
|
|
149
|
+
|
|
150
|
+
```js
|
|
151
|
+
worldParameters: {
|
|
152
|
+
selectors: { files: ['my-app.json'] }
|
|
153
|
+
}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Then in features:
|
|
157
|
+
|
|
158
|
+
```gherkin
|
|
159
|
+
When I add "primary cta" selector for "main .hero button.cta-primary" css selector
|
|
160
|
+
# (or skip the inline registration since the JSON file is loaded)
|
|
161
|
+
Then I see visible primary cta
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Better yet, use a CMS preset that already names the common parts —
|
|
165
|
+
see [03 Selector registry](03-selector-registry.md).
|
|
166
|
+
|
|
167
|
+
## How do I mock an external API?
|
|
168
|
+
|
|
169
|
+
```gherkin
|
|
170
|
+
Given the URL "**/api/users" returns the JSON:
|
|
171
|
+
"""
|
|
172
|
+
{"users": [{"id": 1, "name": "Alice"}]}
|
|
173
|
+
"""
|
|
174
|
+
When I am on "/users"
|
|
175
|
+
Then "<#user-list>" should have a count of 1 within 5 seconds
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Full reference: [06 Networking & dialogs](06-network-and-dialogs.md).
|
|
179
|
+
|
|
180
|
+
## How do I test as a logged-in user without retyping the login?
|
|
181
|
+
|
|
182
|
+
One-time setup feature saves the session. Every other feature restores it.
|
|
183
|
+
|
|
184
|
+
```gherkin
|
|
185
|
+
@auth-setup
|
|
186
|
+
Scenario: Save admin auth state
|
|
187
|
+
Given I am on "/login"
|
|
188
|
+
When I fill in "Email" with "admin@example.com"
|
|
189
|
+
And I fill in "Password" with "..."
|
|
190
|
+
And I press "Sign in"
|
|
191
|
+
And I wait until the URL contains "/dashboard"
|
|
192
|
+
Then I save the auth state to "tests/auth/admin.json"
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Run once: `npx cucumber-js --tags @auth-setup`.
|
|
196
|
+
|
|
197
|
+
In every other feature:
|
|
198
|
+
|
|
199
|
+
```gherkin
|
|
200
|
+
Background:
|
|
201
|
+
Given I restore the auth state from "tests/auth/admin.json"
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
Saves 1-3 seconds per scenario. Full reference: [07 Auth state](07-auth-state.md).
|
|
205
|
+
|
|
206
|
+
## How do I run accessibility audits?
|
|
207
|
+
|
|
208
|
+
```gherkin
|
|
209
|
+
Scenario: Page passes WCAG AA
|
|
210
|
+
Given I am on "/checkout"
|
|
211
|
+
Then the page should pass an accessibility audit at level "AA"
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Layered approach in [10 Accessibility](10-accessibility.md). Combine
|
|
215
|
+
custom probes (heading order, skip link, ARIA validity) with axe-core
|
|
216
|
+
full audits.
|
|
217
|
+
|
|
218
|
+
## How do I make CI fast?
|
|
219
|
+
|
|
220
|
+
```bash
|
|
221
|
+
SLOW_MO=0 npx cucumber-js --parallel 4 --retry 1 --retry-tag-filter @flaky
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Set `--parallel 4` (or however many cores) to fan scenarios across
|
|
225
|
+
processes. Each scenario gets its own browser context — no shared
|
|
226
|
+
state, safe to parallelise.
|
|
227
|
+
|
|
228
|
+
## I want AI to write tests for me.
|
|
229
|
+
|
|
230
|
+
Read [12 AI agent guide](12-ai-agent-guide.md) and follow the
|
|
231
|
+
Test-Drive-Develop loop: human writes the feature file (the
|
|
232
|
+
specification), AI implements code, tests verify. The varbase-e2e
|
|
233
|
+
suite is the validation layer that prevents AI from "looking right"
|
|
234
|
+
while doing the wrong thing.
|
|
235
|
+
|
|
236
|
+
For a structured prompt template, see Recipe AI-1 in that page.
|