webship-js 2.0.10 → 2.0.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.circleci/config.yml +10 -1
- package/.drone.yml +7 -7
- package/.github/ISSUE_TEMPLATE/issue.md +37 -0
- package/.github/PULL_REQUEST_TEMPLATE.md +24 -0
- package/.github/workflows/github-actions.yml +101 -4
- package/.gitlab-ci.yml +9 -4
- package/.harness/webship-js-pipeline.yml +5 -6
- package/.semaphore/semaphore.yml +6 -9
- package/.teamcity/settings.kts +7 -6
- package/.travis.yml +6 -1
- package/AGENTS.md +1 -1
- package/CLAUDE.md +69 -24
- package/Jenkinsfile +5 -3
- package/README.md +13 -1
- package/azure-pipelines.yml +17 -8
- package/bamboo-specs/bamboo.yml +9 -15
- package/bin/generate-reports.js +51 -7
- package/bin/init-webship.js +127 -23
- package/bitbucket-pipelines.yml +5 -1
- package/buildspec.yml +6 -7
- package/cloudbuild.yaml +28 -12
- package/codefresh.yml +6 -6
- package/cucumber.js +46 -11
- package/docs/00-quick-start.md +38 -7
- package/docs/02-bbr-smart-waits.md +21 -10
- package/docs/03-selector-registry.md +24 -0
- package/docs/04-step-reference.md +53 -27
- package/docs/05-web-first-assertions.md +6 -0
- package/docs/07-auth-state.md +1 -1
- package/docs/08-clock-mocking.md +21 -2
- package/docs/09-api-testing.md +40 -2
- package/docs/10-accessibility.md +2 -0
- package/docs/11-debugging.md +24 -1
- package/docs/12-ai-agent-guide.md +8 -1
- package/docs/13-faq.md +75 -1
- package/docs/14-recipes-cookbook.md +2 -0
- package/docs/15-tag-conventions.md +5 -1
- package/docs/16-ci-cd.md +161 -124
- package/docs/README.md +5 -5
- package/docs/advanced-screenshots.md +3 -1
- package/docs/api-step-definitions.md +16 -2
- package/docs/global-settings.md +70 -18
- package/docs/install-webship-js/ddev-webship-js.md +3 -0
- package/docs/install-webship-js.md +16 -2
- package/docs/step-definitions/then-i-should-see.md +14 -1
- package/docs/step-definitions/then-item-checkbox.md +2 -0
- package/docs/step-definitions/when-i-fill.md +4 -0
- package/docs/step-definitions/when-i-select-option-select-list.md +2 -0
- package/examples/a11y-legacy-widget.html +34 -0
- package/examples/auth-state.html +42 -0
- package/examples/choice-controls.html +29 -0
- package/examples/clock.html +33 -0
- package/examples/datetime-fields.html +52 -0
- package/examples/downloads/webship-archive.zip +0 -0
- package/examples/downloads/webship-notes.txt +4 -0
- package/examples/downloads/webship-report.csv +4 -0
- package/examples/downloads.html +16 -0
- package/examples/element-selector-hint.html +16 -0
- package/examples/element-viewport.html +42 -0
- package/examples/field-states.html +27 -0
- package/examples/field-types.html +98 -0
- package/examples/fill-in-empty.html +26 -0
- package/examples/fill-unreachable.html +27 -0
- package/examples/iframe-inner.html +20 -0
- package/examples/iframe-nested.html +16 -0
- package/examples/iframe-newsletter.html +29 -0
- package/examples/iframe-order.html +26 -0
- package/examples/iframe-outer.html +15 -0
- package/examples/iframe.html +21 -0
- package/examples/javascript-errors.html +35 -0
- package/examples/links.html +1 -0
- package/examples/metatags.html +18 -0
- package/examples/mouse-and-pointer.html +83 -0
- package/examples/multi-value-fields.html +54 -0
- package/examples/native-dialogs.html +8 -0
- package/examples/network.html +83 -0
- package/examples/registered-names.html +15 -0
- package/examples/scroll-horizontal.html +50 -0
- package/examples/select-by-label.html +33 -0
- package/examples/selectors-position.html +29 -0
- package/examples/settle-budget.html +21 -0
- package/examples/sorted-tables.html +38 -0
- package/examples/storage.html +57 -0
- package/examples/text-matching.html +48 -0
- package/examples/text-order.html +25 -0
- package/examples/touch.html +22 -0
- package/examples/viewport-size.html +24 -0
- package/examples/viewport-top.html +36 -0
- package/examples/wait-conditions.html +69 -0
- package/examples/web-first.html +99 -0
- package/package.json +24 -2
- package/playwright.config.ts +3 -1
- package/tests/assets/catalog.xml +19 -0
- package/tests/assets/feed-namespaced.xml +13 -0
- package/tests/assets/sitemap.xml +14 -0
- package/tests/assets/webship-site-expected.yaml +21 -0
- package/tests/assets/webship-site.schema.json +46 -0
- package/tests/assets/webship-site.yaml +21 -0
- package/tests/check-api-launch-url.js +106 -0
- package/tests/check-assets.js +103 -0
- package/tests/check-attachments.js +105 -0
- package/tests/check-breakpoints.js +117 -0
- package/tests/check-clock.js +143 -0
- package/tests/check-config.js +90 -0
- package/tests/check-context.js +215 -0
- package/tests/check-dry-run.js +104 -0
- package/tests/check-examples.js +241 -0
- package/tests/check-exit-report.js +141 -0
- package/tests/check-javascript-errors.js +105 -0
- package/tests/check-must-fail.js +85 -0
- package/tests/check-paths.js +82 -0
- package/tests/check-report.js +119 -0
- package/tests/check-scaffold.js +246 -0
- package/tests/check-screenshots.js +117 -0
- package/tests/check-settle-verbs.js +137 -0
- package/tests/check-time-limit.js +104 -0
- package/tests/check-video.js +156 -0
- package/tests/check-world.js +167 -0
- package/tests/features/a11y.feature +18 -0
- package/tests/features/api-base-url.feature +25 -0
- package/tests/features/api-form-data.feature +18 -0
- package/tests/features/api-json-paths.feature +24 -0
- package/tests/features/api-response-contains.feature +21 -0
- package/tests/features/api-response-header-presence.feature +17 -0
- package/tests/features/api-response-header.feature +17 -0
- package/tests/features/api-saved-placeholders.feature +37 -0
- package/tests/features/api-step-definitions-individual.feature +1 -1
- package/tests/features/api-tables.feature +40 -0
- package/tests/features/auth-state.feature +32 -0
- package/tests/features/breakpoints.feature +25 -0
- package/tests/features/choice-controls.feature +35 -0
- package/tests/features/clock.feature +27 -0
- package/tests/features/complete-api-testing-examples.feature +1 -1
- package/tests/features/cookie.feature +9 -0
- package/tests/features/datetime-field-lookup.feature +29 -0
- package/tests/features/dialog.feature +27 -0
- package/tests/features/element-selector-hint.feature +14 -0
- package/tests/features/element.feature +47 -0
- package/tests/features/field-state.feature +21 -0
- package/tests/features/field.feature +104 -0
- package/tests/features/file-download.feature +64 -0
- package/tests/features/fill-in-empty-value.feature +41 -0
- package/tests/features/fill-reachable.feature +13 -0
- package/tests/features/iframe-nested.feature +43 -0
- package/tests/features/iframe.feature +83 -0
- package/tests/features/javascript-tolerated-errors.feature +20 -0
- package/tests/features/javascript.feature +47 -0
- package/tests/features/link.feature +11 -0
- package/tests/features/metatag.feature +38 -0
- package/tests/features/modal-open.feature +13 -0
- package/tests/features/mouse-and-pointer.feature +86 -0
- package/tests/features/multi-value-field-lookup.feature +30 -0
- package/tests/features/network-mocking.feature +80 -0
- package/tests/features/registered-names.feature +14 -0
- package/tests/features/responsive.feature +27 -0
- package/tests/features/scroll-horizontal.feature +55 -0
- package/tests/features/select-by-label.feature +31 -0
- package/tests/features/selector-files.feature +14 -0
- package/tests/features/selectors-position.feature +41 -0
- package/tests/features/selectors-table-header.feature +35 -0
- package/tests/features/settle-budget.feature +11 -0
- package/tests/features/sorted-tables.feature +19 -0
- package/tests/features/storage-quoted-values.feature +26 -0
- package/tests/features/storage.feature +98 -0
- package/tests/features/test--then--i-see-locator.feature +13 -0
- package/tests/features/test--then--i-should--see-text-in-element.feature +9 -1
- package/tests/features/test--then--the-field-should-contain-value.feature +8 -0
- package/tests/features/text-matching.feature +38 -0
- package/tests/features/text-order.feature +14 -0
- package/tests/features/touch.feature +21 -0
- package/tests/features/video.feature +1 -0
- package/tests/features/viewport-top.feature +13 -0
- package/tests/features/wait.feature +89 -0
- package/tests/features/web-first.feature +100 -0
- package/tests/features/xml-namespaces.feature +34 -0
- package/tests/features/xml.feature +96 -0
- package/tests/features/yaml.feature +75 -0
- package/tests/must-fail/api-json-paths.feature +21 -0
- package/tests/must-fail/api-response-contains.feature +23 -0
- package/tests/must-fail/api-response-header-presence.feature +17 -0
- package/tests/must-fail/api-response-header.feature +21 -0
- package/tests/must-fail/api-without-base-url.feature +11 -0
- package/tests/must-fail/assertions.feature +96 -0
- package/tests/must-fail/breakpoints.feature +15 -0
- package/tests/must-fail/choice-controls.feature +25 -0
- package/tests/must-fail/clock.feature +11 -0
- package/tests/must-fail/datetime-field.feature +10 -0
- package/tests/must-fail/element-selector-hint.feature +17 -0
- package/tests/must-fail/field-state.feature +20 -0
- package/tests/must-fail/fill-field.feature +30 -0
- package/tests/must-fail/iframe-text.feature +17 -0
- package/tests/must-fail/modal-open.feature +26 -0
- package/tests/must-fail/multi-value-field.feature +18 -0
- package/tests/must-fail/registered-names.feature +25 -0
- package/tests/must-fail/select-list.feature +15 -0
- package/tests/must-fail/selector-files.feature +10 -0
- package/tests/must-fail/sorted-tables.feature +20 -0
- package/tests/must-fail/storage-before-a-page.feature +13 -0
- package/tests/must-fail/text-matching.feature +16 -0
- package/tests/must-fail/text-order.feature +15 -0
- package/tests/must-fail/viewport-top.feature +20 -0
- package/tests/must-fail/waits.feature +51 -0
- package/tests/must-fail/yaml-numbers.feature +48 -0
- package/tests/reports/README.md +2 -2
- package/tests/step-definitions/a11y.steps.js +36 -23
- package/tests/step-definitions/action.steps.js +22 -8
- package/tests/step-definitions/api.steps.js +349 -67
- package/tests/step-definitions/assertion.steps.js +327 -63
- package/tests/step-definitions/auth.steps.js +15 -18
- package/tests/step-definitions/clock.steps.js +22 -0
- package/tests/step-definitions/debug.steps.js +8 -0
- package/tests/step-definitions/dialog.steps.js +9 -2
- package/tests/step-definitions/element.steps.js +135 -18
- package/tests/step-definitions/field.steps.js +523 -80
- package/tests/step-definitions/file-download.steps.js +74 -10
- package/tests/step-definitions/form.steps.js +76 -29
- package/tests/step-definitions/iframe.steps.js +89 -29
- package/tests/step-definitions/input.steps.js +26 -7
- package/tests/step-definitions/javascript.steps.js +64 -22
- package/tests/step-definitions/modal.steps.js +79 -13
- package/tests/step-definitions/navigation.steps.js +14 -0
- package/tests/step-definitions/network.steps.js +15 -5
- package/tests/step-definitions/response.steps.js +6 -4
- package/tests/step-definitions/responsive.steps.js +26 -33
- package/tests/step-definitions/screenshot.steps.js +26 -9
- package/tests/step-definitions/scroll.steps.js +20 -0
- package/tests/step-definitions/selectors.steps.js +104 -51
- package/tests/step-definitions/storage.steps.js +41 -16
- package/tests/step-definitions/table.steps.js +81 -6
- package/tests/step-definitions/video.steps.js +52 -55
- package/tests/step-definitions/wait.steps.js +209 -54
- package/tests/step-definitions/web-first.steps.js +69 -35
- package/tests/step-definitions/webship.js +807 -128
- package/tests/step-definitions/xml.steps.js +124 -28
- package/tests/step-definitions/yaml.steps.js +90 -46
- package/.scratch/commit.txt +0 -14
- package/.scratch/docs.py +0 -42
- package/.scratch/echo-server.js +0 -8
- package/.scratch/echo.js +0 -35
- package/.scratch/echo.pid +0 -1
- package/.scratch/features/cookies.feature +0 -22
- package/.scratch/http.pid +0 -1
- package/.scratch/https.pid +0 -1
- package/.scratch/issue.md +0 -49
- package/.scratch/one.js +0 -2
- package/.scratch/one.json +0 -280
- package/.scratch/patch.py +0 -163
- package/.scratch/pr.md +0 -30
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict';
|
|
3
|
+
|
|
4
|
+
// Runs the steps whose length the scenario gives under a step timeout
|
|
5
|
+
// shorter than that length (WEBSHIP_STEP_TIMEOUT=4000) and checks that each
|
|
6
|
+
// one runs past it: "I wait 10 seconds" on a page that never settles waits
|
|
7
|
+
// longer than 4 seconds and passes, a web-first assertion "within 9 seconds"
|
|
8
|
+
// waits for an element that shows after 7 seconds, and one that never holds
|
|
9
|
+
// fails with its own message after its 6 seconds — none of them ends with
|
|
10
|
+
// cucumber's "function timed out".
|
|
11
|
+
//
|
|
12
|
+
// The pages are data: URLs: no server is needed.
|
|
13
|
+
//
|
|
14
|
+
// Usage: node tests/check-time-limit.js (npm run test:time-limit)
|
|
15
|
+
|
|
16
|
+
const fs = require('fs');
|
|
17
|
+
const os = require('os');
|
|
18
|
+
const path = require('path');
|
|
19
|
+
const { spawnSync } = require('child_process');
|
|
20
|
+
|
|
21
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
22
|
+
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'webship-js-time-limit-'));
|
|
23
|
+
const report = path.join(tmp, 'time-limit.json');
|
|
24
|
+
const config = path.join(tmp, 'cucumber.js');
|
|
25
|
+
const steps = path.join(tmp, 'time-limit.steps.js');
|
|
26
|
+
const feature = path.join(tmp, 'time-limit.feature');
|
|
27
|
+
|
|
28
|
+
// A page that never settles: a timer that reschedules itself and changes the
|
|
29
|
+
// page every 100 ms. And one whose "#late" element shows after 7 seconds.
|
|
30
|
+
const NEVER_SETTLES = '<p id="beat">0</p><script>var n = 0; (function tick() { document.getElementById("beat").textContent = String(++n); setTimeout(tick, 100); })();</script>';
|
|
31
|
+
const SHOWS_LATE = '<p id="late" hidden>Late</p><script>setTimeout(function () { document.getElementById("late").hidden = false; }, 7000);</script>';
|
|
32
|
+
|
|
33
|
+
const cucumber = path.join(ROOT, 'node_modules', '@cucumber', 'cucumber');
|
|
34
|
+
fs.writeFileSync(steps, [
|
|
35
|
+
`const { Given } = require(${JSON.stringify(cucumber)});`,
|
|
36
|
+
`Given('a page that never settles', async function () { await this.page.goto('data:text/html,' + encodeURIComponent(${JSON.stringify(NEVER_SETTLES)})); });`,
|
|
37
|
+
`Given('a page whose element shows late', async function () { await this.page.goto('data:text/html,' + encodeURIComponent(${JSON.stringify(SHOWS_LATE)})); });`,
|
|
38
|
+
'',
|
|
39
|
+
].join('\n'));
|
|
40
|
+
|
|
41
|
+
// name, steps, the status of the last step, its shortest duration in seconds, a text of its failure
|
|
42
|
+
const scenarios = [
|
|
43
|
+
['A wait in seconds runs past the step timeout', 'Given a page that never settles\n When I wait 10 seconds', 'passed', 4.5, null],
|
|
44
|
+
['A wait with a maximum runs past the step timeout', 'Given a page that never settles\n When I wait max of 10 seconds', 'passed', 4.5, null],
|
|
45
|
+
['A wait for a number of seconds runs past the step timeout', 'Given a page that never settles\n When I wait for 10 seconds', 'passed', 4.5, null],
|
|
46
|
+
['A web-first assertion waits as long as it is told', 'Given a page whose element shows late\n Then "#late" should be visible within 9 seconds', 'passed', 6.5, null],
|
|
47
|
+
['A web-first assertion that never holds fails with its own message', 'Given a page that never settles\n Then "#never" should be visible within 6 seconds', 'failed', 5.5, 'Expected "#never" to be visible'],
|
|
48
|
+
];
|
|
49
|
+
fs.writeFileSync(feature, ['Feature: Steps that run for the time the scenario gives', ''].concat(
|
|
50
|
+
scenarios.map(([name, body]) => ` Scenario: ${name}\n ${body}\n`)
|
|
51
|
+
).join('\n'));
|
|
52
|
+
|
|
53
|
+
const base = require(path.join(ROOT, 'cucumber.js')).default;
|
|
54
|
+
fs.writeFileSync(config, 'module.exports = { default: ' + JSON.stringify({
|
|
55
|
+
requireModule: base.requireModule,
|
|
56
|
+
require: base.require.concat([steps]),
|
|
57
|
+
paths: [feature],
|
|
58
|
+
format: [`json:${report}`],
|
|
59
|
+
worldParameters: base.worldParameters,
|
|
60
|
+
}) + ' };');
|
|
61
|
+
|
|
62
|
+
spawnSync(process.execPath, [path.join(cucumber, 'bin', 'cucumber.js'), '--config', path.relative(ROOT, config)], {
|
|
63
|
+
cwd: ROOT,
|
|
64
|
+
stdio: ['ignore', 'ignore', 'inherit'],
|
|
65
|
+
env: Object.assign({ SLOW_MO: '0' }, process.env, {
|
|
66
|
+
WEBSHIP_REPORT_DISABLE: '1',
|
|
67
|
+
WEBSHIP_VIDEO: 'off',
|
|
68
|
+
WEBSHIP_JS_ERROR_MODE: 'off',
|
|
69
|
+
WEBSHIP_STEP_TIMEOUT: '4000',
|
|
70
|
+
}),
|
|
71
|
+
});
|
|
72
|
+
|
|
73
|
+
const problems = [];
|
|
74
|
+
if (!fs.existsSync(report) || !fs.readFileSync(report, 'utf8').trim()) {
|
|
75
|
+
problems.push('The run wrote no report: cucumber-js did not start.');
|
|
76
|
+
} else {
|
|
77
|
+
const found = {};
|
|
78
|
+
for (const f of JSON.parse(fs.readFileSync(report, 'utf8'))) {
|
|
79
|
+
for (const scenario of f.elements || []) found[scenario.name] = scenario;
|
|
80
|
+
}
|
|
81
|
+
for (const [name, , status, seconds, text] of scenarios) {
|
|
82
|
+
const scenario = found[name];
|
|
83
|
+
if (!scenario) { problems.push(`"${name}" did not run.`); continue; }
|
|
84
|
+
const visible = (scenario.steps || []).filter((s) => !s.hidden);
|
|
85
|
+
const last = visible[visible.length - 1];
|
|
86
|
+
const result = (last && last.result) || {};
|
|
87
|
+
const message = String(result.error_message || '');
|
|
88
|
+
const took = (result.duration || 0) / 1e9;
|
|
89
|
+
if (result.status !== status) {
|
|
90
|
+
problems.push(`"${name}": the step ${result.status}, it must have ${status}${message ? `: ${message.split('\n')[0]}` : ''}.`);
|
|
91
|
+
continue;
|
|
92
|
+
}
|
|
93
|
+
if (took < seconds) problems.push(`"${name}": the step ended after ${took.toFixed(1)} s; it must run past the 4-second step timeout (${seconds} s or more).`);
|
|
94
|
+
if (/function timed out/.test(message)) problems.push(`"${name}": the step was cut by the step timeout: ${message.split('\n')[0]}`);
|
|
95
|
+
if (text && !message.includes(text)) problems.push(`"${name}": the failure does not say "${text}": ${message.split('\n')[0]}`);
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
fs.rmSync(tmp, { recursive: true, force: true });
|
|
100
|
+
if (problems.length) {
|
|
101
|
+
console.error('Steps with a length of their own:\n - ' + problems.join('\n - '));
|
|
102
|
+
process.exit(1);
|
|
103
|
+
}
|
|
104
|
+
console.log('Steps with a length of their own: each ran for the time the scenario gave it, past the step timeout.');
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict';
|
|
3
|
+
|
|
4
|
+
// Runs video-recording scenarios in three modes and checks the files they
|
|
5
|
+
// leave in the video folder:
|
|
6
|
+
// - mode "on": "I start video recording" on a scenario that already
|
|
7
|
+
// records leaves one video under its final name, no page@<hash>.webm;
|
|
8
|
+
// the status in the name is in lower case; "print video path" prints
|
|
9
|
+
// the path the video is saved under, not a temporary file;
|
|
10
|
+
// - mode "on-failure": a passing scenario whose video has a reserved name
|
|
11
|
+
// ("I save the current video as") keeps it, another passing one keeps none;
|
|
12
|
+
// - mode "off": "I stop video recording" saves under the filename pattern;
|
|
13
|
+
// WEBSHIP_VIDEO_SIZE sets the recording size.
|
|
14
|
+
//
|
|
15
|
+
// The scenarios write their own page, so no server is needed.
|
|
16
|
+
//
|
|
17
|
+
// Usage: node tests/check-video.js (npm run test:video)
|
|
18
|
+
|
|
19
|
+
const fs = require('fs');
|
|
20
|
+
const os = require('os');
|
|
21
|
+
const path = require('path');
|
|
22
|
+
const { spawnSync } = require('child_process');
|
|
23
|
+
|
|
24
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
25
|
+
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'webship-js-video-'));
|
|
26
|
+
const cucumber = path.join(ROOT, 'node_modules', '@cucumber', 'cucumber');
|
|
27
|
+
const steps = path.join(tmp, 'video.steps.js');
|
|
28
|
+
const defaults = require(path.join(ROOT, 'cucumber.js')).default;
|
|
29
|
+
|
|
30
|
+
fs.writeFileSync(steps, [
|
|
31
|
+
`const { Given, Then } = require(${JSON.stringify(cucumber)});`,
|
|
32
|
+
"Given('a page that reads {string}', async function (text) {",
|
|
33
|
+
' await this.page.setContent(`<h1>${text}</h1>`);',
|
|
34
|
+
'});',
|
|
35
|
+
"Then('the recording size should be {int} by {int}', function (width, height) {",
|
|
36
|
+
' const size = this._videoSettings && this._videoSettings.size;',
|
|
37
|
+
' if (!size || size.width !== width || size.height !== height) throw new Error(`The recording size is ${JSON.stringify(size)}.`);',
|
|
38
|
+
'});',
|
|
39
|
+
'',
|
|
40
|
+
].join('\n'));
|
|
41
|
+
|
|
42
|
+
const problems = [];
|
|
43
|
+
const strip = (text) => String(text || '').replace(/\u001b\[[0-9;]*m/g, '');
|
|
44
|
+
|
|
45
|
+
function run(name, mode, lines, extraEnv) {
|
|
46
|
+
const dir = path.join(tmp, name);
|
|
47
|
+
const videos = path.join(dir, 'videos');
|
|
48
|
+
fs.mkdirSync(dir, { recursive: true });
|
|
49
|
+
const feature = path.join(dir, `${name}.feature`);
|
|
50
|
+
const report = path.join(dir, 'report.json');
|
|
51
|
+
const config = path.join(dir, 'cucumber.js');
|
|
52
|
+
fs.writeFileSync(feature, lines.join('\n') + '\n');
|
|
53
|
+
const world = JSON.parse(JSON.stringify(defaults.worldParameters));
|
|
54
|
+
world.video = Object.assign({}, world.video, { mode, dir: videos });
|
|
55
|
+
fs.writeFileSync(config, 'module.exports = { default: ' + JSON.stringify({
|
|
56
|
+
requireModule: defaults.requireModule,
|
|
57
|
+
require: defaults.require.concat([steps]),
|
|
58
|
+
paths: [feature],
|
|
59
|
+
format: [`json:${report}`],
|
|
60
|
+
worldParameters: world,
|
|
61
|
+
}) + ' };');
|
|
62
|
+
const env = Object.assign({ SLOW_MO: '0' }, process.env, {
|
|
63
|
+
WEBSHIP_REPORT_DISABLE: '1',
|
|
64
|
+
WEBSHIP_JS_ERROR_MODE: 'off',
|
|
65
|
+
WEBSHIP_SCREENSHOT_ON_FAILED: 'false',
|
|
66
|
+
}, extraEnv || {});
|
|
67
|
+
delete env.WEBSHIP_VIDEO;
|
|
68
|
+
delete env.WEBSHIP_VIDEO_DIR;
|
|
69
|
+
if (!extraEnv || !extraEnv.WEBSHIP_VIDEO_SIZE) delete env.WEBSHIP_VIDEO_SIZE;
|
|
70
|
+
const result = spawnSync(process.execPath, [path.join(cucumber, 'bin', 'cucumber.js'), '--config', path.relative(ROOT, config)], {
|
|
71
|
+
cwd: ROOT,
|
|
72
|
+
encoding: 'utf8',
|
|
73
|
+
stdio: ['ignore', 'pipe', 'pipe'],
|
|
74
|
+
timeout: 180000,
|
|
75
|
+
env,
|
|
76
|
+
});
|
|
77
|
+
if (!fs.existsSync(report) || !fs.readFileSync(report, 'utf8').trim()) {
|
|
78
|
+
problems.push(`${name}: the run wrote no report.\n${strip(result.stderr).slice(-1500)}`);
|
|
79
|
+
} else {
|
|
80
|
+
for (const f of JSON.parse(fs.readFileSync(report, 'utf8'))) {
|
|
81
|
+
for (const scenario of f.elements || []) {
|
|
82
|
+
const failed = (scenario.steps || []).find((step) => step.result && step.result.status !== 'passed');
|
|
83
|
+
if (failed) problems.push(`${name}: "${scenario.name}": ${failed.hidden ? 'a hook' : `"${failed.keyword}${failed.name}"`} ${failed.result.status}: ${strip(failed.result.error_message).split('\n')[0]}`);
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
return {
|
|
88
|
+
files: fs.existsSync(videos) ? fs.readdirSync(videos).sort() : [],
|
|
89
|
+
videos,
|
|
90
|
+
stdout: strip(result.stdout),
|
|
91
|
+
};
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
// Mode "on".
|
|
95
|
+
const on = run('on', 'on', [
|
|
96
|
+
'Feature: Recording that is on',
|
|
97
|
+
'',
|
|
98
|
+
' Scenario: Starting a recording that is already on',
|
|
99
|
+
' Given a page that reads "One"',
|
|
100
|
+
' When I start video recording',
|
|
101
|
+
' And print video path',
|
|
102
|
+
' Then I should see "One"',
|
|
103
|
+
'',
|
|
104
|
+
]);
|
|
105
|
+
if (on.files.some((f) => /^page@/.test(f))) problems.push(`on: a video was left under a temporary name: ${on.files.join(', ')}.`);
|
|
106
|
+
if (on.files.length !== 1) problems.push(`on: ${on.files.length} videos were saved, not 1: ${on.files.join(', ') || 'none'}.`);
|
|
107
|
+
else if (!/\.passed\.webm$/.test(on.files[0])) problems.push(`on: the video name does not end in ".passed.webm": ${on.files[0]}.`);
|
|
108
|
+
const printed = (on.stdout.match(/--- Video path: (\S+)/) || [])[1];
|
|
109
|
+
if (!printed) {
|
|
110
|
+
problems.push('on: "print video path" printed no path.');
|
|
111
|
+
} else if (on.files.length === 1 && path.resolve(printed.replace('<status>', 'passed')) !== path.resolve(on.videos, on.files[0])) {
|
|
112
|
+
problems.push(`on: "print video path" printed ${printed}, but the video was saved as ${path.join(on.videos, on.files[0])}.`);
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
// Mode "on-failure".
|
|
116
|
+
const onFailure = run('on-failure', 'on-failure', [
|
|
117
|
+
'Feature: Recording kept on failure',
|
|
118
|
+
'',
|
|
119
|
+
' Scenario: A passing scenario with a reserved video name',
|
|
120
|
+
' Given a page that reads "Kept"',
|
|
121
|
+
' When I save the current video as "kept.webm"',
|
|
122
|
+
' Then I should see "Kept"',
|
|
123
|
+
'',
|
|
124
|
+
' Scenario: A passing scenario without one',
|
|
125
|
+
' Given a page that reads "Dropped"',
|
|
126
|
+
' Then I should see "Dropped"',
|
|
127
|
+
'',
|
|
128
|
+
]);
|
|
129
|
+
if (onFailure.files.join(',') !== 'kept.webm') problems.push(`on-failure: the folder holds ${onFailure.files.join(', ') || 'nothing'}, not only kept.webm.`);
|
|
130
|
+
|
|
131
|
+
// Mode "off", with a recording size from the environment.
|
|
132
|
+
const off = run('off', 'off', [
|
|
133
|
+
'Feature: Recording started and stopped by steps',
|
|
134
|
+
'',
|
|
135
|
+
' Scenario: Stop and save',
|
|
136
|
+
' When I start video recording',
|
|
137
|
+
' And a page that reads "Stopped"',
|
|
138
|
+
' When I stop video recording',
|
|
139
|
+
'',
|
|
140
|
+
' Scenario: The recording size',
|
|
141
|
+
' When I start video recording',
|
|
142
|
+
' Then the recording size should be 640 by 360',
|
|
143
|
+
' When I stop video recording',
|
|
144
|
+
'',
|
|
145
|
+
], { WEBSHIP_VIDEO_SIZE: '640x360' });
|
|
146
|
+
const stopped = off.files.filter((f) => /Stop_and_save/.test(f));
|
|
147
|
+
if (stopped.length !== 1 || !/\.off\.Stop_and_save\.manual\.webm$/.test(stopped[0])) {
|
|
148
|
+
problems.push(`off: the stopped recording was saved as ${off.files.join(', ') || 'nothing'}, not under the filename pattern with the status "manual".`);
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
fs.rmSync(tmp, { recursive: true, force: true });
|
|
152
|
+
if (problems.length) {
|
|
153
|
+
console.error('Video recording:\n - ' + problems.join('\n - '));
|
|
154
|
+
process.exit(1);
|
|
155
|
+
}
|
|
156
|
+
console.log('Video recording: final names in lower case, no temporary file left, the printed path is the saved one, reserved names kept on failure, stopped recordings named by the pattern, WEBSHIP_VIDEO_SIZE read.');
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict';
|
|
3
|
+
|
|
4
|
+
// Runs a project World that extends the PlaywrightWorld webship-js exports,
|
|
5
|
+
// with a project step that calls this.smartSettle(), and checks that the
|
|
6
|
+
// extended World keeps what webship-js puts on a page (the @mobile viewport,
|
|
7
|
+
// the settle of the smart waits); then runs a World that replaces it
|
|
8
|
+
// without extending it and checks that the run says so.
|
|
9
|
+
//
|
|
10
|
+
// The check serves its own page on a free local port, so no other server is
|
|
11
|
+
// needed.
|
|
12
|
+
//
|
|
13
|
+
// Usage: node tests/check-world.js (npm run test:world)
|
|
14
|
+
|
|
15
|
+
const fs = require('fs');
|
|
16
|
+
const os = require('os');
|
|
17
|
+
const path = require('path');
|
|
18
|
+
const http = require('http');
|
|
19
|
+
const { spawn } = require('child_process');
|
|
20
|
+
|
|
21
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
22
|
+
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'webship-js-world-'));
|
|
23
|
+
const cucumber = path.join(ROOT, 'node_modules', '@cucumber', 'cucumber');
|
|
24
|
+
const webship = path.join(ROOT, 'tests', 'step-definitions', 'webship.js');
|
|
25
|
+
const defaults = require(path.join(ROOT, 'cucumber.js')).default;
|
|
26
|
+
|
|
27
|
+
const PAGE = `<!doctype html><html><head><title>World</title></head><body>
|
|
28
|
+
<h1>World</h1>
|
|
29
|
+
<button id="add" type="button">Add</button>
|
|
30
|
+
<p id="cart">Empty</p>
|
|
31
|
+
<script>
|
|
32
|
+
document.getElementById('add').addEventListener('click', function () {
|
|
33
|
+
setTimeout(function () { document.getElementById('cart').textContent = '1 item'; }, 600);
|
|
34
|
+
});
|
|
35
|
+
</script></body></html>`;
|
|
36
|
+
|
|
37
|
+
const extending = [
|
|
38
|
+
`const { setWorldConstructor, When, Then } = require(${JSON.stringify(cucumber)});`,
|
|
39
|
+
`const { PlaywrightWorld } = require(${JSON.stringify(webship)});`,
|
|
40
|
+
'class ProjectWorld extends PlaywrightWorld {',
|
|
41
|
+
" greeting() { return 'hello from the project World'; }",
|
|
42
|
+
'}',
|
|
43
|
+
'setWorldConstructor(ProjectWorld);',
|
|
44
|
+
"When('I put one item in the cart', async function () {",
|
|
45
|
+
" await this.page.click('#add');",
|
|
46
|
+
' await this.smartSettle();',
|
|
47
|
+
'});',
|
|
48
|
+
"Then('the World should be the project one', function () {",
|
|
49
|
+
" if (this.greeting() !== 'hello from the project World') throw new Error('Not the project World.');",
|
|
50
|
+
'});',
|
|
51
|
+
"Then('the window should be {int} by {int}', async function (width, height) {",
|
|
52
|
+
' const size = await this.page.evaluate(() => `${window.innerWidth} by ${window.innerHeight}`);',
|
|
53
|
+
' if (size !== `${width} by ${height}`) throw new Error(`The window is ${size}.`);',
|
|
54
|
+
'});',
|
|
55
|
+
'',
|
|
56
|
+
].join('\n');
|
|
57
|
+
|
|
58
|
+
const replacing = [
|
|
59
|
+
`const { setWorldConstructor, World, Then } = require(${JSON.stringify(cucumber)});`,
|
|
60
|
+
`const playwright = require(${JSON.stringify(path.join(ROOT, 'node_modules', 'playwright'))});`,
|
|
61
|
+
'class OwnWorld extends World {',
|
|
62
|
+
' async openBrowser() {',
|
|
63
|
+
' this.playwrightBrowser = await playwright.chromium.launch();',
|
|
64
|
+
' this.context = await this.playwrightBrowser.newContext();',
|
|
65
|
+
' this.page = await this.context.newPage();',
|
|
66
|
+
' }',
|
|
67
|
+
'}',
|
|
68
|
+
'setWorldConstructor(OwnWorld);',
|
|
69
|
+
'',
|
|
70
|
+
].join('\n');
|
|
71
|
+
|
|
72
|
+
function run(name, support, lines, base) {
|
|
73
|
+
const dir = path.join(tmp, name);
|
|
74
|
+
fs.mkdirSync(dir, { recursive: true });
|
|
75
|
+
const steps = path.join(dir, 'world.steps.js');
|
|
76
|
+
const feature = path.join(dir, `${name}.feature`);
|
|
77
|
+
const report = path.join(dir, 'report.json');
|
|
78
|
+
const config = path.join(dir, 'cucumber.js');
|
|
79
|
+
fs.writeFileSync(steps, support);
|
|
80
|
+
fs.writeFileSync(feature, lines.join('\n') + '\n');
|
|
81
|
+
const world = JSON.parse(JSON.stringify(defaults.worldParameters));
|
|
82
|
+
world.launchUrl = base;
|
|
83
|
+
fs.writeFileSync(config, 'module.exports = { default: ' + JSON.stringify({
|
|
84
|
+
requireModule: defaults.requireModule,
|
|
85
|
+
require: defaults.require.concat([steps]),
|
|
86
|
+
paths: [feature],
|
|
87
|
+
format: [`json:${report}`],
|
|
88
|
+
worldParameters: world,
|
|
89
|
+
}) + ' };');
|
|
90
|
+
const env = Object.assign({ SLOW_MO: '0' }, process.env, {
|
|
91
|
+
LAUNCH_URL: base,
|
|
92
|
+
WEBSHIP_REPORT_DISABLE: '1',
|
|
93
|
+
WEBSHIP_VIDEO: 'off',
|
|
94
|
+
WEBSHIP_JS_ERROR_MODE: 'off',
|
|
95
|
+
WEBSHIP_SCREENSHOT_ON_FAILED: 'false',
|
|
96
|
+
WEBSHIP_AUTO_SETTLE: 'off',
|
|
97
|
+
});
|
|
98
|
+
return new Promise((resolve) => {
|
|
99
|
+
const child = spawn(process.execPath, [path.join(cucumber, 'bin', 'cucumber.js'), '--config', path.relative(ROOT, config)], {
|
|
100
|
+
cwd: ROOT,
|
|
101
|
+
stdio: ['ignore', 'pipe', 'pipe'],
|
|
102
|
+
env,
|
|
103
|
+
});
|
|
104
|
+
let output = '';
|
|
105
|
+
child.stdout.on('data', (chunk) => { output += chunk; });
|
|
106
|
+
child.stderr.on('data', (chunk) => { output += chunk; });
|
|
107
|
+
const timer = setTimeout(() => child.kill('SIGKILL'), 120000);
|
|
108
|
+
child.on('exit', () => {
|
|
109
|
+
clearTimeout(timer);
|
|
110
|
+
const results = [];
|
|
111
|
+
if (fs.existsSync(report) && fs.readFileSync(report, 'utf8').trim()) {
|
|
112
|
+
for (const f of JSON.parse(fs.readFileSync(report, 'utf8'))) {
|
|
113
|
+
for (const scenario of f.elements || []) {
|
|
114
|
+
const failed = (scenario.steps || []).find((step) => step.result && step.result.status !== 'passed');
|
|
115
|
+
results.push({ name: scenario.name, failed });
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
}
|
|
119
|
+
resolve({ output: output.replace(/\u001b\[[0-9;]*m/g, ''), results });
|
|
120
|
+
});
|
|
121
|
+
});
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
const server = http.createServer((req, res) => {
|
|
125
|
+
res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' });
|
|
126
|
+
res.end(PAGE);
|
|
127
|
+
});
|
|
128
|
+
|
|
129
|
+
server.listen(0, '127.0.0.1', async () => {
|
|
130
|
+
const base = `http://127.0.0.1:${server.address().port}`;
|
|
131
|
+
const problems = [];
|
|
132
|
+
// The auto-settle is off in both runs: only this.smartSettle() waits.
|
|
133
|
+
const extended = await run('extending', extending, [
|
|
134
|
+
'Feature: A project World that extends PlaywrightWorld',
|
|
135
|
+
'',
|
|
136
|
+
' @mobile',
|
|
137
|
+
' Scenario: The extended World keeps the viewport and settles a project step',
|
|
138
|
+
' Given I am on "/"',
|
|
139
|
+
' Then the World should be the project one',
|
|
140
|
+
' And the window should be 375 by 667',
|
|
141
|
+
' When I put one item in the cart',
|
|
142
|
+
' Then I should see "1 item" in the "#cart" element',
|
|
143
|
+
], base);
|
|
144
|
+
if (extended.results.length !== 1) {
|
|
145
|
+
problems.push(`The extended World ran ${extended.results.length} scenarios, not 1:\n${extended.output.slice(-1200)}`);
|
|
146
|
+
} else if (extended.results[0].failed) {
|
|
147
|
+
const f = extended.results[0].failed;
|
|
148
|
+
problems.push(`The extended World: ${f.hidden ? 'a hook' : `"${f.keyword}${f.name}"`} ${f.result.status}: ${String(f.result.error_message || '').replace(/\u001b\[[0-9;]*m/g, '').split('\n')[0]}`);
|
|
149
|
+
}
|
|
150
|
+
const replaced = await run('replacing', replacing, [
|
|
151
|
+
'Feature: A World of its own',
|
|
152
|
+
'',
|
|
153
|
+
' Scenario: A World set by the project alone',
|
|
154
|
+
' Given I am on "/"',
|
|
155
|
+
' Then I should see "World"',
|
|
156
|
+
], base);
|
|
157
|
+
if (!/\[webship-js\] The World of this run does not extend PlaywrightWorld/.test(replaced.output)) {
|
|
158
|
+
problems.push('A World that replaces PlaywrightWorld ran without a word about what it loses.');
|
|
159
|
+
}
|
|
160
|
+
server.close();
|
|
161
|
+
fs.rmSync(tmp, { recursive: true, force: true });
|
|
162
|
+
if (problems.length) {
|
|
163
|
+
console.error('Extending the World:\n - ' + problems.join('\n - '));
|
|
164
|
+
process.exit(1);
|
|
165
|
+
}
|
|
166
|
+
console.log('Extending the World: an extended PlaywrightWorld keeps the viewport tag and this.smartSettle(); a replacing World is reported.');
|
|
167
|
+
});
|
|
@@ -59,3 +59,21 @@ Feature: Accessibility (a11y) smoke checks
|
|
|
59
59
|
And the page should pass the full accessibility check at level "AA"
|
|
60
60
|
And the element "main" should not violate the accessibility rule "image-alt"
|
|
61
61
|
And I print the full accessibility check
|
|
62
|
+
|
|
63
|
+
Scenario: A skip link that leads to the main content
|
|
64
|
+
Given I am on "/a11y-legacy-widget.html"
|
|
65
|
+
Then the page should have a skip link
|
|
66
|
+
And the page should have a main landmark
|
|
67
|
+
|
|
68
|
+
Scenario: The whole page passes the WCAG audit
|
|
69
|
+
Given I am on "/a11y.html"
|
|
70
|
+
Then the page should pass an accessibility audit
|
|
71
|
+
And the page should pass an accessibility audit at level "A"
|
|
72
|
+
And the page should pass an accessibility audit at level "AA"
|
|
73
|
+
And the page should pass an accessibility audit at level "AAA"
|
|
74
|
+
|
|
75
|
+
Scenario: Audit one part of a page, or everything but one part
|
|
76
|
+
Given I am on "/a11y-legacy-widget.html"
|
|
77
|
+
Then the element "main" should pass an accessibility audit
|
|
78
|
+
And the element "#main-content form" should pass an accessibility audit
|
|
79
|
+
And the page should pass an accessibility audit excluding ".legacy-widget"
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
Feature: The API base URL belongs to its scenario
|
|
2
|
+
As a tester,
|
|
3
|
+
I want each scenario to send its API requests where it says
|
|
4
|
+
So that a scenario gives the same result alone and among others
|
|
5
|
+
|
|
6
|
+
Scenario: A path is joined to the base URL the scenario set
|
|
7
|
+
Given the API base URL is "/"
|
|
8
|
+
When I send a GET request to "/example-api.json"
|
|
9
|
+
Then the API response code should be 200
|
|
10
|
+
And the JSON response should have "data.0.type" equal to "articles"
|
|
11
|
+
|
|
12
|
+
Scenario: A base URL with a path of its own
|
|
13
|
+
Given the API base URL is "/examples-that-do-not-exist"
|
|
14
|
+
When I send a GET request to "/example-api.json"
|
|
15
|
+
Then the API response code should be 404
|
|
16
|
+
|
|
17
|
+
Scenario: A whole URL is sent as written, with no base URL
|
|
18
|
+
When I send a GET request to "http://localhost:8080/example-api.json"
|
|
19
|
+
Then the API response code should be 200
|
|
20
|
+
And the JSON response should have "included.0.attributes.name" equal to "John"
|
|
21
|
+
|
|
22
|
+
Scenario: A whole URL is sent as written, past the base URL
|
|
23
|
+
Given the API base URL is "https://api.example.invalid"
|
|
24
|
+
When I send a GET request to "http://localhost:8080/example-api.json"
|
|
25
|
+
Then the API response code should be 200
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
Feature: The API form data step sends each field whole
|
|
2
|
+
As a tester,
|
|
3
|
+
I want a form field whose value contains "=" to reach the server as written
|
|
4
|
+
So that a redirect, a query string or a base64 token is not cut
|
|
5
|
+
|
|
6
|
+
Scenario: A value that contains "=" is sent whole
|
|
7
|
+
Given the API base URL is "https://jsonplaceholder.typicode.com"
|
|
8
|
+
When I send a POST request to "/posts" with form data:
|
|
9
|
+
"""
|
|
10
|
+
redirect=/search?q=shoes
|
|
11
|
+
token=YWJj==
|
|
12
|
+
|
|
13
|
+
title=Form data
|
|
14
|
+
"""
|
|
15
|
+
Then the API response code should be 201
|
|
16
|
+
And the JSON response should have "redirect" equal to "/search?q=shoes"
|
|
17
|
+
And the JSON response should have "token" equal to "YWJj=="
|
|
18
|
+
And the JSON response should have "title" equal to "Form data"
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
Feature: The API JSON property steps take a dot path or a JSON Pointer
|
|
2
|
+
As a tester,
|
|
3
|
+
I want to name a property of the response the way the YAML steps do too
|
|
4
|
+
So that a path written either way reads the same property
|
|
5
|
+
|
|
6
|
+
Background:
|
|
7
|
+
Given the API base URL is "/"
|
|
8
|
+
When I send a GET request to "/example-api.json"
|
|
9
|
+
|
|
10
|
+
Scenario: A dot path, with an index written either way
|
|
11
|
+
Then the JSON response should have "data.0.type" equal to "articles"
|
|
12
|
+
And the JSON response should have "data[0].attributes.title" equal to "JSON:API paints my bikeshed!"
|
|
13
|
+
And the JSON property "included.0.attributes.age" should be 80
|
|
14
|
+
|
|
15
|
+
Scenario: A JSON Pointer
|
|
16
|
+
Then the JSON response should have "/data/0/type" equal to "articles"
|
|
17
|
+
And the API response should have "/included/0/attributes/name" equal to "John"
|
|
18
|
+
And the JSON property "/included/0/attributes/age" should be 80
|
|
19
|
+
And the JSON response should have property "/data/0/relationships/author"
|
|
20
|
+
And the JSON response should not have property "/data/0/password"
|
|
21
|
+
|
|
22
|
+
Scenario: A JSON Pointer to a property that is there is not taken for a missing key
|
|
23
|
+
Then the API response should contain property "/data/0/type"
|
|
24
|
+
And the JSON response should not have property "/data/0/discount"
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
Feature: The API response text checks read the response the same way in both polarities
|
|
2
|
+
As a tester,
|
|
3
|
+
I want "should contain" and "should not contain" to agree on letter case and placeholders
|
|
4
|
+
So that a negative check fails on the text a positive check finds
|
|
5
|
+
|
|
6
|
+
Background:
|
|
7
|
+
Given the API base URL is "/"
|
|
8
|
+
When I send a GET request to "/example-api.json"
|
|
9
|
+
|
|
10
|
+
Scenario: Letter case does not matter to either polarity
|
|
11
|
+
Then the API response code should be 200
|
|
12
|
+
And the API response should contain "JSON:API paints my bikeshed!"
|
|
13
|
+
And the API response should contain "json:api paints my bikeshed!"
|
|
14
|
+
And the API response should not contain "Jane"
|
|
15
|
+
And the API response should not contain "jane"
|
|
16
|
+
|
|
17
|
+
Scenario: A placeholder in the expected text is replaced
|
|
18
|
+
Given I set placeholder "{{author}}" to "John"
|
|
19
|
+
And I set placeholder "{{nobody}}" to "Jane"
|
|
20
|
+
Then the API response should contain "{{author}}"
|
|
21
|
+
And the API response should not contain "{{nobody}}"
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
Feature: The API steps tell whether a response header is there
|
|
2
|
+
As a tester,
|
|
3
|
+
I want to check that an API response has a header, or has not, whatever its value
|
|
4
|
+
So that a missing ETag or a leaking server header is caught
|
|
5
|
+
|
|
6
|
+
Background:
|
|
7
|
+
Given the API base URL is "/"
|
|
8
|
+
When I send a GET request to "/example-api.json"
|
|
9
|
+
|
|
10
|
+
Scenario: Headers the response has
|
|
11
|
+
Then the API response should contain the header "ETag"
|
|
12
|
+
And the API response should contain the header "content-type"
|
|
13
|
+
And API response should contain the header "Accept-Ranges"
|
|
14
|
+
|
|
15
|
+
Scenario: Headers the response has not
|
|
16
|
+
Then the API response should not contain the header "X-Powered-By"
|
|
17
|
+
And the API response should not contain the header "Set-Cookie"
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
Feature: The API response header steps tell "should be" from "should contain"
|
|
2
|
+
As a tester,
|
|
3
|
+
I want "should be" to compare the whole value of a header
|
|
4
|
+
So that a wrong value that contains the right text does not pass
|
|
5
|
+
|
|
6
|
+
Background:
|
|
7
|
+
Given the API base URL is "/"
|
|
8
|
+
When I send a GET request to "/example-api.json"
|
|
9
|
+
|
|
10
|
+
Scenario: The whole value of a header
|
|
11
|
+
Then the response header "Content-Type" should be "application/json; charset=UTF-8"
|
|
12
|
+
And the header "content-type" should be "application/json; charset=UTF-8"
|
|
13
|
+
And the response header "Accept-Ranges" should be "bytes"
|
|
14
|
+
|
|
15
|
+
Scenario: A part of the value of a header
|
|
16
|
+
Then the response header "Content-Type" should contain "application/json"
|
|
17
|
+
And the header "CONTENT-TYPE" should contain "charset"
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
Feature: A value of an API response carried into later requests
|
|
2
|
+
As a tester,
|
|
3
|
+
I want to save a property of a response as a placeholder
|
|
4
|
+
So that a token or an id the API returned goes into the next requests
|
|
5
|
+
|
|
6
|
+
Scenario: A property saved as a placeholder, in a URL and a body
|
|
7
|
+
Given the API base URL is "https://jsonplaceholder.typicode.com"
|
|
8
|
+
When I send a GET request to "/posts/1"
|
|
9
|
+
Then the API response code should be 200
|
|
10
|
+
When I save the JSON property "userId" as placeholder "{{author}}"
|
|
11
|
+
And we save the JSON property "title" as placeholder "{{title}}"
|
|
12
|
+
And I send a GET request to "/users/{{author}}"
|
|
13
|
+
Then the API response code should be 200
|
|
14
|
+
And the JSON response should have "id" equal to 1
|
|
15
|
+
When I send a POST request to "/posts" with body:
|
|
16
|
+
"""
|
|
17
|
+
{"title": "{{title}}", "userId": {{author}}}
|
|
18
|
+
"""
|
|
19
|
+
Then the API response code should be 201
|
|
20
|
+
And the JSON response should have "userId" equal to 1
|
|
21
|
+
And the API response should contain "{{title}}"
|
|
22
|
+
|
|
23
|
+
Scenario: A header set before its placeholder has a value gets it when the request is sent
|
|
24
|
+
Given the API base URL is "/"
|
|
25
|
+
And I set the header "Range" to "bytes=0-{{last}}"
|
|
26
|
+
And I set placeholder "{{last}}" to "9"
|
|
27
|
+
When I send a GET request to "/example-api.json"
|
|
28
|
+
Then the API response code should be 206
|
|
29
|
+
|
|
30
|
+
Scenario: A value with characters that mean something in a pattern
|
|
31
|
+
Given I set placeholder "{{price}}" to "$1.50 (incl. $&)"
|
|
32
|
+
And the API base URL is "https://jsonplaceholder.typicode.com"
|
|
33
|
+
When I send a POST request to "/posts" with body:
|
|
34
|
+
"""
|
|
35
|
+
{"title": "{{price}}", "userId": 1}
|
|
36
|
+
"""
|
|
37
|
+
Then the JSON response should have "title" equal to "$1.50 (incl. $&)"
|
|
@@ -139,7 +139,7 @@ Feature: API Step Definitions Individual
|
|
|
139
139
|
Scenario: Header Assertions
|
|
140
140
|
Given the API base URL is "https://jsonplaceholder.typicode.com"
|
|
141
141
|
When I send a GET request to "/posts/1"
|
|
142
|
-
Then the response header "Content-Type" should be "application/json"
|
|
142
|
+
Then the response header "Content-Type" should be "application/json; charset=utf-8"
|
|
143
143
|
# Alternative syntax:
|
|
144
144
|
# Then the header "Cache-Control" should contain "no-cache"
|
|
145
145
|
|