testaro 72.5.1 → 74.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +1 -1
- package/CLAUDE.md +9 -0
- package/CONTRIBUTING.md +1 -1
- package/README.md +17 -14
- package/VALIDATION.md +1 -1
- package/aceconfig.js +1 -1
- package/actSpecs-doc.md +9 -0
- package/actSpecs.js +8 -0
- package/call.js +3 -2
- package/env.example +10 -2
- package/package.json +1 -1
- package/procs/config.js +4 -13
- package/procs/doActs.js +19 -28
- package/procs/job.js +8 -1
- package/procs/launch.js +1 -0
- package/procs/shoot.js +33 -9
- package/run.js +48 -8
- package/testaro/motion.js +1 -6
- package/testaro/shoot0.js +2 -2
- package/testaro/shoot1.js +2 -7
- package/tests/nuVnu.js +4 -8
- package/tests/testaro.js +10 -2
- package/validation/jobs/todo/240101T1300-shoot-example.json +46 -0
- package/scratch/README.md +0 -12
package/AGENTS.md
CHANGED
package/CLAUDE.md
CHANGED
|
@@ -114,3 +114,12 @@ Long comments are not broken into multiple lines per paragraph.
|
|
|
114
114
|
| `tests/testaro.js` | `allRules` registry for the testaro tool |
|
|
115
115
|
| `testaro/<ruleID>.js` | One file per Testaro rule |
|
|
116
116
|
| `validation/validateTest.js` | Core validation harness |
|
|
117
|
+
|
|
118
|
+
## License
|
|
119
|
+
|
|
120
|
+
© 2026 Jonathan Robert Pool.
|
|
121
|
+
|
|
122
|
+
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
123
|
+
at the project root for details.
|
|
124
|
+
|
|
125
|
+
SPDX-License-Identifier: MIT
|
package/CONTRIBUTING.md
CHANGED
|
@@ -69,7 +69,7 @@ From 12 February 2024 through 30 September 2025, contributors of code to Testaro
|
|
|
69
69
|
## License
|
|
70
70
|
|
|
71
71
|
© 2023–2024 CVS Health and/or one of its affiliates. All rights reserved.
|
|
72
|
-
© 2025 Jonathan Robert Pool.
|
|
72
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
73
73
|
|
|
74
74
|
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
75
75
|
at the project root for details.
|
package/README.md
CHANGED
|
@@ -83,13 +83,11 @@ The main concepts of Testaro are:
|
|
|
83
83
|
|
|
84
84
|
### Operating system and Node.js version
|
|
85
85
|
|
|
86
|
-
Testaro can be installed under a MacOS, Windows, Debian, or Ubuntu operating system.
|
|
87
|
-
|
|
88
|
-
Testaro is tested with the latest long-term-support version of [Node.js](https://nodejs.org/en/).
|
|
86
|
+
Testaro can be installed under a MacOS, Windows, Debian, or Ubuntu operating system with the latest long-term-support version of [Node.js](https://nodejs.org/en/).
|
|
89
87
|
|
|
90
88
|
### Browser security
|
|
91
89
|
|
|
92
|
-
Testaro is configured so that, when Playwright or Puppeteer (a dependency of Playwright and of some tools, including QualWeb) launches a `chromium` browser, the browser is [sandboxed](https://www.geeksforgeeks.org/ethical-hacking/what-is-browser-sandboxing/) for improved security. That is the default for Playwright and Puppeteer, and Testaro does not override that default. The host must therefore permit sandboxed browsers.
|
|
90
|
+
Testaro is configured so that, when Playwright or Puppeteer (a dependency of Playwright and of some tools, including QualWeb) launches a `chromium` browser, the browser is [sandboxed](https://www.geeksforgeeks.org/ethical-hacking/what-is-browser-sandboxing/) for improved security. That is the default for Playwright and Puppeteer, and Testaro does not override that default. The host must therefore permit sandboxed browsers. If you try to run Testaro on a host that prohibits sandboxed browsers, each attempted launch of a `chromium` browser will throw an error with a message complaining about the unavailability of a sandbox.
|
|
93
91
|
|
|
94
92
|
In some operating systems a sandboxed browser requires an [unprivileged user namespace](https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces). In one case, a `…userns.conf` file in the `/etc/sysctl.d` directory with the content `kernel.apparmor_restrict_unprivileged_userns = 1` prohibits unprivileged user namespaces and thereby makes sandboxed browsers unlaunchable.
|
|
95
93
|
|
|
@@ -97,7 +95,7 @@ In some operating systems a sandboxed browser requires an [unprivileged user nam
|
|
|
97
95
|
|
|
98
96
|
One way to cope with this prohibition is to configure Playwright and Puppeteer to launch `chromium` non-sandboxed. In both cases, launch arguments `'--no-sandbox'` and `'--disable-setuid-sandbox'` are available to specify this.
|
|
99
97
|
|
|
100
|
-
- For Playwright, `'--no-sandbox'` and `'--disable-setuid-sandbox'`
|
|
98
|
+
- For Playwright, `'--no-sandbox'` and `'--disable-setuid-sandbox'` are added to the arguments of `browserOptionArgs.push` in the Testaro `run.js` file.
|
|
101
99
|
- For the `qualWeb` tool, this is done in the Testaro `tests/qualweb.js` file, where the `qualWeb.start` method is called with an options argument. Its `args` array property is modified to include `'--no-sandbox'` and `'--disable-setuid-sandbox'`.
|
|
102
100
|
- The `ibm` tool, too, can launch a Puppeteer `chromium` browser, if page content instead of a Playwright page is passed to the `accessibilityChecker.getCompliance` method, or if the implementation of the tool is changed in the future. For anticipation of such a case, the Testaro `aceconfig.js` file is modified. That file defines a `module.exports` object with a `puppeteerArgs` property, and, `--no-sandbox` and `--disable-setuid-sandbox` are added to its array value.
|
|
103
101
|
|
|
@@ -105,7 +103,7 @@ Non-sandboxed browsers are less secure than sandboxed ones, particularly when th
|
|
|
105
103
|
|
|
106
104
|
### Option B
|
|
107
105
|
|
|
108
|
-
|
|
106
|
+
Another solution is to leave the `chromium` configuration unchanged, but configure the operating system to permit a sandboxed browser to be launched. In one case, this is implemented with:
|
|
109
107
|
|
|
110
108
|
```bash
|
|
111
109
|
sudo sysctl -w kernel.unprivileged_userns_clone=1
|
|
@@ -117,6 +115,8 @@ EOF
|
|
|
117
115
|
sudo sysctl --system
|
|
118
116
|
```
|
|
119
117
|
|
|
118
|
+
This repository implements option B.
|
|
119
|
+
|
|
120
120
|
## Installation
|
|
121
121
|
|
|
122
122
|
### As an independent application
|
|
@@ -214,7 +214,7 @@ There are 18 act types. They and their options are documented in the `etc` prope
|
|
|
214
214
|
|
|
215
215
|
#### Job as an object
|
|
216
216
|
|
|
217
|
-
An application can execute a job with
|
|
217
|
+
An application can execute a job with:
|
|
218
218
|
|
|
219
219
|
```javascript
|
|
220
220
|
const {doJob} = require('testaro/run');
|
|
@@ -291,8 +291,8 @@ A report is a job with information about the results of the performance of the j
|
|
|
291
291
|
|
|
292
292
|
As Testaro performs a job, information about the job as a whole is inserted into the job. That information is organized into one or two properties:
|
|
293
293
|
|
|
294
|
-
- `jobData`:
|
|
295
|
-
- `catalog`: A collection of data about the elements on the target that are relevant to any test failures
|
|
294
|
+
- `jobData`: Facts about the performance of the job
|
|
295
|
+
- `catalog`: A collection of data about the HTML elements on the target that are relevant to any test failures
|
|
296
296
|
|
|
297
297
|
Testaro inserts the `jobData` property into every job, but inserts the `catalog` property only into jobs that instruct Testaro to produce standard results.
|
|
298
298
|
|
|
@@ -327,7 +327,7 @@ Testaro uses the following techniques to make the tools calculate XPaths:
|
|
|
327
327
|
- `alfa` and `aslint`: They report XPaths, so Testaro needs only to normalize them.
|
|
328
328
|
- `ed11y`: Testaro adds it and a `window.getXPath` method to the page; when the tool reports an element, Testaro computes its XPath.
|
|
329
329
|
- `wave`: It reports a selector for each element; Testaro finds each element in the page via its selector and executes `window.getXPath` on the element.
|
|
330
|
-
- `axe`, `htmlcs`, `ibm`, `nuVal`, `nuVnu`, `qualWeb`: Testaro adds `data-xpath` attributes to all elements; the tools include code excerpts, with the `data-
|
|
330
|
+
- `axe`, `htmlcs`, `ibm`, `nuVal`, `nuVnu`, `qualWeb`: Testaro adds `data-xpath` attributes to all elements; the tools include code excerpts, with the `data-xpath` attributes, in the reported violations.
|
|
331
331
|
- `testaro`: Testaro designs each of its own tests to report element XPaths.
|
|
332
332
|
|
|
333
333
|
By attaching a catalog entry to each reported element, Testaro allows an application that uses Testaro to tell users, for any particular HTML element, which tools ascribed violations of which rules to that element. An application could, for example, use a screenshot or a text-fragment link or could ask the user to paste the XPath into a browser developer tool.
|
|
@@ -477,7 +477,7 @@ Thus, when the `rules` argument is omitted, QualWeb will test for all of the rul
|
|
|
477
477
|
|
|
478
478
|
The target can be provided to QualWeb either as HTML or as a URL. Experience indicates that the results can differ between these methods, with each method reporting some rule violations or some instances that the other method does not report. For at least some cases, more rules are reported violated when HTML is provided (`withNewItems: false`).
|
|
479
479
|
|
|
480
|
-
QualWeb creates sandboxed Puppeteer pages to perform its tests on. Therefore, the host must permit sandboxed browsers to be launched. See the pertinent [Kilotest documentation](https://github.com/jrpool/kilotest/blob/main/SERVICE.md#browser-privileges) for information about the configuration of an Ubuntu Linux host for this purpose.
|
|
480
|
+
QualWeb creates sandboxed Puppeteer pages to perform its tests on. Therefore, the host must permit sandboxed browsers to be launched. See the discussion above about browser security. Also see the pertinent [Kilotest documentation](https://github.com/jrpool/kilotest/blob/main/SERVICE.md#browser-privileges) for information about the configuration of an Ubuntu Linux host for this purpose.
|
|
481
481
|
|
|
482
482
|
### Testaro
|
|
483
483
|
|
|
@@ -571,7 +571,7 @@ Testaro normally performs tests with headless browsers. Some experiments appear
|
|
|
571
571
|
|
|
572
572
|
## Repository exclusions
|
|
573
573
|
|
|
574
|
-
|
|
574
|
+
Any files in the `temp` or `tmp` directory are presumed ephemeral and are not tracked by `git`. Jobs create temporary files in subdirectories of `tmp` and delete those subdirectories on termination.
|
|
575
575
|
|
|
576
576
|
## Related work
|
|
577
577
|
|
|
@@ -588,6 +588,10 @@ Testilo contains procedures that reorganize report data by issue and by element,
|
|
|
588
588
|
|
|
589
589
|
Report standardization could be performed by other software rather than by Testaro. That would require sending the original reports to the server. They are typically larger than standardized reports. Whenever users want only standardized reports, the fact that Testaro standardizes them eliminates the need to send the original reports anywhere.
|
|
590
590
|
|
|
591
|
+
### Kilotest
|
|
592
|
+
|
|
593
|
+
[Kilotest](https://www.npmjs.com/package/kilotest) is an application that offers a simplified interface to Testaro. At present it is [deployed as a public service](https://kilotest.com/).
|
|
594
|
+
|
|
591
595
|
## Code style
|
|
592
596
|
|
|
593
597
|
The JavaScript code in this project generally conforms to the ESLint configuration file `.eslintrc.json`. However, the `htmlcs/HTMLCS.js` file implements an older version of JavaScript. Its style is regulated by the `htmlcs/.eslintrc.json` file.
|
|
@@ -617,7 +621,6 @@ Future work on this project is being considered. Strategic recommendations for s
|
|
|
617
621
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
618
622
|
© 2025–2026 Jonathan Robert Pool.
|
|
619
623
|
|
|
620
|
-
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
621
|
-
at the project root for details.
|
|
624
|
+
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file at the project root for details.
|
|
622
625
|
|
|
623
626
|
SPDX-License-Identifier: MIT
|
package/VALIDATION.md
CHANGED
|
@@ -94,7 +94,7 @@ Preparing a PR:
|
|
|
94
94
|
## License
|
|
95
95
|
|
|
96
96
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
97
|
-
© 2025 Jonathan Robert Pool.
|
|
97
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
98
98
|
|
|
99
99
|
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
100
100
|
at the project root for details.
|
package/aceconfig.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
-
© 2025 Jonathan Robert Pool.
|
|
3
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
4
4
|
|
|
5
5
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
6
6
|
https://opensource.org/license/mit/ for details.
|
package/actSpecs-doc.md
CHANGED
|
@@ -51,3 +51,12 @@ The validity criterion named in item 2 may be any of these:
|
|
|
51
51
|
- `'isTest'`: is the name of a tool
|
|
52
52
|
- `'isWaitable'`: is `'url'`, `'title'`, or `'body'`
|
|
53
53
|
- `'areStrings'`: is an array of strings
|
|
54
|
+
|
|
55
|
+
## License
|
|
56
|
+
|
|
57
|
+
© 2026 Jonathan Robert Pool.
|
|
58
|
+
|
|
59
|
+
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
60
|
+
at the project root for details.
|
|
61
|
+
|
|
62
|
+
SPDX-License-Identifier: MIT
|
package/actSpecs.js
CHANGED
|
@@ -121,6 +121,14 @@ exports.actSpecs = {
|
|
|
121
121
|
what: [true, 'string', 'hasLength', 'substring of option text content']
|
|
122
122
|
}
|
|
123
123
|
],
|
|
124
|
+
shoot: [
|
|
125
|
+
'Save a full-page screenshot to <tmpdir>/testaro-shoot-<which>.png',
|
|
126
|
+
{
|
|
127
|
+
which: [true, 'string', 'hasLength', 'screenshot label, used in the filename'],
|
|
128
|
+
exclusion: [false, 'string', 'hasLength', 'CSS selector for an element to mask'],
|
|
129
|
+
what: [false, 'string', 'hasLength', 'comment']
|
|
130
|
+
}
|
|
131
|
+
],
|
|
124
132
|
state: [
|
|
125
133
|
'Wait until the page reaches a load state',
|
|
126
134
|
{
|
package/call.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2022–2024 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
+
© 2026 Jonathan Robert Pool.
|
|
3
4
|
|
|
4
5
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
5
6
|
https://opensource.org/license/mit/ for details.
|
|
@@ -42,7 +43,7 @@ const rawDir = `${reportDir}/raw`;
|
|
|
42
43
|
|
|
43
44
|
// FUNCTIONS
|
|
44
45
|
|
|
45
|
-
// Fulfills a
|
|
46
|
+
// Fulfills a request to perform a job.
|
|
46
47
|
const callRun = async jobIDStart => {
|
|
47
48
|
// Find the job.
|
|
48
49
|
const jobDirFileNames = await fs.readdir(todoDir);
|
|
@@ -56,7 +57,7 @@ const callRun = async jobIDStart => {
|
|
|
56
57
|
// Get it.
|
|
57
58
|
const jobJSON = await fs.readFile(`${todoDir}/${jobFileName}`, 'utf8');
|
|
58
59
|
let report = JSON.parse(jobJSON);
|
|
59
|
-
// Run
|
|
60
|
+
// Run the job.
|
|
60
61
|
report = await doJob(report);
|
|
61
62
|
// Archive it.
|
|
62
63
|
await fs.rename(`${todoDir}/${jobFileName}`, `${jobDir}/done/${jobFileName}`);
|
package/env.example
CHANGED
|
@@ -26,7 +26,15 @@ JOBDIR=__placeholder__
|
|
|
26
26
|
REPORTDIR=__placeholder__
|
|
27
27
|
# Multiplier for time limits (normally 1).
|
|
28
28
|
TIMEOUT_MULTIPLIER=1
|
|
29
|
-
# Name of the directory at the project root used for temporary files.
|
|
30
|
-
TMPDIRNAME=scratch
|
|
31
29
|
# Whether to abort the job when any test act fork crashes (default false).
|
|
32
30
|
ABORT_ASSERTIVELY=false
|
|
31
|
+
|
|
32
|
+
## License
|
|
33
|
+
|
|
34
|
+
# © 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
35
|
+
# © 2026 Jeff Witt.
|
|
36
|
+
# © 2026 Jonathan Robert Pool.
|
|
37
|
+
|
|
38
|
+
# Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file at the project root for details.
|
|
39
|
+
|
|
40
|
+
# SPDX-License-Identifier: MIT
|
package/package.json
CHANGED
package/procs/config.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
/*
|
|
2
|
-
© 2025 Jonathan Robert Pool.
|
|
2
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
3
3
|
|
|
4
4
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
5
5
|
https://opensource.org/license/mit/ for details.
|
|
@@ -12,16 +12,7 @@
|
|
|
12
12
|
Shared configuration values for Testaro.
|
|
13
13
|
*/
|
|
14
14
|
|
|
15
|
-
//
|
|
16
|
-
// Set TIMEOUT_MULTIPLIER > 1 for slow networks/sites, < 1 for fast environments.
|
|
15
|
+
// Amount to multiply by specified time limits (normally 1) to adapt to network/site speed.
|
|
17
16
|
const timeoutMultiplier = Number.parseFloat(process.env.TIMEOUT_MULTIPLIER) || 1;
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
// Use for navigation, interaction, and long-running operation timeouts.
|
|
21
|
-
// Do NOT use for very short "fail-fast" timeouts (< 100ms).
|
|
22
|
-
const applyMultiplier = (baseTimeout) => Math.round(baseTimeout * timeoutMultiplier);
|
|
23
|
-
|
|
24
|
-
module.exports = {
|
|
25
|
-
timeoutMultiplier,
|
|
26
|
-
applyMultiplier
|
|
27
|
-
};
|
|
17
|
+
// Multiplies a time limit by the configured amount.
|
|
18
|
+
exports.applyMultiplier = (baseTimeout) => Math.round(baseTimeout * timeoutMultiplier);
|
package/procs/doActs.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
+
© 2026 Jeff Witt.
|
|
3
4
|
© 2025–2026 Jonathan Robert Pool.
|
|
4
5
|
|
|
5
6
|
Licensed under the MIT License. See LICENSE file at the project root or https://opensource.org/license/mit/ for details.
|
|
@@ -18,9 +19,13 @@ const {addError} = require('./error');
|
|
|
18
19
|
const {getNonce, goTo, launch, wait} = require('./launch');
|
|
19
20
|
const {tools} = require('./job');
|
|
20
21
|
const {fork} = require('child_process');
|
|
21
|
-
const os = require('os');
|
|
22
22
|
const {pruneCatalog} = require('./catalog');
|
|
23
|
+
// Function to take a full-page screenshot.
|
|
24
|
+
const {shoot} = require('./shoot');
|
|
25
|
+
// Module to handle file system operations.
|
|
26
|
+
const {applyMultiplier} = require('./config');
|
|
23
27
|
const fs = require('fs/promises');
|
|
28
|
+
const path = require('path');
|
|
24
29
|
|
|
25
30
|
// CONSTANTS
|
|
26
31
|
|
|
@@ -51,8 +56,6 @@ const timeLimits = {
|
|
|
51
56
|
testaro: 200 + Math.round(6 * waits / 1000),
|
|
52
57
|
wave: 25
|
|
53
58
|
};
|
|
54
|
-
// Timeout multiplier.
|
|
55
|
-
const timeoutMultiplier = Number.parseFloat(process.env.TIMEOUT_MULTIPLIER) || 1;
|
|
56
59
|
// Abort aggressiveness.
|
|
57
60
|
const abortAssertively = process.env.ABORT_ASSERTIVELY === 'true';
|
|
58
61
|
|
|
@@ -231,31 +234,11 @@ exports.doActs = async report => {
|
|
|
231
234
|
let {acts} = tempReport;
|
|
232
235
|
// Get the standardization specification.
|
|
233
236
|
const standard = tempReport.standard || 'only';
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
for (const tmpDirAlternative of tmpDirs) {
|
|
238
|
-
try {
|
|
239
|
-
// Verify that it is writable.
|
|
240
|
-
await fs.access(tmpDirAlternative, fs.constants.W_OK);
|
|
241
|
-
tmpDir = tmpDirAlternative;
|
|
242
|
-
break;
|
|
243
|
-
}
|
|
244
|
-
// If it is not:
|
|
245
|
-
catch(error) {
|
|
246
|
-
// Report this.
|
|
247
|
-
console.log(`ERROR: ${tmpDirAlternative} is not a writable directory for temporary reports`);
|
|
248
|
-
}
|
|
249
|
-
}
|
|
250
|
-
// If no writable temporary directory was found:
|
|
251
|
-
if (! tmpDir) {
|
|
252
|
-
// Report this.
|
|
253
|
-
console.log('ERROR: No writable temporary directory was found; quitting');
|
|
254
|
-
// Quit.
|
|
255
|
-
process.exit(1);
|
|
256
|
-
}
|
|
237
|
+
// Get the path to a writable temporary directory.
|
|
238
|
+
const {tmpDir} = report.jobData;
|
|
239
|
+
let reportPath;
|
|
257
240
|
// Get a path for temporary reports.
|
|
258
|
-
|
|
241
|
+
reportPath = path.join(tmpDir, `${tempReport.id}.json`);
|
|
259
242
|
// Initialize the count of completed acts.
|
|
260
243
|
let actCount = 0;
|
|
261
244
|
// For each act in the temporary report:
|
|
@@ -332,7 +315,7 @@ exports.doActs = async report => {
|
|
|
332
315
|
// Save a copy of the temporary report, which the child process will read.
|
|
333
316
|
await fs.writeFile(reportPath, tempReportJSON);
|
|
334
317
|
let timedOut = false;
|
|
335
|
-
const limitMs =
|
|
318
|
+
const limitMs = applyMultiplier(1000 * (timeLimits[act.which] || 15));
|
|
336
319
|
const actResult = await new Promise(resolve => {
|
|
337
320
|
let closed = false;
|
|
338
321
|
// Create a child process to perform the act.
|
|
@@ -656,6 +639,14 @@ exports.doActs = async report => {
|
|
|
656
639
|
act.result.success = false;
|
|
657
640
|
});
|
|
658
641
|
}
|
|
642
|
+
// Otherwise, if the act is a screenshot:
|
|
643
|
+
else if (type === 'shoot') {
|
|
644
|
+
const exclusion = act.exclusion ? page.locator(act.exclusion) : null;
|
|
645
|
+
const pngPath = await shoot(page, act.which, {exclusion});
|
|
646
|
+
act.result = pngPath
|
|
647
|
+
? {success: true, path: pngPath}
|
|
648
|
+
: {success: false, prevented: true};
|
|
649
|
+
}
|
|
659
650
|
// Otherwise, if the act is a move:
|
|
660
651
|
else if (moves[type]) {
|
|
661
652
|
const selector = typeof moves[type] === 'string' ? moves[type] : act.what;
|
package/procs/job.js
CHANGED
|
@@ -181,7 +181,8 @@ exports.isValidJob = job => {
|
|
|
181
181
|
executionTimeStamp,
|
|
182
182
|
target,
|
|
183
183
|
sources,
|
|
184
|
-
acts
|
|
184
|
+
acts,
|
|
185
|
+
jobData
|
|
185
186
|
} = job;
|
|
186
187
|
// Return an error for the first missing or invalid property.
|
|
187
188
|
if (! id || typeof id !== 'string') {
|
|
@@ -260,6 +261,12 @@ exports.isValidJob = job => {
|
|
|
260
261
|
error: `Invalid act:\n${JSON.stringify(invalidAct, null, 2)}`
|
|
261
262
|
};
|
|
262
263
|
}
|
|
264
|
+
if (jobData && typeof jobData !== 'object') {
|
|
265
|
+
return {
|
|
266
|
+
isValid: false,
|
|
267
|
+
error: 'Bad job jobData'
|
|
268
|
+
};
|
|
269
|
+
}
|
|
263
270
|
return {
|
|
264
271
|
isValid: true
|
|
265
272
|
};
|
package/procs/launch.js
CHANGED
package/procs/shoot.js
CHANGED
|
@@ -6,6 +6,17 @@
|
|
|
6
6
|
/*
|
|
7
7
|
shoot
|
|
8
8
|
Makes and saves as a PNG buffer file a full-page screenshot and returns the file path.
|
|
9
|
+
|
|
10
|
+
Call shape:
|
|
11
|
+
shoot(page, label, options?)
|
|
12
|
+
label: string|number used in the saved filename. Sanitized to
|
|
13
|
+
testaro-shoot-<safe>.png — characters outside [A-Za-z0-9._-]
|
|
14
|
+
collapse to '_', leading/trailing dots and underscores are
|
|
15
|
+
stripped, length is capped at 100, and an empty result becomes
|
|
16
|
+
'unnamed'.
|
|
17
|
+
options: optional object:
|
|
18
|
+
exclusion: a Playwright Locator to mask in the screenshot.
|
|
19
|
+
dir: output directory (defaults to the OS temp dir).
|
|
9
20
|
*/
|
|
10
21
|
|
|
11
22
|
// IMPORTS
|
|
@@ -13,16 +24,27 @@
|
|
|
13
24
|
// Shared configuration for timeout multiplier.
|
|
14
25
|
const {applyMultiplier} = require('./config');
|
|
15
26
|
const fs = require('fs/promises');
|
|
16
|
-
const os = require('os');
|
|
17
27
|
const path = require('path');
|
|
18
28
|
const {PNG} = require('pngjs');
|
|
19
29
|
|
|
20
|
-
// CONSTANTS
|
|
21
|
-
|
|
22
|
-
const tmpDir = os.tmpdir();
|
|
23
|
-
|
|
24
30
|
// FUNCTIONS
|
|
25
31
|
|
|
32
|
+
// Coerces a label into a filesystem-safe string. Runs of any character outside
|
|
33
|
+
// [A-Za-z0-9._-] collapse to one underscore; leading and trailing dots and
|
|
34
|
+
// underscores are stripped (no hidden files, no traversal); capped at 100
|
|
35
|
+
// characters; falls back to 'unnamed' if nothing usable remains.
|
|
36
|
+
const sanitizeLabel = (label) => {
|
|
37
|
+
const raw = String(label);
|
|
38
|
+
const cleaned = raw
|
|
39
|
+
.replace(/[^A-Za-z0-9._-]+/g, '_')
|
|
40
|
+
.replace(/^[._]+|[._]+$/g, '')
|
|
41
|
+
.slice(0, 100) || 'unnamed';
|
|
42
|
+
if (cleaned !== raw) {
|
|
43
|
+
console.log(`>> shoot: label sanitized from "${raw}" to "${cleaned}"`);
|
|
44
|
+
}
|
|
45
|
+
return cleaned;
|
|
46
|
+
};
|
|
47
|
+
|
|
26
48
|
// Creates and returns a screenshot.
|
|
27
49
|
const screenShot = async (page, exclusion = null) => {
|
|
28
50
|
const options = {
|
|
@@ -40,9 +62,11 @@ const screenShot = async (page, exclusion = null) => {
|
|
|
40
62
|
return '';
|
|
41
63
|
});
|
|
42
64
|
};
|
|
43
|
-
exports.shoot = async (page,
|
|
65
|
+
exports.shoot = async (page, label, tmpDir, options = {}) => {
|
|
66
|
+
const exclusion = options.exclusion || null;
|
|
67
|
+
const dir = options.dir || tmpDir;
|
|
44
68
|
// Make and get a screenshot as a buffer.
|
|
45
|
-
let shot = await screenShot(page);
|
|
69
|
+
let shot = await screenShot(page, exclusion);
|
|
46
70
|
// If it succeeded:
|
|
47
71
|
if (shot.length) {
|
|
48
72
|
// Get the screenshot as an object representation of a PNG image.
|
|
@@ -54,8 +78,8 @@ exports.shoot = async (page, index) => {
|
|
|
54
78
|
if (global.gc) {
|
|
55
79
|
global.gc();
|
|
56
80
|
}
|
|
57
|
-
const fileName = `testaro-shoot-${
|
|
58
|
-
const pngPath = path.join(
|
|
81
|
+
const fileName = `testaro-shoot-${sanitizeLabel(label)}.png`;
|
|
82
|
+
const pngPath = path.join(dir, fileName);
|
|
59
83
|
// Save the PNG buffer.
|
|
60
84
|
await fs.writeFile(pngPath, pngBuffer);
|
|
61
85
|
// Return the result.
|
package/run.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
+
© 2026 Jeff Witt.
|
|
3
4
|
© 2025–2026 Jonathan Robert Pool.
|
|
4
5
|
|
|
5
6
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
@@ -15,19 +16,14 @@
|
|
|
15
16
|
|
|
16
17
|
// IMPORTS
|
|
17
18
|
|
|
18
|
-
// Module to perform acts.
|
|
19
19
|
const {doActs} = require('./procs/doActs');
|
|
20
|
-
// Module to keep secrets.
|
|
21
20
|
require('dotenv').config({quiet: true});
|
|
22
|
-
// Function to validate jobs.
|
|
23
21
|
const {isValidJob} = require('./procs/job');
|
|
24
|
-
// Module to create catalogs.
|
|
25
22
|
const {getCatalog} = require('./procs/catalog');
|
|
26
|
-
// Module to process dates and times.
|
|
27
23
|
const {nowString} = require('./procs/dateTime');
|
|
28
|
-
// Module to create browsers.
|
|
29
24
|
const {chromium, webkit, firefox} = require('playwright-extra');
|
|
30
|
-
|
|
25
|
+
const fs = require('fs').promises;
|
|
26
|
+
const os = require('os');
|
|
31
27
|
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
|
|
32
28
|
chromium.use(StealthPlugin());
|
|
33
29
|
webkit.use(StealthPlugin());
|
|
@@ -35,11 +31,51 @@ firefox.use(StealthPlugin());
|
|
|
35
31
|
|
|
36
32
|
// FUNCTIONS
|
|
37
33
|
|
|
34
|
+
// Returns an operating-system-compatible absolute path to a temporary directory.
|
|
35
|
+
const getTmpDirPath = async jobName => {
|
|
36
|
+
let jobTmpDir = `${__dirname}/tmp/${jobName}`;
|
|
37
|
+
try {
|
|
38
|
+
// Ensure that a temporary directory exists.
|
|
39
|
+
await fs.mkdir(jobTmpDir, {recursive: true});
|
|
40
|
+
}
|
|
41
|
+
catch (error) {
|
|
42
|
+
console.log(`ERROR: Could not create temporary directory (${error.message})`);
|
|
43
|
+
jobTmpDir = null;
|
|
44
|
+
}
|
|
45
|
+
const tmpDirs = [os.tmpdir(), '/tmp'];
|
|
46
|
+
if (jobTmpDir) {
|
|
47
|
+
tmpDirs.unshift(jobTmpDir);
|
|
48
|
+
}
|
|
49
|
+
let tmpDir = null;
|
|
50
|
+
// For each potential temporary directory:
|
|
51
|
+
for (const tmpDirAlternative of tmpDirs) {
|
|
52
|
+
try {
|
|
53
|
+
// Verify that it is writable.
|
|
54
|
+
await fs.access(tmpDirAlternative, fs.constants.W_OK);
|
|
55
|
+
tmpDir = tmpDirAlternative;
|
|
56
|
+
// If it is, stop checking alternatives.
|
|
57
|
+
break;
|
|
58
|
+
}
|
|
59
|
+
// If it is not:
|
|
60
|
+
catch(error) {
|
|
61
|
+
// Report this and continue checking alternatives.
|
|
62
|
+
console.log(`ERROR: ${tmpDirAlternative} not writable for temporary files`);
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
// If no writable temporary directory was found:
|
|
66
|
+
if (! tmpDir) {
|
|
67
|
+
// Report this.
|
|
68
|
+
console.log('ERROR: No writable temporary directory was found');
|
|
69
|
+
}
|
|
70
|
+
// Return the directory path or a failure.
|
|
71
|
+
return tmpDir;
|
|
72
|
+
};
|
|
38
73
|
// Runs a job and returns a report.
|
|
39
74
|
exports.doJob = async (job, opts = {}) => {
|
|
40
75
|
// Initialize a report as a copy of the job.
|
|
41
76
|
let report = JSON.parse(JSON.stringify(job));
|
|
42
|
-
|
|
77
|
+
report.jobData ??= {};
|
|
78
|
+
const {jobData} = report;
|
|
43
79
|
// Get whether the job is valid and, if not, why not.
|
|
44
80
|
const jobInvalidity = isValidJob(job);
|
|
45
81
|
// If it is invalid:
|
|
@@ -54,9 +90,11 @@ exports.doJob = async (job, opts = {}) => {
|
|
|
54
90
|
else {
|
|
55
91
|
// Report this.
|
|
56
92
|
console.log(`Starting job ${job.id} (${job.target.what})`);
|
|
93
|
+
const tmpDir = await getTmpDirPath(job.id);
|
|
57
94
|
// Add initialized job data to the report.
|
|
58
95
|
const startTime = new Date();
|
|
59
96
|
report.jobData = {
|
|
97
|
+
tmpDir,
|
|
60
98
|
startTime: nowString(),
|
|
61
99
|
endTime: '',
|
|
62
100
|
elapsedSeconds: 0,
|
|
@@ -89,6 +127,8 @@ exports.doJob = async (job, opts = {}) => {
|
|
|
89
127
|
}
|
|
90
128
|
// Perform the acts and revise the report.
|
|
91
129
|
report = await doActs(report, opts);
|
|
130
|
+
// Delete the temporary directory.
|
|
131
|
+
await fs.rm(tmpDir, {recursive: true, force: true});
|
|
92
132
|
// Add the end time and duration to the report.
|
|
93
133
|
const endTime = new Date();
|
|
94
134
|
report.jobData.endTime = nowString();
|
package/testaro/motion.js
CHANGED
|
@@ -16,18 +16,13 @@
|
|
|
16
16
|
|
|
17
17
|
const {getXPathCatalogIndex} = require('../procs/xPath');
|
|
18
18
|
const fs = require('fs/promises');
|
|
19
|
-
const os = require('os');
|
|
20
19
|
const blazediff = require('@blazediff/core').diff;
|
|
21
20
|
const {PNG} = require('pngjs');
|
|
22
21
|
|
|
23
|
-
// CONSTANTS
|
|
24
|
-
|
|
25
|
-
const tmpDir = os.tmpdir();
|
|
26
|
-
|
|
27
22
|
// FUNCTIONS
|
|
28
23
|
|
|
29
24
|
// Runs the test and returns the result.
|
|
30
|
-
exports.reporter = async (
|
|
25
|
+
exports.reporter = async (_0, catalog, _1, tmpDir) => {
|
|
31
26
|
// Initialize the totals and standard instances.
|
|
32
27
|
const data = {};
|
|
33
28
|
const totals = [0, 0, 0, 0];
|
package/testaro/shoot0.js
CHANGED
|
@@ -15,9 +15,9 @@ const {shoot} = require('../procs/shoot');
|
|
|
15
15
|
// FUNCTIONS
|
|
16
16
|
|
|
17
17
|
// Makes and saves the first screenshot.
|
|
18
|
-
exports.reporter = async page => {
|
|
18
|
+
exports.reporter = async (page, _, __, tmpDir) => {
|
|
19
19
|
// Make and save the screenshot.
|
|
20
|
-
const pngPath = await shoot(page, 0);
|
|
20
|
+
const pngPath = await shoot(page, 0, tmpDir);
|
|
21
21
|
// Return whether the screenshot was prevented.
|
|
22
22
|
return {
|
|
23
23
|
data: {
|
package/testaro/shoot1.js
CHANGED
|
@@ -11,23 +11,18 @@
|
|
|
11
11
|
// IMPORTS
|
|
12
12
|
|
|
13
13
|
const fs = require('fs/promises');
|
|
14
|
-
const os = require('os');
|
|
15
14
|
const {shoot} = require('../procs/shoot');
|
|
16
15
|
|
|
17
|
-
// CONSTANTS
|
|
18
|
-
|
|
19
|
-
const tmpDir = os.tmpdir();
|
|
20
|
-
|
|
21
16
|
// FUNCTIONS
|
|
22
17
|
|
|
23
18
|
// Make and save the second screenshot.
|
|
24
|
-
exports.reporter = async page => {
|
|
19
|
+
exports.reporter = async (page, _0, _1, tmpDir) => {
|
|
25
20
|
const tempFileNames = await fs.readdir(tmpDir);
|
|
26
21
|
let pngPath = '';
|
|
27
22
|
// If there is a shoot0 file:
|
|
28
23
|
if (tempFileNames.includes('testaro-shoot-0.png')) {
|
|
29
24
|
// Make and save the screenshot.
|
|
30
|
-
pngPath = await shoot(page, 1);
|
|
25
|
+
pngPath = await shoot(page, 1, tmpDir);
|
|
31
26
|
}
|
|
32
27
|
// Return whether the screenshot was prevented.
|
|
33
28
|
return {
|
package/tests/nuVnu.js
CHANGED
|
@@ -16,15 +16,11 @@
|
|
|
16
16
|
|
|
17
17
|
// IMPORTS
|
|
18
18
|
|
|
19
|
-
const fs = require('fs/promises');
|
|
20
|
-
const os = require('os');
|
|
21
|
-
const {vnu} = require('vnu-jar');
|
|
22
19
|
const {curate, getContent} = require('../procs/nu');
|
|
23
20
|
const {getAttributeXPath, getXPathCatalogIndex} = require('../procs/xPath');
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
const tmpDir = os.tmpdir();
|
|
21
|
+
const fs = require('fs/promises');
|
|
22
|
+
const path = require('path');
|
|
23
|
+
const {vnu} = require('vnu-jar');
|
|
28
24
|
|
|
29
25
|
// FUNCTIONS
|
|
30
26
|
|
|
@@ -58,7 +54,7 @@ exports.reporter = async (page, report, actIndex) => {
|
|
|
58
54
|
const {testTarget} = content;
|
|
59
55
|
// If it was obtained and contains a test target:
|
|
60
56
|
if (testTarget) {
|
|
61
|
-
const pagePath =
|
|
57
|
+
const pagePath = path.join(report.jobData.tmpDir, 'nuVnu-page.html');
|
|
62
58
|
// Save the test target in a temporary file.
|
|
63
59
|
await fs.writeFile(pagePath, testTarget);
|
|
64
60
|
let nuData;
|
package/tests/testaro.js
CHANGED
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
// IMPORTS
|
|
17
17
|
|
|
18
18
|
// Shared configuration for timeout multiplier.
|
|
19
|
-
const {
|
|
19
|
+
const {applyMultiplier} = require('../procs/config');
|
|
20
20
|
const {launch} = require('../procs/launch');
|
|
21
21
|
|
|
22
22
|
// CONSTANTS
|
|
@@ -28,6 +28,7 @@ const allRules = [
|
|
|
28
28
|
what: 'first page screenshot',
|
|
29
29
|
contaminates: false,
|
|
30
30
|
needsAccessibleName: false,
|
|
31
|
+
needsTmpDir: true,
|
|
31
32
|
timeOut: 5,
|
|
32
33
|
defaultOn: true
|
|
33
34
|
},
|
|
@@ -332,6 +333,7 @@ const allRules = [
|
|
|
332
333
|
what: 'second page screenshot',
|
|
333
334
|
contaminates: false,
|
|
334
335
|
needsAccessibleName: false,
|
|
336
|
+
needsTmpDir: true,
|
|
335
337
|
timeOut: 5,
|
|
336
338
|
defaultOn: true
|
|
337
339
|
},
|
|
@@ -340,6 +342,7 @@ const allRules = [
|
|
|
340
342
|
what: 'motion without user request, measured across tests',
|
|
341
343
|
contaminates: false,
|
|
342
344
|
needsAccessibleName: false,
|
|
345
|
+
needsTmpDir: true,
|
|
343
346
|
timeOut: 5,
|
|
344
347
|
defaultOn: true
|
|
345
348
|
},
|
|
@@ -569,6 +572,11 @@ exports.reporter = async (page, report, actIndex) => {
|
|
|
569
572
|
}
|
|
570
573
|
// Initialize an argument array for the reporter.
|
|
571
574
|
const ruleArgs = [page, report.catalog, withItems];
|
|
575
|
+
// If the rule needs a temporary directory:
|
|
576
|
+
if (rule.needsTmpDir) {
|
|
577
|
+
// Add its path to the argument array.
|
|
578
|
+
ruleArgs.push(report.jobData.tmpDir);
|
|
579
|
+
}
|
|
572
580
|
// If the rule has extra arguments:
|
|
573
581
|
if (argRules?.includes(ruleResult.id)) {
|
|
574
582
|
// Add them to the argument array.
|
|
@@ -578,7 +586,7 @@ exports.reporter = async (page, report, actIndex) => {
|
|
|
578
586
|
let timer;
|
|
579
587
|
try {
|
|
580
588
|
// Apply a time limit to the test.
|
|
581
|
-
const timeLimit = 1000 *
|
|
589
|
+
const timeLimit = applyMultiplier(1000 * rule.timeOut);
|
|
582
590
|
let timeout;
|
|
583
591
|
// If the time limit expires during the test:
|
|
584
592
|
timer = new Promise(resolve => {
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
{
|
|
2
|
+
"id": "240101T1300-shoot-example",
|
|
3
|
+
"what": "Demonstrate the shoot action: full-page screenshots interleaved with scripted steps",
|
|
4
|
+
"strict": true,
|
|
5
|
+
"timeLimit": 30,
|
|
6
|
+
"acts": [
|
|
7
|
+
{
|
|
8
|
+
"type": "launch",
|
|
9
|
+
"which": "chromium",
|
|
10
|
+
"url": "https://example.edu/",
|
|
11
|
+
"what": "Chromium browser"
|
|
12
|
+
},
|
|
13
|
+
{
|
|
14
|
+
"type": "shoot",
|
|
15
|
+
"which": "before",
|
|
16
|
+
"what": "Snapshot the landing page before any interaction"
|
|
17
|
+
},
|
|
18
|
+
{
|
|
19
|
+
"type": "shoot",
|
|
20
|
+
"which": "before-masked",
|
|
21
|
+
"exclusion": "header",
|
|
22
|
+
"what": "Same view with the page header masked out"
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"type": "test",
|
|
26
|
+
"which": "testaro",
|
|
27
|
+
"withItems": false,
|
|
28
|
+
"stopOnFail": true,
|
|
29
|
+
"rules": [
|
|
30
|
+
"y",
|
|
31
|
+
"bulk"
|
|
32
|
+
]
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
"type": "shoot",
|
|
36
|
+
"which": "after-tests",
|
|
37
|
+
"what": "Snapshot the page after Testaro tests have run"
|
|
38
|
+
}
|
|
39
|
+
],
|
|
40
|
+
"sources": {},
|
|
41
|
+
"standard": "only",
|
|
42
|
+
"observe": false,
|
|
43
|
+
"sendReportTo": "http://localhost:3007/api",
|
|
44
|
+
"timeStamp": "240101T1300",
|
|
45
|
+
"creationTimeStamp": "240101T1300"
|
|
46
|
+
}
|
package/scratch/README.md
DELETED
|
@@ -1,12 +0,0 @@
|
|
|
1
|
-
# scratch
|
|
2
|
-
|
|
3
|
-
This directory is used for temporary files produced during execution of jobs.
|
|
4
|
-
|
|
5
|
-
## License
|
|
6
|
-
|
|
7
|
-
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
8
|
-
© 2025–2026 Jonathan Robert Pool.
|
|
9
|
-
|
|
10
|
-
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file at the project root for details.
|
|
11
|
-
|
|
12
|
-
SPDX-License-Identifier: MIT
|