testaro 72.5.0 → 73.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +1 -1
- package/CLAUDE.md +9 -0
- package/CONTRIBUTING.md +1 -1
- package/README.md +18 -44
- package/VALIDATION.md +1 -1
- package/aceconfig.js +1 -1
- package/actSpecs-doc.md +9 -0
- package/call.js +3 -2
- package/env.example +10 -2
- package/package.json +1 -1
- package/procs/config.js +4 -13
- package/procs/doActs.js +8 -28
- package/procs/job.js +8 -1
- package/procs/launch.js +1 -0
- package/procs/shoot.js +1 -6
- package/run.js +48 -8
- package/testaro/motion.js +1 -6
- package/testaro/shoot0.js +2 -2
- package/testaro/shoot1.js +2 -7
- package/tests/nuVnu.js +4 -8
- package/tests/testaro.js +10 -2
package/AGENTS.md
CHANGED
package/CLAUDE.md
CHANGED
|
@@ -114,3 +114,12 @@ Long comments are not broken into multiple lines per paragraph.
|
|
|
114
114
|
| `tests/testaro.js` | `allRules` registry for the testaro tool |
|
|
115
115
|
| `testaro/<ruleID>.js` | One file per Testaro rule |
|
|
116
116
|
| `validation/validateTest.js` | Core validation harness |
|
|
117
|
+
|
|
118
|
+
## License
|
|
119
|
+
|
|
120
|
+
© 2026 Jonathan Robert Pool.
|
|
121
|
+
|
|
122
|
+
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
123
|
+
at the project root for details.
|
|
124
|
+
|
|
125
|
+
SPDX-License-Identifier: MIT
|
package/CONTRIBUTING.md
CHANGED
|
@@ -69,7 +69,7 @@ From 12 February 2024 through 30 September 2025, contributors of code to Testaro
|
|
|
69
69
|
## License
|
|
70
70
|
|
|
71
71
|
© 2023–2024 CVS Health and/or one of its affiliates. All rights reserved.
|
|
72
|
-
© 2025 Jonathan Robert Pool.
|
|
72
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
73
73
|
|
|
74
74
|
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
75
75
|
at the project root for details.
|
package/README.md
CHANGED
|
@@ -83,13 +83,11 @@ The main concepts of Testaro are:
|
|
|
83
83
|
|
|
84
84
|
### Operating system and Node.js version
|
|
85
85
|
|
|
86
|
-
Testaro can be installed under a MacOS, Windows, Debian, or Ubuntu operating system.
|
|
87
|
-
|
|
88
|
-
Testaro is tested with the latest long-term-support version of [Node.js](https://nodejs.org/en/).
|
|
86
|
+
Testaro can be installed under a MacOS, Windows, Debian, or Ubuntu operating system with the latest long-term-support version of [Node.js](https://nodejs.org/en/).
|
|
89
87
|
|
|
90
88
|
### Browser security
|
|
91
89
|
|
|
92
|
-
Testaro is configured so that, when Playwright or Puppeteer (a dependency of Playwright and of some tools, including QualWeb) launches a `chromium` browser, the browser is [sandboxed](https://www.geeksforgeeks.org/ethical-hacking/what-is-browser-sandboxing/) for improved security. That is the default for Playwright and Puppeteer, and Testaro does not override that default. The host must therefore permit sandboxed browsers.
|
|
90
|
+
Testaro is configured so that, when Playwright or Puppeteer (a dependency of Playwright and of some tools, including QualWeb) launches a `chromium` browser, the browser is [sandboxed](https://www.geeksforgeeks.org/ethical-hacking/what-is-browser-sandboxing/) for improved security. That is the default for Playwright and Puppeteer, and Testaro does not override that default. The host must therefore permit sandboxed browsers. If you try to run Testaro on a host that prohibits sandboxed browsers, each attempted launch of a `chromium` browser will throw an error with a message complaining about the unavailability of a sandbox.
|
|
93
91
|
|
|
94
92
|
In some operating systems a sandboxed browser requires an [unprivileged user namespace](https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces). In one case, a `…userns.conf` file in the `/etc/sysctl.d` directory with the content `kernel.apparmor_restrict_unprivileged_userns = 1` prohibits unprivileged user namespaces and thereby makes sandboxed browsers unlaunchable.
|
|
95
93
|
|
|
@@ -97,7 +95,7 @@ In some operating systems a sandboxed browser requires an [unprivileged user nam
|
|
|
97
95
|
|
|
98
96
|
One way to cope with this prohibition is to configure Playwright and Puppeteer to launch `chromium` non-sandboxed. In both cases, launch arguments `'--no-sandbox'` and `'--disable-setuid-sandbox'` are available to specify this.
|
|
99
97
|
|
|
100
|
-
- For Playwright, `'--no-sandbox'` and `'--disable-setuid-sandbox'`
|
|
98
|
+
- For Playwright, `'--no-sandbox'` and `'--disable-setuid-sandbox'` are added to the arguments of `browserOptionArgs.push` in the Testaro `run.js` file.
|
|
101
99
|
- For the `qualWeb` tool, this is done in the Testaro `tests/qualweb.js` file, where the `qualWeb.start` method is called with an options argument. Its `args` array property is modified to include `'--no-sandbox'` and `'--disable-setuid-sandbox'`.
|
|
102
100
|
- The `ibm` tool, too, can launch a Puppeteer `chromium` browser, if page content instead of a Playwright page is passed to the `accessibilityChecker.getCompliance` method, or if the implementation of the tool is changed in the future. For anticipation of such a case, the Testaro `aceconfig.js` file is modified. That file defines a `module.exports` object with a `puppeteerArgs` property, and, `--no-sandbox` and `--disable-setuid-sandbox` are added to its array value.
|
|
103
101
|
|
|
@@ -105,7 +103,7 @@ Non-sandboxed browsers are less secure than sandboxed ones, particularly when th
|
|
|
105
103
|
|
|
106
104
|
### Option B
|
|
107
105
|
|
|
108
|
-
|
|
106
|
+
Another solution is to leave the `chromium` configuration unchanged, but configure the operating system to permit a sandboxed browser to be launched. In one case, this is implemented with:
|
|
109
107
|
|
|
110
108
|
```bash
|
|
111
109
|
sudo sysctl -w kernel.unprivileged_userns_clone=1
|
|
@@ -117,6 +115,8 @@ EOF
|
|
|
117
115
|
sudo sysctl --system
|
|
118
116
|
```
|
|
119
117
|
|
|
118
|
+
This repository implements option B.
|
|
119
|
+
|
|
120
120
|
## Installation
|
|
121
121
|
|
|
122
122
|
### As an independent application
|
|
@@ -137,36 +137,7 @@ You can make `testaro` a dependency in another application. As noted at the begi
|
|
|
137
137
|
|
|
138
138
|
## Environment configuration
|
|
139
139
|
|
|
140
|
-
The `.env` file stores your decisions about the environment in which Testaro runs. The variables that can be defined there are
|
|
141
|
-
|
|
142
|
-
```conf
|
|
143
|
-
# Whether the browsers launched by Testaro should have visible windows.
|
|
144
|
-
HEADED_BROWSER=false
|
|
145
|
-
# Whether console logging in launched browsers should be mirrored to the Testaro console.
|
|
146
|
-
DEBUG=false
|
|
147
|
-
# Whether to disable Puppeteer log warnings of a future headless-mode deprecation.
|
|
148
|
-
PUPPETEER_DISABLE_HEADLESS_WARNING=true
|
|
149
|
-
# How much time, in milliseconds, to insert between Playwright operations for debugging.
|
|
150
|
-
WAITS=0
|
|
151
|
-
# API key to enable the WAVE tool.
|
|
152
|
-
WAVE_KEY=yourwavekey (get it from [WebAim](https://wave.webaim.org/api/)).
|
|
153
|
-
#----------------------------
|
|
154
|
-
# When Testaro listens for new jobs in a directory:
|
|
155
|
-
# Directory where it listens for them.
|
|
156
|
-
JOBDIR=../testing/jobs
|
|
157
|
-
# Directory into which Testaro saves the reports of those jobs.
|
|
158
|
-
REPORTDIR=../testing/reports
|
|
159
|
-
# Name of this Testaro instance when it listens for jobs and sends reports to requesting hosts.
|
|
160
|
-
AGENT=agentabc
|
|
161
|
-
#----------------------------
|
|
162
|
-
# When Testaro polls a network host to ask for new jobs, data on the host.
|
|
163
|
-
# URL to poll.
|
|
164
|
-
NETWATCH_JOB=http://localhost:3000/api/assignJob/agentabc
|
|
165
|
-
# URL to which to send completed job reports.
|
|
166
|
-
NETWATCH_REPORT=http://localhost:3000/api/takeReport/agentabc
|
|
167
|
-
# Password to give to the host to authenticate this instance.
|
|
168
|
-
NETWATCH_AUTH=abcxyz
|
|
169
|
-
```
|
|
140
|
+
The `.env` file stores your decisions about the environment in which Testaro runs. The variables that can be defined there are documented in the `env.example` file.
|
|
170
141
|
|
|
171
142
|
## Jobs
|
|
172
143
|
|
|
@@ -243,7 +214,7 @@ There are 18 act types. They and their options are documented in the `etc` prope
|
|
|
243
214
|
|
|
244
215
|
#### Job as an object
|
|
245
216
|
|
|
246
|
-
An application can execute a job with
|
|
217
|
+
An application can execute a job with:
|
|
247
218
|
|
|
248
219
|
```javascript
|
|
249
220
|
const {doJob} = require('testaro/run');
|
|
@@ -320,8 +291,8 @@ A report is a job with information about the results of the performance of the j
|
|
|
320
291
|
|
|
321
292
|
As Testaro performs a job, information about the job as a whole is inserted into the job. That information is organized into one or two properties:
|
|
322
293
|
|
|
323
|
-
- `jobData`:
|
|
324
|
-
- `catalog`: A collection of data about the elements on the target that are relevant to any test failures
|
|
294
|
+
- `jobData`: Facts about the performance of the job
|
|
295
|
+
- `catalog`: A collection of data about the HTML elements on the target that are relevant to any test failures
|
|
325
296
|
|
|
326
297
|
Testaro inserts the `jobData` property into every job, but inserts the `catalog` property only into jobs that instruct Testaro to produce standard results.
|
|
327
298
|
|
|
@@ -356,7 +327,7 @@ Testaro uses the following techniques to make the tools calculate XPaths:
|
|
|
356
327
|
- `alfa` and `aslint`: They report XPaths, so Testaro needs only to normalize them.
|
|
357
328
|
- `ed11y`: Testaro adds it and a `window.getXPath` method to the page; when the tool reports an element, Testaro computes its XPath.
|
|
358
329
|
- `wave`: It reports a selector for each element; Testaro finds each element in the page via its selector and executes `window.getXPath` on the element.
|
|
359
|
-
- `axe`, `htmlcs`, `ibm`, `nuVal`, `nuVnu`, `qualWeb`: Testaro adds `data-xpath` attributes to all elements; the tools include code excerpts, with the `data-
|
|
330
|
+
- `axe`, `htmlcs`, `ibm`, `nuVal`, `nuVnu`, `qualWeb`: Testaro adds `data-xpath` attributes to all elements; the tools include code excerpts, with the `data-xpath` attributes, in the reported violations.
|
|
360
331
|
- `testaro`: Testaro designs each of its own tests to report element XPaths.
|
|
361
332
|
|
|
362
333
|
By attaching a catalog entry to each reported element, Testaro allows an application that uses Testaro to tell users, for any particular HTML element, which tools ascribed violations of which rules to that element. An application could, for example, use a screenshot or a text-fragment link or could ask the user to paste the XPath into a browser developer tool.
|
|
@@ -506,7 +477,7 @@ Thus, when the `rules` argument is omitted, QualWeb will test for all of the rul
|
|
|
506
477
|
|
|
507
478
|
The target can be provided to QualWeb either as HTML or as a URL. Experience indicates that the results can differ between these methods, with each method reporting some rule violations or some instances that the other method does not report. For at least some cases, more rules are reported violated when HTML is provided (`withNewItems: false`).
|
|
508
479
|
|
|
509
|
-
QualWeb creates sandboxed Puppeteer pages to perform its tests on. Therefore, the host must permit sandboxed browsers to be launched. See the pertinent [Kilotest documentation](https://github.com/jrpool/kilotest/blob/main/SERVICE.md#browser-privileges) for information about the configuration of an Ubuntu Linux host for this purpose.
|
|
480
|
+
QualWeb creates sandboxed Puppeteer pages to perform its tests on. Therefore, the host must permit sandboxed browsers to be launched. See the discussion above about browser security. Also see the pertinent [Kilotest documentation](https://github.com/jrpool/kilotest/blob/main/SERVICE.md#browser-privileges) for information about the configuration of an Ubuntu Linux host for this purpose.
|
|
510
481
|
|
|
511
482
|
### Testaro
|
|
512
483
|
|
|
@@ -600,7 +571,7 @@ Testaro normally performs tests with headless browsers. Some experiments appear
|
|
|
600
571
|
|
|
601
572
|
## Repository exclusions
|
|
602
573
|
|
|
603
|
-
|
|
574
|
+
Any files in the `temp` or `tmp` directory are presumed ephemeral and are not tracked by `git`. Jobs create temporary files in subdirectories of `tmp` and delete those subdirectories on termination.
|
|
604
575
|
|
|
605
576
|
## Related work
|
|
606
577
|
|
|
@@ -617,6 +588,10 @@ Testilo contains procedures that reorganize report data by issue and by element,
|
|
|
617
588
|
|
|
618
589
|
Report standardization could be performed by other software rather than by Testaro. That would require sending the original reports to the server. They are typically larger than standardized reports. Whenever users want only standardized reports, the fact that Testaro standardizes them eliminates the need to send the original reports anywhere.
|
|
619
590
|
|
|
591
|
+
### Kilotest
|
|
592
|
+
|
|
593
|
+
[Kilotest](https://www.npmjs.com/package/kilotest) is an application that offers a simplified interface to Testaro. At present it is [deployed as a public service](https://kilotest.com/).
|
|
594
|
+
|
|
620
595
|
## Code style
|
|
621
596
|
|
|
622
597
|
The JavaScript code in this project generally conforms to the ESLint configuration file `.eslintrc.json`. However, the `htmlcs/HTMLCS.js` file implements an older version of JavaScript. Its style is regulated by the `htmlcs/.eslintrc.json` file.
|
|
@@ -646,7 +621,6 @@ Future work on this project is being considered. Strategic recommendations for s
|
|
|
646
621
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
647
622
|
© 2025–2026 Jonathan Robert Pool.
|
|
648
623
|
|
|
649
|
-
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
650
|
-
at the project root for details.
|
|
624
|
+
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file at the project root for details.
|
|
651
625
|
|
|
652
626
|
SPDX-License-Identifier: MIT
|
package/VALIDATION.md
CHANGED
|
@@ -94,7 +94,7 @@ Preparing a PR:
|
|
|
94
94
|
## License
|
|
95
95
|
|
|
96
96
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
97
|
-
© 2025 Jonathan Robert Pool.
|
|
97
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
98
98
|
|
|
99
99
|
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
100
100
|
at the project root for details.
|
package/aceconfig.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
-
© 2025 Jonathan Robert Pool.
|
|
3
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
4
4
|
|
|
5
5
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
6
6
|
https://opensource.org/license/mit/ for details.
|
package/actSpecs-doc.md
CHANGED
|
@@ -51,3 +51,12 @@ The validity criterion named in item 2 may be any of these:
|
|
|
51
51
|
- `'isTest'`: is the name of a tool
|
|
52
52
|
- `'isWaitable'`: is `'url'`, `'title'`, or `'body'`
|
|
53
53
|
- `'areStrings'`: is an array of strings
|
|
54
|
+
|
|
55
|
+
## License
|
|
56
|
+
|
|
57
|
+
© 2026 Jonathan Robert Pool.
|
|
58
|
+
|
|
59
|
+
Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file
|
|
60
|
+
at the project root for details.
|
|
61
|
+
|
|
62
|
+
SPDX-License-Identifier: MIT
|
package/call.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2022–2024 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
+
© 2026 Jonathan Robert Pool.
|
|
3
4
|
|
|
4
5
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
5
6
|
https://opensource.org/license/mit/ for details.
|
|
@@ -42,7 +43,7 @@ const rawDir = `${reportDir}/raw`;
|
|
|
42
43
|
|
|
43
44
|
// FUNCTIONS
|
|
44
45
|
|
|
45
|
-
// Fulfills a
|
|
46
|
+
// Fulfills a request to perform a job.
|
|
46
47
|
const callRun = async jobIDStart => {
|
|
47
48
|
// Find the job.
|
|
48
49
|
const jobDirFileNames = await fs.readdir(todoDir);
|
|
@@ -56,7 +57,7 @@ const callRun = async jobIDStart => {
|
|
|
56
57
|
// Get it.
|
|
57
58
|
const jobJSON = await fs.readFile(`${todoDir}/${jobFileName}`, 'utf8');
|
|
58
59
|
let report = JSON.parse(jobJSON);
|
|
59
|
-
// Run
|
|
60
|
+
// Run the job.
|
|
60
61
|
report = await doJob(report);
|
|
61
62
|
// Archive it.
|
|
62
63
|
await fs.rename(`${todoDir}/${jobFileName}`, `${jobDir}/done/${jobFileName}`);
|
package/env.example
CHANGED
|
@@ -26,7 +26,15 @@ JOBDIR=__placeholder__
|
|
|
26
26
|
REPORTDIR=__placeholder__
|
|
27
27
|
# Multiplier for time limits (normally 1).
|
|
28
28
|
TIMEOUT_MULTIPLIER=1
|
|
29
|
-
# Name of the directory at the project root used for temporary files.
|
|
30
|
-
TMPDIRNAME=scratch
|
|
31
29
|
# Whether to abort the job when any test act fork crashes (default false).
|
|
32
30
|
ABORT_ASSERTIVELY=false
|
|
31
|
+
|
|
32
|
+
## License
|
|
33
|
+
|
|
34
|
+
# © 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
35
|
+
# © 2026 Jeff Witt.
|
|
36
|
+
# © 2026 Jonathan Robert Pool.
|
|
37
|
+
|
|
38
|
+
# Licensed under the [MIT License](https://opensource.org/license/mit/). See [LICENSE](../../LICENSE) file at the project root for details.
|
|
39
|
+
|
|
40
|
+
# SPDX-License-Identifier: MIT
|
package/package.json
CHANGED
package/procs/config.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
/*
|
|
2
|
-
© 2025 Jonathan Robert Pool.
|
|
2
|
+
© 2025–2026 Jonathan Robert Pool.
|
|
3
3
|
|
|
4
4
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
5
5
|
https://opensource.org/license/mit/ for details.
|
|
@@ -12,16 +12,7 @@
|
|
|
12
12
|
Shared configuration values for Testaro.
|
|
13
13
|
*/
|
|
14
14
|
|
|
15
|
-
//
|
|
16
|
-
// Set TIMEOUT_MULTIPLIER > 1 for slow networks/sites, < 1 for fast environments.
|
|
15
|
+
// Amount to multiply by specified time limits (normally 1) to adapt to network/site speed.
|
|
17
16
|
const timeoutMultiplier = Number.parseFloat(process.env.TIMEOUT_MULTIPLIER) || 1;
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
// Use for navigation, interaction, and long-running operation timeouts.
|
|
21
|
-
// Do NOT use for very short "fail-fast" timeouts (< 100ms).
|
|
22
|
-
const applyMultiplier = (baseTimeout) => Math.round(baseTimeout * timeoutMultiplier);
|
|
23
|
-
|
|
24
|
-
module.exports = {
|
|
25
|
-
timeoutMultiplier,
|
|
26
|
-
applyMultiplier
|
|
27
|
-
};
|
|
17
|
+
// Multiplies a time limit by the configured amount.
|
|
18
|
+
exports.applyMultiplier = (baseTimeout) => Math.round(baseTimeout * timeoutMultiplier);
|
package/procs/doActs.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
+
© 2026 Jeff Witt.
|
|
3
4
|
© 2025–2026 Jonathan Robert Pool.
|
|
4
5
|
|
|
5
6
|
Licensed under the MIT License. See LICENSE file at the project root or https://opensource.org/license/mit/ for details.
|
|
@@ -18,9 +19,10 @@ const {addError} = require('./error');
|
|
|
18
19
|
const {getNonce, goTo, launch, wait} = require('./launch');
|
|
19
20
|
const {tools} = require('./job');
|
|
20
21
|
const {fork} = require('child_process');
|
|
21
|
-
const os = require('os');
|
|
22
22
|
const {pruneCatalog} = require('./catalog');
|
|
23
|
+
const {applyMultiplier} = require('./config');
|
|
23
24
|
const fs = require('fs/promises');
|
|
25
|
+
const path = require('path');
|
|
24
26
|
|
|
25
27
|
// CONSTANTS
|
|
26
28
|
|
|
@@ -51,8 +53,6 @@ const timeLimits = {
|
|
|
51
53
|
testaro: 200 + Math.round(6 * waits / 1000),
|
|
52
54
|
wave: 25
|
|
53
55
|
};
|
|
54
|
-
// Timeout multiplier.
|
|
55
|
-
const timeoutMultiplier = Number.parseFloat(process.env.TIMEOUT_MULTIPLIER) || 1;
|
|
56
56
|
// Abort aggressiveness.
|
|
57
57
|
const abortAssertively = process.env.ABORT_ASSERTIVELY === 'true';
|
|
58
58
|
|
|
@@ -231,31 +231,11 @@ exports.doActs = async report => {
|
|
|
231
231
|
let {acts} = tempReport;
|
|
232
232
|
// Get the standardization specification.
|
|
233
233
|
const standard = tempReport.standard || 'only';
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
for (const tmpDirAlternative of tmpDirs) {
|
|
238
|
-
try {
|
|
239
|
-
// Verify that it is writable.
|
|
240
|
-
await fs.access(tmpDirAlternative, fs.constants.W_OK);
|
|
241
|
-
tmpDir = tmpDirAlternative;
|
|
242
|
-
break;
|
|
243
|
-
}
|
|
244
|
-
// If it is not:
|
|
245
|
-
catch(error) {
|
|
246
|
-
// Report this.
|
|
247
|
-
console.log(`ERROR: ${tmpDirAlternative} is not a writable directory for temporary reports`);
|
|
248
|
-
}
|
|
249
|
-
}
|
|
250
|
-
// If no writable temporary directory was found:
|
|
251
|
-
if (! tmpDir) {
|
|
252
|
-
// Report this.
|
|
253
|
-
console.log('ERROR: No writable temporary directory was found; quitting');
|
|
254
|
-
// Quit.
|
|
255
|
-
process.exit(1);
|
|
256
|
-
}
|
|
234
|
+
// Get the path to a writable temporary directory.
|
|
235
|
+
const {tmpDir} = report.jobData;
|
|
236
|
+
let reportPath;
|
|
257
237
|
// Get a path for temporary reports.
|
|
258
|
-
|
|
238
|
+
reportPath = path.join(tmpDir, `${tempReport.id}.json`);
|
|
259
239
|
// Initialize the count of completed acts.
|
|
260
240
|
let actCount = 0;
|
|
261
241
|
// For each act in the temporary report:
|
|
@@ -332,7 +312,7 @@ exports.doActs = async report => {
|
|
|
332
312
|
// Save a copy of the temporary report, which the child process will read.
|
|
333
313
|
await fs.writeFile(reportPath, tempReportJSON);
|
|
334
314
|
let timedOut = false;
|
|
335
|
-
const limitMs =
|
|
315
|
+
const limitMs = applyMultiplier(1000 * (timeLimits[act.which] || 15));
|
|
336
316
|
const actResult = await new Promise(resolve => {
|
|
337
317
|
let closed = false;
|
|
338
318
|
// Create a child process to perform the act.
|
package/procs/job.js
CHANGED
|
@@ -181,7 +181,8 @@ exports.isValidJob = job => {
|
|
|
181
181
|
executionTimeStamp,
|
|
182
182
|
target,
|
|
183
183
|
sources,
|
|
184
|
-
acts
|
|
184
|
+
acts,
|
|
185
|
+
jobData
|
|
185
186
|
} = job;
|
|
186
187
|
// Return an error for the first missing or invalid property.
|
|
187
188
|
if (! id || typeof id !== 'string') {
|
|
@@ -260,6 +261,12 @@ exports.isValidJob = job => {
|
|
|
260
261
|
error: `Invalid act:\n${JSON.stringify(invalidAct, null, 2)}`
|
|
261
262
|
};
|
|
262
263
|
}
|
|
264
|
+
if (jobData && typeof jobData !== 'object') {
|
|
265
|
+
return {
|
|
266
|
+
isValid: false,
|
|
267
|
+
error: 'Bad job jobData'
|
|
268
|
+
};
|
|
269
|
+
}
|
|
263
270
|
return {
|
|
264
271
|
isValid: true
|
|
265
272
|
};
|
package/procs/launch.js
CHANGED
package/procs/shoot.js
CHANGED
|
@@ -13,14 +13,9 @@
|
|
|
13
13
|
// Shared configuration for timeout multiplier.
|
|
14
14
|
const {applyMultiplier} = require('./config');
|
|
15
15
|
const fs = require('fs/promises');
|
|
16
|
-
const os = require('os');
|
|
17
16
|
const path = require('path');
|
|
18
17
|
const {PNG} = require('pngjs');
|
|
19
18
|
|
|
20
|
-
// CONSTANTS
|
|
21
|
-
|
|
22
|
-
const tmpDir = os.tmpdir();
|
|
23
|
-
|
|
24
19
|
// FUNCTIONS
|
|
25
20
|
|
|
26
21
|
// Creates and returns a screenshot.
|
|
@@ -40,7 +35,7 @@ const screenShot = async (page, exclusion = null) => {
|
|
|
40
35
|
return '';
|
|
41
36
|
});
|
|
42
37
|
};
|
|
43
|
-
exports.shoot = async (page, index) => {
|
|
38
|
+
exports.shoot = async (page, index, tmpDir) => {
|
|
44
39
|
// Make and get a screenshot as a buffer.
|
|
45
40
|
let shot = await screenShot(page);
|
|
46
41
|
// If it succeeded:
|
package/run.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
/*
|
|
2
2
|
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
+
© 2026 Jeff Witt.
|
|
3
4
|
© 2025–2026 Jonathan Robert Pool.
|
|
4
5
|
|
|
5
6
|
Licensed under the MIT License. See LICENSE file at the project root or
|
|
@@ -15,19 +16,14 @@
|
|
|
15
16
|
|
|
16
17
|
// IMPORTS
|
|
17
18
|
|
|
18
|
-
// Module to perform acts.
|
|
19
19
|
const {doActs} = require('./procs/doActs');
|
|
20
|
-
// Module to keep secrets.
|
|
21
20
|
require('dotenv').config({quiet: true});
|
|
22
|
-
// Function to validate jobs.
|
|
23
21
|
const {isValidJob} = require('./procs/job');
|
|
24
|
-
// Module to create catalogs.
|
|
25
22
|
const {getCatalog} = require('./procs/catalog');
|
|
26
|
-
// Module to process dates and times.
|
|
27
23
|
const {nowString} = require('./procs/dateTime');
|
|
28
|
-
// Module to create browsers.
|
|
29
24
|
const {chromium, webkit, firefox} = require('playwright-extra');
|
|
30
|
-
|
|
25
|
+
const fs = require('fs').promises;
|
|
26
|
+
const os = require('os');
|
|
31
27
|
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
|
|
32
28
|
chromium.use(StealthPlugin());
|
|
33
29
|
webkit.use(StealthPlugin());
|
|
@@ -35,11 +31,51 @@ firefox.use(StealthPlugin());
|
|
|
35
31
|
|
|
36
32
|
// FUNCTIONS
|
|
37
33
|
|
|
34
|
+
// Returns an operating-system-compatible absolute path to a temporary directory.
|
|
35
|
+
const getTmpDirPath = async jobName => {
|
|
36
|
+
let jobTmpDir = `${__dirname}/tmp/${jobName}`;
|
|
37
|
+
try {
|
|
38
|
+
// Ensure that a temporary directory exists.
|
|
39
|
+
await fs.mkdir(jobTmpDir, {recursive: true});
|
|
40
|
+
}
|
|
41
|
+
catch (error) {
|
|
42
|
+
console.log(`ERROR: Could not create temporary directory (${error.message})`);
|
|
43
|
+
jobTmpDir = null;
|
|
44
|
+
}
|
|
45
|
+
const tmpDirs = [os.tmpdir(), '/tmp'];
|
|
46
|
+
if (jobTmpDir) {
|
|
47
|
+
tmpDirs.unshift(jobTmpDir);
|
|
48
|
+
}
|
|
49
|
+
let tmpDir = null;
|
|
50
|
+
// For each potential temporary directory:
|
|
51
|
+
for (const tmpDirAlternative of tmpDirs) {
|
|
52
|
+
try {
|
|
53
|
+
// Verify that it is writable.
|
|
54
|
+
await fs.access(tmpDirAlternative, fs.constants.W_OK);
|
|
55
|
+
tmpDir = tmpDirAlternative;
|
|
56
|
+
// If it is, stop checking alternatives.
|
|
57
|
+
break;
|
|
58
|
+
}
|
|
59
|
+
// If it is not:
|
|
60
|
+
catch(error) {
|
|
61
|
+
// Report this and continue checking alternatives.
|
|
62
|
+
console.log(`ERROR: ${tmpDirAlternative} not writable for temporary files`);
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
// If no writable temporary directory was found:
|
|
66
|
+
if (! tmpDir) {
|
|
67
|
+
// Report this.
|
|
68
|
+
console.log('ERROR: No writable temporary directory was found');
|
|
69
|
+
}
|
|
70
|
+
// Return the directory path or a failure.
|
|
71
|
+
return tmpDir;
|
|
72
|
+
};
|
|
38
73
|
// Runs a job and returns a report.
|
|
39
74
|
exports.doJob = async (job, opts = {}) => {
|
|
40
75
|
// Initialize a report as a copy of the job.
|
|
41
76
|
let report = JSON.parse(JSON.stringify(job));
|
|
42
|
-
|
|
77
|
+
report.jobData ??= {};
|
|
78
|
+
const {jobData} = report;
|
|
43
79
|
// Get whether the job is valid and, if not, why not.
|
|
44
80
|
const jobInvalidity = isValidJob(job);
|
|
45
81
|
// If it is invalid:
|
|
@@ -54,9 +90,11 @@ exports.doJob = async (job, opts = {}) => {
|
|
|
54
90
|
else {
|
|
55
91
|
// Report this.
|
|
56
92
|
console.log(`Starting job ${job.id} (${job.target.what})`);
|
|
93
|
+
const tmpDir = await getTmpDirPath(job.id);
|
|
57
94
|
// Add initialized job data to the report.
|
|
58
95
|
const startTime = new Date();
|
|
59
96
|
report.jobData = {
|
|
97
|
+
tmpDir,
|
|
60
98
|
startTime: nowString(),
|
|
61
99
|
endTime: '',
|
|
62
100
|
elapsedSeconds: 0,
|
|
@@ -89,6 +127,8 @@ exports.doJob = async (job, opts = {}) => {
|
|
|
89
127
|
}
|
|
90
128
|
// Perform the acts and revise the report.
|
|
91
129
|
report = await doActs(report, opts);
|
|
130
|
+
// Delete the temporary directory.
|
|
131
|
+
await fs.rm(tmpDir, {recursive: true, force: true});
|
|
92
132
|
// Add the end time and duration to the report.
|
|
93
133
|
const endTime = new Date();
|
|
94
134
|
report.jobData.endTime = nowString();
|
package/testaro/motion.js
CHANGED
|
@@ -16,18 +16,13 @@
|
|
|
16
16
|
|
|
17
17
|
const {getXPathCatalogIndex} = require('../procs/xPath');
|
|
18
18
|
const fs = require('fs/promises');
|
|
19
|
-
const os = require('os');
|
|
20
19
|
const blazediff = require('@blazediff/core').diff;
|
|
21
20
|
const {PNG} = require('pngjs');
|
|
22
21
|
|
|
23
|
-
// CONSTANTS
|
|
24
|
-
|
|
25
|
-
const tmpDir = os.tmpdir();
|
|
26
|
-
|
|
27
22
|
// FUNCTIONS
|
|
28
23
|
|
|
29
24
|
// Runs the test and returns the result.
|
|
30
|
-
exports.reporter = async (
|
|
25
|
+
exports.reporter = async (_0, catalog, _1, tmpDir) => {
|
|
31
26
|
// Initialize the totals and standard instances.
|
|
32
27
|
const data = {};
|
|
33
28
|
const totals = [0, 0, 0, 0];
|
package/testaro/shoot0.js
CHANGED
|
@@ -15,9 +15,9 @@ const {shoot} = require('../procs/shoot');
|
|
|
15
15
|
// FUNCTIONS
|
|
16
16
|
|
|
17
17
|
// Makes and saves the first screenshot.
|
|
18
|
-
exports.reporter = async page => {
|
|
18
|
+
exports.reporter = async (page, _, __, tmpDir) => {
|
|
19
19
|
// Make and save the screenshot.
|
|
20
|
-
const pngPath = await shoot(page, 0);
|
|
20
|
+
const pngPath = await shoot(page, 0, tmpDir);
|
|
21
21
|
// Return whether the screenshot was prevented.
|
|
22
22
|
return {
|
|
23
23
|
data: {
|
package/testaro/shoot1.js
CHANGED
|
@@ -11,23 +11,18 @@
|
|
|
11
11
|
// IMPORTS
|
|
12
12
|
|
|
13
13
|
const fs = require('fs/promises');
|
|
14
|
-
const os = require('os');
|
|
15
14
|
const {shoot} = require('../procs/shoot');
|
|
16
15
|
|
|
17
|
-
// CONSTANTS
|
|
18
|
-
|
|
19
|
-
const tmpDir = os.tmpdir();
|
|
20
|
-
|
|
21
16
|
// FUNCTIONS
|
|
22
17
|
|
|
23
18
|
// Make and save the second screenshot.
|
|
24
|
-
exports.reporter = async page => {
|
|
19
|
+
exports.reporter = async (page, _0, _1, tmpDir) => {
|
|
25
20
|
const tempFileNames = await fs.readdir(tmpDir);
|
|
26
21
|
let pngPath = '';
|
|
27
22
|
// If there is a shoot0 file:
|
|
28
23
|
if (tempFileNames.includes('testaro-shoot-0.png')) {
|
|
29
24
|
// Make and save the screenshot.
|
|
30
|
-
pngPath = await shoot(page, 1);
|
|
25
|
+
pngPath = await shoot(page, 1, tmpDir);
|
|
31
26
|
}
|
|
32
27
|
// Return whether the screenshot was prevented.
|
|
33
28
|
return {
|
package/tests/nuVnu.js
CHANGED
|
@@ -16,15 +16,11 @@
|
|
|
16
16
|
|
|
17
17
|
// IMPORTS
|
|
18
18
|
|
|
19
|
-
const fs = require('fs/promises');
|
|
20
|
-
const os = require('os');
|
|
21
|
-
const {vnu} = require('vnu-jar');
|
|
22
19
|
const {curate, getContent} = require('../procs/nu');
|
|
23
20
|
const {getAttributeXPath, getXPathCatalogIndex} = require('../procs/xPath');
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
const tmpDir = os.tmpdir();
|
|
21
|
+
const fs = require('fs/promises');
|
|
22
|
+
const path = require('path');
|
|
23
|
+
const {vnu} = require('vnu-jar');
|
|
28
24
|
|
|
29
25
|
// FUNCTIONS
|
|
30
26
|
|
|
@@ -58,7 +54,7 @@ exports.reporter = async (page, report, actIndex) => {
|
|
|
58
54
|
const {testTarget} = content;
|
|
59
55
|
// If it was obtained and contains a test target:
|
|
60
56
|
if (testTarget) {
|
|
61
|
-
const pagePath =
|
|
57
|
+
const pagePath = path.join(report.jobData.tmpDir, 'nuVnu-page.html');
|
|
62
58
|
// Save the test target in a temporary file.
|
|
63
59
|
await fs.writeFile(pagePath, testTarget);
|
|
64
60
|
let nuData;
|
package/tests/testaro.js
CHANGED
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
// IMPORTS
|
|
17
17
|
|
|
18
18
|
// Shared configuration for timeout multiplier.
|
|
19
|
-
const {
|
|
19
|
+
const {applyMultiplier} = require('../procs/config');
|
|
20
20
|
const {launch} = require('../procs/launch');
|
|
21
21
|
|
|
22
22
|
// CONSTANTS
|
|
@@ -28,6 +28,7 @@ const allRules = [
|
|
|
28
28
|
what: 'first page screenshot',
|
|
29
29
|
contaminates: false,
|
|
30
30
|
needsAccessibleName: false,
|
|
31
|
+
needsTmpDir: true,
|
|
31
32
|
timeOut: 5,
|
|
32
33
|
defaultOn: true
|
|
33
34
|
},
|
|
@@ -332,6 +333,7 @@ const allRules = [
|
|
|
332
333
|
what: 'second page screenshot',
|
|
333
334
|
contaminates: false,
|
|
334
335
|
needsAccessibleName: false,
|
|
336
|
+
needsTmpDir: true,
|
|
335
337
|
timeOut: 5,
|
|
336
338
|
defaultOn: true
|
|
337
339
|
},
|
|
@@ -340,6 +342,7 @@ const allRules = [
|
|
|
340
342
|
what: 'motion without user request, measured across tests',
|
|
341
343
|
contaminates: false,
|
|
342
344
|
needsAccessibleName: false,
|
|
345
|
+
needsTmpDir: true,
|
|
343
346
|
timeOut: 5,
|
|
344
347
|
defaultOn: true
|
|
345
348
|
},
|
|
@@ -569,6 +572,11 @@ exports.reporter = async (page, report, actIndex) => {
|
|
|
569
572
|
}
|
|
570
573
|
// Initialize an argument array for the reporter.
|
|
571
574
|
const ruleArgs = [page, report.catalog, withItems];
|
|
575
|
+
// If the rule needs a temporary directory:
|
|
576
|
+
if (rule.needsTmpDir) {
|
|
577
|
+
// Add its path to the argument array.
|
|
578
|
+
ruleArgs.push(report.jobData.tmpDir);
|
|
579
|
+
}
|
|
572
580
|
// If the rule has extra arguments:
|
|
573
581
|
if (argRules?.includes(ruleResult.id)) {
|
|
574
582
|
// Add them to the argument array.
|
|
@@ -578,7 +586,7 @@ exports.reporter = async (page, report, actIndex) => {
|
|
|
578
586
|
let timer;
|
|
579
587
|
try {
|
|
580
588
|
// Apply a time limit to the test.
|
|
581
|
-
const timeLimit = 1000 *
|
|
589
|
+
const timeLimit = applyMultiplier(1000 * rule.timeOut);
|
|
582
590
|
let timeout;
|
|
583
591
|
// If the time limit expires during the test:
|
|
584
592
|
timer = new Promise(resolve => {
|