testaro 11.0.0 → 12.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (160) hide show
  1. package/.eslintrc.json +41 -41
  2. package/LICENSE +21 -21
  3. package/README.md +57 -61
  4. package/actSpecs.js +364 -358
  5. package/call.js +85 -85
  6. package/continuum/AccessEngine.community.js +791 -791
  7. package/continuum/Continuum.community.js +2697 -2697
  8. package/continuum/continuum.conf.js +19 -19
  9. package/data/roles.txt +62 -62
  10. package/htmlcs/.eslintrc.json +67 -67
  11. package/htmlcs/HTMLCS.js +8204 -8204
  12. package/htmlcs/licence.txt +24 -24
  13. package/package.json +2 -1
  14. package/procs/allText.js +76 -76
  15. package/procs/allVis.js +17 -17
  16. package/procs/linksByType.js +54 -54
  17. package/procs/textOf.txt +73 -73
  18. package/procs/wavedoc.json +1617 -1617
  19. package/run.js +1549 -1548
  20. package/samples/00000-ts18-example.json +231 -231
  21. package/tests/alfa.js +138 -138
  22. package/tests/allHidden.js +49 -49
  23. package/tests/attVal.js +48 -48
  24. package/tests/axe.js +118 -118
  25. package/tests/bulk.js +22 -22
  26. package/tests/continuum.js +71 -71
  27. package/tests/docType.js +14 -14
  28. package/tests/elements.js +159 -159
  29. package/tests/embAc.js +36 -36
  30. package/tests/filter.js +47 -47
  31. package/tests/focAll.js +57 -57
  32. package/tests/focInd.js +167 -167
  33. package/tests/focOp.js +138 -138
  34. package/tests/focVis.js +29 -29
  35. package/tests/hover.js +381 -381
  36. package/tests/htmlcs.js +90 -90
  37. package/tests/labClash.js +159 -159
  38. package/tests/linkTo.js +25 -25
  39. package/tests/linkUl.js +68 -68
  40. package/tests/menuNav.js +264 -264
  41. package/tests/miniText.js +48 -48
  42. package/tests/motion.js +125 -125
  43. package/tests/nonTable.js +67 -67
  44. package/tests/nuVal.js +92 -92
  45. package/tests/qualWeb.js +85 -0
  46. package/tests/radioSet.js +97 -97
  47. package/tests/role.js +483 -483
  48. package/tests/styleDiff.js +153 -153
  49. package/tests/tabNav.js +328 -328
  50. package/tests/tenon.js +130 -130
  51. package/tests/textNodes.js +140 -140
  52. package/tests/title.js +11 -11
  53. package/tests/titledEl.js +31 -31
  54. package/tests/wave.js +59 -59
  55. package/tests/zIndex.js +49 -49
  56. package/validation/done/README.md +3 -3
  57. package/validation/executors/run.js +40 -40
  58. package/validation/executors/test.js +7 -7
  59. package/validation/executors/tests.js +22 -22
  60. package/validation/executors/watchDir.js +28 -28
  61. package/validation/executors/watchNet.js +117 -117
  62. package/validation/jobs/done/README.md +3 -3
  63. package/validation/jobs/todo/00000-simple-example.json +35 -35
  64. package/validation/jobs/todo/README.md +3 -3
  65. package/validation/tests/jobs/allHidden.json +314 -314
  66. package/validation/tests/jobs/bulk.json +48 -48
  67. package/validation/tests/jobs/docType.json +46 -46
  68. package/validation/tests/jobs/elements.json +140 -140
  69. package/validation/tests/jobs/embAc.json +54 -54
  70. package/validation/tests/jobs/filter.json +55 -55
  71. package/validation/tests/jobs/focAll.json +68 -68
  72. package/validation/tests/jobs/focInd.json +69 -69
  73. package/validation/tests/jobs/focOp.json +62 -62
  74. package/validation/tests/jobs/focVis.json +35 -35
  75. package/validation/tests/jobs/hover.json +118 -118
  76. package/validation/tests/jobs/labClash.json +52 -52
  77. package/validation/tests/jobs/linkTo.json +35 -35
  78. package/validation/tests/jobs/linkUl.json +71 -71
  79. package/validation/tests/jobs/menuNav.json +106 -106
  80. package/validation/tests/jobs/miniText.json +36 -36
  81. package/validation/tests/jobs/motion.json +62 -62
  82. package/validation/tests/jobs/nonTable.json +37 -37
  83. package/validation/tests/jobs/radioSet.json +52 -52
  84. package/validation/tests/jobs/role.json +60 -60
  85. package/validation/tests/jobs/styleDiff.json +71 -71
  86. package/validation/tests/jobs/tabNav.json +106 -106
  87. package/validation/tests/jobs/textNodes.json +98 -98
  88. package/validation/tests/jobs/title.json +46 -46
  89. package/validation/tests/jobs/titledEl.json +37 -37
  90. package/validation/tests/jobs/zIndex.json +49 -49
  91. package/validation/tests/targets/allHidden/ariaHiddenMain.html +15 -15
  92. package/validation/tests/targets/allHidden/good.html +15 -15
  93. package/validation/tests/targets/allHidden/hiddenMain.html +16 -16
  94. package/validation/tests/targets/allHidden/mixedHidden0.html +16 -16
  95. package/validation/tests/targets/allHidden/mixedHidden1.html +16 -16
  96. package/validation/tests/targets/allHidden/noBody.html +11 -11
  97. package/validation/tests/targets/allHidden/noMain.html +13 -13
  98. package/validation/tests/targets/allHidden/noneDoc.html +15 -15
  99. package/validation/tests/targets/allHidden/noneMain.html +15 -15
  100. package/validation/tests/targets/allHidden/semiHidden.html +15 -15
  101. package/validation/tests/targets/allHidden/visHiddenMain.html +15 -15
  102. package/validation/tests/targets/bulk/bad.html +48 -48
  103. package/validation/tests/targets/bulk/good.html +15 -15
  104. package/validation/tests/targets/docType/bad.html +14 -14
  105. package/validation/tests/targets/docType/good.html +15 -15
  106. package/validation/tests/targets/elements/index.html +40 -40
  107. package/validation/tests/targets/embAc/bad.html +21 -21
  108. package/validation/tests/targets/embAc/good.html +15 -15
  109. package/validation/tests/targets/filter/bad.html +27 -27
  110. package/validation/tests/targets/filter/good.html +19 -19
  111. package/validation/tests/targets/focAll/good.html +15 -15
  112. package/validation/tests/targets/focAll/less.html +15 -15
  113. package/validation/tests/targets/focAll/more.html +16 -16
  114. package/validation/tests/targets/focInd/bad.html +66 -66
  115. package/validation/tests/targets/focInd/good.html +22 -22
  116. package/validation/tests/targets/focOp/bad.html +18 -18
  117. package/validation/tests/targets/focOp/good.html +15 -15
  118. package/validation/tests/targets/focVis/index.html +23 -23
  119. package/validation/tests/targets/hover/bad.html +25 -25
  120. package/validation/tests/targets/hover/good.html +20 -20
  121. package/validation/tests/targets/hover/large.html +27 -27
  122. package/validation/tests/targets/hover/styleBad.html +35 -35
  123. package/validation/tests/targets/labClash/bad.html +24 -24
  124. package/validation/tests/targets/labClash/good.html +22 -22
  125. package/validation/tests/targets/linkTo/index.html +15 -15
  126. package/validation/tests/targets/linkUl/bad.html +35 -35
  127. package/validation/tests/targets/linkUl/good.html +30 -30
  128. package/validation/tests/targets/linkUl/na.html +20 -20
  129. package/validation/tests/targets/menuNav/bad.html +106 -106
  130. package/validation/tests/targets/menuNav/bad.js +348 -348
  131. package/validation/tests/targets/menuNav/good.html +106 -106
  132. package/validation/tests/targets/menuNav/good.js +365 -365
  133. package/validation/tests/targets/menuNav/style.css +22 -22
  134. package/validation/tests/targets/miniText/index.html +33 -33
  135. package/validation/tests/targets/motion/bad.css +15 -15
  136. package/validation/tests/targets/motion/bad.html +16 -16
  137. package/validation/tests/targets/motion/good.html +15 -15
  138. package/validation/tests/targets/nonTable/index.html +65 -65
  139. package/validation/tests/targets/radioSet/bad.html +34 -34
  140. package/validation/tests/targets/radioSet/good.html +27 -27
  141. package/validation/tests/targets/role/bad.html +48 -48
  142. package/validation/tests/targets/role/good.html +22 -22
  143. package/validation/tests/targets/styleDiff/bad.html +35 -35
  144. package/validation/tests/targets/styleDiff/good.html +36 -36
  145. package/validation/tests/targets/tabNav/bad.html +51 -51
  146. package/validation/tests/targets/tabNav/bad.js +35 -35
  147. package/validation/tests/targets/tabNav/good.html +53 -53
  148. package/validation/tests/targets/tabNav/good.js +83 -83
  149. package/validation/tests/targets/tabNav/goodMoz.js +205 -205
  150. package/validation/tests/targets/tabNav/style.css +34 -34
  151. package/validation/tests/targets/textNodes/index.html +40 -40
  152. package/validation/tests/targets/title/bad.html +14 -14
  153. package/validation/tests/targets/title/good.html +15 -15
  154. package/validation/tests/targets/titledEl/index.html +17 -17
  155. package/validation/tests/targets/zIndex/bad.html +17 -17
  156. package/validation/tests/targets/zIndex/good.html +15 -15
  157. package/validation/validateTest.js +44 -44
  158. package/validation/watch/done/README.md +3 -3
  159. package/validation/watch/todo/README.md +3 -3
  160. package/watch.js +4 -1
package/.eslintrc.json CHANGED
@@ -1,41 +1,41 @@
1
- {
2
- "env": {
3
- "browser": true,
4
- "commonjs": true,
5
- "es2021": true,
6
- "node": true
7
- },
8
- "extends": "eslint:recommended",
9
- "parserOptions": {
10
- "ecmaVersion": 2021
11
- },
12
- "rules": {
13
- "indent": [
14
- "error",
15
- 2,
16
- {
17
- "MemberExpression": 0,
18
- "ObjectExpression": "first"
19
- }
20
- ],
21
- "linebreak-style": [
22
- "error",
23
- "unix"
24
- ],
25
- "quotes": [
26
- "error",
27
- "single"
28
- ],
29
- "semi": [
30
- "error",
31
- "always"
32
- ],
33
- "no-use-before-define": [
34
- "error"
35
- ],
36
- "brace-style": [
37
- "error",
38
- "stroustrup"
39
- ]
40
- }
41
- }
1
+ {
2
+ "env": {
3
+ "browser": true,
4
+ "commonjs": true,
5
+ "es2021": true,
6
+ "node": true
7
+ },
8
+ "extends": "eslint:recommended",
9
+ "parserOptions": {
10
+ "ecmaVersion": 2021
11
+ },
12
+ "rules": {
13
+ "indent": [
14
+ "error",
15
+ 2,
16
+ {
17
+ "MemberExpression": 0,
18
+ "ObjectExpression": "first"
19
+ }
20
+ ],
21
+ "linebreak-style": [
22
+ "error",
23
+ "unix"
24
+ ],
25
+ "quotes": [
26
+ "error",
27
+ "single"
28
+ ],
29
+ "semi": [
30
+ "error",
31
+ "always"
32
+ ],
33
+ "no-use-before-define": [
34
+ "error"
35
+ ],
36
+ "brace-style": [
37
+ "error",
38
+ "stroustrup"
39
+ ]
40
+ }
41
+ }
package/LICENSE CHANGED
@@ -1,21 +1,21 @@
1
- MIT License
2
-
3
- Copyright (c) 2021 Jonathan Pool
4
-
5
- Permission is hereby granted, free of charge, to any person obtaining a copy
6
- of this software and associated documentation files (the "Software"), to deal
7
- in the Software without restriction, including without limitation the rights
8
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
- copies of the Software, and to permit persons to whom the Software is
10
- furnished to do so, subject to the following conditions:
11
-
12
- The above copyright notice and this permission notice shall be included in all
13
- copies or substantial portions of the Software.
14
-
15
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
- SOFTWARE.
1
+ MIT License
2
+
3
+ Copyright (c) 2021 Jonathan Pool
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -4,9 +4,9 @@ Federated accessibility test automation
4
4
 
5
5
  ## Summary
6
6
 
7
- Testaro is a collection of collections of web accessibility tests.
7
+ Testaro is a collection of web accessibility testing tools.
8
8
 
9
- The purpose of Testaro is to provide programmatic access to accessibility tests defined in several test packages, including Testaro itself.
9
+ The purpose of Testaro is to provide programmatic access to accessibility tests defined by several tools, including Testaro itself.
10
10
 
11
11
  Testaro launches and controls web browsers, performing operations, conducting tests, and recording results.
12
12
 
@@ -31,30 +31,32 @@ Testaro uses:
31
31
  - [Playwright](https://playwright.dev/) to launch browsers, perform user actions in them, and perform tests
32
32
  - [pixelmatch](https://www.npmjs.com/package/pixelmatch) to measure motion
33
33
 
34
- Testaro includes some of its own accessibility tests. In addition, it performs the tests in:
34
+ Testaro includes some of its own accessibility tests. In addition, it performs the tests of these tools:
35
35
  - [accessibility-checker](https://www.npmjs.com/package/accessibility-checker) (the IBM Equal Access Accessibility Checker)
36
36
  - [alfa](https://alfa.siteimprove.com/) (Siteimprove alfa)
37
+ - [axe-playwright](https://www.npmjs.com/package/axe-playwright) (Deque Axe-core)
37
38
  - [Continuum Community Edition](https://www.webaccessibility.com/tools/)
38
39
  - [HTML CodeSniffer](https://www.npmjs.com/package/html_codesniffer) (Squiz HTML CodeSniffer)
39
- - [axe-playwright](https://www.npmjs.com/package/axe-playwright) (Deque Axe-core)
40
+ - [Nu Html Checker](https://github.com/validator/validator)
41
+ - [QualWeb core](https://www.npmjs.com/package/@qualweb/core)
40
42
  - [Tenon](https://tenon.io/documentation/what-tenon-tests.php) (Level Access)
41
43
  - [WAVE API](https://wave.webaim.org/api/) (WebAIM WAVE)
42
- - [Nu Html Checker](https://github.com/validator/validator)
43
44
 
44
45
  Some of the Testaro tests are derived from tests performed by the [BBC Accessibility Standards Checker](https://github.com/bbc/bbc-a11y).
45
46
 
46
- As of this version, the counts of tests in the packages referenced above were:
47
+ As of this version, the counts of tests of the tools referenced above were:
47
48
  - Alfa: 103
48
49
  - Axe-core: 138
49
50
  - Continuum Community Edition: 267
50
51
  - Equal Access: 163
51
52
  - HTML CodeSniffer: 98
53
+ - Nu Html Checker: 147
54
+ - QualWeb core: 121
52
55
  - Tenon: 180
53
56
  - WAVE: 110
54
- - Nu Html Checker: 147
55
- - subtotal: 1206
57
+ - subtotal: 1327
56
58
  - Testaro tests: 24
57
- - grand total: 1230
59
+ - grand total: 1351
58
60
 
59
61
  ## Quasi-tests
60
62
 
@@ -103,7 +105,7 @@ However, if the Playwright dependency is ever updated to a newer version, you mu
103
105
 
104
106
  ## Payment
105
107
 
106
- All of the tests that Testaro can perform are free of cost, except those in the Tenon and WAVE packages. The owner of each of those packages gives new registrants a free allowance of credits before it becomes necessary to pay for use of the API of the package. The required environment variables for authentication and payment are described below under “Environment variables”.
108
+ All of the tests that Testaro can perform are free of cost, except those performed by the Tenon and WAVE tools. The owner of each of those tools gives new registrants a free allowance of credits before it becomes necessary to pay for use of the API of the tool. The required environment variables for authentication and payment are described below under “Environment variables”.
107
109
 
108
110
  ## Process objects
109
111
 
@@ -111,7 +113,7 @@ All of the tests that Testaro can perform are free of cost, except those in the
111
113
 
112
114
  A _job_ is an object containing instructions for Testaro.
113
115
 
114
- A _report_ is a job with added properties describing the results from Testaro running the job.
116
+ A _report_ is a job with properties added by Testaro, describing the results.
115
117
 
116
118
  ### Jobs
117
119
 
@@ -138,7 +140,7 @@ Here is an example of a job:
138
140
  {
139
141
  type: 'test',
140
142
  which: 'alfa',
141
- what: 'Siteimprove alfa package'
143
+ what: 'Siteimprove alfa tool'
142
144
  }
143
145
  ],
144
146
  sources: {
@@ -157,21 +159,21 @@ Here is an example of a job:
157
159
 
158
160
  This job contains three `acts`, telling Testaro to:
159
161
  1. open a page in the Chromium browser
160
- 1. navigate to some URL
161
- 1. perform the tests in the `alfa` package on that URL
162
+ 1. navigate to a specified URL
163
+ 1. perform the tests of the `alfa` tool on that URL
162
164
 
163
165
  Job properties:
164
- - `id`: This is a string consisting of alphanumeric ASCII characters and hyphen-minus (-), intended to be unique. When this job is saved as a JSON file, the file name is `be76p-sp25-w3c.json`. Typically, a job is created from a _script_, and the job ID adds a timestamp prefix and a target suffix to the script ID. Here the script ID would have been `ts25`.
166
+ - `id`: This is a string consisting of alphanumeric ASCII characters and hyphen-minus (-), intended to be unique. When the above example job is saved as a JSON file, the file name is `be76p-ts25-w3c.json`. Typically, a job is created from a _script_, and the job ID adds a timestamp prefix and a target suffix to the script ID. Here the script ID would have been `ts25`.
165
167
  - `what`: This is a description of the job.
166
168
  - `strict`: This is `true` or `false`, indicating whether _substantive redirections_ should be treated as failures. These are redirections that do more than add or subtract a final slash. For example, if `strict` is true, a redirection from `xyz.com` to `www.xyz.com` or to `xyz.com/en` will abort the job.
167
169
  - `timeLimit`: This property is the number of seconds allowed for the execution of the job.
168
170
  - `acts`: This is an array of the acts to be performed. Acts are documented below.
169
171
  - `sources`: This object has properties describing where the job came from:
170
- - `script`: This is the ID of the script from which the job was made. Other applications, such as Testilo, can make jobs from scripts. When Testilo creates a job, the job inherits its `id`, `what`, `strict`, `timeLimit`, and `acts` properties from the script. However, Testilo can create multiple jobs from a single script, replacing acts of type `placeholder` with one or more target-specific acts. Examples of scripts can be found in the Testilo package.
172
+ - `script`: This is the ID of the script from which the job was made, if it was made from a script, or is otherwise an empty string. Other applications, such as Testilo, can make jobs from scripts. When Testilo creates a job, the job inherits its `id`, `what`, `strict`, `timeLimit`, and `acts` properties from the script. However, Testilo can create multiple jobs from a single script, replacing acts of type `placeholder` with one or more target-specific acts. Examples of scripts can be found in the Testilo package.
171
173
  - `batch`: If the job was one of a set of jobs created by a merger of a script and a batch of targets, this property’s value is the ID of the batch, or otherwise is an empty string.
172
- - `target`: If the job was made from a script with placeholder acts, this property describes the target whose target-specific acts have replaced the placeholder acts. Otherwise `target` is an empty object. Testilo also makes the `id` property of the target the third segment of the job ID.
174
+ - `target`: If the job was made from a script with placeholder acts, this property describes the target whose target-specific acts have replaced the placeholder acts. Otherwise `target` is an empty object. Testilo also uses the `id` property of the target as the third segment of the job ID.
173
175
  - `requester`: This string is the email address to receive a notice of completion of the running of the job.
174
- - `creationTime`: This is the time when the job was created from a script.
176
+ - `creationTime`: This is the time when the job was created.
175
177
  - `timeStamp`: This string is a compact representation of the job creation time, suitable for inclusion in the ID of the job.
176
178
 
177
179
  ### Reports
@@ -198,7 +200,7 @@ The subsequent acts can tell Testaro to perform any of:
198
200
  - _moves_ (clicks, text inputs, hovers, etc.)
199
201
  - _navigations_ (browser launches, visits to URLs, waits for page conditions, etc.)
200
202
  - _alterations_ (changes to the page)
201
- - _tests_ (whether in dependency packages or defined within Testaro)
203
+ - _tests_ (whether in dependency tools or defined within Testaro)
202
204
  - _branching_ (continuing from an act other than the next one)
203
205
 
204
206
  ##### Moves
@@ -222,9 +224,9 @@ When the texts of multiple elements of the same type will contain the same `whic
222
224
 
223
225
  ##### Navigations
224
226
 
225
- An example of a **navigation** is the command of type `url` above.
227
+ An example of a **navigation** is the act of type `url` above.
226
228
 
227
- Once you have included a `url` command in a job, you do not need to add more `url` commands unless you want the browser to visit a different URL or revisit the same URL.
229
+ Once you have included a `url` act in a job, you do not need to add more `url` acts unless you want the browser to visit a different URL or revisit the same URL.
228
230
 
229
231
  If any act alters the page, you can restore the page to its original state for the next act by inserting new `launch` and `url` acts (and, if necessary, additional page-specific acts) between them.
230
232
 
@@ -240,7 +242,7 @@ Another navigation example is:
240
242
 
241
243
  In this case, Testaro waits until the page title contains the string “travel” (case-insensitively).
242
244
 
243
- The `launch` navigation command allows you to specify a “lowMotion” property as `true`. If you do, then the browser creates tabs with the `reduce-motion` option set to `reduce` instead of `no-preference`. This makes the browser act as if the user has chosen a [motion-reduction option in the settings of the operating system or browser](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-reduced-motion#user_preferences). However, there are often motions on web pages that this option fails to suppress, such as those on the [Inditex](https://www.inditex.com/itxcomweb/en/home) and [Rescuing Leftover Cuisine](https://www.rescuingleftovercuisine.org) home pages. Carousel motion is also not suppressed.
245
+ The `launch` navigation act allows you to specify a “lowMotion” property as `true`. If you do, then the browser creates tabs with the `reduce-motion` option set to `reduce` instead of `no-preference`. This makes the browser act as if the user has chosen a [motion-reduction option in the settings of the operating system or browser](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-reduced-motion#user_preferences). However, there are often motions on web pages that this option fails to suppress, such as those on the [Inditex](https://www.inditex.com/itxcomweb/en/home) and [Rescuing Leftover Cuisine](https://www.rescuingleftovercuisine.org) home pages. Carousel motion is also not suppressed.
244
246
 
245
247
  ##### Alterations
246
248
 
@@ -253,19 +255,15 @@ An example of an **alteration** is:
253
255
  }
254
256
  ```
255
257
 
256
- This command causes Testaro to alter the `display` and `visibility` style properties of all elements, where necessary, so those properties do not make any element invisible.
258
+ This act causes Testaro to alter the `display` and `visibility` style properties of all elements, where necessary, so those properties do not make any element invisible.
257
259
 
258
260
  ##### Tests
259
261
 
260
262
  ###### Introduction
261
263
 
262
- A test performs operations and reports results. The results may or may not directly indicate that a page passes or fails requirements. Typically, accessibility tests report successes and failures. But a test in Testaro is defined less restrictively, so it can report any results. As one example, the Testaro `elements` test reports facts about certain elements on a page, without asserting that those facts are successes or failures.
264
+ An act of type `test` performs operations and reports a result. The result may indicate that a page passes or fails requirements. Typically, accessibility tests report successes and failures. But a test in Testaro is defined less restrictively, so it can report any result. As one example, the Testaro `elements` test reports facts about certain elements on a page, without asserting that those facts are successes or failures.
263
265
 
264
- The term test has two meanings for Testaro:
265
- - An act is a test (_test command_) if its `type` property has the value `test`.
266
- - An act with type `test` whose `which` value is the name of a package, such as Continuum, performs multiple tests defined by that _test package_.
267
-
268
- Thus, if a command of type `test` runs Continuum, Continuum performs multiple tests and reports their results.
266
+ The `which` property of a `test` act identifies the operations to perform. If the value of `which` is the name of one of the tools, such as `alfa`, then the operations are some or all of the tests of that tool. If the value is the name of a Testaro test, then the operations are those of that single Testaro test. Thus, a single `test` act may specify performing anything from a single test to hundreds of tests.
269
267
 
270
268
  ###### Configuration
271
269
 
@@ -285,13 +283,13 @@ test: [
285
283
  ],
286
284
  ```
287
285
 
288
- That means that a test (i.e. a command with a `type` property having the value `'test'`) must have a string-valued `which` property naming a test and may optionally have a string-valued `what` property describing the test.
286
+ That means that a test (i.e. an act with a `type` property having the value `'test'`) must have a string-valued `which` property naming a test and may optionally have a string-valued `what` property describing the test.
289
287
 
290
288
  If a particular test either must have or may have any other properties, those properties must be specified in the `tests` property in `actSpecs.js`.
291
289
 
292
290
  ###### Examples
293
291
 
294
- An example of a **packaged test** is:
292
+ An example of a `test` act invoking a **tool** is:
295
293
 
296
294
  ```json
297
295
  {
@@ -304,7 +302,7 @@ An example of a **packaged test** is:
304
302
 
305
303
  In this case, Testaro runs the WAVE test with report type 1.
306
304
 
307
- An example of a **Testaro-defined** test is:
305
+ An example of a `test` act invoking a **Testaro** test is:
308
306
 
309
307
  ```json
310
308
  {
@@ -321,15 +319,15 @@ In this case, Testaro runs the `motion` test with the specified parameters.
321
319
 
322
320
  ###### Continuum
323
321
 
324
- The `continuum` packaged test makes use of the files in the `continuum` directory. The test inserts the contents of all three files into the page as scripts and then uses them to perform the tests of the Continuum package.
322
+ The `continuum` tests makes use of the files in the `continuum` directory. The test inserts the contents of all three files into the page as scripts and then uses them to perform the tests of the Continuum tool.
325
323
 
326
324
  Level Access on 22 August 2022 granted authorization for the copying of the `AccessEngine.community.js` file insofar as necessary for allowing Continuum community edition tests to be included in Testaro.
327
325
 
328
326
  ###### IBM Equal Access
329
327
 
330
- The `ibm` packaged test requires the `aceconfig.js` file.
328
+ The `ibm` tests require the `aceconfig.js` file.
331
329
 
332
- As of 2 March 2023 (version 3.1.45 of `accessibility-checker`), the `ibm` packaged test threw errors when hosted under the Windows operating system. To prevent these errors, it was possible to edit two files in the package as follows:
330
+ As of 2 March 2023 (version 3.1.45 of `accessibility-checker`), the `ibm` tool threw errors when hosted under the Windows operating system. To prevent these errors, it was possible to edit two files in the `accessibility-checker` package as follows:
333
331
 
334
332
  In `node_modules/accessibility-checker/lib/ACEngineManager.js`, remove or comment out these lines starting on line 169:
335
333
 
@@ -348,11 +346,11 @@ results.label = results.label.replace(/:/g, '-');
348
346
 
349
347
  These changes were proposed as pull requests 1333 and 1334 (https://github.com/IBMa/equal-access/pulls).
350
348
 
351
- If you choose to invoke this packaged test with the `withNewContent` property specified, you will choose whether the tested content is the content of the existing page or is retrieved anew with the document URL. Typically, both methods succeed and deliver similar results. However, sometimes one method succeeds and the other fails, or one method reports more violations than the other does. In those cases, most often the success, or the larger violation count, arises from the existing page (`withNewContent: false`).
349
+ If you choose to invoke the `ibm` tests with the `withNewContent` property specified, you will choose whether the tested content is the content of the existing page or is retrieved anew with the document URL. Typically, both methods succeed and deliver similar results. However, sometimes one method succeeds and the other fails, or one method reports more violations than the other does. In those cases, most often the success, or the larger violation count, arises from the existing page (`withNewContent: false`).
352
350
 
353
351
  ###### HTML CodeSniffer
354
352
 
355
- The `htmlcs` packaged test makes use of the`htmlcs/HTMLCS.js` file. That file was created, and can be recreated if necessary, as follows:
353
+ The `htmlcs` tests make use of the `htmlcs/HTMLCS.js` file. That file was created, and can be recreated if necessary, as follows:
356
354
 
357
355
  1. Clone the [HTML CodeSniffer package](https://github.com/squizlabs/HTML_CodeSniffer).
358
356
  1. Make that package’s directory the active directory.
@@ -384,7 +382,7 @@ The changes in `htmlcs/HTMLCS.js` are:
384
382
 
385
383
  ###### Tenon
386
384
 
387
- Most packaged tests require only one act, but the `tenon` packaged test requires two acts:
385
+ Most tools require only one act, but the `tenon` tool requires two acts:
388
386
  - An act of type `tenonRequest`.
389
387
  - An act of type `test` with `tenon` as the value of `which`.
390
388
 
@@ -410,23 +408,23 @@ Example:
410
408
  }
411
409
  ```
412
410
 
413
- The reason for this is that the Tenon API operates asynchronously. You ask it to perform a test, and it puts your request into a queue. To learn whether Tenon has completed your test, you make a status request. You can continue making status requests until Tenon replies that your test has been completed. Then you submit a request for the test result, and Tenon replies with the result. (As of May 2022, however, status requests were observed to misreport still-running tests as completed. The `tenon` test works around that by requesting only the result and using the response to determine whether the tests have been completed.)
411
+ The reason for this is that the Tenon API operates asynchronously. You ask it to perform a test, and it puts your request into a queue. To learn whether Tenon has completed your test, you make a status request. You can continue making status requests until Tenon replies that your test has been completed. Then you submit a request for the test result, and Tenon replies with the result. (As of May 2022, however, status requests were observed to misreport still-running tests as completed. The `tenon` test act works around that by requesting only the result and using the response to determine whether the tests have been completed.)
414
412
 
415
413
  Tenon says that tests are typically completed in 3 to 6 seconds but that the latency can be longer, depending on demand.
416
414
 
417
415
  Therefore, you can include a `tenonRequest` act early in your job, and a `tenon` test act late in your job. Tenon will move your request through its queue while Testaro is processing your job. When Testaro reaches your `tenon` test act, Tenon will most likely have completed your test. If not, the `tenon` test will wait and then make a second request before giving up.
418
416
 
419
- Thus, a `tenon` test actually does not perform any test; it merely collects the result. The page that was active when the `tenonRequest` act was performed is the one that Tenon tests.
417
+ Thus, a `tenon` test act actually does not perform any test; it merely collects the result. The page that was active when the `tenonRequest` act was performed is the one that Tenon tests.
420
418
 
421
- In case you want to perform more than one `tenon` test with the same job, you can do so. Just give each pair of acts a distinct `id` property, so each `tenon` test act will request the correct result.
419
+ In case you want to perform the Tenon tests more than once in the same job, you can do so. Just give each pair of acts a distinct `id` property, so each `tenon` test act will request the correct result.
422
420
 
423
421
  Tenon recommends giving it a public URL rather than giving it the content of a page, if possible. So, it is best to give the `withNewContent` property of the `tenonRequest` act the value `true`, unless the page is not public.
424
422
 
425
- If a `tenon` test is included in a job, environment variables named `TENON_USER` and `TENON_PASSWORD` must exist, with your Tenon username and password, respectively, as their values. You can obtain those from [Tenon](https://tenon.io/documentation/overview).
423
+ If a `tenon` test act is included in a job, environment variables named `TENON_USER` and `TENON_PASSWORD` must exist, with your Tenon username and password, respectively, as their values. These could be obtained from [Tenon](https://tenon.io/documentation/overview) until Tenon was closed to new subscribers in or about October 2022.
426
424
 
427
425
  ###### WAVE
428
426
 
429
- If a `wave` test is included in the job, an environment variable named `WAVE_KEY` must exist, with your WAVE API key as its value. You can get it from [WebAIM](https://wave.webaim.org/api/).
427
+ If a `wave` test act is included in the job, an environment variable named `WAVE_KEY` must exist, with your WAVE API key as its value. You can get it from [WebAIM](https://wave.webaim.org/api/).
430
428
 
431
429
  The `wave` API does not accept a transmitted document for testing. WAVE must be given only a URL, which it then visits to perform its tests. Therefore, you cannot manipulate a page and then have WAVE test it, or ask WAVE to test a page that cannot be reached directly with a URL.
432
430
 
@@ -438,7 +436,7 @@ The BBC Accessibility Standards Checker has obsolete dependencies with security
438
436
 
439
437
  ##### Branching
440
438
 
441
- An example of a **branching** command is:
439
+ An example of a **branching** act is:
442
440
 
443
441
  ```json
444
442
  {
@@ -449,9 +447,9 @@ An example of a **branching** command is:
449
447
  }
450
448
  ```
451
449
 
452
- This command checks the result of the previous act to determine whether its `result.totals.invalid` property has a positive value. If so, it changes the next command to be performed, specifying the command 4 commands before this one.
450
+ This act checks the result of the previous act to determine whether its `result.totals.invalid` property has a positive value. If so, it changes the next act to be performed, specifying the act 4 acts before this one.
453
451
 
454
- A `next`-type command can use a `next` property instead of a `jump` property. The value of the `next` property is a command name. It tells Testaro to continue performing commands starting with the command having that value as the value of its `name` property.
452
+ A `next` act can use a `next` property instead of a `jump` property. The value of the `next` property is an act name. It tells Testaro to continue performing acts starting with the act having that value as the value of its `name` property.
455
453
 
456
454
  #### `actSpecs` file
457
455
 
@@ -507,7 +505,7 @@ The validity criterion named in item 2 may be any of these:
507
505
 
508
506
  Any `test` act can also (in addition to the requirements in `actSpecs.js`) contain an `expect` property. If it does, the value of that property must be an array of arrays. Each array specifies expectations about the results of the test.
509
507
 
510
- For example, a `test` command might have this `expect` property:
508
+ For example, a `test` act might have this `expect` property:
511
509
 
512
510
  ```javaScript
513
511
  'expect': [
@@ -533,7 +531,7 @@ The second item in each array, if there are 3 items in the array, is an operator
533
531
 
534
532
  The third item in each array, if there are 3 items in the array, is the criterion with which the value of the first property is compared.
535
533
 
536
- A typical use for an `expect` property is checking the correctness of a Testaro test. Thus, the validation jobs in the `validation/tests/jobs` directory all contain `test` commands with `expect` properties. See the “Validation” section below.
534
+ A typical use for an `expect` property is checking the correctness of a Testaro test. Thus, the validation jobs in the `validation/tests/jobs` directory all contain `test` acts with `expect` properties. See the “Validation” section below.
537
535
 
538
536
  ## Execution
539
537
 
@@ -636,7 +634,7 @@ The URL to which Testaro sends reports is given by the `sources.sendReportTo` pr
636
634
 
637
635
  ### Environment variables
638
636
 
639
- In addition to their uses described above, environment variables can be used by commands of type `text`, as documented in the `commands.js` file.
637
+ In addition to their uses described above, environment variables can be used by acts of type `text`, as documented in the `actSpecs.js` file.
640
638
 
641
639
  Before making Testaro run a job, you can optionally also set `process.env.DEBUG` (to `'true'` or anything else) and/or `process.env.WAITS` (to a non-negative integer). The effects of these variables are described in the `run.js` file.
642
640
 
@@ -677,13 +675,13 @@ The `tests` executor makes use of the jobs in the `validation/tests/jobs` direct
677
675
 
678
676
  ## Contribution
679
677
 
680
- You can define additional Testaro commands and functionality. Contributions are welcome.
678
+ You can define additional Testaro acts and functionality. Contributions are welcome.
681
679
 
682
680
  Please report any issues, including feature requests, at the [repository](https://github.com/jrpool/testaro/issues).
683
681
 
684
682
  ## Accessibility principles
685
683
 
686
- The rationales motivating the Testaro-defined tests can be found in comments within the files of those tests, in the `tests` directory. Unavoidably, each test is opinionated. Testaro itself, however, can accommodate other tests representing different opinions. Testaro is intended to be neutral with respect to questions such as the criteria for accessibility, the severities of accessibility issues, whether accessibility is binary or graded, and the distinction between usability and accessibility.
684
+ The rationales motivating the Testaro-defined tests can be found in comments within the files of those tests, in the `tests` directory. Unavoidably, each test is opinionated. Testaro itself, however, can accommodate other tests representing different opinions. Testaro is intended to be neutral with respect to questions such as the criteria for accessibility, the severities of accessibility defects, whether accessibility is binary or graded, and the distinction between usability and accessibility.
687
685
 
688
686
  ## Testing challenges
689
687
 
@@ -697,18 +695,16 @@ Testing to determine what happens when a control or link is activated is straigh
697
695
 
698
696
  The Playwright “Receives Events” actionability check does **not** check whether an event is dispatched on an element. It checks only whether a click on the location of the element makes the element the target of that click, rather than some other element occupying the same location.
699
697
 
700
- ### Test-package duplicativity
701
-
702
- Test packages sometimes do redundant testing, in that two or more packages test for the same issues, although such duplications are not necessarily perfect. This fact creates three problems:
703
- - One cannot be confident in excluding some tests of some packages on the assumption that they perfectly duplicate tests of other packages.
704
- - The Testaro report from a job documents each package’s results separately, so a single defect may be documented in multiple locations within the report, making the direct consumption of the report inefficient.
705
- - An effort to aggregate the results into a single score may distort the scores by inflating the weights of defects that happen to be discovered by multiple packages.
698
+ ### Tool duplicativity
706
699
 
707
- The tests provided with Testaro do not exclude any apparently duplicative tests from packages.
700
+ Tools sometimes do redundant testing, in that two or more tools test for the same defects, although such duplications are not necessarily perfect. This fact creates three problems:
701
+ - One cannot be confident in excluding some tests of some tools on the assumption that they perfectly duplicate tests of other tools.
702
+ - The Testaro report from a job documents each tool’s results separately, so a single defect may be documented in multiple locations within the report, making the direct consumption of the report inefficient.
703
+ - An effort to aggregate the results into a single score may distort the scores by inflating the weights of defects that happen to be discovered by multiple tools.
708
704
 
709
705
  To deal with the above problems, you can:
710
- - revise package `test` commands to exclude tests that you consider duplicative
711
- - create derivative reports that organize results by defect types rather than by package
706
+ - configure `test` acts for tools to exclude tests that you consider duplicative
707
+ - create derivative reports that organize results by defect types rather than by tool
712
708
  - take duplication into account when defining scoring rules
713
709
 
714
710
  Some measures of these kinds are included in the scoring and reporting features of the Testilo package.
@@ -725,7 +721,7 @@ The files in the `temp` directory are presumed ephemeral and are not tracked by
725
721
  - produces human-oriented HTML digests from scored reports
726
722
  - produces human-oriented HTML comparisons of the scores of targets
727
723
 
728
- Testilo contains procedures that reorganize report data by defect rather than test package, and that compensate for duplicative tests when computing scores.
724
+ Testilo contains procedures that reorganize report data by defect rather than tool, and that compensate for duplicative tests when computing scores.
729
725
 
730
726
  Testaro is derived from [Autotest](https://github.com/jrpool/autotest). Autotest was created as a monolithic accessibility testing package, but that forced functionalities to be hosted on a workstation merely because it was impractical to host Playwright elsewhere. Testaro embodies an architectural decision to isolate workstation-dependent functionalities.
731
727