testilo 2.1.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -77
- package/package.json +3 -4
- package/procs/score/tsp09.js +565 -0
- package/score.js +33 -0
- package/scoring/correlation.js +76 -0
- package/scoring/correlations.json +327 -0
- package/scoring/data.json +26021 -0
- package/scoring/dupCounts.js +39 -0
- package/scoring/dupCounts.json +112 -0
- package/scoring/duplications.json +253 -0
- package/scoring/issues.json +304 -0
- package/scoring/packageData.js +171 -0
- package/scoring/packageIssues.js +34 -0
- package/scoring/packageRules/aatt.js +543 -0
- package/scoring/rulesetData.json +15 -0
- package/aceconfig.js +0 -6
- package/index.js +0 -88
package/README.md
CHANGED
|
@@ -1,93 +1,30 @@
|
|
|
1
1
|
# testilo
|
|
2
|
-
|
|
2
|
+
Scorer and digester of Testaro reports
|
|
3
3
|
|
|
4
4
|
## Introduction
|
|
5
5
|
|
|
6
|
-
This application
|
|
7
|
-
|
|
8
|
-
[Testaro](https://www.npmjs.com/package/testaro) is a dependency that performs digital accessibility tests on Web resources.
|
|
6
|
+
This application enriches [Testaro](https://www.npmjs.com/package/testaro) reports. Testaro performs digital accessibility tests on Web resources and creates reports in JSON format of the results. To make those reports more useful, this application, Testilo, computes scores and converts the scored reports to human-readable web pages.
|
|
9
7
|
|
|
10
8
|
## Dependencies
|
|
11
9
|
|
|
12
10
|
The `dotenv` dependency lets you set environment variables in an untracked `.env` file.
|
|
13
11
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
```bash
|
|
17
|
-
@siteimprove:registry=https://npm.pkg.github.com
|
|
18
|
-
//npm.pkg.github.com/:username=abc
|
|
19
|
-
//npm.pkg.github.com/:_authToken=def
|
|
20
|
-
```
|
|
21
|
-
|
|
22
|
-
In this content, replace `abc` with your Github username and `def` with a Github personal access token that has `read:packages` scope.
|
|
23
|
-
|
|
24
|
-
## Operation
|
|
25
|
-
|
|
26
|
-
### General
|
|
27
|
-
|
|
28
|
-
Testilo orders a Testaro job by calling Testaro’s `handleRequest` function with an object argument. The argument has this structure:
|
|
29
|
-
|
|
30
|
-
```javascript
|
|
31
|
-
{
|
|
32
|
-
id,
|
|
33
|
-
script: {…},
|
|
34
|
-
log: [],
|
|
35
|
-
acts: []
|
|
36
|
-
}
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
The `script` property has a Testaro script as its value. See the Testaro `README.md` file for documentation on scripts.
|
|
40
|
-
|
|
41
|
-
If a script is represented as JSON in a file `scripts/scriptX.json`, you can incorporate it into the options object of a Testaro call by executing the statement
|
|
42
|
-
|
|
43
|
-
```javascript
|
|
44
|
-
node index scriptX
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
### Batches
|
|
48
|
-
|
|
49
|
-
You may wish to have Testaro perform the same sequence of tests on multiple web pages. In that case, you can create a _batch_, with the following structure:
|
|
50
|
-
|
|
51
|
-
```javascript
|
|
52
|
-
{
|
|
53
|
-
what: 'Web leaders',
|
|
54
|
-
hosts: {
|
|
55
|
-
id: 'w3c',
|
|
56
|
-
which: 'https://www.w3.org/',
|
|
57
|
-
what: 'W3C'
|
|
58
|
-
},
|
|
59
|
-
{
|
|
60
|
-
id: 'wikimedia',
|
|
61
|
-
which: 'https://www.wikimedia.org/',
|
|
62
|
-
what: 'Wikimedia'
|
|
63
|
-
}
|
|
64
|
-
}
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
With a batch, you can execute a single statement to call Testaro multiple times, one per host. On each call, Testilo takes one of the hosts in the batch and substitutes it for each host specified in a `url` command of the script. Testilo waits for each Testaro job to finish before calling the next Testaro job.
|
|
68
|
-
|
|
69
|
-
If a batch is represented as a JSON file `batches/batchY.json`, you can use it to call a set of Testaro jobs with the statement
|
|
70
|
-
|
|
71
|
-
```javascript
|
|
72
|
-
node index scriptX batchY
|
|
73
|
-
```
|
|
74
|
-
|
|
75
|
-
Given that statement, Testilo replaces the hosts in the script with the first host in the batch and calls Testaro. When Testaro finishes performing that script, Testilo replaces the script hosts with the second batch host and calls Testaro again. And so on.
|
|
76
|
-
|
|
77
|
-
### Reports
|
|
78
|
-
|
|
79
|
-
When you execute a `node index …` statement, Testilo begins populating the object argument by giving its `id` property a value. If there is no batch, the value of that property is a string encoding the date and time when you executed the statement (e.g., `eh9q7r`). If there is a batch, the value is the same, except that it is suffixed with a hyphen-minus character followed by the `id` value of the host (e.g., `eh9q7r-wikimedia`).
|
|
12
|
+
## Scoring
|
|
80
13
|
|
|
81
|
-
|
|
14
|
+
To score a Testaro report, execute the statement `node score xyz`, replacing `xyz` with the base of the name of the file containing the report.
|
|
82
15
|
|
|
83
|
-
|
|
16
|
+
This procedure has some preconditions:
|
|
17
|
+
- The full name of the report file is `xyz.json`.
|
|
18
|
+
- The report file is located in the directory whose relative path (relative to the project directory of Testilo) is the value of the `REPORTDIR` environment variable.
|
|
19
|
+
- Testilo can read and write in that report directory.
|
|
20
|
+
- The `procs/score` directory contains a file named `tspnn.js`, where `tspnn` is replaced with the value of the `id` property of the `script` property of the report.
|
|
84
21
|
|
|
85
|
-
|
|
22
|
+
Thus, the script that Testaro ran in order to produce the report must be one that Testilo has a scoring algorithm (_score proc_) for. If so, Testilo can score the report.
|
|
86
23
|
|
|
87
|
-
|
|
24
|
+
If Testilo scores a report, Testilo saves the scored report alongside the original report. The scored report file has the same name as the original, plus `-scored` added to the base name.
|
|
88
25
|
|
|
89
|
-
|
|
26
|
+
## Digesting
|
|
90
27
|
|
|
91
|
-
|
|
28
|
+
To make a scored Testaro report more useful for humans, Testilo can create a digest of the report. This is an HTML document (a web page) summarizing the findings.
|
|
92
29
|
|
|
93
|
-
|
|
30
|
+
The digesting functionality currently exists in a different application and is being ported to Testilo.
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "testilo",
|
|
3
|
-
"version": "
|
|
4
|
-
"description": "Client that
|
|
3
|
+
"version": "3.0.0",
|
|
4
|
+
"description": "Client that scores and digests Testaro reports",
|
|
5
5
|
"main": "index.js",
|
|
6
6
|
"scripts": {
|
|
7
7
|
"test": "echo \"Error: no test specified\" && exit 1"
|
|
@@ -18,8 +18,7 @@
|
|
|
18
18
|
},
|
|
19
19
|
"homepage": "https://github.com/jrpool/testilo",
|
|
20
20
|
"dependencies": {
|
|
21
|
-
"dotenv": "*"
|
|
22
|
-
"testaro": "*"
|
|
21
|
+
"dotenv": "*"
|
|
23
22
|
},
|
|
24
23
|
"devDependencies": {
|
|
25
24
|
"eslint": "*"
|
|
@@ -0,0 +1,565 @@
|
|
|
1
|
+
/*
|
|
2
|
+
tsp09
|
|
3
|
+
Testilo score proc 9
|
|
4
|
+
Computes scores from script tsp09 and adds them to a report.
|
|
5
|
+
*/
|
|
6
|
+
exports.scorer = report => {
|
|
7
|
+
// CONSTANTS
|
|
8
|
+
const {acts} = report;
|
|
9
|
+
// Define the configuration disclosures.
|
|
10
|
+
const logWeights = {
|
|
11
|
+
count: 0.5,
|
|
12
|
+
size: 0.01,
|
|
13
|
+
prohibited: 15,
|
|
14
|
+
visitTimeout: 10,
|
|
15
|
+
visitRejection: 10
|
|
16
|
+
};
|
|
17
|
+
const rules = {
|
|
18
|
+
aatt: '',
|
|
19
|
+
alfa:'',
|
|
20
|
+
axe: '',
|
|
21
|
+
bulk: '',
|
|
22
|
+
embAc: '',
|
|
23
|
+
focAll: '',
|
|
24
|
+
focInd: '',
|
|
25
|
+
focOp: '',
|
|
26
|
+
hover: '',
|
|
27
|
+
ibm: '',
|
|
28
|
+
labClash: '',
|
|
29
|
+
linkUl: '',
|
|
30
|
+
log: 'multiply log items by respective logWeights; sum',
|
|
31
|
+
menuNav: '',
|
|
32
|
+
motion: '',
|
|
33
|
+
radioSet: '',
|
|
34
|
+
role: '',
|
|
35
|
+
styleDiff: '',
|
|
36
|
+
tabNav: '',
|
|
37
|
+
wave: '',
|
|
38
|
+
zIndex: ''
|
|
39
|
+
};
|
|
40
|
+
let duplications = {};
|
|
41
|
+
const diffStyles = [
|
|
42
|
+
'borderStyle',
|
|
43
|
+
'borderWidth',
|
|
44
|
+
'fontStyle',
|
|
45
|
+
'fontWeight',
|
|
46
|
+
'lineHeight',
|
|
47
|
+
'maxHeight',
|
|
48
|
+
'maxWidth',
|
|
49
|
+
'minHeight',
|
|
50
|
+
'minWidth',
|
|
51
|
+
'opacity',
|
|
52
|
+
'outlineOffset',
|
|
53
|
+
'outlineStyle',
|
|
54
|
+
'outlineWidth',
|
|
55
|
+
'textDecorationLine',
|
|
56
|
+
'textDecorationStyle',
|
|
57
|
+
'textDecorationThickness'
|
|
58
|
+
];
|
|
59
|
+
// Initialize the score.
|
|
60
|
+
const inferences = {};
|
|
61
|
+
let scores = {
|
|
62
|
+
total: 0,
|
|
63
|
+
aatt: null,
|
|
64
|
+
alfa: null,
|
|
65
|
+
axe: null,
|
|
66
|
+
bulk: null,
|
|
67
|
+
embAc: null,
|
|
68
|
+
focAll: null,
|
|
69
|
+
focInd: null,
|
|
70
|
+
focOp: null,
|
|
71
|
+
hover: null,
|
|
72
|
+
ibm: null,
|
|
73
|
+
labClash: null,
|
|
74
|
+
linkUl: null,
|
|
75
|
+
log: null,
|
|
76
|
+
menuNav: null,
|
|
77
|
+
motion: null,
|
|
78
|
+
radioSet: null,
|
|
79
|
+
role: null,
|
|
80
|
+
styleDiff: null,
|
|
81
|
+
tabNav: null,
|
|
82
|
+
wave: null,
|
|
83
|
+
zIndex: null
|
|
84
|
+
};
|
|
85
|
+
// VARIABLES
|
|
86
|
+
let facts;
|
|
87
|
+
// If there are any acts:
|
|
88
|
+
if (Array.isArray(acts)) {
|
|
89
|
+
// If any of them are tests:
|
|
90
|
+
const tests = acts.filter(act => act.type === 'test');
|
|
91
|
+
if (tests.length) {
|
|
92
|
+
// CONSTANTS
|
|
93
|
+
// Empirically derived counts of duplications of package rules.
|
|
94
|
+
duplications = {
|
|
95
|
+
'aatt': {
|
|
96
|
+
'e:F77': 1,
|
|
97
|
+
'e:H36': 4,
|
|
98
|
+
'e:H37': 2,
|
|
99
|
+
'e:H57': 3,
|
|
100
|
+
'e:H58': 2,
|
|
101
|
+
'w:G141': 3,
|
|
102
|
+
'w:H98': 1,
|
|
103
|
+
'e:ARIA6+H53': 1,
|
|
104
|
+
'e:H24': 2,
|
|
105
|
+
'e:G1+G123+G124': 1,
|
|
106
|
+
'w:G90': 1,
|
|
107
|
+
'w:H44': 1
|
|
108
|
+
},
|
|
109
|
+
'alfa': {
|
|
110
|
+
'r3': 2,
|
|
111
|
+
'r28': 4,
|
|
112
|
+
'r2': 2,
|
|
113
|
+
'r4': 3,
|
|
114
|
+
'r7': 2,
|
|
115
|
+
'r53': 3,
|
|
116
|
+
'r10': 1,
|
|
117
|
+
'r11': 1,
|
|
118
|
+
'r12': 2,
|
|
119
|
+
'r20': 1,
|
|
120
|
+
'r42': 1,
|
|
121
|
+
'r43': 1,
|
|
122
|
+
'r47': 1,
|
|
123
|
+
'r5': 2,
|
|
124
|
+
'r68': 1,
|
|
125
|
+
'r93': 1,
|
|
126
|
+
'r13': 1
|
|
127
|
+
},
|
|
128
|
+
'axe': {
|
|
129
|
+
'input-image-alt': 4,
|
|
130
|
+
'html-has-lang': 3,
|
|
131
|
+
'valid-lang': 2,
|
|
132
|
+
'heading-order': 3,
|
|
133
|
+
'link-name': 2,
|
|
134
|
+
'aria-command-name': 2,
|
|
135
|
+
'dlitem': 1,
|
|
136
|
+
'image-alt': 2,
|
|
137
|
+
'duplicate-id': 1,
|
|
138
|
+
'aria-required-parent': 1,
|
|
139
|
+
'svg-img-alt': 1,
|
|
140
|
+
'meta-viewport': 1,
|
|
141
|
+
'html-lang-valid': 2,
|
|
142
|
+
'aria-required-children': 1,
|
|
143
|
+
'avoid-inline-spacing': 1,
|
|
144
|
+
'area-alt': 2,
|
|
145
|
+
'aria-allowed-role': 1,
|
|
146
|
+
'aria-required-attr': 1,
|
|
147
|
+
'aria-valid-attr': 1,
|
|
148
|
+
'autocomplete-valid': 1,
|
|
149
|
+
'color-contrast': 1,
|
|
150
|
+
'empty-heading': 2,
|
|
151
|
+
'frame-title': 1,
|
|
152
|
+
'image-redundant-alt': 1,
|
|
153
|
+
'landmark-complementary-is-top-level': 2,
|
|
154
|
+
'landmark-no-duplicate-banner': 1,
|
|
155
|
+
'landmark-no-duplicate-main': 2,
|
|
156
|
+
'document-title': 1,
|
|
157
|
+
'object-alt': 1,
|
|
158
|
+
'page-has-heading-one': 1,
|
|
159
|
+
'select-name': 1
|
|
160
|
+
},
|
|
161
|
+
'ibm': {
|
|
162
|
+
'v:WCAG20_Object_HasText': 1,
|
|
163
|
+
'v:WCAG20_Area_HasAlt': 2,
|
|
164
|
+
'v:WCAG20_Input_ExplicitLabelImage': 4,
|
|
165
|
+
'v:WCAG20_Frame_HasTitle': 2,
|
|
166
|
+
'v:WCAG20_Elem_Lang_Valid': 2,
|
|
167
|
+
'v:HAAC_Img_UsemapAlt': 1,
|
|
168
|
+
'v:aria_semantics_role': 1,
|
|
169
|
+
'v:Rpt_Aria_RequiredProperties': 1,
|
|
170
|
+
'v:Rpt_Aria_ValidProperty': 1,
|
|
171
|
+
'v:WCAG21_Input_Autocomplete': 1,
|
|
172
|
+
'v:IBMA_Color_Contrast_WCAG2AA': 2,
|
|
173
|
+
'v:RPT_Header_HasContent': 2,
|
|
174
|
+
'v:WCAG20_Img_HasAlt': 2,
|
|
175
|
+
'v:WCAG20_Img_LinkTextNotRedundant': 1,
|
|
176
|
+
'v:Rpt_Aria_ComplementaryRequiredLabel_Implicit': 1,
|
|
177
|
+
'v:Rpt_Aria_MultipleComplementaryLandmarks_Implicit': 1,
|
|
178
|
+
'v:Rpt_Aria_MultipleBannerLandmarks_Implicit': 1,
|
|
179
|
+
'r:Rpt_Aria_MultipleMainsVisibleLabel_Implicit': 1,
|
|
180
|
+
'v:Rpt_Aria_MultipleMainsRequireLabel_Implicit_2': 1,
|
|
181
|
+
'v:WCAG20_A_HasText': 1,
|
|
182
|
+
'v:Rpt_Aria_ValidIdRef': 1,
|
|
183
|
+
'v:WCAG20_Elem_UniqueAccessKey': 1,
|
|
184
|
+
'v:WCAG20_Input_RadioChkInFieldSet': 1
|
|
185
|
+
},
|
|
186
|
+
'wave': {
|
|
187
|
+
'a:link_internal_broken': 1,
|
|
188
|
+
'e:alt_area_missing': 3,
|
|
189
|
+
'e:alt_input_missing': 4,
|
|
190
|
+
'e:alt_missing': 2,
|
|
191
|
+
'e:language_missing': 3,
|
|
192
|
+
'a:heading_skipped': 3,
|
|
193
|
+
'a:event_handler': 1,
|
|
194
|
+
'a:label_orphaned': 1,
|
|
195
|
+
'e:title_invalid': 1,
|
|
196
|
+
'e:heading_empty': 2,
|
|
197
|
+
'a:plugin': 1,
|
|
198
|
+
'a:h1_missing': 1,
|
|
199
|
+
'a:select_missing_label': 1,
|
|
200
|
+
'c:contrast': 1,
|
|
201
|
+
'e:aria_reference_broken': 1,
|
|
202
|
+
'a:accesskey': 1,
|
|
203
|
+
'a:fieldset_missing': 1
|
|
204
|
+
}
|
|
205
|
+
};
|
|
206
|
+
// FUNCTIONS
|
|
207
|
+
// Adds the actual or inferred score of a test to the total score.
|
|
208
|
+
const increment = test => {
|
|
209
|
+
scores.total += typeof scores[test] === 'number' ? scores[test] : inferences[test];
|
|
210
|
+
};
|
|
211
|
+
// OPERATION
|
|
212
|
+
// For each test:
|
|
213
|
+
tests.forEach(test => {
|
|
214
|
+
const {which} = test;
|
|
215
|
+
// Compute its score.
|
|
216
|
+
if (which === 'alfa') {
|
|
217
|
+
facts = test.result;
|
|
218
|
+
if (facts && Array.isArray(facts)) {
|
|
219
|
+
rules.alfa = 'multiply cantTell by 2*, failed by 4* (*discounted); sum';
|
|
220
|
+
scores.alfa = Math.round(facts.reduce((total, issue) => {
|
|
221
|
+
const rawScore = [4, 2][['failed', 'cantTell'].indexOf(issue.verdict)] || 0;
|
|
222
|
+
const divisor = duplications.alfa[issue.rule.ruleID] + 1 || 1;
|
|
223
|
+
return total + rawScore / divisor;
|
|
224
|
+
}, 0));
|
|
225
|
+
scores.total += scores.alfa;
|
|
226
|
+
}
|
|
227
|
+
}
|
|
228
|
+
else if (which === 'aatt') {
|
|
229
|
+
facts = test.result;
|
|
230
|
+
if (facts && Array.isArray(facts)) {
|
|
231
|
+
rules.aatt = 'multiply warning by 2*, error by 4* (*discounted); sum';
|
|
232
|
+
const issues = facts.filter(fact => fact.type);
|
|
233
|
+
scores.aatt = Math.round(issues.reduce((total, issue) => {
|
|
234
|
+
const rawScore = [4, 2][['error', 'warning'].indexOf(issue.type)] || 0;
|
|
235
|
+
const divisor = duplications.aatt[`${issue.type.slice(0, 1)}:${issue.id}`] + 1 || 1;
|
|
236
|
+
return total + rawScore / divisor;
|
|
237
|
+
}, 0));
|
|
238
|
+
scores.total += scores.aatt;
|
|
239
|
+
}
|
|
240
|
+
}
|
|
241
|
+
else if (which === 'axe') {
|
|
242
|
+
facts = test.result && test.result.items;
|
|
243
|
+
if (facts) {
|
|
244
|
+
rules.axe = 'multiply minor by 2*, moderate by 3*, serious by 4*, critical by 5* (*discounted); sum';
|
|
245
|
+
scores.axe = Math.round(facts.reduce((total, item) => {
|
|
246
|
+
const rawScore = item.elements.length * (
|
|
247
|
+
[5, 4, 3, 2][['critical', 'serious', 'moderate', 'minor'].indexOf(item.impact)] || 0
|
|
248
|
+
);
|
|
249
|
+
const divisor = duplications.axe[item.rule] + 1 || 1;
|
|
250
|
+
return total + rawScore / divisor;
|
|
251
|
+
}, 0));
|
|
252
|
+
scores.total += scores.axe;
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
else if (which === 'ibm') {
|
|
256
|
+
facts = test.result;
|
|
257
|
+
if (facts && facts.content && facts.url && (facts.content.totals || facts.url.totals)) {
|
|
258
|
+
rules.ibm = 'multiply violations by 4*, recommendations by 2* (*discounted); sum';
|
|
259
|
+
const ibmScores = {
|
|
260
|
+
content: null,
|
|
261
|
+
url: null
|
|
262
|
+
};
|
|
263
|
+
['content', 'url'].forEach(type => {
|
|
264
|
+
const totals = facts[type].totals;
|
|
265
|
+
if (totals) {
|
|
266
|
+
const items = facts[type].items || [];
|
|
267
|
+
ibmScores[type] = Math.round(items.reduce((total, item) => {
|
|
268
|
+
const {ruleId, level} = item;
|
|
269
|
+
const rawScore = [4, 2][['violation', 'recommendation'].indexOf(level)] || 0;
|
|
270
|
+
const divisor = duplications.ibm[`${level.slice(0, 1)}:${ruleId}`] + 1 || 1;
|
|
271
|
+
return total + rawScore / divisor;
|
|
272
|
+
}, 0));
|
|
273
|
+
}
|
|
274
|
+
});
|
|
275
|
+
if (ibmScores.content !== null || ibmScores.url !== null) {
|
|
276
|
+
scores.ibm = Math.max(ibmScores.content || 0, ibmScores.url || 0);
|
|
277
|
+
scores.total += scores.ibm;
|
|
278
|
+
}
|
|
279
|
+
}
|
|
280
|
+
}
|
|
281
|
+
else if (which === 'wave') {
|
|
282
|
+
facts = test.result && test.result.categories;
|
|
283
|
+
if (facts) {
|
|
284
|
+
rules.wave
|
|
285
|
+
= 'multiply alerts by 2*, contrast errors by 3*, errors by 4* (*discounted); sum';
|
|
286
|
+
const weights = {
|
|
287
|
+
error: 4,
|
|
288
|
+
contrast: 3,
|
|
289
|
+
alert: 2
|
|
290
|
+
};
|
|
291
|
+
const waveScores = {
|
|
292
|
+
error: 0,
|
|
293
|
+
contrast: 0,
|
|
294
|
+
alert: 0
|
|
295
|
+
};
|
|
296
|
+
['error', 'contrast', 'alert'].forEach(level => {
|
|
297
|
+
const {items} = facts[level];
|
|
298
|
+
waveScores[level] = Math.round(Object.keys(items).reduce((total, ruleID) => {
|
|
299
|
+
const rawScore = items[ruleID].count * weights[level];
|
|
300
|
+
const divisor = duplications.wave[`${level.slice(0, 1)}:${ruleID}`] + 1 || 1;
|
|
301
|
+
return total + rawScore / divisor;
|
|
302
|
+
}, 0));
|
|
303
|
+
});
|
|
304
|
+
scores.wave = waveScores.error + waveScores.contrast + waveScores.alert;
|
|
305
|
+
scores.total += scores.wave;
|
|
306
|
+
}
|
|
307
|
+
}
|
|
308
|
+
else if (which === 'bulk') {
|
|
309
|
+
facts = test.result && test.result.visibleElements;
|
|
310
|
+
if (typeof facts === 'number') {
|
|
311
|
+
rules.bulk = 'subtract 250 from visible elements; make 0 if negative; raise to 0.9th power; multiply by 0.15';
|
|
312
|
+
// Deficit: 15% of the excess, to the 0.9th power, of the element count over 250.
|
|
313
|
+
scores.bulk = Math.floor(0.15 * Math.pow(Math.max(0, facts - 250), 0.9));
|
|
314
|
+
}
|
|
315
|
+
else {
|
|
316
|
+
inferences.bulk = 100;
|
|
317
|
+
}
|
|
318
|
+
increment('bulk');
|
|
319
|
+
}
|
|
320
|
+
else if (which === 'embAc') {
|
|
321
|
+
facts = test.result && test.result.totals;
|
|
322
|
+
if (facts) {
|
|
323
|
+
rules.embAc = 'multiply link- or button-contained links, buttons, inputs, and selects by 3 (discounted)';
|
|
324
|
+
scores.embAc = 3 * (facts.links + facts.buttons + facts.inputs + facts.selects);
|
|
325
|
+
}
|
|
326
|
+
else {
|
|
327
|
+
inferences.embAc = 150;
|
|
328
|
+
}
|
|
329
|
+
increment('embAc');
|
|
330
|
+
}
|
|
331
|
+
else if (which === 'focAll') {
|
|
332
|
+
facts = test.result;
|
|
333
|
+
if (facts && typeof facts === 'object') {
|
|
334
|
+
rules.focAll= 'multiply discrepancy between focusable and focused element counts by 3';
|
|
335
|
+
scores.focAll = 3 * Math.abs(facts.discrepancy);
|
|
336
|
+
}
|
|
337
|
+
else {
|
|
338
|
+
inferences.focAll = 150;
|
|
339
|
+
}
|
|
340
|
+
increment('focAll');
|
|
341
|
+
}
|
|
342
|
+
else if (which === 'focInd') {
|
|
343
|
+
facts = test.result && test.result.totals;
|
|
344
|
+
facts = facts ? facts.types : null;
|
|
345
|
+
if (facts) {
|
|
346
|
+
rules.focInd = 'multiply indicatorless-when-focused elements by 5';
|
|
347
|
+
scores.focInd = 5 * facts.indicatorMissing.total + 3 * facts.nonOutlinePresent.total;
|
|
348
|
+
}
|
|
349
|
+
else {
|
|
350
|
+
inferences.focInd = 150;
|
|
351
|
+
}
|
|
352
|
+
increment('focInd');
|
|
353
|
+
}
|
|
354
|
+
else if (which === 'focOl') {
|
|
355
|
+
facts = test.result && test.result.totals;
|
|
356
|
+
facts = facts ? facts.types : null;
|
|
357
|
+
facts = facts ? facts.outlineMissing : null;
|
|
358
|
+
if (facts) {
|
|
359
|
+
rules.focOl = 'multiply non-outline focus indicators by 3, missing focus indicators by 5; sum';
|
|
360
|
+
scores.focOl = 3 * facts.total;
|
|
361
|
+
}
|
|
362
|
+
else {
|
|
363
|
+
inferences.focOl = 100;
|
|
364
|
+
}
|
|
365
|
+
increment('focOl');
|
|
366
|
+
}
|
|
367
|
+
else if (which === 'focOp') {
|
|
368
|
+
facts = test.result && test.result.totals;
|
|
369
|
+
if (facts) {
|
|
370
|
+
rules.focOp = 'multiply nonfocusable operable elements by 4, nonoperable focusable by 1; sum';
|
|
371
|
+
scores.focOp
|
|
372
|
+
= 4 * facts.types.onlyOperable.total + 1 * facts.types.onlyFocusable.total;
|
|
373
|
+
}
|
|
374
|
+
else {
|
|
375
|
+
inferences.focOp = 150;
|
|
376
|
+
}
|
|
377
|
+
increment('focOp');
|
|
378
|
+
}
|
|
379
|
+
else if (which === 'hover') {
|
|
380
|
+
facts = test.result && test.result.totals;
|
|
381
|
+
if (facts) {
|
|
382
|
+
rules.hover = 'multiply elements changing page on hover by 4, made visible by 2, with directly changed opacity by 0.1, with indirectly changed opacity by 0.2, unhoverable by 2; sum';
|
|
383
|
+
scores.hover
|
|
384
|
+
= 4 * facts.triggers
|
|
385
|
+
+ 2 * facts.madeVisible
|
|
386
|
+
+ Math.floor(0.1 * facts.opacityChanged)
|
|
387
|
+
+ Math.floor(0.2 * facts.opacityAffected)
|
|
388
|
+
+ 2 * facts.unhoverables;
|
|
389
|
+
}
|
|
390
|
+
else {
|
|
391
|
+
inferences.hover = 150;
|
|
392
|
+
}
|
|
393
|
+
increment('hover');
|
|
394
|
+
}
|
|
395
|
+
else if (which === 'labClash') {
|
|
396
|
+
facts = test.result && test.result.totals;
|
|
397
|
+
if (facts) {
|
|
398
|
+
rules.labClash = 'multiply conflictually labeled elements by 2, unlabeled elements by 2; sum';
|
|
399
|
+
// Unlabeled elements discounted.
|
|
400
|
+
scores.labClash = 2 * facts.mislabeled + 2 * facts.unlabeled;
|
|
401
|
+
}
|
|
402
|
+
else {
|
|
403
|
+
inferences.labClash = 100;
|
|
404
|
+
}
|
|
405
|
+
increment('labClash');
|
|
406
|
+
}
|
|
407
|
+
else if (which === 'linkUl') {
|
|
408
|
+
facts = test.result && test.result.totals;
|
|
409
|
+
facts = facts ? facts.inline : null;
|
|
410
|
+
if (facts) {
|
|
411
|
+
rules.linkUl = 'multiply nonunderlined inline links by 3';
|
|
412
|
+
scores.linkUl = 3 * (facts.total - facts.underlined);
|
|
413
|
+
}
|
|
414
|
+
else {
|
|
415
|
+
inferences.linkUl = 150;
|
|
416
|
+
}
|
|
417
|
+
increment('linkUl');
|
|
418
|
+
}
|
|
419
|
+
else if (which === 'menuNav') {
|
|
420
|
+
facts = test.result && test.result.totals && test.result.totals.navigations;
|
|
421
|
+
if (facts) {
|
|
422
|
+
rules.menuNav = 'multiply Home and End errors by 1 and other key-navigation errors by 3; sum';
|
|
423
|
+
scores.menuNav
|
|
424
|
+
= 3 * facts.all.incorrect
|
|
425
|
+
- 2 * (facts.specific.home.incorrect + facts.specific.end.incorrect);
|
|
426
|
+
}
|
|
427
|
+
else {
|
|
428
|
+
inferences.menuNav = 150;
|
|
429
|
+
}
|
|
430
|
+
increment('menuNav');
|
|
431
|
+
}
|
|
432
|
+
else if (which === 'motion') {
|
|
433
|
+
facts = test.result;
|
|
434
|
+
if (facts && facts.bytes) {
|
|
435
|
+
rules.motion = 'get PNG screenshot sizes (sss); get differing-pixel counts between adjacent PNG screenshots (pd); “sssd” = sss difference ÷ smaller sss - 1; multiply mean adjacent sssd by 5, maximum adjacent sssd by 2, maximum over-all ssd by 1; divide mean pd by 10,000, maximum pd by 25,000; multiply count of non-0 pd by 30; sum';
|
|
436
|
+
scores.motion = Math.floor(
|
|
437
|
+
5 * (facts.meanLocalRatio - 1)
|
|
438
|
+
+ 2 * (facts.maxLocalRatio - 1)
|
|
439
|
+
+ facts.globalRatio - 1
|
|
440
|
+
+ facts.meanPixelChange / 10000
|
|
441
|
+
+ facts.maxPixelChange / 25000
|
|
442
|
+
+ 30 * facts.changeFrequency
|
|
443
|
+
);
|
|
444
|
+
}
|
|
445
|
+
else {
|
|
446
|
+
inferences.motion = 150;
|
|
447
|
+
}
|
|
448
|
+
increment('motion');
|
|
449
|
+
}
|
|
450
|
+
else if (which === 'radioSet') {
|
|
451
|
+
facts = test.result && test.result.totals;
|
|
452
|
+
if (facts) {
|
|
453
|
+
rules.radioSet = 'multiply radio buttons not in fieldsets with legends and no other-name radio buttons by 2';
|
|
454
|
+
// Defects discounted.
|
|
455
|
+
scores.radioSet = 2 * (facts.total - facts.inSet);
|
|
456
|
+
}
|
|
457
|
+
else {
|
|
458
|
+
inferences.radioSet = 100;
|
|
459
|
+
}
|
|
460
|
+
increment('radioSet');
|
|
461
|
+
}
|
|
462
|
+
else if (which === 'role') {
|
|
463
|
+
facts = test.result && test.result.badRoleElements;
|
|
464
|
+
if (typeof facts === 'number') {
|
|
465
|
+
rules.role = 'multiple role attributes with invalid or native-HTML-equivalent values by 2';
|
|
466
|
+
// Defects discounted.
|
|
467
|
+
scores.role = 2 * facts;
|
|
468
|
+
}
|
|
469
|
+
else {
|
|
470
|
+
inferences.role = 100;
|
|
471
|
+
}
|
|
472
|
+
increment('role');
|
|
473
|
+
}
|
|
474
|
+
else if (which === 'styleDiff') {
|
|
475
|
+
facts = test.result && test.result.totals;
|
|
476
|
+
if (facts) {
|
|
477
|
+
rules.styleDiff = 'for each of element classes block a, inline a, button, h1, h2, h3, h4, h5, and h6, get diffStyles-distinct styles; multiply their count minus 1 by 2; multiply count of elements with non-plurality styles by 0.2; sum';
|
|
478
|
+
// Identify objects having the tag-name totals and style distributions as properties.
|
|
479
|
+
const tagNameCounts = Object.values(facts);
|
|
480
|
+
// Identify an array of pairs of counts of excess styles and nonplurality elements.
|
|
481
|
+
const deficits = tagNameCounts.map(
|
|
482
|
+
item => {
|
|
483
|
+
const subtotals = item.subtotals ? item.subtotals : [item.total];
|
|
484
|
+
return [subtotals.length - 1, item.total - subtotals[0]];
|
|
485
|
+
}
|
|
486
|
+
);
|
|
487
|
+
// Deficit: 2 per excess style + 0.2 per nonplurality element.
|
|
488
|
+
scores.styleDiff = Math.floor(deficits.reduce(
|
|
489
|
+
(total, currentPair) => total + 2 * currentPair[0] + 0.2 * currentPair[1], 0
|
|
490
|
+
));
|
|
491
|
+
}
|
|
492
|
+
else {
|
|
493
|
+
inferences.styleDiff = 100;
|
|
494
|
+
}
|
|
495
|
+
increment('styleDiff');
|
|
496
|
+
}
|
|
497
|
+
else if (which === 'tabNav') {
|
|
498
|
+
facts = test.result && test.result.totals && test.result.totals.navigations;
|
|
499
|
+
if (facts) {
|
|
500
|
+
rules.tabNav = 'multiply Home and End errors by 1 and other key-navigation errors by 3; sum';
|
|
501
|
+
scores.tabNav
|
|
502
|
+
= 3 * facts.all.incorrect
|
|
503
|
+
- 2 * (facts.specific.home.incorrect + facts.specific.end.incorrect);
|
|
504
|
+
}
|
|
505
|
+
else {
|
|
506
|
+
inferences.tabNav = 150;
|
|
507
|
+
}
|
|
508
|
+
increment('tabNav');
|
|
509
|
+
}
|
|
510
|
+
else if (which === 'zIndex') {
|
|
511
|
+
facts = test.result && test.result.totals;
|
|
512
|
+
if (facts) {
|
|
513
|
+
rules.zIndex = 'multiply non-auto z indexes by 3';
|
|
514
|
+
scores.zIndex = 3 * facts.total;
|
|
515
|
+
}
|
|
516
|
+
else {
|
|
517
|
+
inferences.zIndex = 100;
|
|
518
|
+
}
|
|
519
|
+
increment('zIndex');
|
|
520
|
+
}
|
|
521
|
+
});
|
|
522
|
+
// Compute the inferred scores of package tests that failed and adjust the total score.
|
|
523
|
+
const estimate = (tests, penalty) => {
|
|
524
|
+
const packageScores = tests.map(test => scores[test]).filter(score => score !== null);
|
|
525
|
+
const scoreCount = packageScores.length;
|
|
526
|
+
let meanScore;
|
|
527
|
+
if (scoreCount) {
|
|
528
|
+
meanScore = Math.floor(
|
|
529
|
+
packageScores.reduce((sum, current) => sum + current) / packageScores.length
|
|
530
|
+
);
|
|
531
|
+
}
|
|
532
|
+
else {
|
|
533
|
+
meanScore = 100;
|
|
534
|
+
}
|
|
535
|
+
tests.forEach(test => {
|
|
536
|
+
if (scores[test] === null) {
|
|
537
|
+
inferences[test] = meanScore + penalty;
|
|
538
|
+
scores.total += inferences[test];
|
|
539
|
+
}
|
|
540
|
+
});
|
|
541
|
+
};
|
|
542
|
+
estimate(['alfa', 'aatt', 'axe', 'ibm', 'wave'], 100);
|
|
543
|
+
}
|
|
544
|
+
}
|
|
545
|
+
logScore = Math.floor(
|
|
546
|
+
logWeights.count * report.logCount
|
|
547
|
+
+ logWeights.size * report.logSize
|
|
548
|
+
+ logWeights.prohibited * report.prohibitedCount
|
|
549
|
+
+ logWeights.visitTimeout * report.visitTimeoutCount
|
|
550
|
+
+ logWeights.visitRejection * report.visitRejectionCount
|
|
551
|
+
);
|
|
552
|
+
scores.log = logScore;
|
|
553
|
+
scores.total += logScore;
|
|
554
|
+
// Add the score facts to the report.
|
|
555
|
+
report.score = {
|
|
556
|
+
scoreProc: 'tsp',
|
|
557
|
+
version: '9',
|
|
558
|
+
duplications,
|
|
559
|
+
rules,
|
|
560
|
+
diffStyles,
|
|
561
|
+
logWeights,
|
|
562
|
+
inferences,
|
|
563
|
+
scores
|
|
564
|
+
};
|
|
565
|
+
};
|