testilo 41.4.0 → 41.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -121,6 +121,67 @@
|
|
|
121
121
|
<p>How <a href="https://www.w3.org/WAI/">accessible</a> is the __org__ web page at <a href="__url__"><code>__url__</code></a>?</p>
|
|
122
122
|
<p>This digest can help answer that question. Ten different tools (Alfa, ASLint, Axe, Editoria11y, Equal Access, HTML CodeSniffer, Nu Html Checker, QualWeb, Testaro, and WAVE) tested the page to check its compliance with their accessibility rules. In all, the tools define about 990 rules, which are classified here into about 310 accessibility issues.</p>
|
|
123
123
|
<p>The results were interpreted to yield a score, with 0 being ideal. The score for this page was __total__, the sum of __issueCount__ for the count of issues, __issue__ for specific issues, __solo__ for unclassified rule violations, __tool__ for tool-by-tool ratings, __element__ for the count of violating elements, __prevention__ for the page preventing tools from running, __log__ for browser warnings, and __latency__ for delayed page responses.</p>
|
|
124
|
+
<details>
|
|
125
|
+
<summary>How the page was scored</summary>
|
|
126
|
+
<h3>Introduction</h3>
|
|
127
|
+
<p>This is an explanation of the scoring of the page.</p>
|
|
128
|
+
<h3>Motivations for scoring</h3>
|
|
129
|
+
<p>Why score? Specifically, why aggregate many facts about a page into a single number?</p>
|
|
130
|
+
<p>One motivation is to simplify comparison and tracking. If you want to compare a page with other pages, or if you want to track changes in a page over time, aggregating many test results for each page into one page score simplifies the task. It becomes possible to say things like <q>Page A is more accessible than page B</q> or <q>Page A is becoming more accessible over time</q>.</p>
|
|
131
|
+
<p>Another motivation is to influence behavior. A score arises from decisions about importance, urgency, and other attributes. People may use scores to award benefits, impose costs, or prioritize work. So, scoring can influence who gets and does what.</p>
|
|
132
|
+
<h3>How to score?</h3>
|
|
133
|
+
<p>Scoring is subjective. Accessibility testing tools use various scoring procedures, based on various ideas.</p>
|
|
134
|
+
<p>The built-in scoring procedures of Testilo are based on the idea that multiple attributes should affect a web-page accessibility score, including:</p>
|
|
135
|
+
<ul>
|
|
136
|
+
<li>conformity to standards</li>
|
|
137
|
+
<li>adherence to best practices</li>
|
|
138
|
+
<li>how many different issues exist</li>
|
|
139
|
+
<li>how many instances of each issue exist</li>
|
|
140
|
+
<li>how many tools report violations of their rules</li>
|
|
141
|
+
<li>how trustworthy each test of each tool is</li>
|
|
142
|
+
<li>how serious each violation of a rule is</li>
|
|
143
|
+
<li>simplicity</li>
|
|
144
|
+
<li>speed of page responses</li>
|
|
145
|
+
<li>testability</li>
|
|
146
|
+
<li>conformity to expectations of browsers</li>
|
|
147
|
+
</ul>
|
|
148
|
+
<p>Some ideas found elsewhere in accessibility scoring are <strong>rejected</strong> by the Testilo procedures:</p>
|
|
149
|
+
<ul>
|
|
150
|
+
<li>density: that an accessibility score should be based on the count of accessibility faults as a fraction of the count of potential accessibility faults (so, the score of a large page with particular faults should be better than the score of a smaller page with exactly those faults)</li>
|
|
151
|
+
<li>applicability: that the score of a page should depend on the types of its content (so, the score of a page with accessible video content should be better than the score of an otherwise identical page without video content)</li>
|
|
152
|
+
</ul>
|
|
153
|
+
<h3>Scoring method summary</h3>
|
|
154
|
+
<p>The scoring method is found in the code of the procedure. The file for any procedure <code>tspnn</code> is in the file <code>procs/score/tspnn.js</code> within the <a href="https://www.npmjs.com/package/testilo?activeTab=code">Testilo package</a>.</p>
|
|
155
|
+
<p>The score computed by this procedure for any page is a non-negative number. The numbers represent accessibility faults, so a higher-scoring page is considered <strong>less</strong> accessible than a lower-scoring page. The best possible score is 0.</p>
|
|
156
|
+
<p>The procedure makes the score of a page the sum of 7 components:</p>
|
|
157
|
+
<ul>
|
|
158
|
+
<li>issue score: how many instances of issues were reported</li>
|
|
159
|
+
<li>solo score: how many violations of rules not yet classified into issues were reported</li>
|
|
160
|
+
<li>tool score: how many violations of their rules did tools report</li>
|
|
161
|
+
<li>element score: how many HTML elements did any tool identify as violating at least one tool rule</li>
|
|
162
|
+
<li>prevention score: how many tools were unable to perform their tests on the page</li>
|
|
163
|
+
<li>log score: how much logging of page errors and other events did the browser do</li>
|
|
164
|
+
<li>latency score: how long did the page take to load and become stable</li>
|
|
165
|
+
</ul>
|
|
166
|
+
<h3>Scoring method details</h3>
|
|
167
|
+
<p>The above component descriptions omit various details.</p>
|
|
168
|
+
<p>About a thousand tool rules are classified into <dfn>issues</dfn> in the <code>tic45.js</code> file used by this procedure. Each of those rules has a <dfn>quality</dfn>, ranging from 0 to 1. When the <strong>issue score</strong> is computed, the count of violations of each rule is multiplied by the quality of that rule. Whichever tool has the largest quality-weighted violation count for an issue, that count is treated as the instance count for the issue. Thus, if 8 tools each report 15 violations within the issue, and their rule qualities are all 1, the instance count is 15, not 120. Moreover, the issue itself has a <dfn>weight</dfn>, ranging from 1 to 4, representing its importance. This instance count is multiplied by that weight. That product is further multiplied by the <code>issueCountWeight</code> value, namely 10. That final product is further adjusted if the issue is inherently limited in instance count. For example, if the issue is that the page <code>html</code> element has no <code>lang</code> attribute, the instance count is limited to 1. If there is such a limit, the <code>maxWeight</code> value, namely 30, is divided by the actual instance count and the quotient is added to 1. That product (or 1 if there is no limit) is multiplied by the instance count, and then the result is treated as the contribution of the issue to the issue score.</p>
|
|
169
|
+
<p>Each <dfn>solo</dfn> (not yet issue-classified) rule violation is multiplied by the sum of 1 and the ordinal severity of the rule, to produce the <strong>solo score</strong>.</p>
|
|
170
|
+
<p>Each rule violation reported by each tool is severity-weighted in the same way as solo rule violations are. Then the sum of those violations is multiplied by the <code>toolWeight</code> value, namely 0.1, to produce the <strong>tool score</strong>.</p>
|
|
171
|
+
<p>The count of elements reported as violators of any rule is multiplied by the <code>elementWeight</code> value, namely 2, to produce the <strong>element score</strong>.</p>
|
|
172
|
+
<p>The count of prevented Testaro rules is multiplied by the <code>testaroRulePreventionWeight</code> quantity, namely 30, and the count of other prevented tools is multiplied by the <code>preventionWeight</code> value, namely 300, to produce the <strong>prevention score</strong>.</p>
|
|
173
|
+
<p>The <strong>log score</strong> is the sum of several components:</p>
|
|
174
|
+
<ul>
|
|
175
|
+
<li>log count: how many times the browser logged a message</li>
|
|
176
|
+
<li>log size: how many characters large the collection of browser log messages was</li>
|
|
177
|
+
<li>error log count: how many browser log messages reported page errors</li>
|
|
178
|
+
<li>error log size: how many characters large the collection of browser log messages reporting page errors was</li>
|
|
179
|
+
<li>prohibition count: how many times the browser logged a prohibited response status (403)</li>
|
|
180
|
+
<li>visit rejection count: how many times the browser logged an abnormal response status</li>
|
|
181
|
+
</ul>
|
|
182
|
+
<p>Each of these components is multiplied by a weight found in the <code>logWeights</code> object.</p>
|
|
183
|
+
<p>Finally, the <strong>latency score</strong> is based on how much longer it took for the page to become loaded and stable than expected. The expected total duration in seconds is the <code>normalLatency</code> value, namely 22 seconds (that is 2 seconds per visit, multiplied by the 11 visits of the 11 tools). This is subtracted from the actual total latency, and that difference is multiplied by the <code>latencyWeight</code> amount, namely 2.</p>
|
|
184
|
+
</details>
|
|
124
185
|
<h2 id="summary">Issue summary</h2>
|
|
125
186
|
<h3>Details about this summary</h3>
|
|
126
187
|
<ul>
|
|
@@ -27,7 +27,7 @@
|
|
|
27
27
|
// Module to keep secrets.
|
|
28
28
|
require('dotenv').config();
|
|
29
29
|
// Module to classify tool rules into issues
|
|
30
|
-
const {issues} = require('../../score/
|
|
30
|
+
const {issues} = require('../../score/tic49');
|
|
31
31
|
// Module to process files.
|
|
32
32
|
const fs = require('fs/promises');
|
|
33
33
|
// Utility module.
|
|
@@ -36,7 +36,7 @@ const {getNowDate, getNowDateSlash} = require('../../util');
|
|
|
36
36
|
// CONSTANTS
|
|
37
37
|
|
|
38
38
|
// Digester ID.
|
|
39
|
-
const digesterID = '
|
|
39
|
+
const digesterID = 'tdp49';
|
|
40
40
|
// Newline with indentations.
|
|
41
41
|
const innerJoiner = '\n ';
|
|
42
42
|
const outerJoiner = '\n ';
|