testilo 21.4.1 → 21.5.0

package/README.md

@@ -137,9 +137,13 @@ Here is a script:
 ```javaScript
 {
   id: 'ts99',
-  what: 'Axe and QualWeb on account page',
+  what: 'aside mislocation',
   strict: true,
+  isolate: true,
   timeLimit: 60,
+  standard: 'also',
+  observe: false,
+  timeStamp: '240115T1200',
   acts: [
     {
       type: 'placeholder',
@@ -149,27 +153,38 @@ Here is a script:
     {
       type: 'test',
       which: 'axe',
-      detailLevel: 1,
-      rules: [],
-      what: 'Axe core, all rules'
+      detailLevel: 2,
+      rules: ['landmark-complementary-is-top-level'],
+      what: 'Axe'
     },
     {
       type: 'test',
       which: 'qualWeb',
       withNewContent: false,
-      what: 'QualWeb, all rules'
+      rules: ['QW-BP25', 'QW-BP26']
+      what: 'QualWeb'
     }
   ]
 }
 ```
 
-This script has 3 acts. The first is a placeholder act. The above batch can be merged with this script to create jobs. In that case, the first job would launch a Chromium browser, navigate to the Acme login page, complete and submit the login form, wait for the account page to load, run the Axe tests, and then run the QualWeb tests. If the batch contained additional targets, additional jobs would be created, with the login actions for each target specified in the `private` array of the `acts` object of that target.
+A script has several properties that specify facts about the jobs to be created. They include:
+- `id`: an ID that uniquely distinguishes the script from other scripts.
+- `what`: a description of the script.
+- `strict`: You decide whether Testaro is to throw an error on an attempt to navigate to a URL if the server redirects the request to a URL differing substantially from the specified URL. All differences are considered substantial unless the URLs differ only in the presence and absence of a trailing slash.
+- `isolate`: You decide whether to isolate test acts, as needed, from effects of previous test acts. If `true`, the `merge` module will add a copy of the latest placeholder after each target-modifying test act before the immediately following test act.
+- `standard`: You choose how the reports of test acts should be standardized. The alternatives are `'no'` (do not standardize), `'also'` (standardize and report both the original and the standardized results), and `'only'` (standardize and report only the standardized results).
+- `observe`: You decide how granularly Testaro will allow a server to observe job progress. If `true`, Testaro sends a message to the server to announce each test act (identifying the tool), and the `testaro` tool sends a message to the server to announce each rule when its test is performed. The server can further update the requesting client on the basis of these messages. It is generally user-friendly to make `observe` `true` if the user application makes the user wait while the job is assigned and performed. If the application allows the user to leave and sends the user a message when the job has been completed, `observe` can be set to `false`.
+- `timeStamp`: You can specify the date and time that the job is to wait for before it is performed. This specification is a string with the format `yymmddThhMM`.
+- `acts`: an array of acts.
+
+The first act in this example script is a placeholder, whose `which` property is `'private'`. If the above batch were merged with this script, in each job the placeholder would be replaced with the `private` acts of a target. For example, the first act of the first job would launch a Chromium browser, navigate to the Acme login page, complete and submit the login form, wait for the account page to load, run the Axe tests, and then run the QualWeb tests. If the batch contained additional targets, additional jobs would be created, with the login actions for each target specified in the `private` array of the `acts` object of that target.
 
-As shown in this example, when a browser is launched by placeholder substitution, the script can determine the browser type (`chromium`, `firefox`, or `webkit`) by assigning a value to a `launch` property of the placeholder. That is useful, because sometimes it is the actions specified in a script that dictate which browser type is appropriate.
+As shown in this example, when a browser is launched by placeholder substitution, the script can determine the browser type (`chromium`, `firefox`, or `webkit`) by assigning a value to a `launch` property of the placeholder.
 
 ### Target list to batch
 
-If you have a target list, the `batch` module of Testilo can convert it to a batch. The batch will contain, for each target, one array of acts named `main`, containing a `launch` act (depending on the script to specify the browser type) and a `url` act.
+If you have a target list, the `batch` module of Testilo can convert it to a batch. The batch will contain, for each target, one array of acts named `main`, containing a `launch` act (depending on the script to specify the browser type and the target to specify the URL).
 
 #### Invocation
 
@@ -203,11 +218,13 @@ The `call` module will save the batch as a JSON file in the `batches` subdirecto
 
 ### Issues to script
 
-Testilo classifies issues. The built-in issue classifications are located in the `procs/score` directory, in files whose names begin with `tic` (for “Testilo issue classification”). You can create additional `tic` files with custom issue classifications.
+Testilo classifies tool rules into _issues_. The built-in classifications are located in the `procs/score` directory, in files whose names begin with `tic` (for “Testilo issue classification”). You can create additional `tic` files with custom classifications.
+
+For example, one of the issues in the `tic38.js` file is `mainNot1`. Four rules are classified as belonging to that issue: rule `main_element_only_one` of `aslint` and 3 more rules defined by 3 other tools.
 
-If you want Testaro to test targets for particular issues, you can name those issues and use the Testilo `script` module to create a script.
+If you want Testaro to test targets for only particular issues, you can name those issues and use the Testilo `script` module to create a script for that purpose. The only tools called by the script will be tools that define rules that are classified as belonging to one or more issues named.
 
-If you want Testaro to test targets for **all** the rules of all the available tools, without regard to any issue classification, you can use the `script` module to create a script that does not impose any issue restrictions.
+If you want Testaro to test targets for **all** the rules of all the available tools, you can use the `script` module to create a script that does not impose any issue restrictions.
 
 #### Invocation
 
@@ -223,13 +240,13 @@ const scriptObj = script(scriptID, issues, issueID0, issueID1, …);
 ```
 
 This invocation references `scriptID`, `issues`, and `issueID` variables.
-- The `scriptID` variable is an alphanumeric string.
-- The `issues` variable is an object that classifies issues, such as the `issues` object in a `tic` file.
-- The `issueID` variables are strings, such as `'regionNoText'`, that name of properties of the `issues` object.
+- The `scriptID` variable is an arbitrary alphanumeric string.
+- The `issues` variable (if present) is an object that classifies issues, such as the `issues` object in a `tic` file.
+- The `issueID` variables (if any) are strings, such as `'regionNoText'`, that name properties of the `issues` object.
 
-The `script()` function of the `script` module generates a script and returns it as an object. The invoking module can further dispose of the script as needed.
+The `script()` function of the `script` module generates a script and returns it as an object. The invoking module can further modify and use the script as needed.
 
-To create a script without issue restrictions, a module can use this invocation:
+To create a script **without** issue restrictions, a module can use this invocation:
 
 ```javaScript
 const {script} = require('testilo/script');
@@ -241,11 +258,11 @@ const scriptObj = script(scriptID);
 A user can invoke `script` in this way: In the Testilo project directory, execute the statement `node call script s c i0 i1 i2 i3 …`.
 
 In this statement:
-- Replace `s` with an ID for the script, such as `headings`.
+- Replace `s` with an arbitrary ID for the script, such as `headings`.
 - Replace `c` with the base name, such as `tic99`, of an issue classification file in the `score` subdirectory of the `process.env.FUNCTIONDIR` directory.
 - Replace the remaining arguments (`i0` etc.) with issue IDs from that classification file.
 
-The `call` module will retrieve the named classification from its directory.
+The `call` module will retrieve the named classification.
 The `script` module will create a script.
 The `call` module will save the script as a JSON file in the `scripts` subdirectory of the `process.env.SPECDIR` directory.
 
@@ -259,7 +276,7 @@ When the `script` module creates a script for you, it does not ask you for all o
 
 Testilo merges batches with scripts, producing jobs, by means of the `merge` module.
 
-The `merge` module needs to be given a batch and a script. In addition, `merge` offers an isolation option. If you exercise it, the `merge` module will act as if the latest placeholder were **again** inserted after each target-modifying test act, except where that test act is the last act or where the next act after it is a placeholder.
+The `merge` module needs to be given a batch and a script.
 
 #### Output
 
@@ -362,7 +379,7 @@ A module can invoke `merge` in this way:
 
 ```javaScript
 const {merge} = require('testilo/merge');
-const jobs = merge(script, batch, requester, true, 'only', false);
+const jobs = merge(script, batch, requester, true, 'only', false, '240115T1200');
 ```
 
 This invocation references `script`, `batch`, and `requester` variables that the module must have already defined. The `script` and `batch` variables are a script object and a batch object, respectively. The `requester` variable is an email address. The fourth argument is a boolean, specifying whether to perform test isolation. The fifth argument is a string that specifies the Testaro standardization option ('also', 'only', or 'no'). The sixth argument is a boolean, specifying whether Testaro will allow granular network watching of the job. The `merge()` function of the `merge` module generates jobs and returns them in an array. The invoking module can further dispose of the jobs as needed.

package/batch.js

@@ -1,10 +1,6 @@
 /*
   batch.js
   Converts a target list to a batch.
-  Arguments:
-    0. batch ID
-    1. batch description
-    2. target list
 */
 
 // ########## FUNCTIONS

package/merge.js

@@ -8,6 +8,7 @@
     3. whether to provide test isolation
     4. value of the standard property
     5. whether reporting is to be granular
+    6. date and time as a compact timestamp for job execution, if not now
 */
 
 // ########## IMPORTS
@@ -17,34 +18,85 @@ require('dotenv').config();
 
 // ########## CONSTANTS
 
-const stdRequester = process.env.REQUESTER;
+// Standard requester.
+const stdRequester = process.env.REQUESTER || 'nobody@nodomain.tld';
+// Length of the random part of a job ID, as a string.
+const randomIDLength = process.env.RANDOM_ID_LENGTH || '3';
 // Tools that alter the page.
 const contaminantNames = new Set([
   'alfa',
   'aslint',
   'axe',
   'htmlcs',
-  'ibm',
   'testaro'
 ]);
+const randomIDChars = (() => {
+  const digits = Array(10).fill('').map((digit, index) => index.toString());
+  const uppers = Array(26).fill('').map((letter, index) => String.fromCodePoint(65 + index));
+  const lowers = Array(26).fill('').map((letter, index) => String.fromCodePoint(97 + index));
+  return digits.concat(uppers, lowers);
+})();
+
 
 // ########## FUNCTIONS
 
+// Inserts a character periodically in a string.
+const punctuate = (string, insertion, chunkSize) => {
+  const segments = [];
+  let startIndex = 0;
+  while (startIndex < string.length) {
+    segments.push(string.slice(startIndex, startIndex + chunkSize));
+    startIndex += chunkSize;
+  }
+  return segments.join(insertion);
+};
+// Converts a compact timestamp to a date.
+const dateOf = timeStamp => {
+  if (/^\d{6}T\d{4}$/.test(timeStamp)) {
+    const dateString = punctuate(timeStamp.slice(0, 6), '-', 2);
+    const timeString = punctuate(timeStamp.slice(7, 11), ':', 2);
+    return new Date(`20${dateString}T${timeString}Z`);
+  } else {
+    return null;
+  }
+};
+// Converts a date and time to a compact timestamp.
+const stampTime = date => date.toISOString().replace(/[-:]/g, '').slice(2, 13);
+// Generates a random string.
+const getRandomID = length => {
+  const chars = [];
+  for (let i = 0; i < length; i++) {
+    chars.push(randomIDChars[Math.floor(62 * Math.random())]);
+  }
+  return chars.join('');
+};
 // Merges a script and a batch and returns jobs.
-exports.merge = (script, batch, requester, isolate, standard, isGranular) => {
+exports.merge = (script, batch, requester, isolate, standard, observe, timeStamp) => {
   if (isolate === 'false') {
     isolate = false;
   }
-  // If the requester is unspecified, make it the standard requester.
+  // If a timestamp was specified:
+  if (timeStamp) {
+    // If it is invalid:
+    if (! dateOf(timeStamp)) {
+      // Report this and quit.
+      console.log(`ERROR: Timestamp invalid`);
+      return [];
+    }
+  }
+  // Otherwise, i.e. if no timestamp was specified:
+  else {
+    // Create one for the job.
+    timeStamp = stampTime(new Date());
+  }
+  // If the requester is blank or unspecified, make it the standard requester.
   requester ||= stdRequester;
-  // Create a timestamp.
-  const timeStamp = new Date().toISOString().replace(/[-:]/g, '').slice(2, 15);
-  // Create a time description.
-  const creationTime = (new Date()).toISOString().slice(0, 19);
+  // Create a creation-time description.
+  const creationTime = (new Date()).toISOString().slice(0, 16);
   // Initialize a target-independent job.
   const protoJob = JSON.parse(JSON.stringify(script));
-  // Add the timestamp to the ID of the job.
-  protoJob.id = `${timeStamp}-${protoJob.id}-`;
+  // Make the timestamp and a random string the ID of the job.
+  protoJob.id = `${timeStamp}-${getRandomID(Number.parseInt(randomIDLength, 10))}`;
   // Add a sources property to the job.
   protoJob.sources = {
     script: script.id,
@@ -61,10 +113,10 @@ exports.merge = (script, batch, requester, isolate, standard, isGranular) => {
   protoJob.creationTime = creationTime;
   protoJob.timeStamp = timeStamp;
   protoJob.standard = standard || 'only';
-  protoJob.observe = isGranular || false;
+  protoJob.observe = observe || false;
   // If isolation was requested:
   if (isolate) {
-    // Perform it.
+    // Configure the job for it.
     let {acts} = protoJob;
     let lastPlaceholder = {};
     for (const actIndexString in acts) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testilo",
-  "version": "21.4.1",
+  "version": "21.5.0",
   "description": "Prepares and processes Testaro reports",
   "main": "aim.js",
   "scripts": {

package/procs/digest/{tdp38 → tdp39}/index.html

@@ -26,13 +26,15 @@
           <tr><th>Tested by</th><td>Testaro, procedure <code>__ts__</code></td></tr>
           <tr><th>Scored by</th><td>Testilo, procedure <code>__sp__</code></td></tr>
           <tr><th>Digested by</th><td>Testilo, procedure <code>__dp__</code></td></tr>
-          <tr><th>Full report</th><td><a href="__reportURL__"><code>__reportURL__</code></a></td></tr>
+          <tr>
+            <th>Full report</th><td><a href="__reportURL__"><code>__reportURL__</code></a></td>
+          </tr>
         </table>
       </header>
       <h2>Introduction</h2>
       <p>How <a href="https://www.w3.org/WAI/">accessible</a> is the __org__ web page at <a href="__url__"><code>__url__</code></a>?</p>
       <p>This digest can help answer that question. Nine different tools (Alfa, ASLint, Axe, Equal Access, HTML CodeSniffer, Nu Html Checker, QualWeb, Testaro, and WAVE) tested the page to check its compliance with their accessibility rules. In all, the tools define about 915 rules, which are classified here into about 300 accessibility issues.</p>
-      <p>The results were interpreted to yield a score, with 0 being ideal. The score for this page was __total__, the sum of __issueCount__ for the count of issues, __issue__ for specific issues, __tool__ for tool-by-tool ratings, __prevention__ for the page preventing tools from running, __log__ for browser warnings, and __latency__ for delayed page responses.</p>
+      <p>The results were interpreted to yield a score, with 0 being ideal. The score for this page was __total__, the sum of __issueCount__ for the count of issues, __issue__ for specific issues, __solo__ for unclassified rule violations, __tool__ for tool-by-tool ratings, __prevention__ for the page preventing tools from running, __log__ for browser warnings, and __latency__ for delayed page responses.</p>
       <h2>Issue summary</h2>
       <table class="allBorder thirdCellRight">
         <caption>Summary of issues</caption>

package/procs/digest/{tdp38 → tdp39}/index.js

@@ -43,7 +43,8 @@ const populateQuery = (report, query) => {
     summaryRows: [],
     issueRows: []
   };
-  ['total', 'issueCount', 'issue', 'tool', 'prevention', 'log', 'latency'].forEach(sumItem => {
+  ['total', 'issueCount', 'issue', 'solo', 'tool', 'prevention', 'log', 'latency']
+  .forEach(sumItem => {
     query[sumItem] = summary[sumItem];
     rows.summaryRows.push(getScoreRow(sumItem, query[sumItem]));
   });

package/procs/score/{tic38.js → tic39.js}

@@ -1,6 +1,6 @@
 /*
-  tic37
-  Testilo issue classification 38
+  tic39
+  Testilo issue classification 39
 
   Classifies about 900 rules of the tools of Testaro into about 300 issues.
 
@@ -163,7 +163,7 @@ exports.issues = {
         'QW-WCAG-T20': {
           variable: false,
           quality: 0,
-          what: 'Link text is not supplemented with a title attribute [invalid]'
+          what: 'Link title may fail to describe the link correctly [speculative]'
         },
         'QW-WCAG-T23': {
           variable: false,
@@ -4363,28 +4363,12 @@ exports.issues = {
       }
     }
   },
-  h1Multiple: {
-    summary: 'multiple h1 headings',
+  h1Not1: {
+    summary: 'not exactly 1 h1 heading',
     why: 'User cannot understand the topic of the document',
     wcag: '1.3.1',
     weight: 2,
     max: 1,
-    tools: {
-      nuVal: {
-        'Consider using the h1 element as a top-level heading only (all h1 elements are treated as top-level headings by many screen readers and other tools).': {
-          variable: false,
-          quality: 1,
-          what: 'Page contains more than 1 h1 element'
-        }
-      }
-    }
-  },
-  h1Missing: {
-    summary: 'h1 missing',
-    why: 'User cannot understand the topic of the document',
-    wcag: '1.3.1',
-    weight: 3,
-    max: 1,
     tools: {
       aslint: {
         'h1_must_be': {
@@ -4400,6 +4384,20 @@ exports.issues = {
           what: 'Document contains no level-one heading'
         }
       },
+      nuVal: {
+        'Consider using the h1 element as a top-level heading only (all h1 elements are treated as top-level headings by many screen readers and other tools).': {
+          variable: false,
+          quality: 1,
+          what: 'Page contains more than 1 h1 element'
+        }
+      },
+      qualWeb: {
+        'QW-BP28': {
+          variable: false,
+          quality: 1,
+          what: 'h1 element missing or used more than once'
+        }
+      },
       wave: {
         'h1_missing': {
           variable: false,
@@ -4696,6 +4694,13 @@ exports.issues = {
           quality: 1,
           what: 'Element is not contained by a ul or ol element'
         }
+      },
+      qualWeb: {
+        'QW-BP23': {
+          variable: false,
+          quality: 0.1,
+          what: 'Element is not in a true list'
+        }
       }
     }
   },

package/procs/score/{tsp38.js → tsp39.js}

@@ -7,12 +7,12 @@
 
 // IMPORTS
 
-const {issues} = require('./tic38');
+const {issues} = require('./tic39');
 
 // CONSTANTS
 
 // ID of this proc.
-const scoreProcID = 'tsp38';
+const scoreProcID = 'tsp39';
 // Latency weight (how much each second of excess latency adds to the score).
 const latencyWeight = 1;
 // Normal latency (6 visits, with 1.5 second per visit).

package/script.js

@@ -1,9 +1,6 @@
 /*
   script.js
-  Creates and returns a script.
-  Arguments:
-    0. issue classification
-    1. issue IDs
+  Creates and returns a script to perform the tests for issues.
 */
 
 // ########## IMPORTS
@@ -13,6 +10,7 @@ require('dotenv').config();
 
 // ########## VARIABLES
 
+// List of presumptively needed tools.
 let toolIDs = [
   'alfa', 'aslint', 'axe', 'htmlcs', 'ibm', 'nuVal', 'qualWeb', 'testaro', 'wave'
 ];
@@ -21,7 +19,7 @@ let toolIDs = [
 
 // Creates and returns a script.
 exports.script = (id, issues = null, ... issueIDs) => {
-  // Initialize data on the tools and rules for the specified issues.
+  // Initialize data on the tools and their rules for the specified issues, if any.
   const neededTools = {};
   // If an issue classification and any issues were specified:
   if (issues && issueIDs.length) {
@@ -53,42 +51,47 @@ exports.script = (id, issues = null, ... issueIDs) => {
             }
           });
         });
+        // Remove unneeded tools from the tool list.
         toolIDs = Object.keys(neededTools);
       }
       // Otherwise, i.e. if it does not exist in the classification:
       else {
-        // Report this.
+        // Report this and quit.
         console.log(`ERROR: Issue ${issueID} not in issue classification`);
         return {};
       }
     });
   }
-  // If any rules have been identified:
+  // If, after any issue-based pruning, any needed tools remain:
   if (toolIDs.length) {
     // Initialize a script.
     const scriptObj = {
       id,
       what: `accessibility tests`,
       strict: true,
+      isolate: true,
       timeLimit: 30 + (10 * issueIDs.length || 30 * toolIDs.length),
+      standard: 'only',
+      observe: true,
+      timeStamp: '',
       acts: [
         {
           "type": "placeholder",
           "which": "main",
-          "launch": "chromium"
+          "launch": "webkit"
         }
       ]
     };
-    // For each identified tool:
+    // For each needed tool:
     toolIDs.forEach(toolID => {
       // Initialize a test act for it.
       const toolAct = {
         type: 'test',
         which: toolID
       };
-      // If rules were specified:
+      // If issues were specified:
       if (issues && issueIDs.length) {
-        // Add a rules property to the act.
+        // Add a rules array as a property to the act.
         toolAct.rules = neededTools[toolID];
         // If the tool is Testaro:
         if (toolID === 'testaro') {
@@ -96,13 +99,13 @@ exports.script = (id, issues = null, ... issueIDs) => {
           toolAct.rules.unshift('y');
         }
       }
-      // Add option specifications if necessary.
+      // Add any needed option specifications to the act.
       if (toolID === 'axe') {
         toolAct.detailLevel = 2;
       }
       else if (toolID === 'ibm') {
         toolAct.withItems = true;
-        toolAct.withNewContent = false;
+        toolAct.withNewContent = true;
       }
       else if (toolID === 'qualWeb') {
         toolAct.withNewContent = false;

package/scripts/{ts36.json → ts39.json}

@@ -1,8 +1,12 @@
 {
-  "id": "ts36",
+  "id": "ts39",
   "what": "comprehensive accessibility tests with webkit and redirection permitted",
   "strict": false,
-  "timeLimit": 330,
+  "isolate": true,
+  "timeLimit": 600,
+  "standard": "only",
+  "observe": true,
+  "timeStamp": "",
   "acts": [
     {
       "type": "placeholder",
@@ -30,7 +34,7 @@
       "type": "test",
       "which": "ibm",
       "withItems": true,
-      "withNewContent": false
+      "withNewContent": true
     },
     {
       "type": "test",

package/procs/score/tsp38-old.js

@@ -1,300 +0,0 @@
-/*
-  tsp37
-  Testilo score proc 37
-
-  Computes target score data and adds them to a ts37 report.
-*/
-
-// IMPORTS
-
-const {issues} = require('./tic38');
-
-// CONSTANTS
-
-// ID of this proc.
-const scoreProcID = 'tsp38';
-// Latency weight (how much each second of excess latency adds to the score).
-const latencyWeight = 1;
-// Normal latency (6 visits, with 1.5 second per visit).
-const normalLatency = 9;
-// Prevention weight (how much each prevention adds to the score).
-const preventionWeight = 300;
-// Maximum instance count addition weight (divisor of max).
-const maxWeight = 30;
-// Other weights.
-const severityWeights = [1, 2, 3, 4];
-const toolWeight = 0.1;
-const logWeights = {
-  logCount: 0.1,
-  logSize: 0.002,
-  errorLogCount: 0.2,
-  errorLogSize: 0.004,
-  prohibitedCount: 3,
-  visitRejectionCount: 2
-};
-// Initialize a table of tool rules.
-const issueIndex = {};
-// Initialize an array of variably named tool rules.
-const issueMatcher = [];
-// For each issue:
-Object.keys(issues).forEach(issueName => {
-  // For each tool with rules belonging to that issue:
-  Object.keys(issues[issueName].tools).forEach(toolName => {
-    // For each of those rules:
-    Object.keys(issues[issueName].tools[toolName]).forEach(ruleID => {
-      // Add it to the table of tool rules.
-      if (! issueIndex[toolName]) {
-        issueIndex[toolName] = {};
-      }
-      issueIndex[toolName][ruleID] = issueName;
-      // If it is variably named:
-      if (issues[issueName].tools[toolName][ruleID].variable) {
-        // Add it to the array of variably named tool rules.
-        issueMatcher.push(ruleID);
-      }
-    })
-  });
-});
-
-// FUNCTIONS
-
-// Scores a report.
-exports.scorer = report => {
-  // If there are any acts in the report:
-  const {acts} = report;
-  if (Array.isArray(acts) && acts.length) {
-    // If any of them are test acts:
-    const testActs = acts.filter(act => act.type === 'test');
-    if (testActs.length) {
-      // Initialize the score data.
-      const score = {
-        scoreProcID,
-        weights: {
-          severities: severityWeights,
-          tool: toolWeight,
-          log: logWeights,
-          latency: latencyWeight,
-          prevention: preventionWeight,
-          maxInstanceCount: maxWeight
-        },
-        normalLatency,
-        summary: {
-          total: 0,
-          issue: 0,
-          solo: 0,
-          tool: 0,
-          prevention: 0,
-          log: 0,
-          latency: 0
-        },
-        details: {
-          severity: {
-            total: [0, 0, 0, 0],
-            byTool: {}
-          },
-          prevention: {},
-          issue: {},
-          solo: {},
-          tool: {}
-        }
-      };
-      const {summary, details} = score;
-      // For each test act:
-      testActs.forEach(act => {
-        // If the page prevented the tool from operating:
-        const {which, standardResult} = act;
-        if (! standardResult || standardResult.prevented) {
-          // Add this to the score.
-          details.prevention[which] = preventionWeight;
-        }
-        // Otherwise, if a valid standard result exists:
-        else if (
-          standardResult
-          && standardResult.totals
-          && standardResult.totals.length === 4
-          && standardResult.instances
-        ) {
-          // Add the severity totals of the tool to the score.
-          const {totals} = standardResult;
-          details.severity.byTool[which] = totals;
-          // Add the severity-weighted tool totals to the score.
-          details.tool[which] = totals.reduce(
-            (sum, current, index) => sum + severityWeights[index] * current, 0
-          );
-          // For each instance of the tool:
-          standardResult.instances.forEach(instance => {
-            // Get the rule ID.
-            const {ruleID, ordinalSeverity, count} = instance;
-            // If it is not in the table of tool rules:
-            let canonicalRuleID = ruleID;
-            if (! issueIndex[which][ruleID]) {
-              // Convert it to the variably named tool rule that it matches, if any.
-              canonicalRuleID = issueMatcher.find(pattern => {
-                const patternRE = new RegExp(pattern);
-                return patternRE.test(instance.ruleID);
-              });
-            }
-            // If the rule ID belongs to an issue:
-            if (canonicalRuleID) {
-              // Get the issue.
-              const issueName = issueIndex[which][canonicalRuleID];
-              // Add the instance to the issue details of the score data.
-              if (! details.issue[issueName]) {
-                details.issue[issueName] = {
-                  summary: issues[issueName].summary,
-                  score: 0,
-                  maxCount: 0,
-                  weight: issues[issueName].weight,
-                  countLimit: issues[issueName].max,
-                  tools: {}
-                };
-                if (! details.issue[issueName].countLimit) {
-                  delete details.issue[issueName].countLimit;
-                }
-              }
-              if (! details.issue[issueName].tools[which]) {
-                details.issue[issueName].tools[which] = {};
-              }
-              if (! details.issue[issueName].tools[which][canonicalRuleID]) {
-                const ruleData = issues[issueName].tools[which][canonicalRuleID];
-                details.issue[issueName].tools[which][canonicalRuleID] = {
-                  quality: ruleData.quality,
-                  what: ruleData.what,
-                  complaints: {
-                    countTotal: 0,
-                    texts: []
-                  }
-                };
-              }
-              details
-              .issue[issueName]
-              .tools[which][canonicalRuleID]
-              .complaints
-              .countTotal += instance.count || 1;
-              if (
-                ! details
-                .issue[issueName]
-                .tools[which][canonicalRuleID]
-                .complaints
-                .texts
-                .includes(instance.what)
-              ) {
-                details
-                .issue[issueName]
-                .tools[which][canonicalRuleID]
-                .complaints
-                .texts
-                .push(instance.what);
-              }
-            }
-            // Otherwise, i.e. if the rule ID belongs to no issue:
-            else {
-              // Add the instance to the solo details of the score data.
-              if (! details.solo[which]) {
-                details.solo[which] = {};
-              }
-              if (! details.solo[which][ruleID]) {
-                details.solo[which][ruleID] = 0;
-              }
-              details.solo[which][ruleID] += (count || 1) * (ordinalSeverity + 1);
-              // Report this.
-              console.log(`ERROR: ${instance.ruleID} of ${which} not found in issues`);
-            }
-          });
-        }
-        // Otherwise, i.e. if a failed standard result exists:
-        else {
-          // Add an inferred prevention to the score.
-          details.prevention[which] = preventionWeight;
-        }
-      });
-      // For each issue with any complaints:
-      Object.keys(details.issue).forEach(issueName => {
-        const issueData = details.issue[issueName];
-        // For each tool with any complaints for the issue:
-        Object.keys(issueData.tools).forEach(toolID => {
-          // Get the sum of the quality-weighted counts of its issue rules.
-          let weightedCount = 0;
-          Object.values(issueData.tools[toolID]).forEach(ruleData => {
-            weightedCount += ruleData.quality * ruleData.complaints.countTotal;
-          });
-          // If the sum exceeds the existing maximum weighted count for the issue:
-          if (weightedCount > issueData.maxCount) {
-            // Change the maximum count for the issue to the sum.
-            issueData.maxCount = weightedCount;
-          }
-        });
-        // Get the score for the issue, including any addition for the instance count limit.
-        const maxAddition = issueData.countLimit ? maxWeight / issueData.countLimit : 0;
-        issueData.score = Math.round(issueData.weight * issueData.maxCount * (1 + maxAddition));
-      });
-      // Add the severity detail totals to the score.
-      details.severity.total = Object.keys(details.severity.byTool).reduce((severityTotals, toolID) => {
-        details.severity.byTool[toolID].forEach((severityScore, index) => {
-          severityTotals[index] += severityScore;
-        });
-        return severityTotals;
-      }, details.severity.total);
-      // Add the summary issue total to the score.
-      summary.issue = Object
-      .values(details.issue)
-      .reduce((total, current) => total + current.score, 0);
-      // Add the summary solo total to the score.
-      Object.keys(details.solo).forEach(tool => {
-        summary.solo += Object
-        .values(details.solo[tool])
-        .reduce((total, current) => total + current);
-      });
-      // Add the summary tool total to the score.
-      summary.tool = toolWeight * details.severity.total.reduce(
-        (total, current, index) => total + severityWeights[index] * current, 0
-      );
-      // Add the summary prevention total to the score.
-      summary.prevention = Object.values(details.prevention).reduce(
-        (total, current) => total + current, 0
-      );
-      // Add the summary log score to the score.
-      const {jobData} = report;
-      if (jobData) {
-        summary.log = Math.max(0, Math.round(
-          logWeights.logCount * jobData.logCount
-          + logWeights.logSize * jobData.logSize +
-          + logWeights.errorLogCount * jobData.errorLogCount
-          + logWeights.errorLogSize * jobData.errorLogSize
-          + logWeights.prohibitedCount * jobData.prohibitedCount +
-          + logWeights.visitRejectionCount * jobData.visitRejectionCount
-        ));
-        // Add the summary latency score to the score.
-        summary.latency = Math.round(
-          latencyWeight * (Math.max(0, jobData.visitLatency - normalLatency))
-        );
-      }
-      // Round the unrounded scores.
-      Object.keys(summary).forEach(summaryTypeName => {
-        summary[summaryTypeName] = Math.round(summary[summaryTypeName]);
-      });
-      details.severity.total.forEach((severityTotal, index) => {
-        details.severity.total[index] = Math.round(severityTotal);
-      });
-      // Add the summary total score to the score.
-      summary.total = summary.issue
-      + summary.solo
-      + summary.tool
-      + summary.prevention
-      + summary.log
-      + summary.latency;
-      // Add the score to the report or replace the existing score of the report.
-      report.score = score;
-    }
-    // Otherwise, i.e. if none of them is a test act:
-    else {
-      // Report this.
-      console.log('ERROR: No test acts');
-    }
-  }
-  // Otherwise, i.e. if there are no acts in the report:
-  else {
-    // Report this.
-    console.log('ERROR: No acts');
-  }
-};

package/scripts/ts31.json

@@ -1,56 +0,0 @@
-{
-  "id": "ts31",
-  "what": "accessibility tests",
-  "strict": true,
-  "timeLimit": 330,
-  "acts": [
-    {
-      "type": "placeholder",
-      "which": "main",
-      "launch": "chromium"
-    },
-    {
-      "type": "test",
-      "which": "alfa"
-    },
-    {
-      "type": "test",
-      "which": "axe",
-      "detailLevel": 2
-    },
-    {
-      "type": "test",
-      "which": "continuum"
-    },
-    {
-      "type": "test",
-      "which": "htmlcs"
-    },
-    {
-      "type": "test",
-      "which": "ibm",
-      "withItems": true,
-      "withNewContent": false
-    },
-    {
-      "type": "test",
-      "which": "nuVal"
-    },
-    {
-      "type": "test",
-      "which": "qualWeb",
-      "withNewContent": false
-    },
-    {
-      "type": "test",
-      "which": "testaro",
-      "withItems": true,
-      "stopOnFail": false
-    },
-    {
-      "type": "test",
-      "which": "wave",
-      "reportType": 4
-    }
-  ]
-}

package/scripts/ts33.json

@@ -1,56 +0,0 @@
-{
-  "id": "ts33",
-  "what": "comprehensive accessibility tests",
-  "strict": true,
-  "timeLimit": 330,
-  "acts": [
-    {
-      "type": "placeholder",
-      "which": "main",
-      "launch": "chromium"
-    },
-    {
-      "type": "test",
-      "which": "alfa"
-    },
-    {
-      "type": "test",
-      "which": "axe",
-      "detailLevel": 2
-    },
-    {
-      "type": "test",
-      "which": "continuum"
-    },
-    {
-      "type": "test",
-      "which": "htmlcs"
-    },
-    {
-      "type": "test",
-      "which": "ibm",
-      "withItems": true,
-      "withNewContent": false
-    },
-    {
-      "type": "test",
-      "which": "nuVal"
-    },
-    {
-      "type": "test",
-      "which": "qualWeb",
-      "withNewContent": false
-    },
-    {
-      "type": "test",
-      "which": "testaro",
-      "withItems": true,
-      "stopOnFail": false
-    },
-    {
-      "type": "test",
-      "which": "wave",
-      "reportType": 4
-    }
-  ]
-}

package/scripts/ts34.json

@@ -1,77 +0,0 @@
-{
-  "id": "ts34",
-  "what": "comprehensive accessibility tests",
-  "strict": true,
-  "timeLimit": 330,
-  "acts": [
-    {
-      "type": "placeholder",
-      "which": "main",
-      "launch": "chromium"
-    },
-    {
-      "type": "test",
-      "which": "alfa"
-    },
-    {
-      "type": "test",
-      "which": "axe",
-      "detailLevel": 2
-    },
-    {
-      "type": "test",
-      "which": "continuum"
-    },
-    {
-      "type": "test",
-      "which": "htmlcs"
-    },
-    {
-      "type": "test",
-      "which": "ibm",
-      "withItems": true,
-      "withNewContent": false
-    },
-    {
-      "type": "test",
-      "which": "nuVal"
-    },
-    {
-      "type": "test",
-      "which": "qualWeb",
-      "withNewContent": false
-    },
-    {
-      "type": "test",
-      "which": "wave",
-      "reportType": 4
-    },
-    {
-      "type": "test",
-      "which": "testaro",
-      "withItems": true,
-      "stopOnFail": false,
-      "rules": [
-        "n",
-        "hover",
-        "motion"
-      ]
-    },
-    {
-      "type": "placeholder",
-      "which": "main",
-      "launch": "webkit"
-    },
-    {
-      "type": "test",
-      "which": "testaro",
-      "withItems": true,
-      "stopOnFail": false,
-      "rules": [
-        "y",
-        "hover",
-        "motion"
-      ]
-    }
-  ]
-}