testilo 21.4.0 → 21.4.2

package/README.md

@@ -137,9 +137,13 @@ Here is a script:
 ```javaScript
 {
   id: 'ts99',
-  what: 'Axe and QualWeb on account page',
+  what: 'aside mislocation',
   strict: true,
+  isolate: true,
   timeLimit: 60,
+  standard: 'also',
+  observe: false,
+  timeStamp: '240115T1200',
   acts: [
     {
       type: 'placeholder',
@@ -149,27 +153,35 @@ Here is a script:
     {
       type: 'test',
       which: 'axe',
-      detailLevel: 1,
-      rules: [],
-      what: 'Axe core, all rules'
+      detailLevel: 2,
+      rules: ['landmark-complementary-is-top-level'],
+      what: 'Axe'
     },
     {
       type: 'test',
       which: 'qualWeb',
       withNewContent: false,
-      what: 'QualWeb, all rules'
+      rules: ['QW-BP25', 'QW-BP26']
+      what: 'QualWeb'
     }
   ]
 }
 ```
 
-This script has 3 acts. The first is a placeholder act. The above batch can be merged with this script to create jobs. In that case, the first job would launch a Chromium browser, navigate to the Acme login page, complete and submit the login form, wait for the account page to load, run the Axe tests, and then run the QualWeb tests. If the batch contained additional targets, additional jobs would be created, with the login actions for each target specified in the `private` array of the `acts` object of that target.
+A script has several properties that specify facts about the jobs to be created. They include:
+- `isolate`: You decide whether to isolate test acts, as needed, from effects of previous test acts. If `true`, the `merge` module will add a copy of the latest placeholder after each target-modifying test act before the immediately following test act.
+- `standardize`: You choose how the reports of test acts should be standardized. The alternatives are `'no'` (do not standardize), `'also'` (standardize and report both the original and the standardized results), and `'only'` (standardize and report only the standardized results).
+- `observe`: You decide how granularly Testaro will allow a server to observe job progress. If `true`, Testaro sends a message to the server to announce each test act (identifying the tool), and the `testaro` tool sends a message to the server to announce each rule when its test is performed. The server can further update the requesting client on the basis of these messages.
+- `timeStamp`: You can specify the date and time that the job is to wait for before it is performed. This specification is a string with the format `yymmddThhMM`.
+- `acts`: an array of acts.
+
+The first act in this script is a placeholder, whose `which` property is `'private'`. If the above batch were merged with this script, in each job the placeholder would be replaced with the `private` acts of a target. For example, the first act of the first job would launch a Chromium browser, navigate to the Acme login page, complete and submit the login form, wait for the account page to load, run the Axe tests, and then run the QualWeb tests. If the batch contained additional targets, additional jobs would be created, with the login actions for each target specified in the `private` array of the `acts` object of that target.
 
-As shown in this example, when a browser is launched by placeholder substitution, the script can determine the browser type (`chromium`, `firefox`, or `webkit`) by assigning a value to a `launch` property of the placeholder. That is useful, because sometimes it is the actions specified in a script that dictate which browser type is appropriate.
+As shown in this example, when a browser is launched by placeholder substitution, the script can determine the browser type (`chromium`, `firefox`, or `webkit`) by assigning a value to a `launch` property of the placeholder.
 
 ### Target list to batch
 
-If you have a target list, the `batch` module of Testilo can convert it to a batch. The batch will contain, for each target, one array of acts named `main`, containing a `launch` act (depending on the script to specify the browser type) and a `url` act.
+If you have a target list, the `batch` module of Testilo can convert it to a batch. The batch will contain, for each target, one array of acts named `main`, containing a `launch` act (depending on the script to specify the browser type and the target to specify the URL).
 
 #### Invocation
 
@@ -203,11 +215,13 @@ The `call` module will save the batch as a JSON file in the `batches` subdirecto
 
 ### Issues to script
 
-Testilo classifies issues. The built-in issue classifications are located in the `procs/score` directory, in files whose names begin with `tic` (for “Testilo issue classification”). You can create additional `tic` files with custom issue classifications.
+Testilo classifies tool rules into _issues_. The built-in classifications are located in the `procs/score` directory, in files whose names begin with `tic` (for “Testilo issue classification”). You can create additional `tic` files with custom classifications.
+
+For example, one of the issues in the `tic38.js` file is `mainNot1`. Four rules are classified as belonging to that issue: rule `main_element_only_one` of `aslint` and 3 more rules defined by 3 other tools.
 
-If you want Testaro to test targets for particular issues, you can name those issues and use the Testilo `script` module to create a script.
+If you want Testaro to test targets for only particular issues, you can name those issues and use the Testilo `script` module to create a script for that purpose. The only tools called by the script will be tools that define rules that are classified as belonging to one or more issues named.
 
-If you want Testaro to test targets for **all** the rules of all the available tools, without regard to any issue classification, you can use the `script` module to create a script that does not impose any issue restrictions.
+If you want Testaro to test targets for **all** the rules of all the available tools, you can use the `script` module to create a script that does not impose any issue restrictions.
 
 #### Invocation
 
@@ -223,13 +237,13 @@ const scriptObj = script(scriptID, issues, issueID0, issueID1, …);
 ```
 
 This invocation references `scriptID`, `issues`, and `issueID` variables.
-- The `scriptID` variable is an alphanumeric string.
-- The `issues` variable is an object that classifies issues, such as the `issues` object in a `tic` file.
-- The `issueID` variables are strings, such as `'regionNoText'`, that name of properties of the `issues` object.
+- The `scriptID` variable is an arbitrary alphanumeric string.
+- The `issues` variable (if present) is an object that classifies issues, such as the `issues` object in a `tic` file.
+- The `issueID` variables (if any) are strings, such as `'regionNoText'`, that name properties of the `issues` object.
 
-The `script()` function of the `script` module generates a script and returns it as an object. The invoking module can further dispose of the script as needed.
+The `script()` function of the `script` module generates a script and returns it as an object. The invoking module can further modify and use the script as needed.
 
-To create a script without issue restrictions, a module can use this invocation:
+To create a script **without** issue restrictions, a module can use this invocation:
 
 ```javaScript
 const {script} = require('testilo/script');
@@ -241,11 +255,11 @@ const scriptObj = script(scriptID);
 A user can invoke `script` in this way: In the Testilo project directory, execute the statement `node call script s c i0 i1 i2 i3 …`.
 
 In this statement:
-- Replace `s` with an ID for the script, such as `headings`.
+- Replace `s` with an arbitrary ID for the script, such as `headings`.
 - Replace `c` with the base name, such as `tic99`, of an issue classification file in the `score` subdirectory of the `process.env.FUNCTIONDIR` directory.
 - Replace the remaining arguments (`i0` etc.) with issue IDs from that classification file.
 
-The `call` module will retrieve the named classification from its directory.
+The `call` module will retrieve the named classification.
 The `script` module will create a script.
 The `call` module will save the script as a JSON file in the `scripts` subdirectory of the `process.env.SPECDIR` directory.
 
@@ -259,7 +273,7 @@ When the `script` module creates a script for you, it does not ask you for all o
 
 Testilo merges batches with scripts, producing jobs, by means of the `merge` module.
 
-The `merge` module needs to be given a batch and a script. In addition, `merge` offers an isolation option. If you exercise it, the `merge` module will act as if the latest placeholder were **again** inserted after each target-modifying test act, except where that test act is the last act or where the next act after it is a placeholder.
+The `merge` module needs to be given a batch and a script.
 
 #### Output
 
@@ -362,7 +376,7 @@ A module can invoke `merge` in this way:
 
 ```javaScript
 const {merge} = require('testilo/merge');
-const jobs = merge(script, batch, requester, true, 'only', false);
+const jobs = merge(script, batch, requester, true, 'only', false, '240115T1200');
 ```
 
 This invocation references `script`, `batch`, and `requester` variables that the module must have already defined. The `script` and `batch` variables are a script object and a batch object, respectively. The `requester` variable is an email address. The fourth argument is a boolean, specifying whether to perform test isolation. The fifth argument is a string that specifies the Testaro standardization option ('also', 'only', or 'no'). The sixth argument is a boolean, specifying whether Testaro will allow granular network watching of the job. The `merge()` function of the `merge` module generates jobs and returns them in an array. The invoking module can further dispose of the jobs as needed.

package/batch.js

@@ -1,10 +1,6 @@
 /*
   batch.js
   Converts a target list to a batch.
-  Arguments:
-    0. batch ID
-    1. batch description
-    2. target list
 */
 
 // ########## FUNCTIONS

package/merge.js

@@ -8,6 +8,7 @@
     3. whether to provide test isolation
     4. value of the standard property
     5. whether reporting is to be granular
+    6. date and time as a compact timestamp for job execution, if not now
 */
 
 // ########## IMPORTS
@@ -24,27 +25,75 @@ const contaminantNames = new Set([
   'aslint',
   'axe',
   'htmlcs',
-  'ibm',
   'testaro'
 ]);
+const randomIDChars = (() => {
+  const digits = Array(10).fill('').map((digit, index) => index.toString());
+  const uppers = Array(26).fill('').map((letter, index) => String.fromCodePoint(65 + index));
+  const lowers = Array(26).fill('').map((letter, index) => String.fromCodePoint(97 + index));
+  return digits.concat(uppers, lowers);
+})();
+  
 
 // ########## FUNCTIONS
 
+// Inserts a character periodically in a string.
+const punctuate = (string, insertion, chunkSize) => {
+  const segments = [];
+  let startIndex = 0;
+  while (startIndex < string.length) {
+    segments.push(string.slice(startIndex, startIndex + chunkSize));
+    startIndex += chunkSize;
+  }
+  return segments.join(insertion);
+};
+// Converts a compact timestamp to a date.
+const dateOf = timeStamp => {
+  if (/^\d{6}T\d{4}$/.test(timeStamp)) {
+    const dateString = punctuate(timeStamp.slice(0, 6), '-', 2);
+    const timeString = punctuate(timeStamp.slice(7, 11), ':', 2);
+    return new Date(`20${dateString}T${timeString}Z`);
+  } else {
+    return null;
+  }
+};
+// Converts a date and time to a compact timestamp.
+const stampTime = date => date.toISOString().replace(/[-:]/g, '').slice(2, 13);
+// Generates a random string.
+const getRandomID = length => {
+  const chars = [];
+  for (let i = 0; i < length; i++) {
+    chars.push(randomIDChars[Math.floor(62 * Math.random())]);
+  }
+  return chars.join('');
+};
 // Merges a script and a batch and returns jobs.
-exports.merge = (script, batch, requester, isolate, standard, isGranular) => {
+exports.merge = (script, batch, requester, isolate, standard, isGranular, timeStamp) => {
   if (isolate === 'false') {
     isolate = false;
   }
-  // If the requester is unspecified, make it the standard requester.
+  // If a timestamp was specified:
+  if (timeStamp) {
+    // If it is invalid:
+    if (! dateOf(timeStamp)) {
+      // Report this and quit.
+      console.log(`ERROR: Timestamp invalid`);
+      return [];
+    }
+  }
+  // Otherwise, i.e. if no timestamp was specified:
+  else {
+    // Create one for the job.
+    timeStamp = stampTime(new Date());
+  }
+  // If the requester is blank or unspecified, make it the standard requester.
   requester ||= stdRequester;
-  // Create a timestamp.
-  const timeStamp = new Date().toISOString().replace(/[-:]/g, '').slice(2, 15);
-  // Create a time description.
+  // Create a creation-time description.
   const creationTime = (new Date()).toISOString().slice(0, 19);
   // Initialize a target-independent job.
   const protoJob = JSON.parse(JSON.stringify(script));
-  // Add the timestamp to the ID of the job.
-  protoJob.id = `${timeStamp}-${protoJob.id}-`;
+  // Make the timestamp and a random string the ID of the job.
+  protoJob.id = `${timeStamp}-${getRandomID(Number.parseInt(process.env.RANDOM_ID_LENGTH, 10))}`;
   // Add a sources property to the job.
   protoJob.sources = {
     script: script.id,
@@ -64,7 +113,7 @@ exports.merge = (script, batch, requester, isolate, standard, isGranular) => {
   protoJob.observe = isGranular || false;
   // If isolation was requested:
   if (isolate) {
-    // Perform it.
+    // Configure the job for it.
     let {acts} = protoJob;
     let lastPlaceholder = {};
     for (const actIndexString in acts) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testilo",
-  "version": "21.4.0",
+  "version": "21.4.2",
   "description": "Prepares and processes Testaro reports",
   "main": "aim.js",
   "scripts": {

package/procs/digest/tdp38/index.html

@@ -26,13 +26,15 @@
           <tr><th>Tested by</th><td>Testaro, procedure <code>__ts__</code></td></tr>
           <tr><th>Scored by</th><td>Testilo, procedure <code>__sp__</code></td></tr>
           <tr><th>Digested by</th><td>Testilo, procedure <code>__dp__</code></td></tr>
-          <tr><th>Full report</th><td><a href="__reportURL__"><code>__reportURL__</code></a></td></tr>
+          <tr>
+            <th>Full report</th><td><a href="__reportURL__"><code>__reportURL__</code></a></td>
+          </tr>
         </table>
       </header>
       <h2>Introduction</h2>
       <p>How <a href="https://www.w3.org/WAI/">accessible</a> is the __org__ web page at <a href="__url__"><code>__url__</code></a>?</p>
       <p>This digest can help answer that question. Nine different tools (Alfa, ASLint, Axe, Equal Access, HTML CodeSniffer, Nu Html Checker, QualWeb, Testaro, and WAVE) tested the page to check its compliance with their accessibility rules. In all, the tools define about 915 rules, which are classified here into about 300 accessibility issues.</p>
-      <p>The results were interpreted to yield a score, with 0 being ideal. The score for this page was __total__, the sum of __issue__ for issues, __tool__ for tool-by-tool ratings, __prevention__ for the page preventing tools from running, __log__ for browser warnings, and __latency__ for delayed page responses.</p>
+      <p>The results were interpreted to yield a score, with 0 being ideal. The score for this page was __total__, the sum of __issueCount__ for the count of issues, __issue__ for specific issues, __tool__ for tool-by-tool ratings, __prevention__ for the page preventing tools from running, __log__ for browser warnings, and __latency__ for delayed page responses.</p>
       <h2>Issue summary</h2>
       <table class="allBorder thirdCellRight">
         <caption>Summary of issues</caption>

package/procs/digest/tdp38/index.js

@@ -43,7 +43,7 @@ const populateQuery = (report, query) => {
     summaryRows: [],
     issueRows: []
   };
-  ['total', 'issue', 'tool', 'prevention', 'log', 'latency'].forEach(sumItem => {
+  ['total', 'issueCount', 'issue', 'tool', 'prevention', 'log', 'latency'].forEach(sumItem => {
     query[sumItem] = summary[sumItem];
     rows.summaryRows.push(getScoreRow(sumItem, query[sumItem]));
   });

package/procs/score/tic38.js

@@ -163,7 +163,7 @@ exports.issues = {
         'QW-WCAG-T20': {
           variable: false,
           quality: 0,
-          what: 'Link text is not supplemented with a title attribute [invalid]'
+          what: 'Link title may fail to describe the link correctly [speculative]'
         },
         'QW-WCAG-T23': {
           variable: false,

package/procs/score/tsp38.js

@@ -22,7 +22,7 @@ const preventionWeight = 300;
 // Maximum instance count addition weight (divisor of max).
 const maxWeight = 30;
 // Issue count weight.
-const issueCountWeight = 15;
+const issueCountWeight = 10;
 // Other weights.
 const severityWeights = [1, 2, 3, 4];
 const toolWeight = 0.1;

package/script.js

@@ -1,9 +1,6 @@
 /*
   script.js
-  Creates and returns a script.
-  Arguments:
-    0. issue classification
-    1. issue IDs
+  Creates and returns a script to perform the tests for issues.
 */
 
 // ########## IMPORTS
@@ -13,6 +10,7 @@ require('dotenv').config();
 
 // ########## VARIABLES
 
+// List of presumptively needed tools.
 let toolIDs = [
   'alfa', 'aslint', 'axe', 'htmlcs', 'ibm', 'nuVal', 'qualWeb', 'testaro', 'wave'
 ];
@@ -21,7 +19,7 @@ let toolIDs = [
 
 // Creates and returns a script.
 exports.script = (id, issues = null, ... issueIDs) => {
-  // Initialize data on the tools and rules for the specified issues.
+  // Initialize data on the tools and their rules for the specified issues, if any.
   const neededTools = {};
   // If an issue classification and any issues were specified:
   if (issues && issueIDs.length) {
@@ -53,42 +51,47 @@ exports.script = (id, issues = null, ... issueIDs) => {
             }
           });
         });
+        // Remove unneeded tools from the tool list.
         toolIDs = Object.keys(neededTools);
       }
       // Otherwise, i.e. if it does not exist in the classification:
       else {
-        // Report this.
+        // Report this and quit.
         console.log(`ERROR: Issue ${issueID} not in issue classification`);
         return {};
       }
     });
   }
-  // If any rules have been identified:
+  // If, after any issue-based pruning, any needed tools remain:
   if (toolIDs.length) {
     // Initialize a script.
     const scriptObj = {
       id,
       what: `accessibility tests`,
       strict: true,
+      isolate: true,
       timeLimit: 30 + (10 * issueIDs.length || 30 * toolIDs.length),
+      standard: 'only',
+      observe: true,
+      timeStamp: '',
       acts: [
         {
           "type": "placeholder",
           "which": "main",
-          "launch": "chromium"
+          "launch": "webkit"
         }
       ]
     };
-    // For each identified tool:
+    // For each needed tool:
     toolIDs.forEach(toolID => {
       // Initialize a test act for it.
       const toolAct = {
         type: 'test',
         which: toolID
       };
-      // If rules were specified:
+      // If issues were specified:
       if (issues && issueIDs.length) {
-        // Add a rules property to the act.
+        // Add a rules array as a property to the act.
         toolAct.rules = neededTools[toolID];
         // If the tool is Testaro:
         if (toolID === 'testaro') {
@@ -96,13 +99,13 @@ exports.script = (id, issues = null, ... issueIDs) => {
           toolAct.rules.unshift('y');
         }
       }
-      // Add option specifications if necessary.
+      // Add any needed option specifications to the act.
       if (toolID === 'axe') {
         toolAct.detailLevel = 2;
       }
       else if (toolID === 'ibm') {
         toolAct.withItems = true;
-        toolAct.withNewContent = false;
+        toolAct.withNewContent = true;
       }
       else if (toolID === 'qualWeb') {
         toolAct.withNewContent = false;

package/scripts/ts37.json

@@ -0,0 +1,60 @@
+{
+  "id": "ts37",
+  "what": "comprehensive accessibility tests with webkit and redirection permitted",
+  "strict": false,
+  "isolate": true,
+  "timeLimit": 600,
+  "standard": "only",
+  "observe": true,
+  "timeStamp": "",
+  "acts": [
+    {
+      "type": "placeholder",
+      "which": "main",
+      "launch": "webkit"
+    },
+    {
+      "type": "test",
+      "which": "alfa"
+    },
+    {
+      "type": "test",
+      "which": "aslint"
+    },
+    {
+      "type": "test",
+      "which": "axe",
+      "detailLevel": 2
+    },
+    {
+      "type": "test",
+      "which": "htmlcs"
+    },
+    {
+      "type": "test",
+      "which": "ibm",
+      "withItems": true,
+      "withNewContent": true
+    },
+    {
+      "type": "test",
+      "which": "nuVal"
+    },
+    {
+      "type": "test",
+      "which": "qualWeb",
+      "withNewContent": false
+    },
+    {
+      "type": "test",
+      "which": "testaro",
+      "withItems": true,
+      "stopOnFail": false
+    },
+    {
+      "type": "test",
+      "which": "wave",
+      "reportType": 4
+    }
+  ]
+}