testilo 31.3.1 → 32.0.0

package/README.md

@@ -230,13 +230,52 @@ The `call` module will retrieve the named target list.
 The `batch` module will convert the target list to a batch.
 The `call` module will save the batch as a JSON file named `x.json` (replacing `x` with the list ID) in the `batches` subdirectory of the `SPECDIR` directory.
 
-### Issues to script
+### Script creation
 
 You can use the `script()` function of the `script` module to simplify the creation of scripts.
 
-In its simplest form, `script()` requires only one argument, a string that will serve as the ID of the script. Called in this way, `script()` produces a script that tells Testaro to perform all of the tests defined by all of the tools integrated by Testaro.
+#### Without options
 
-If you want a more focused script, you can add additional arguments to `script()`. First you must add the ID of a Testilo issue classification (such as `tic99`). After that, you must add one or more arguments giving the IDs of issues in that classification. The latest Testilo issue classification classifies about a thousand rules of the 10 Testaro tools into about 300 _issues_. The classification is found in a file whose name begins with `tic` in the `procs/score` directory. For example, one issue in the `tic40.js` file is `mainNot1`. Four rules are classified as belonging to that issue: rule `main_element_only_one` of the `aslint` tool and 3 more rules defined by 3 other tools. You can also create custom classifications and save them in a `score` subdirectory of the `FUNCTIONDIR` directory.
+In its simplest form, `script()` requires two string arguments:
+- An ID for the script
+- A description of the script
+
+Called in this way, `script()` produces a script that tells Testaro to perform the tests for all of the evaluation rules defined by all of the tools integrated by Testaro.
+
+#### With options
+
+If you want a more focused script, you can add an additional option argument to `script()`. The option argument lets you restrict the rules to be tested for. You may choose between restrictions of two types:
+- Tools
+- Issues
+
+The option argument is an object. Its properties depend on the restriction type.
+
+For a tool restriction, it has this structure:
+
+```javascript
+{
+  type: 'tools',
+  specs: ['toolID0', 'toolID1', 'toolIDn']
+}
+```
+
+For an issue restriction, it has this structure:
+
+```javascript
+{
+  type: 'issues',
+  specs: {
+    issues: issueClassification,
+    issueIDs: ['issue0', 'issue1', 'issueN']
+  }
+}
+```
+
+If you specify tool options, the script will prescribe the tests for all evaluation rules of the tools that you specify.
+
+If you specify issue options, the script will prescribe the tests for all evaluation rules that are classified in the issues whose IDs you specify. Any tools that do not have any of those rules will be omitted. The value of `specs/issues` is an issue classification object, with a structure like the one in `procs/score/tic40.js`. That one classifies about 1000 rules into about 300 issues.
+
+For example, one issue in the `tic40.js` file is `mainNot1`. Four rules are classified as belonging to that issue: rule `main_element_only_one` of the `aslint` tool and 3 more rules defined by 3 other tools. You can also create custom classifications and save them in a `score` subdirectory of the `FUNCTIONDIR` directory.
 
 #### Invocation
 
@@ -244,34 +283,40 @@ There are two ways to use the `script` module.
 
 ##### By a module
 
-A module can invoke `script()` in this way:
+A module can invoke `script()` in one of these ways:
+
+```javaScript
+const {script} = require('testilo/script');
+const scriptObj = script('monthly', 'landmarks');
+```
 
 ```javaScript
 const {script} = require('testilo/script');
-const {issues} = require('testilo/procs/score/tic99');
-const scriptObj = script('monthly', 'landmarks', issues, 'regionNoText', 'mainNot1');
+const options = {…};
+const scriptObj = script('monthly', 'landmarks', options);
 ```
 
-In this example, the script will have `'monthly'` as its ID and `'landmarks'` as its description. It will tell Testaro to test for all, and only, the rules that are classified into either the `regionNoText` or the `mainNot1` issue.
+In this example, the script will have `'monthly'` as its ID and `'landmarks'` as its description. It will tell Testaro to test for all evaluation rules if the first form is used, or for all rules specified by the options argument if the second form is used.
 
 The invoking module can further modify and use the script (`scriptObj`) as needed.
 
-To create a script **without** issue restrictions, a module can call `script()` with only the first two arguments (the ID and the description).
-
 ##### By a user
 
 A user can invoke `script()` by executing one of these statements in the Testilo project directory:
 
 ```javascript
-node call script id what ticnn issuea issueb …
 node call script id what
+node call script id what tools toolID0 toolID1 toolID2 …
+node call script id what issues tic99 issueID0 issueID1 …
 ```
 
-In this statement, replace `id` with an ID for the script, such as `headings1`, consisting of ASCII alphanumeric characters.
+The first form will create a script with no restrictions.
 
-If specifying issues:
-- Replace `ticnn` with the base, such as `tic99`, of the name of an issue classification file in the `score` subdirectory of the `FUNCTIONDIR` directory.
-- Replace the remaining arguments (`issuea` etc.) with issue names from that classification file.
+The second form will create a script that prescribes tests for all the rules of the specified tools.
+
+The third form will create a script that prescribes tests for all the rules classified by the specified issue classification into any of the specified issues.
+
+In this statement, replace `id` with an ID for the script, such as `headings1`, consisting of ASCII alphanumeric characters.
 
 The `call` module will retrieve the named classification, if any.
 The `script` module will create a script.
@@ -281,7 +326,7 @@ The `call` module will save the script as a JSON file in the `scripts` subdirect
 
 The `script` module will use the value of the `SEND_REPORT_TO` environment variable as the value of the `sendReportTo` property of the script, if that variable exists, and otherwise will leave that property with an empty-string value.
 
-When the `script` module creates a script for you, it does not ask you for all of the options that the script may require. Instead, it chooses default options. For example, it sets the values of `isolate` and `strict` to `true`. After you invoke `script`, you can edit the script that it creates to revise options.
+When the `script` module creates a script for you, it does not ask you for all of the other options that the script may require. Instead, it chooses default options. For example, it sets the values of `isolate` and `strict` to `true`. After you invoke `script`, you can edit the script that it creates to revise options.
 
 ### Merge
 

package/call.js

@@ -23,7 +23,7 @@ const fs = require('fs/promises');
 // Function to process a list-to-batch conversion.
 const {batch} = require('./batch');
 // Function to create a script from rule specifications.
-const {script} = require('./script');
+const {script, toolIDs} = require('./script');
 // Function to process a merger.
 const {merge} = require('./merge');
 // Function to score reports.
@@ -96,27 +96,77 @@ const callBatch = async (id, what) => {
   }
 };
 // Fulfills a script-creation request.
-const callScript = async (scriptID, what, classificationID = null, ... issueIDs) => {
-  try {
-    // Get any issue classification.
-    const issues = classificationID
-    ? require(`${functionDir}/score/${classificationID}`).issues
-    : null;
-    // Sanitize the ID.
-    scriptID = scriptID.replace(/[^a-zA-Z0-9]/g, '');
-    if (scriptID === '') {
-      scriptID = `script-${getRandomString(2)}`;
+const callScript = async (scriptID, what, optionType, ... optionDetails) => {
+  // Sanitize the script ID.
+  scriptID = scriptID.replace(/[^a-zA-Z0-9]/g, '');
+  if (scriptID === '') {
+    scriptID = `script-${getRandomString(2)}`;
+  }
+  // Create the option argument.
+  const optionArg = {};
+  if (optionType) {
+    if (optionType === 'tools') {
+      if (
+        optionDetails.length === new Set(optionDetails).size
+        && optionDetails.every(toolID => toolIDs.includes(toolID))
+      ) {
+        optionArg.type = 'tools';
+        optionArg.specs = optionDetails;
+      }
+      else {
+        console.log('ERROR: Tool IDs invalid');
+        return;
+      }
     }
-    // Create a script.
-    const scriptObj = script(scriptID, what, issues, ... issueIDs);
-    // Save the script.
+    else if (optionType === 'issues') {
+      if (optionDetails.length > 1) {
+        if (optionDetails[0].startsWith('tic')) {
+          try {
+            const {issues} = require(`${functionDir}/score/${optionDetails[0]}`);
+            const issueIDs = Object.keys(issues);
+            if (optionDetails.slice(1).every(issueID => issueIDs.includes(issueID))) {
+              optionArg.type = 'issues';
+              optionArg.specs = {
+                issues,
+                issueIDs: optionDetails.slice(1)
+              };
+            }
+            else {
+              console.log('ERROR: Issue IDs invalid');
+              return;
+            }
+          }
+          catch(error) {
+            console.log(`ERROR getting issue classification (${error.message})`);
+            return;
+          }
+        }
+        else {
+          console.log('ERROR: Issue classification ID invalid');
+          return;
+        }
+      }
+      else {
+        console.log('ERROR: No issue IDs specified');
+        return;
+      }
+    }
+    else {
+      console.log('ERROR: Option type invalid');
+      return;
+    }
+  }
+  // Create a script.
+  const scriptObj = script(scriptID, what, optionArg);
+  try {
+    // Save it.
     const scriptJSON = JSON.stringify(scriptObj, null, 2);
     const scriptPath = `${specDir}/scripts/${scriptID}.json`;
     await fs.writeFile(scriptPath, `${scriptJSON}\n`);
     console.log(`Script created and saved as ${scriptPath}`);
   }
   catch(error) {
-    console.log(`ERROR creating script (${error.message})`);
+    console.log(`ERROR saving script (${error.message})`);
   }
 };
 // Fulfills a merging request.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testilo",
-  "version": "31.3.1",
+  "version": "32.0.0",
   "description": "Prepares and processes Testaro reports",
   "main": "call.js",
   "scripts": {

package/procs/score/tic40.js

@@ -289,7 +289,7 @@ exports.issues = {
       }
     }
   },
-  IDUnique: {
+  duplicateID: {
     summary: 'ID not unique',
     why: 'User may be pointed to the wrong item',
     wcag: '4.1.1',

package/script.js

@@ -3,145 +3,164 @@
   Creates and returns a script to perform the tests for issues.
 */
 
-// ########## IMPORTS
-
-// Module to keep secrets.
-require('dotenv').config();
-
 // ########## VARIABLES
 
-// List of presumptively needed tools.
-let toolIDs = [
+// Testaro tool IDs.
+const toolIDs = exports.toolIDs = [
   'alfa', 'aslint', 'axe', 'ed11y', 'htmlcs', 'ibm', 'nuVal', 'qualWeb', 'testaro', 'wave'
 ];
 
 // ########## FUNCTIONS
 
 // Creates and returns a script.
-exports.script = (id, what, issues = null, ... issueIDs) => {
-  // Initialize data on the tools and their rules for the specified issues, if any.
-  const neededTools = {};
-  // If an issue classification and any issues were specified:
-  if (issues && issueIDs.length) {
-    // For each specified issue:
-    issueIDs.forEach(issueID => {
-      // If it exists in the classification:
-      const issueData = issues[issueID];
-      if (issueData) {
-        // For each tool that tests for the issue:
-        const issueToolIDs = Object.keys(issueData.tools);
-        issueToolIDs.forEach(issueToolID => {
-          // For each of the rules of the tool for the issue:
-          if (! neededTools[issueToolID]) {
-            neededTools[issueToolID] = [];
-          }
-          Object.keys(issueData.tools[issueToolID]).forEach(ruleID => {
-            // Add data on the rule.
-            const ruleData = issueData.tools[issueToolID][ruleID];
-            if (issueToolID === 'nuVal') {
-              if (ruleData.variable) {
-                neededTools[issueToolID].push(`~${ruleID}`);
+exports.script = (id, what, options = {}) => {
+  const toolsRulesData = {};
+  // If options are specified:
+  if (options.type && options.specs) {
+    const {type, specs} = options;
+    // If the option type is tools and is valid:
+    if (
+      type === 'tools'
+      && Array.isArray(specs)
+      && specs.length
+      && specs.every(spec => toolIDs.includes(spec))
+    ) {
+      // Populate the data on tools and rules.
+      specs.forEach(spec => {
+        toolsRulesData[spec] = [];
+      });
+    }
+    // Otherwise, if the option type is issues and is valid:
+    else if (
+      type === 'issues'
+      && typeof specs === 'object'
+      && specs.issues
+      && specs.issueIDs
+      && typeof specs.issues === 'object'
+      && Array.isArray(specs.issueIDs)
+      && specs.issueIDs.length
+    ) {
+      // For each specified issue:
+      const {issueIDs, issues} = specs;
+      issueIDs.forEach(issueID => {
+        // If it exists in the classification:
+        const issueData = issues[issueID];
+        if (issueData) {
+          // For each tool that tests for the issue:
+          const issueToolIDs = Object.keys(issueData.tools);
+          issueToolIDs.forEach(issueToolID => {
+            // For each of the rules of the tool for the issue:
+            toolsRulesData[issueToolID] ??= [];
+            const toolRuleIDs = toolsRulesData[issueToolID];
+            const toolData = issueData.tools[issueToolID];
+            Object.keys(toolData).forEach(ruleID => {
+              // Add the rule to the data on tools and rules.
+              let rulePrefix = '';
+              if (issueToolID === 'nuVal') {
+                rulePrefix = toolData[ruleID].variable ? '~' : '=';
               }
-              else {
-                neededTools[issueToolID].push(`=${ruleID}`);
+              const fullRuleID = `${rulePrefix}${ruleID}`;
+              if (! toolRuleIDs.includes(fullRuleID)) {
+                toolRuleIDs.push(fullRuleID);
               }
-            }
-            else {
-              neededTools[issueToolID].push(ruleID);
-            }
+            });
           });
-        });
-        // Remove unneeded tools from the tool list.
-        toolIDs = Object.keys(neededTools);
-      }
-      // Otherwise, i.e. if it does not exist in the classification:
-      else {
-        // Report this and quit.
-        console.log(`ERROR: Issue ${issueID} not in issue classification`);
-        return {};
-      }
-    });
-  }
-  // If, after any issue-based pruning, any needed tools remain:
-  if (toolIDs.length) {
-    // Initialize a script.
-    const scriptObj = {
-      id,
-      what,
-      strict: true,
-      isolate: true,
-      timeLimit: Math.round(30 + (issueIDs.length || 300) / 2 + 20 * toolIDs.length),
-      acts: [
-        {
-          "type": "placeholder",
-          "which": "main",
-          "launch": "webkit"
         }
-      ]
-    };
-    // For each needed tool:
-    toolIDs.forEach(toolID => {
-      // Initialize a test act for it.
-      const toolAct = {
-        type: 'test',
-        which: toolID
-      };
-      // If issues were specified:
-      if (issues && issueIDs.length) {
-        // Add a rules array as a property to the act.
-        toolAct.rules = neededTools[toolID];
-        // If the tool is QualWeb:
-        if (toolID === 'qualWeb') {
-          // For each QualWeb module:
-          const specs = [];
-          const prefixes = {
-            act: 'QW-ACT-R',
-            wcag: 'QW-WCAG-T',
-            best: 'QW-BP'
-          };
-          Object.keys(prefixes).forEach(prefix => {
-            // Specify the rules of that module to be tested for.
-            const ids = toolAct.rules.filter(id => id.startsWith(prefixes[prefix]));
-            const integers = ids.map(id => id.slice(prefixes[prefix].length));
-            specs.push(`${prefix}:${integers.join(',')}`);
-          });
-          // Replace the generic rule list with the QualWeb-format list.
-          toolAct.rules = specs;
+        // Otherwise, i.e. if it does not exist in the classification:
+        else {
+          // Report this and quit.
+          console.log(`ERROR: Issue ${issueID} not in issue classification`);
+          return {};
         }
-        // Otherwise, if the tool is Testaro:
-        else if (toolID === 'testaro') {
-          // Prepend the inclusion option to the rule array.
-          toolAct.rules.unshift('y');
-        }
-      }
-      // Add any needed option defaults to the act.
-      if (toolID === 'axe') {
-        toolAct.detailLevel = 2;
-      }
-      else if (toolID === 'ibm') {
-        toolAct.withItems = true;
-        toolAct.withNewContent = true;
+      });
+    }
+    // Otherwise, i.e. if the option specification is invalid:
+    else {
+      // Report this and quit.
+      console.log(`ERROR: Options invalid`);
+      return {};
+    }
+  }
+  // Otherwise, i.e. if options are not specified:
+  else {
+    // Populate the data on tools and rules.
+    toolIDs.forEach(toolID => {
+      toolsRulesData[toolID] = [];
+    });
+  }
+  // Initialize a script.
+  const timeLimit = Math.round(50 + 30 * Object.keys(toolsRulesData).length);
+  const scriptObj = {
+    id,
+    what,
+    strict: true,
+    isolate: true,
+    timeLimit,
+    acts: [
+      {
+        "type": "placeholder",
+        "which": "main",
+        "launch": "webkit"
       }
-      else if (toolID === 'qualWeb') {
-        toolAct.withNewContent = false;
+    ]
+  };
+  // For each tool used:
+  Object.keys(toolsRulesData).forEach(toolID => {
+    // Initialize a test act for it.
+    const toolAct = {
+      type: 'test',
+      which: toolID
+    };
+    // If rules were specified:
+    const ruleIDs = toolsRulesData[toolID];
+    if (ruleIDs.length) {
+      // Add a rules array as a property to the act.
+      toolAct.rules = ruleIDs;
+      // If the tool is QualWeb:
+      if (toolID === 'qualWeb') {
+        // For each QualWeb module:
+        const specs = [];
+        const prefixes = {
+          act: 'QW-ACT-R',
+          wcag: 'QW-WCAG-T',
+          best: 'QW-BP'
+        };
+        Object.keys(prefixes).forEach(prefix => {
+          // Specify the rules of that module to be tested for.
+          const ids = toolAct.rules.filter(id => id.startsWith(prefixes[prefix]));
+          const integers = ids.map(id => id.slice(prefixes[prefix].length));
+          specs.push(`${prefix}:${integers.join(',')}`);
+        });
+        // Replace the generic rule list with the QualWeb-format list.
+        toolAct.rules = specs;
       }
+      // Otherwise, if the tool is Testaro:
       else if (toolID === 'testaro') {
-        toolAct.withItems = true;
-        toolAct.stopOnFail = false;
-      }
-      else if (toolID === 'wave') {
-        toolAct.reportType = 4;
+        // Prepend the inclusion option to the rule array.
+        toolAct.rules.unshift('y');
       }
-      // Add the act to the script.
-      scriptObj.acts.push(toolAct);
-    });
-    // Return the script.
-    return scriptObj;
-  }
-  // Otherwise, i.e. if no rules have been identified:
-  else {
-    // Report this.
-    console.log(`ERROR: No rules for the specified issues found`);
-    return {};
-  }
-};
+    }
+    // Add any needed option defaults to the act.
+    if (toolID === 'axe') {
+      toolAct.detailLevel = 2;
+    }
+    else if (toolID === 'ibm') {
+      toolAct.withItems = true;
+      toolAct.withNewContent = true;
+    }
+    else if (toolID === 'qualWeb') {
+      toolAct.withNewContent = false;
+    }
+    else if (toolID === 'testaro') {
+      toolAct.withItems = true;
+      toolAct.stopOnFail = false;
+    }
+    else if (toolID === 'wave') {
+      toolAct.reportType = 4;
+    }
+    // Add the act to the script.
+    scriptObj.acts.push(toolAct);
+  });
+  // Return the script.
+  return scriptObj;
+}