testaro 14.4.0 → 14.5.0

package/README.md

@@ -348,9 +348,9 @@ This act causes Testaro to alter the `display` and `visibility` style properties
 
 ###### Introduction
 
-An act of type `test` performs operations and reports a result. The result may indicate that a page passes or fails requirements. Typically, accessibility tests report successes and failures. But a test in Testaro is defined less restrictively, so it can report any result. As one example, the Testaro `elements` test reports facts about certain elements on a page, without asserting that those facts are successes or failures.
+An act of type `test` performs the tests of a tool and reports a result. The result may indicate that a page passes or fails requirements. Typically, accessibility tests report successes and failures. But a test in Testaro is defined less restrictively, so it can report any result. As one example, the Testaro `elements` test reports facts about certain elements on a page, without asserting that those facts are successes or failures.
 
-The `which` property of a `test` act identifies the operations to perform. The value of `which` is the name of one of the tools, such as `alfa`.
+The `which` property of a `test` act identifies a tool, such as `alfa`.
 
 ###### Configuration
 
@@ -360,20 +360,25 @@ Every test in Testaro must have:
 
 The `actSpecs.js` file (described in detail below) contains a specification for any `test` act, namely:
 
-```json
+```javascript
 test: [
   'Perform a test',
   {
     which: [true, 'string', 'isTest', 'test name'],
+    rules: [false, 'array', 'areStrings', 'rule IDs or specifications, if not all']
     what: [false, 'string', 'hasLength', 'comment']
   }
 ],
 ```
 
-That means that a test act (i.e. an act with a `type` property having the value `'test'`) must have a string-valued `which` property naming a tool and may optionally have a string-valued `what` property describing the tool.
+That means that a test act (i.e. an act with a `type` property having the value `'test'`) must have a string-valued `which` property naming a tool and may optionally have an array-valued `rules` property restricting the tests to be reported and/or a string-valued `what` property describing the tool and/or the tests.
 
 If a particular test act either must have or may have any other properties, those properties are specified in the `tests` property in `actSpecs.js`.
 
+When you include a `rules` property, you limit the tests of the tool that are performed or reported. For some tools (`alfa`, `axe`, `continuum`, `htmlcs`, `qualWeb`, and `testaro`), only the specified tests are performed. Other tools (`ibm`, `nuVal`, `tenon`, and `wave`) do not allow such a limitation, so, for those tools, all tests are performed but results are reported from only the specified tests.
+
+The `nuVal` and `testaro` tools require specific formats for the `rules` property. Those formats are described below in the sections about those tools.
+
 ###### Examples
 
 An example of a `test` act is:
@@ -521,6 +526,12 @@ These changes were proposed as pull requests 1333 and 1334 (https://github.com/I
 
 The `ibm` tool is one of two tools (`testaro` is the other) with a `withItems` property. If you set `withItems` to `false`, the result includes the counts of “violations” and “recommendations”, but no information about the rules that gave rise to them.
 
+###### Nu Html Checker
+
+The `nuVal` tool performs the tests of the Nu Html Checker.
+
+Its `rules` argument is **not** an array of rule IDs, but instead is an array of rule _specifications_. A rule specification for `nuVal` is a string with the format `type:substring`, where `type` is replaced with a message type (namely `info` or `error`) and `substring` is replaced with any substring of a message. This `rules` format arises from the fact that `nuVal` generates customized messages and does not accompany them with rule identifiers. Thus, by choosing a type and a substring, you are deciding that any message of that type that includes that substring will be deemed a `nuVal` rule.
+
 ###### QualWeb
 
 The `qualWeb` tool performs the ACT rules, WCAG Techniques, and best-practices tests of QualWeb. Only failures and warnings are included in the report. The EARL report of QualWeb is not generated, because it is equivalent to the report of the ACT rules tests.
@@ -571,6 +582,8 @@ If a `tenon` test act is included in a job, environment variables named `TENON_U
 
 If you do not specify rules when using the `testaro` tool, Testaro will test for the rules listed in the `evalRules` object of the `tests/testaro.js` file.
 
+The `rules` property for `testaro` is an array whose first item is either `'y'` or `'n'` and whose remaining items are rule IDs. If `'y'`, then only the specified rules’ tests are performed. If `'n'`, then all the evaluative tests are performed, except for the specified rules.
+
 It has been found that the `motion` test of the `testaro` tool measures motion only when the `webkit` browser type is in use. If you want to use `testaro` with different browser types for different tests, you can include 2 or 3 `testaro` test acts in your job, specifying different browser types and different rules.
 
 The `testaro` tool (like the `ibm` tool) has a `withItems` property. If you set it to `false`, the `standardResult` object of `testaro` will contain an `instances` property with summaries that identify issues and instance counts. If you set it to `true`, some of the instances will be itemized.

package/actSpecs.js

@@ -122,10 +122,10 @@ exports.actSpecs = {
       }
     ],
     test: [
-      'Perform test of a tool',
+      'Perform tests of a tool',
       {
-        which: [true, 'string', 'isTest', 'test name'],
-        what: [false, 'string', 'hasLength', 'comment']
+        which: [true, 'string', 'isTest', 'tool name'],
+        rules: [false, 'array', 'areStrings', 'rule IDs or specifications, if not all']
       }
     ],
     text: [
@@ -153,34 +153,10 @@ exports.actSpecs = {
     ]
   },
   tests: {
-    alfa: [
-      'Perform alfa tests',
-      {
-        rules: [false, 'array', 'areStrings', 'rule names (e.g., r25), if not all']
-      }
-    ],
     axe: [
       'Perform Axe tests',
       {
-        detailLevel: [true, 'number', '', '0 = least, 4 = most'],
-        rules: [true, 'array', 'areStrings', 'rule names, or empty if all']
-      }
-    ],
-    continuum: [
-      'Perform Continuum tests',
-      {
-        rules: [false, 'array', 'areNumbers', 'rule numbers (e.g., 25), if not all']
-      }
-    ],
-    htmlcs: [
-      'Perform HTML CodeSniffer tests',
-      {
-        rules: [
-          false,
-          'array',
-          'areStrings',
-          'rule names (e.g., Principle1.Guideline1_4.1_4_9), if not all'
-        ]
+        detailLevel: [true, 'number', '', '0 = least, 4 = most']
       }
     ],
     ibm: [
@@ -189,26 +165,13 @@ exports.actSpecs = {
         withItems: [true, 'boolean', '', 'itemize'],
         withNewContent: [
           true, 'boolean', '', 'true: use a URL; false: use page content'
-        ],
-        rules: [false, 'array', 'areStrings', 'rule names (e.g., RPT_Elem_UniqueId), if not all']
-      }
-    ],
-    nuVal: [
-      'Perform Nu Html Checker tests',
-      {
-        messages: [
-          false,
-          'array',
-          'areStrings',
-          'message specifications (type and start of message, e.g., error:Bad value), if not all'
         ]
       }
     ],
     qualWeb: [
       'Perform QualWeb tests',
       {
-        withNewContent: [true, 'boolean', '', 'whether to use a URL instead of page content'],
-        rules: [false, 'array', 'areStrings', 'QualWeb or ACT IDs of ACT rules to include, if not all']
+        withNewContent: [true, 'boolean', '', 'whether to use a URL instead of page content']
       }
     ],
     tenon: [
@@ -221,7 +184,6 @@ exports.actSpecs = {
       'Perform Testaro tests',
       {
         withItems: [true, 'boolean', '', 'itemize'],
-        rules: [false, 'array', 'areStrings', 'IDs of rules to include if array starts with y or exclude if with n, if not all evaluative rules'],
         args: [false, 'object', 'areArrays', 'extra args (object with rule properties and arrays of argument values as values ({focInd: [false, 250], hover: [-1], motion: [2500, 2500, 5]} by default'],
       }
     ],

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testaro",
-  "version": "14.4.0",
+  "version": "14.5.0",
   "description": "Automation of accessibility testing",
   "main": "index.js",
   "scripts": {

package/run.js

@@ -36,15 +36,12 @@ const tests = {
   alfa: 'alfa',
   axe: 'Axe',
   continuum: 'Level Access Continuum, community edition',
-  elements: 'data on specified elements',
   htmlcs: 'HTML CodeSniffer WCAG 2.1 AA ruleset',
   ibm: 'IBM Accessibility Checker',
   nuVal: 'Nu Html Checker',
   qualWeb: 'QualWeb',
   tenon: 'Tenon',
   testaro: 'Testaro',
-  textNodes: 'data on specified text nodes',
-  title: 'page title',
   wave: 'WAVE',
 };
 // Browser types available in PlayWright.
@@ -164,6 +161,7 @@ const hasSubtype = (variable, subtype) => {
       return isState(variable);
     }
     else {
+      console.log(`ERROR: ${subtype} not a known subtype`);
       return false;
     }
   }
@@ -977,21 +975,18 @@ const doActs = async (report, actIndex, page) => {
                 tenonData.requestIDs[id] = responseID || '';
               }
             }
-            // Otherwise, if the act is a test:
+            // Otherwise, if the act performs the tests of a tool:
             else if (act.type === 'test') {
               // Add a description of the test to the act.
               act.what = tests[act.which];
-              // Initialize the arguments.
-              const args = [act.which === 'tenon' ? tenonData : page];
-              // Identify the additional validator of the test.
-              const testValidator = actSpecs.tests[act.which];
-              // If it exists:
-              if (testValidator) {
-                // Identify its argument properties.
-                const argProperties = Object.keys(testValidator[1]);
-                // Add their values to the arguments.
-                args.push(...argProperties.map(propName => act[propName]));
-              }
+              // Initialize the options argument.
+              const options = {};
+              // Add any specified arguments to it.
+              Object.keys(act).forEach(key => {
+                if (! ['type', 'which'].includes(key)) {
+                  options[key] = act[key];
+                }
+              });
               // Conduct, report, and time the test.
               const startTime = Date.now();
               let testReport = {
@@ -1000,7 +995,8 @@ const doActs = async (report, actIndex, page) => {
                 }
               };
               try {
-                testReport = await require(`./tests/${act.which}`).reporter(...args);
+                const args = [act.which === 'tenon' ? tenonData : page, options];
+                testReport = await require(`./tests/${act.which}`).reporter(... args);
                 const expectations = act.expect;
                 // If the test has expectations:
                 if (expectations) {

package/standardize.js

@@ -18,6 +18,22 @@ const cap = rawString => {
     return '';
   }
 };
+// Returns the tag name and the value of an id attribute from a substring of HTML code.
+const getIdentifiers = code => {
+  let tagName = '';
+  let id = '';
+  if (code && typeof code === 'string' && code.length && /<.+/s.test(code)) {
+    const startTag = code.replace(/^[^<]*<|>.*/sg, '').trim();
+    if (startTag && startTag.length) {
+      tagName = startTag.replace(/\s.+$/s, '').toUpperCase();
+      const idArray = startTag.match(/\sid="([^"<>]+)"/);
+      if (idArray && idArray.length === 2) {
+        id = idArray[1];
+      }
+    }
+  }
+  return [tagName, id];
+};
 // Converts issue instances at an axe certainty level.
 const doAxe = (result, standardResult, certainty) => {
   if (result.details && result.details[certainty]) {
@@ -35,23 +51,13 @@ const doAxe = (result, standardResult, certainty) => {
           critical: 1
         };
         const ordinalSeverity = severityWeights[node.impact] + (certainty === 'violations' ? 2 : 0);
-        const tagName = node.html && node.html.replace(/^<|[ >].*$/sg, '').toUpperCase();
-        let id = '';
-        if (node.target && node.target.length && node.target[0].startsWith('#')) {
-          id = node.target[0].slice(1);
-        }
-        else if (node.html) {
-          const idArray = node.html.match(/\sid="([^"]+)"/);
-          if (idArray && idArray.length === 2) {
-            id = idArray[1];
-          }
-        }
+        const identifiers = getIdentifiers(node.html);
         const instance = {
           issueID: rule.id,
           what: Array.from(whatSet.values()).join('; '), 
           ordinalSeverity,
-          tagName,
-          id,
+          tagName: identifiers[0],
+          id: identifiers[1],
           location: {
             doc: 'dom',
             type: 'selector',
@@ -96,18 +102,13 @@ const doNuVal = (result, standardResult, docType) => {
   const items = result[docType] && result[docType].messages;
   if (items && items.length) {
     items.forEach(item => {
-      let tagName = '';
-      let id = '';
-      if (item.extract) {
-        const tagNameLCArray = item.extract.match(
+      const identifiers = getIdentifiers(item.extract);
+      if (! identifiers[0] && item.message) {
+        const tagNameLCArray = item.message.match(
           /^Element ([^ ]+)|^An (img) element| (meta|script) element| element (script)| tag (script)/
         );
         if (tagNameLCArray && tagNameLCArray.length > 1) {
-          tagName = tagNameLCArray[1].toUpperCase();
-        }
-        const idArray = item.extract.match(/^.+\sid="([^"]+)"/);
-        if (idArray && idArray.length === 2) {
-          id = idArray[1];
+          identifiers[0] = tagNameLCArray[1].toUpperCase();
         }
       }
       // Include the message twice, because in scoring it is likely to be replaced by a pattern.
@@ -115,8 +116,8 @@ const doNuVal = (result, standardResult, docType) => {
         issueID: item.message,
         what: item.message,
         ordinalSeverity: -1,
-        tagName,
-        id,
+        tagName: identifiers[0],
+        id: identifiers[1],
         location: {
           doc: docType === 'pageContent' ? 'dom' : 'source',
           type: 'line',
@@ -158,24 +159,13 @@ const doQualWeb = (result, standardResult, ruleClassName) => {
       ruleResult.results.forEach(item => {
         item.elements.forEach(element => {
           const {htmlCode} = element;
-          let tagName = '';
-          let id = '';
-          if (htmlCode) {
-            const tagNameArray = htmlCode.match(/^<([^ >]+)/);
-            if (tagNameArray && tagNameArray.length === 2) {
-              tagName = tagNameArray[1].toUpperCase();
-            }
-            const idArray = htmlCode.match(/\sid="([^"]+)"/);
-            if (idArray && idArray.length === 2) {
-              id = idArray[1];
-            }
-          }
+          const identifiers = getIdentifiers(htmlCode);
           const instance = {
             issueID: rule,
             what: ruleResult.description,
             ordinalSeverity: severities[ruleClassName][item.verdict],
-            tagName,
-            id,
+            tagName: identifiers[0],
+            id: identifiers[1],
             location: {
               doc: 'dom',
               type: 'selector',
@@ -235,26 +225,27 @@ const convert = (toolName, result, standardResult) => {
     standardResult.totals = [result.totals.warnings, 0, 0, result.totals.failures];
     result.items.forEach(item => {
       const {codeLines} = item.target;
-      let id = '';
-      if (codeLines && codeLines.length) {
-        const code = codeLines[0];
-        const idMatchArray = code.match(/\sid="([^"]+)"/);
-        if (idMatchArray && idMatchArray.length === 2) {
-          id = idMatchArray[1];
+      const code = Array.isArray(codeLines) ? codeLines.join(' ') : '';
+      const identifiers = getIdentifiers(code);
+      let tagName = item.target && item.target.tagName || '';
+      if (! tagName) {
+        const tagNameArray = item.target.path.match(/\/([a-z]+)\[\d+\]\/text\(\)\[\d+\]$/);
+        if (tagNameArray && tagNameArray.length === 2) {
+          tagName = tagNameArray[1];
         }
       }
       const instance = {
         issueID: item.rule.ruleID,
         what: item.rule.ruleSummary,
         ordinalSeverity: ['cantTell', '', '', 'failed'].indexOf(item.verdict),
-        tagName: item.target.tagName.toUpperCase(),
-        id,
+        tagName: tagName.toUpperCase() || identifiers[0],
+        id: identifiers[1],
         location: {
           doc: 'dom',
           type: 'xpath',
           spec: item.target.path
         },
-        excerpt: Array.isArray(codeLines) ? cap(codeLines.join(' ')) : ''
+        excerpt: cap(code)
       };
       standardResult.instances.push(instance);
     });
@@ -321,26 +312,19 @@ const convert = (toolName, result, standardResult) => {
   else if (toolName === 'ibm' && result.totals) {
     standardResult.totals = [0, result.totals.recommendation, 0, result.totals.violation];
     result.items.forEach(item => {
-      let tagName = '';
-      let id = '';
-      if (item.path && item.path.dom) {
+      const identifiers = getIdentifiers(item.snippet);
+      if (! identifiers[0] && item.path && item.path.dom) {
         const tagNameArray = item.path.dom.match(/^.+\/([^/[]+)/s);
         if (tagNameArray && tagNameArray.length === 2) {
-          tagName = tagNameArray[1];
-        }
-        if (item.snippet) {
-          const idArray = item.snippet.match(/^.+\sid="([^"]+)"/s);
-          if (idArray && idArray.length === 2) {
-            id = idArray[1];
-          }
+          identifiers[0] = tagNameArray[1].toUpperCase();
         }
       }
       const instance = {
         issueID: item.ruleId,
         what: item.message,
         ordinalSeverity: ['', 'recommendation', '', 'violation'].indexOf(item.level),
-        tagName,
-        id,
+        tagName: identifiers[0],
+        id: identifiers[1],
         location: {
           doc: 'dom',
           type: 'xpath',
@@ -391,18 +375,13 @@ const convert = (toolName, result, standardResult) => {
   // tenon
   else if (toolName === 'tenon' && result.data && result.data.resultSet) {
     result.data.resultSet.forEach(item => {
-      let tagName = '';
-      if (item.xpath) {
+      const identifiers = getIdentifiers(
+        item.errorSnippet.replace(/&lt;/g, '<').replace(/&gt;/g, '>')
+      );
+      if (! identifiers[0] && item.xpath) {
         const tagNameArray = item.xpath.match(/^.+\/([^/[]+)/);
         if (tagNameArray && tagNameArray.length === 2) {
-          tagName = tagNameArray[1].toUpperCase();
-        }
-      }
-      let id = '';
-      if (item.errorSnippet) {
-        const idArray = item.errorSnippet.match(/^.+\sid="([^"]+)"/);
-        if (idArray && idArray.length === 2) {
-          id = idArray[1];
+          identifiers[0] = tagNameArray[1].toUpperCase();
         }
       }
       const instance = {
@@ -411,8 +390,8 @@ const convert = (toolName, result, standardResult) => {
         ordinalSeverity: Math.min(
           3, Math.max(0, Math.round((item.certainty || 0) * (item.priority || 0) / 3333))
         ),
-        tagName,
-        id,
+        tagName: identifiers[0],
+        id: identifiers[1],
         location: {
           doc: 'dom',
           type: 'xpath',

package/testaro/embAc.js

@@ -32,7 +32,7 @@ exports.reporter = async (page, withItems) => await page.$$eval(
         }
         items.push({
           embeddedElement: bad.tagName,
-          embeddedID: bad.id,
+          embeddedID: bad.id || '',
           excerpt: compact(container.outerHTML)
         });
       }
@@ -49,7 +49,7 @@ exports.reporter = async (page, withItems) => await page.$$eval(
           what: `${item.embeddedElement} element is embedded in a link or button`,
           ordinalSeverity: 2,
           tagName: item.embeddedElement,
-          id: item.id,
+          id: item.embeddedID,
           location: {
             doc: '',
             type: '',

package/tests/alfa.js

@@ -12,7 +12,8 @@ let alfaRules = require('@siteimprove/alfa-rules').default;
 // FUNCTIONS
 
 // Conducts and reports an alfa test.
-exports.reporter = async (page, rules) => {
+exports.reporter = async (page, options) => {
+  const {rules} = options;
   // If only some rules are to be employed:
   if (rules && rules.length) {
     // Remove the other rules.

package/tests/axe.js

@@ -22,7 +22,8 @@
 const {injectAxe, getAxeResults} = require('axe-playwright');
 // FUNCTIONS
 // Conducts and reports an Axe test.
-exports.reporter = async (page, detailLevel, rules = []) => {
+exports.reporter = async (page, options) => {
+  const {detailLevel, rules} = options;
   // Initialize the report.
   let data = {};
   // Inject axe-core into the page.
@@ -38,7 +39,7 @@ exports.reporter = async (page, detailLevel, rules = []) => {
     const axeOptions = {
       resultTypes: ['violations', 'incomplete']
     };
-    if (rules.length) {
+    if (rules && rules.length) {
       axeOptions.runOnly = rules;
     }
     else {

package/tests/continuum.js

@@ -6,7 +6,8 @@
 
 // FUNCTIONS
 // Runs Continuum on the page.
-exports.reporter = async (page, rules) => {
+exports.reporter = async (page, options) => {
+  const {rules} = options;
   let result = {};
   // Inject the continuum scripts into the page, exposing the continuum object.
   for (const fileName of ['continuum.conf', 'AccessEngine.community', 'Continuum.community']) {

package/tests/htmlcs.js

@@ -5,7 +5,8 @@
 
 // FUNCTIONS
 // Runs HTML CodeSniffer on the page.
-exports.reporter = async (page, rules) => {
+exports.reporter = async (page, options) => {
+  const {rules} = options;
   const result = {};
   // Add the HTMLCS script to the page.
   await page.addScriptTag({

package/tests/ibm.js

@@ -122,7 +122,8 @@ const doTest = async (content, withItems, timeLimit, rules) => {
   }
 };
 // Returns results of an IBM test.
-exports.reporter = async (page, withItems, withNewContent, rules) => {
+exports.reporter = async (page, options) => {
+  const {withItems, withNewContent, rules} = options;
   const contentType = withNewContent ? 'new' : 'existing';
   console.log(`>>>>>> Content type: ${contentType}`);
   let result;

package/tests/nuVal.js

@@ -18,7 +18,8 @@ const fs = require('fs/promises');
 
 // ########## FUNCTIONS
 
-exports.reporter = async (page, messages) => {
+exports.reporter = async (page, options) => {
+  const {rules} = options;
   // Get the browser-parsed page.
   const pageContent = await page.content();
   // Get the page source.
@@ -63,12 +64,12 @@ exports.reporter = async (page, messages) => {
       // Delete left and right quotation marks and their erratic invalid replacements.
       data[page[0]] = nuDataClean;
       // If there is a report and restrictions on the report messages were specified:
-      if (! data[page[0]].error && messages && Array.isArray(messages) && messages.length) {
+      if (! data[page[0]].error && rules && Array.isArray(rules) && rules.length) {
         // Remove all messages except those specified.
-        const messageSpecs = messages.map(messageSpec => messageSpec.split(':', 2));
-        data[page[0]].messages = data[page[0]].messages.filter(message => messageSpecs.some(
-          messageSpec => message.type === messageSpec[0]
-          && message.message.startsWith(messageSpec[1])
+        const ruleSpecs = rules.map(ruleSpec => ruleSpec.split(':', 2));
+        data[page[0]].messages = data[page[0]].messages.filter(message => ruleSpecs.some(
+          ruleSpec => message.type === ruleSpec[0]
+          && message.message.includes(ruleSpec[1])
         ));
       }
     }

package/tests/qualWeb.js

@@ -11,11 +11,12 @@ const clusterOptions = {
 };
 // FUNCTIONS
 // Conducts and reports a QualWeb test.
-exports.reporter = async (page, withNewContent, rules = null) => {
+exports.reporter = async (page, options) => {
+  const {withNewContent, rules} = options;
   // Initialize the report.
   // Start the QualWeb core engine.
   await qualWeb.start(clusterOptions);
-  // Specify the page.
+  // Specify the test options.
   const qualWebOptions = {
     log: {
       console: true

package/tests/tenon.js

@@ -5,7 +5,8 @@
 const https = require('https');
 // Wait until a time limit in seconds expires.
 const wait = timeLimit => new Promise(resolve => setTimeout(resolve, 1000 * timeLimit));
-exports.reporter = async (tenonData, id) => {
+exports.reporter = async (tenonData, options) => {
+  const {id, rules} = options;
   if (tenonData && tenonData.accessToken && tenonData.requestIDs && tenonData.requestIDs[id]) {
     // Shared request options.
     const requestOptions = {
@@ -59,6 +60,19 @@ exports.reporter = async (tenonData, id) => {
         });
         resultRequest.end();
       });
+      // If any rules were specified and any test results exist:
+      if (
+        rules
+        && rules.length
+        && testResult.data
+        && testResult.data.resultSet
+        && testResult.data.resultSet.length
+      ) {
+        // Delete the results of tests of rules not specified.
+        const {resultSet} = testResult.data;
+        testResult.data.resultSet = resultSet
+        .filter(result => ! rules.includes(result.tID.toString()));
+      }
       return testResult;
     };
     // Get the test status (not reliable: may say 200 instead of 202).

package/tests/testaro.js

@@ -42,31 +42,50 @@ const etcRules = {
 // FUNCTIONS
 
 // Conducts and reports a Testaro test.
-exports.reporter = async (
-  page, withItems, rules = ['y', ... Object.keys(evalRules)], args = null
-) => {
+exports.reporter = async (page, options) => {
+  const {withItems, args} = options;
+  const argRules = args ? Object.keys(args) : null;
+  const rules = options.rules || ['y', ... Object.keys(evalRules)];
   // Initialize the data.
   const data = {
     rules: {}
   };
-  // For each rule invoked:
-  const argRules = args && Object.keys(args);
-  const realRules = rules[0] === 'y'
-    ? rules.slice(1)
-    : Object.keys(evalRules).filter(ruleName => ! rules.slice(1).includes(ruleName));
-  for (const rule of realRules) {
-    // Initialize an argument array.
-    const ruleArgs = [page, withItems];
-    // If the rule has extra arguments:
-    if (argRules && argRules.includes(rule)) {
-      // Add them to the argument array.
-      ruleArgs.push(... args[rule]);
+  // If the rule specification is valid:
+  if (
+    rules.length > 1
+    && ['y', 'n'].includes(rules[0])
+    && rules.slice(1).every(rule => evalRules[rule] || etcRules[rule])
+  ) {
+    // For each rule invoked:
+    const realRules = rules[0] === 'y'
+      ? rules.slice(1)
+      : Object.keys(evalRules).filter(ruleID => ! rules.slice(1).includes(ruleID));
+    for (const rule of realRules) {
+      // Initialize an argument array.
+      const ruleArgs = [page, withItems];
+      // If the rule has extra arguments:
+      if (argRules && argRules.includes(rule)) {
+        // Add them to the argument array.
+        ruleArgs.push(... args[rule]);
+      }
+      // Test the page.
+      const what = evalRules[rule] || etcRules[rule];
+      if (! data.rules[rule]) {
+        data.rules[rule] = {};
+      }
+      data.rules[rule].what = what;
+      console.log(`>>>>>> ${rule} (${what})`);
+      const report = await require(`../testaro/${rule}`).reporter(... ruleArgs);
+      Object.keys(report).forEach(key => {
+        data.rules[rule][key] = report[key];
+      });
     }
-    // Test the page.
-    data.rules[rule] = await require(`../testaro/${rule}`).reporter(... ruleArgs);
-    const what = evalRules[rule] || etcRules[rule];
-    data.rules[rule].what = what;
-    console.log(`>>>>>> ${rule} (${what})`);
+  }
+  // Otherwise, i.e. if the rule specification is invalid:
+  else {
+    console.log('ERROR: Testaro rule specification invalid');
+    data.prevented = true;
+    data.error = 'ERROR: Rule specification invalid';
   }
   return {result: data};
 };

package/tests/wave.js

@@ -6,7 +6,8 @@
 */
 const fs = require('fs/promises');
 const https = require('https');
-exports.reporter = async (page, reportType) => {
+exports.reporter = async (page, options) => {
+  const {reportType, rules} = options;
   const waveKey = process.env.WAVE_KEY;
   // Get the data from a WAVE test.
   const data = await new Promise(resolve => {
@@ -30,6 +31,23 @@ exports.reporter = async (page, reportType) => {
             delete categories.feature;
             delete categories.structure;
             delete categories.aria;
+            // If rules were specified:
+            if (rules && rules.length) {
+              // Delete the results of tests for other rules.
+              ['error', 'contrast', 'alert'].forEach(category => {
+                if (
+                  categories[category]
+                  && categories[category].items
+                  && categories[category].items.length
+                ) {
+                  Object.keys(categories[category].items).forEach(ruleID => {
+                    if (! rules.includes(ruleID)) {
+                      delete categories[category].items[ruleID];
+                    }
+                  });
+                }
+              });
+            }
             // Add WCAG information from the WAVE documentation.
             const waveDocJSON = await fs.readFile('procs/wavedoc.json');
             const waveDoc = JSON.parse(waveDocJSON);