testaro 14.7.5 → 14.9.0

package/README.md

@@ -6,9 +6,11 @@ Ensemble testing for web accessibility
 
 Testaro is a collection of web accessibility testing tools.
 
-The purpose of Testaro is to provide programmatic access to accessibility tests defined by several tools.
+The purposes of Testaro are to:
+- provide programmatic access to accessibility tests defined by several tools
+- facilitate the integration of the reports of the tools into a unified report
 
-Testaro aims to increase the efficiency of multi-tool integration for accessibility testing. The need for multi-tool integration, and its costs, are analyzed in [Accessibility Metatesting: Comparing Nine Testing Tools](https://arxiv.org/abs/2304.07591).
+The need for multi-tool integration, and its costs, are discussed in [Accessibility Metatesting: Comparing Nine Testing Tools](https://arxiv.org/abs/2304.07591).
 
 Testaro launches and controls web browsers, performing operations, conducting tests, and recording results.
 
@@ -52,27 +54,19 @@ Testaro performs tests of these tools:
 
 The [BBC Accessibility Standards Checker](https://github.com/bbc/bbc-a11y) is not a formal dependency, but some of the tests in the Testaro tool are adaptations of tests of that tool.
 
-As of this version, the counts of tests of the tools referenced above were:
-- Alfa: 103
-- Axe-core: 138
-- Continuum Community Edition: 267
-- Equal Access: 163
-- HTML CodeSniffer: 98
-- Nu Html Checker: 147
-- QualWeb core: 121
-- Tenon: 180
-- WAVE: 110
-- Testaro: 29
-- total: 1356
+## Rules
 
-Of the 29 Testaro tests, 26 are evaluative (they discover accessibility problems), and the other 3 (`elements`, `textNodes`, and `title`) are informative (they report conditions specified by the user).
+Each tool accessed with Testaro defines _rules_ and tests _targets_ for compliance with its rules. The counts of the rules range from about 30, for Testaro itself, to about 270, for Continuum Community Edition. In total, the ten tools define about 1350 rules. Some of the tools are under active development, and their rule counts change over time.
 
-## Quasi-tests
+## Job data
 
-Reports produced by Testaro contain data in addition to the results of these tests. Such data can be used like tests. In particular, the data include:
-- Latency (how long a time each tool takes to run its tests)
-- Test prevention (the failure of tools to run on particular targets)
-- Logging (browser messaging, including about document errors, during testing)
+A report produced by Testaro discloses:
+- raw results of tests conducted by tools
+- standardized results of tests conducted by tools
+- process data, including statistics on:
+    - latency (how long a time each tool takes to run its tests)
+    - test prevention (the failure of tools to run on particular targets)
+    - logging (browser messaging, including about document errors, during testing)
 
 ## Code organization
 
@@ -188,30 +182,31 @@ Job properties:
 #### Introduction
 
 While each tool produces a _tool report_ of the results of its tests, Testaro also produces a _job report_, combining all of the tool reports of a job.
-- Tools append their reports to their acts in a job. For example, if one act in a job specifies some Continuum tests to be run, Continuum appends its report to that act. In that way, the act now describes not only the Continuum tests that were run, but also the results of those tests.
-- Testaro further elaborates a job by reporting comprehensive results there. Testaro can add more facts to each of the tool reports and also adds whole-job facts, in a `jobData` property, to the job. Testaro thereby converts the job to a job report.
+- Tools append their reports to their acts in a job. For example, if one act in a job specifies some Continuum tests to be run, Continuum appends its report to that act. In that way, the act now describes not only the Continuum tests that were run, but also the results of those tests. Testaro does some pruning of tool reports, removing content that is judged unlikely to be useful. You can examine and modify the pruning algorithms in the modules located in the `tests` directory.
+- Testaro further elaborates a job by reporting comprehensive results in a job report. Testaro can add more facts to each of the tool reports and also adds whole-job facts, in a `jobData` property, to the job.
 
 #### Formats
 
-##### Tool formats
+##### Tool-report formats
 
 The tools listed above as dependencies write their tool reports in various formats. They differ in how they organize multiple instances of the same problem, how they classify severity and certainty, how they point to the locations of problems, how they name problems, etc.
 
 ##### Standard format
 
-Testaro helps overcome this format diversity by offering to represent the main facts in the report of each tool in a single standardized format.
+Testaro helps overcome this format diversity by offering to represent the main facts in each tool report in a single standardized format.
 
 In the conceptual scheme underlying the format standardization of Testaro, each tool has its own set of _rules_, where a rule is an algorithm for evaluating a target and determining whether instances of some kind of problem exist in it. With standardization, Testaro reports, in a uniform way, the outcomes from the application of rules by tools to a target.
 
-If the `STANDARD` environment variable has the value `also` (which it has by default) or `only`, Testaro converts some data in each tool report to a standard format. That permits you to ignore the format idiosyncrasies of the tools. If `STANDARD` has the value `also`, the job report includes both formats. If the value is `only`, the job report includes only the standard format. If the value is `no`, the job report includes only the original format of each tool.
+If the `STANDARD` environment variable has the value `also` (which it has by default) or `only`, Testaro converts some data in each tool report to a standard format. That permits you to ignore the format idiosyncrasies of the tools. If `STANDARD` has the value `also`, the job report includes both formats. If the value is `only`, the job report includes only the standard format. If the value is `no`, the job report includes only the original format of each tool report.
 
-The standard format of each tool report has two properties:
+The standard format of each tool report has these properties:
+- `prevented`: `true` if the tool failed to run on the page, or otherwise omitted.
 - `totals`: an array of 4 integers, representing the counts of problem instances classified by the tool into 4 ordinal degrees of severity. For example, `[2, 13, 0, 5]` would mean that the tool reported 2 instances at the lowest severity, 13 at the next-lowest, none at the third-lowest, and 5 at the highest.
 - `instances`: an array of objects describing facts about issue instances reported by the tool. This object has these properties, some of which have empty strings as values when the tool does not provide values:
     - `ruleID`: a code identifying a rule
     - `what`: a description of the rule
     - `count` (optional): the count of instances if this instance represents multiple instances
-    - `ordinalSeverity`: how the tool ranks the severity of the instance, on a 4-point ordinal scale
+    - `ordinalSeverity`: how the tool ranks the severity of the instance, on a 4-point ordinal scale from 0 to 3
     - `tagName`: upper-case tagName of the affected element
     - `id`: value of the `id` property of that element
     - `location`: an object with three properties:
@@ -229,7 +224,7 @@ standardResult: {
     {
       ruleID: 'rule01',
       what: 'Button type invalid',
-      ordinalSeverity: 0,
+      ordinalSeverity: 2,
       tagName: 'BUTTON'
       id: '',
       location: {
@@ -272,7 +267,13 @@ standardResult: {
 
 If a tool has the option to be used without itemization and is being so used, the `instances` array may be empty.
 
-This standard format is not opinionated about issue classifications. A rule ID identifies something deemed to be an issue by a tool. Useful reporting from multi-tool testing still requires the classification of tool **rules** into **issues**. If tool `A` has `alt-incomplete` as a rule ID and tool `B` has `image_alt_stub` as a rule ID, Testaro does not decide whether those are really the same issue or different issues. That decision belongs to you. The standardization of tool reports by Testaro eliminates some of the drudgery in issue classification, but not any of the judgment required for issue classification.
+This standard format reflects some judgments. For example:
+- The `ordinalSeverity` property of an instance may have required interpretation. Tools may report severity, certainty, both, or neither; they may classify severity or certainty ordinally or metrically; and, if they classify ordinally, their scales may have more or fewer than 4 ranks. Testaro coerces each tool’s severity and/or certainty classification into a 4-rank ordinal classification. This classification is deemed to express the most common pattern among the tools.
+- The `tagName` property of an instance may not always be obvious, because in some cases the rule being tested for requires a relationship among more than one element (e.g., “An X element may not have a Y element as its parent”).
+
+You are not dependent on the judgments incorporated into the standard format, because Testaro can give you the original reports from the tools.
+
+The standard format does not express opinions issue classifications. A rule ID identifies something deemed to be an issue by a tool. Useful reporting from multi-tool testing still requires the classification of tool **rules** into **issues**. If tool `A` has `alt-incomplete` as a rule ID and tool `B` has `image_alt_stub` as a rule ID, Testaro does not decide whether those are really the same issue or different issues. That decision belongs to you. The standardization of tool reports by Testaro eliminates some of the drudgery in issue classification, but not any of the judgment required for issue classification.
 
 ### Acts
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testaro",
-  "version": "14.7.5",
+  "version": "14.9.0",
   "description": "Automation of accessibility testing",
   "main": "index.js",
   "scripts": {

package/standardize.js

@@ -116,7 +116,7 @@ const doNuVal = (result, standardResult, docType) => {
           identifiers[0] = tagNameLCArray[1].toUpperCase();
         }
       }
-      // Include the message twice, because in scoring it is likely to be replaced by a pattern.
+      // Include the message twice. A scoring procedure may replace the ruleID with a pattern.
       const instance = {
         ruleID: item.message,
         what: item.message,
@@ -225,26 +225,29 @@ const doWAVE = (result, standardResult, categoryName) => {
 };
 // Converts a result.
 const convert = (toolName, result, standardResult) => {
+  // Prevention.
+  if (result.prevented) {
+    standardResult.prevented = true;
+  }
   // alfa
-  if (toolName === 'alfa' && result.totals) {
+  else if (toolName === 'alfa' && result.totals) {
     standardResult.totals = [result.totals.warnings, 0, 0, result.totals.failures];
     result.items.forEach(item => {
       const {codeLines} = item.target;
       const code = Array.isArray(codeLines) ? codeLines.join(' ') : '';
       const identifiers = getIdentifiers(code);
-      let tagName = item.target && item.target.tagName || '';
-      if (! tagName) {
-        const tagNameArray = item.target.path.match(/\/([a-z]+)\[\d+\]\/text\(\)\[\d+\]$/);
-        if (tagNameArray && tagNameArray.length === 2) {
-          tagName = tagNameArray[1];
-        }
+      const tagNameArray = item.target
+      && item.target.path
+      && item.target.path.match(/^.*\/([a-z]+)\[\d+\]/);
+      if (tagNameArray && tagNameArray.length === 2) {
+        identifiers[0] = tagNameArray[1].toUpperCase();
       }
       const {rule, target} = item;
       const instance = {
         ruleID: rule.ruleID,
         what: rule.ruleSummary,
         ordinalSeverity: ['cantTell', '', '', 'failed'].indexOf(item.verdict),
-        tagName: tagName.toUpperCase() || identifiers[0],
+        tagName: identifiers[0],
         id: identifiers[1],
         location: {
           doc: 'dom',
@@ -453,6 +456,10 @@ const convert = (toolName, result, standardResult) => {
       doWAVE(result, standardResult, categoryName);
     });
   }
+  // Any tool with no reported rule violations:
+  else {
+    standardResult.totals = [0, 0, 0, 0];
+  }
 };
 // Converts the results.
 exports.standardize = act => {

package/testaro/allHidden.js

@@ -76,7 +76,7 @@ exports.reporter = async page => {
       if (data[hider][region]) {
         // Add a standard instance for that combination.
         standardInstances.push({
-          issueID: `allHidden-${hider}-${region}`,
+          ruleID: 'allHidden',
           what: `${reportables[region][1]} ${reportables[hider][1]}`,
           ordinalSeverity: reportables[region][0] + reportables[hider][0] || 0,
           tagName: region.tagName,

package/testaro/attVal.js

@@ -48,7 +48,7 @@ exports.reporter = async (page, withItems, attributeName, areLicit, values) => {
     data.items = badAttributeData;
     badAttributeData.forEach(item => {
       standardInstances.push({
-        issueID: `attVal-${item.tagName}-${attributeName}`,
+        ruleID: 'attVal',
         what:
           `${item.tagName} element has attribute ${attributeName} with illicit value ${item.attributeValue}`,
         ordinalSeverity: 2,
@@ -65,7 +65,7 @@ exports.reporter = async (page, withItems, attributeName, areLicit, values) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'attVal',
+      ruleID: 'attVal',
       what: 'Elements have attributes with illicit values',
       count: data.total,
       ordinalSeverity: 2,

package/testaro/autocomplete.js

@@ -74,7 +74,7 @@ exports.reporter = async (page, withItems) => {
   if (data.items) {
     data.items.forEach(item => {
       standardInstances.push({
-        issueID: `autocomplete-${item[0]}`,
+        ruleID: 'autocomplete',
         what: `Input is missing the required autocomplete attribute with value ${item[0]}`,
         ordinalSeverity: 2,
         tagName: 'INPUT',
@@ -90,7 +90,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'autocomplete',
+      ruleID: 'autocomplete',
       what: 'Inputs are missing required autocomplete attributes',
       ordinalSeverity: 2,
       tagName: '',

package/testaro/bulk.js

@@ -23,7 +23,7 @@ exports.reporter = async page => {
     data,
     totals: [count, 0, 0, 0],
     standardInstances: data.visibleElements < 200 ? [] : [{
-      issueID: 'bulk',
+      ruleID: 'bulk',
       what: 'Page contains a large number of visible elements',
       count,
       ordinalSeverity: 0,

package/testaro/docType.js

@@ -14,7 +14,7 @@ exports.reporter = async page => {
     data: {docHasType},
     totals: [0, 0, 0, docHasType ? 0 : 1],
     standardInstances: docHasType ? [] : [{
-      issueID: 'docType',
+      ruleID: 'docType',
       what: 'Document has no standard HTML doctype preamble',
       ordinalSeverity: 3,
       tagName: 'HTML',

package/testaro/dupAtt.js

@@ -84,7 +84,7 @@ exports.reporter = async (page, withItems) => {
   if (data.items) {
     data.items.forEach(item => {
       standardInstances.push({
-        issueID: 'dupAtt',
+        ruleID: 'dupAtt',
         what: `Element ${item.tagName} has 2 attributes with the same name`,
         ordinalSeverity: 2,
         tagName: item.tagName,
@@ -100,7 +100,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'dupAtt',
+      ruleID: 'dupAtt',
       what: 'In some elements 2 attributes have the same name',
       ordinalSeverity: 2,
       location: {

package/testaro/embAc.js

@@ -45,7 +45,7 @@ exports.reporter = async (page, withItems) => await page.$$eval(
       data.items = items;
       items.forEach(item => {
         standardInstances.push({
-          issueID: `embAc-${item.embeddedElement}`,
+          ruleID: 'embAc',
           what: `${item.embeddedElement} element is embedded in a link or button`,
           ordinalSeverity: 2,
           tagName: item.embeddedElement,
@@ -61,7 +61,7 @@ exports.reporter = async (page, withItems) => await page.$$eval(
     }
     else if (total) {
       standardInstances.push({
-        issueID: 'embAc',
+        ruleID: 'embAc',
         what: 'Interactive elements are contained by links or buttons',
         count: total,
         ordinalSeverity: 2,

package/testaro/filter.js

@@ -53,7 +53,7 @@ exports.reporter = async (page, withItems) => {
   if (data.items) {
     data.items.forEach(item => {
       standardInstances.push({
-        issueID: 'filterStyle',
+        ruleID: 'filterStyle',
         what: `${item.tagName} element has a filter style that impacts ${item.impact} elements`,
         ordinalSeverity: 2,
         tagName: item.toUpperCase(),
@@ -69,7 +69,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.totals.styledElements) {
     standardInstances.push({
-      issueID: 'filterStyle',
+      ruleID: 'filterStyle',
       what: 'Elements have filter styles impacting other elements',
       ordinalSeverity: 2,
       tagName: '',

package/testaro/focAll.js

@@ -65,7 +65,7 @@ exports.reporter = async page => {
     data,
     totals: [0, 0, Math.abs(data.discrepancy), 0],
     standardInstances: data.discrepancy ? [{
-      issueID: 'focAll',
+      ruleID: 'focAll',
       what: 'Some focusable elements are not Tab-focusable or vice versa',
       count: Math.abs(data.discrepancy),
       ordinalSeverity: 2,

package/testaro/focInd.js

@@ -174,7 +174,7 @@ exports.reporter = async (page, withItems, revealAll = false, allowedDelay = 250
       data.items[issueName].forEach(item => {
         const qualifier = issueName === 'nonoutline' ? 'a non-outline' : 'no';
         standardInstances.push({
-          issueID: `focInd-${issueName}`,
+          issueID: 'focInd',
           what: `${item.tagName} element has ${qualifier} focus indicator`,
           ordinalSeverity: issueName === 'nonoutline' ? 2 : 3,
           tagName: item.tagName,
@@ -192,7 +192,7 @@ exports.reporter = async (page, withItems, revealAll = false, allowedDelay = 250
   else {
     if (types.missing.total) {
       standardInstances.push({
-        issueID: 'focInd-missing',
+        issueID: 'focInd',
         what: 'Elements have missing focus indicators',
         count: types.missing.total,
         ordinalSeverity: 3,
@@ -208,7 +208,7 @@ exports.reporter = async (page, withItems, revealAll = false, allowedDelay = 250
     }
     if (types.nonoutline.total) {
       standardInstances.push({
-        issueID: 'focInd-nonoutline',
+        issueID: 'focInd',
         what: 'Elements have non-outline focus indicators',
         count: types.nonoutline.total,
         ordinalSeverity: 2,

package/testaro/focOp.js

@@ -150,17 +150,14 @@ exports.reporter = async (page, withItems) => {
   const standardInstances = [];
   if (data.items && data.items.onlyFocusable && data.items.onlyOperable) {
     ['onlyFocusable', 'onlyOperable'].forEach(issue => {
-      const issueID = issue === 'onlyFocusable'
-        ? 'focOp-focusable-inoperable'
-        : 'focOp-operable-nonfocusable';
       const gripe = issue === 'onlyFocusable'
         ? 'is focusable but not operable'
         : 'is operable but not focusable';
       const ordinalSeverity = issue === 'onlyFocusable' ? 2 : 3;
       data.items[issue].forEach(item => {
         standardInstances.push({
-          issueID,
-          what: `${item.tagName} element ${gripe}`,
+          ruleID: 'focOp',
+          complaint: `${item.tagName} element ${gripe}`,
           ordinalSeverity,
           tagName: item.tagName,
           id: item.id,
@@ -177,8 +174,8 @@ exports.reporter = async (page, withItems) => {
   else {
     if (totals[2]) {
       standardInstances.push({
-        issueID: 'focOp-onlyFocusable',
-        what: 'Focusable elements are inoperable',
+        ruleID: 'focOp',
+        complaint: 'Focusable elements are inoperable',
         count: totals[2],
         ordinalSeverity: 2,
         tagName: '',
@@ -193,8 +190,8 @@ exports.reporter = async (page, withItems) => {
     }
     if (totals[3]) {
       standardInstances.push({
-        issueID: 'focOp-onlyOperable',
-        what: 'Operable elements are nonfocusable',
+        ruleID: 'focOp',
+        complaint: 'Operable elements are nonfocusable',
         count: totals[3],
         ordinalSeverity: 3,
         tagName: '',

package/testaro/focVis.js

@@ -33,7 +33,7 @@ exports.reporter = async (page, withItems) => {
   if (data.items) {
     data.items.forEach(item => {
       standardInstances.push({
-        issueID: 'focVis',
+        ruleID: 'focVis',
         what: 'Visible link is above or to the left of the display',
         ordinalSeverity: 2,
         tagName: 'A',
@@ -49,7 +49,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'focVis',
+      ruleID: 'focVis',
       what: 'Visible links are above or to the left of the display',
       count: data.total,
       ordinalSeverity: 2,

package/testaro/hover.js

@@ -406,7 +406,7 @@ exports.reporter = async (page, withItems, sampleSize = -1) => {
     Object.keys(data.items).forEach(issue => {
       data.items[issue].forEach(item => {
         standardInstances.push({
-          issueID: `hover-${issue}`,
+          ruleID: `hover-${issue}`,
           what: what[issue],
           count: data.totals[issue],
           ordinalSeverity: severity[issue],
@@ -424,7 +424,7 @@ exports.reporter = async (page, withItems, sampleSize = -1) => {
   }
   else if (totals.some(total => total)) {
     standardInstances.push({
-      issueID: 'hover',
+      ruleID: 'hover',
       what: 'Hovering has unexpected impacts',
       count: Object.values(data.totals).reduce((total, current) => total + current),
       ordinalSeverity: totals.reduce((max, current, index) => current ? index : max, 0),

package/testaro/labClash.js

@@ -164,7 +164,7 @@ exports.reporter = async (page, withItems) => {
             excerptTail = item.content || '';
           }
           standardInstances.push({
-            issueID: `labClash-${issue}`,
+            ruleID: 'labClash',
             what: `Element ${item.tagName} ${diagnosis}`,
             ordinalSeverity: issue === 'mislabeled' ? 2 : 3,
             tagName: item.tagName,
@@ -182,7 +182,7 @@ exports.reporter = async (page, withItems) => {
     else {
       if (data.totals.unlabeled) {
         standardInstances.push({
-          issueID: 'labClash-unlabeled',
+          ruleID: 'labClash',
           what: 'Element labels are missing',
           count: data.totals.unlabeled,
           ordinalSeverity: 3,
@@ -198,7 +198,7 @@ exports.reporter = async (page, withItems) => {
       }
       if (data.totals.mislabeled) {
         standardInstances.push({
-          issueID: 'labClash-mislabeled',
+          ruleID: 'labClash',
           what: 'Element labels are conflicting',
           count: data.totals.mislabeled,
           ordinalSeverity: 2,

package/testaro/linkTo.js

@@ -27,7 +27,7 @@ exports.reporter = async (page, withItems) => {
     data.items = badLinkData;
     data.items.forEach(item => {
       standardInstances.push({
-        issueID: 'linkTo',
+        ruleID: 'linkTo',
         what: 'Element a has no href attribute',
         ordinalSeverity: 2,
         tagName: 'A',
@@ -43,7 +43,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'linkTo',
+      ruleID: 'linkTo',
       what: 'Links are missing href attributes',
       count: data.total,
       ordinalSeverity: 2,

package/testaro/linkUl.js

@@ -79,7 +79,7 @@ exports.reporter = async (page, withItems) => {
     if (data.items && data.items.notUnderlined) {
       data.items.notUnderlined.forEach(item => {
         standardInstances.push({
-          issueID: 'linkUl',
+          ruleID: 'linkUl',
           what: 'Link is inline but has no underline',
           ordinalSeverity: 1,
           tagName: 'A',
@@ -95,7 +95,7 @@ exports.reporter = async (page, withItems) => {
     }
     else if (adjacent.total - adjacent.underlined > 0) {
       standardInstances.push({
-        issueID: 'linkUl',
+        ruleID: 'linkUl',
         what: 'Inline links are missing underlines',
         count: adjacent.total - adjacent.underlined,
         ordinalSeverity: 1,

package/testaro/menuNav.js

@@ -275,7 +275,7 @@ exports.reporter = async (page, withItems) => {
   if (data.menuItems && data.menuItems.incorrect) {
     data.menuItems.incorrect.forEach(item => {
       standardInstances.push({
-        issueID: 'menuNav',
+        ruleID: 'menuNav',
         what: `Menu item responds nonstandardly to ${item.navigationErrors.join(', ')}`,
         count: item.navigationErrors.length,
         ordinalSeverity: 1,
@@ -292,7 +292,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.totals.menuItems.incorrect) {
     standardInstances.push({
-      issueID: 'menuNav',
+      ruleID: 'menuNav',
       what: 'Menus and menu items have nonstandard navigation',
       count: data.totals.navigations.all.incorrect,
       ordinalSeverity: 1,

package/testaro/miniText.js

@@ -50,7 +50,7 @@ exports.reporter = async (page, withItems) => {
     data.items = miniTexts;
     miniTexts.forEach(text => {
       standardInstances.push({
-        issueID: 'miniText',
+        ruleID: 'miniText',
         what: 'Text font is smaller than 11 pixels',
         ordinalSeverity: 2,
         tagName: text.tagName,
@@ -66,7 +66,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'miniText',
+      ruleID: 'miniText',
       what: 'Texts have fonts smaller than 11 pixels',
       count: data.total,
       ordinalSeverity: 2,

package/testaro/motion.js

@@ -124,7 +124,7 @@ exports.reporter = async (page, withItems, delay = 2500, interval = 2500, count
         0
       ],
       standardInstances: count ? [{
-        issueID: 'motion',
+        ruleID: 'motion',
         what: 'Content moves or changes without user request',
         count,
         ordinalSeverity: 2,

package/testaro/nonTable.js

@@ -68,7 +68,7 @@ exports.reporter = async (page, withItems) => {
     data.items = badTableTexts;
     data.items.forEach(item => {
       standardInstances.push({
-        issueID: 'nonTable',
+        ruleID: 'nonTable',
         what: 'Table is misused to arrange content',
         ordinalSeverity: 0,
         tagName: 'TABLE',
@@ -84,7 +84,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'nonTable',
+      ruleID: 'nonTable',
       what: 'Tables are misused to arrange content',
       count: data.total,
       ordinalSeverity: 0,

package/testaro/radioSet.js

@@ -88,7 +88,7 @@ exports.reporter = async (page, withItems) => {
         }));
         items.notInSet.forEach(item => {
           standardInstances.push({
-            issueID: 'radioSet',
+            ruleID: 'radioSet',
             what: 'Radio button and its peers are not in a fieldset with a legend',
             ordinalSeverity: 2,
             tagName: 'INPUT',
@@ -104,7 +104,7 @@ exports.reporter = async (page, withItems) => {
       }
       else if (loneRadios > 0) {
         standardInstances.push({
-          issueID: 'radioSet',
+          ruleID: 'radioSet',
           what: 'Radio buttons are not validly grouped in fieldsets with legends',
           count: loneRadios,
           ordinalSeverity: 2,

package/testaro/role.js

@@ -486,7 +486,7 @@ exports.reporter = async page => await page.$eval('body', body => {
       const redCount = pairTotals.redundant;
       if (redCount) {
         standardInstances.push({
-          issueID: 'role-redundant',
+          ruleID: 'role',
           what: `${tagName} elements have redundant explicit role ${role} (count: ${redCount})`,
           count: redCount,
           ordinalSeverity: 1,
@@ -503,7 +503,7 @@ exports.reporter = async page => await page.$eval('body', body => {
       const badCount = pairTotals.bad;
       if (badCount) {
         standardInstances.push({
-          issueID: 'role-bad',
+          ruleID: 'role',
           what:
           `${tagName} elements have invalid or native-replaceable explicit role ${role} (count: ${badCount})`,
           count: badCount,

package/testaro/styleDiff.js

@@ -174,7 +174,7 @@ exports.reporter = async (page, withItems) => {
         const extraCount = elementSubtotals.length - 1;
         totals[severity] += extraCount;
         standardInstances.push({
-          issueID: 'styleDiff',
+          ruleID: 'styleDiff',
           what: `${currentData[1]} have ${elementSubtotals.length} different styles`,
           count: extraCount,
           ordinalSeverity: severity,

package/testaro/tabNav.js

@@ -339,7 +339,7 @@ exports.reporter = async (page, withItems) => {
   if (data.tabElements && data.tabElements.incorrect) {
     data.tabElements.incorrect.forEach(item => {
       standardInstances.push({
-        issueID: 'tabNav',
+        ruleID: 'tabNav',
         what:
         `${item.tagName} element responds nonstandardly to ${item.navigationErrors.join(', ')}`,
         count: item.navigationErrors.length,
@@ -357,7 +357,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.totals.navigations.all.incorrect) {
     standardInstances.push({
-      issueID: 'tabNav',
+      ruleID: 'tabNav',
       what: 'Tablists have nonstandard navigation',
       count: data.totals.navigations.all.incorrect,
       ordinalSeverity: 1,

package/testaro/titledEl.js

@@ -30,7 +30,7 @@ exports.reporter = async (page, withItems) => {
     data.items = badTitleElements;
     badTitleElements.forEach(element => {
       standardInstances.push({
-        issueID: 'titledEl',
+        ruleID: 'titledEl',
         what: `${element.tagName} element has a title attribute`,
         ordinalSeverity: 2,
         tagName: element.tagName,
@@ -46,7 +46,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.total) {
     standardInstances.push({
-      issueID: 'titledEl',
+      ruleID: 'titledEl',
       what: 'Ineligible elements have title attributes',
       count: data.total,
       ordinalSeverity: 2,

package/testaro/zIndex.js

@@ -52,7 +52,7 @@ exports.reporter = async (page, withItems) => {
   if (data.items) {
     data.items.forEach(item => {
       standardInstances.push({
-        issueID: 'zIndex',
+        ruleID: 'zIndex',
         what: `${item.tagName} element has a non-default Z index`,
         ordinalSeverity: 0,
         tagName: item.tagName,
@@ -68,7 +68,7 @@ exports.reporter = async (page, withItems) => {
   }
   else if (data.totals.total) {
     standardInstances.push({
-      issueID: 'zIndex',
+      ruleID: 'zIndex',
       what: 'Elements have non-default Z indexes',
       count: data.totals.total,
       ordinalSeverity: 0,

package/tests/continuum.js

@@ -28,9 +28,9 @@ exports.reporter = async (page, options) => {
       continuum.setUp(null, null, window);
       // If a set of rules to be employed was specified:
       let bigResultPromise;
-      if (rules && Array.isArray(rules) && rules.length && rules.every(rule => typeof rule === 'number')) {
+      if (rules && Array.isArray(rules) && rules.length && rules.every(rule => typeof rule === 'string')) {
         // Run the tests for them.
-        bigResultPromise = continuum.runTests(rules);
+        bigResultPromise = continuum.runTests(rules.map(rule => Number.parseInt(rule)));
       }
       // Otherwise, i.e. if no rules were specified:
       else {

package/tests/testaro.js

@@ -26,7 +26,8 @@ const evalRules = {
   motion: 'motion without user request',
   nonTable: 'table elements used for layout',
   radioSet: 'radio buttons not grouped into standard field sets',
-  role: 'invalid, inadvised, and redundant explicit roles',
+  roleBad: 'invalid and native-replacing explicit roles',
+  roleRedundant: 'redundant explicit roles',
   styleDiff: 'style inconsistencies',
   tabNav: 'nonstandard keyboard navigation between elements with the tab role',
   titledEl: 'title attributes on inappropriate elements',