testaro 38.0.2 → 39.0.0

package/README.md

@@ -40,6 +40,7 @@ One software product that performs some such functions is [Testilo](https://www.
 
 Testaro uses:
 - [Playwright](https://playwright.dev/) to launch browsers, perform user actions in them, and perform tests
+- [plywright-dompath](https://www.npmjs.com/package/playwright-dompath) to retrieve XPaths of elements
 - [pixelmatch](https://www.npmjs.com/package/pixelmatch) to measure motion
 
 Testaro performs tests of these tools:
@@ -212,96 +213,79 @@ Testaro also generates some data about the job and adds those data to the job, i
 
 The tools listed above as dependencies write their tool reports in various formats. They differ in how they organize multiple instances of the same problem, how they classify severity and certainty, how they point to the locations of problems, how they name problems, etc.
 
-##### Standard format
-
-Testaro helps overcome this format diversity by offering to represent the main facts in each tool report in a single standardized format.
-
-In the conceptual scheme underlying the format standardization of Testaro, each tool has its own set of _rules_, where a rule is an algorithm for evaluating a target and determining whether instances of some kind of problem exist in it. With standardization, Testaro reports, in a uniform way, the outcomes from the application of rules by tools to a target.
-
-Each job can specify how Testaro is to handle report standardization. A job can contain a `standard` property. If the value of that property is `'also'` or `'only'`, Testaro converts some data in each tool report to the standard format. That permits you to ignore the format idiosyncrasies of the tools. If `standard` has the value `'also'`, the job report includes both formats. If the value is `'only'`, or there is no value, the job report includes only the standard format. If the value is `'no'`, the job report includes only the original format of each tool report.
-
-The standard format of each tool report has these properties:
-- `prevented`: `true` if the tool failed to run on the page, or otherwise omitted.
-- `totals`: an array of 4 integers, representing the counts of problem instances classified by the tool into 4 ordinal degrees of severity. For example, `[2, 13, 0, 5]` would mean that the tool reported 2 instances at the lowest severity, 13 at the next-lowest, none at the third-lowest, and 5 at the highest.
-- `instances`: an array of objects describing facts about issue instances reported by the tool. This object has these properties, some of which have empty strings as values when the tool does not provide values:
-    - `ruleID`: a code identifying a rule
-    - `what`: a description of the rule
-    - `count` (optional): the count of instances if this instance represents multiple instances
-    - `ordinalSeverity`: how the tool ranks the severity of the instance, on a 4-point ordinal scale from 0 to 3
-    - `tagName`: upper-case tagName of the affected element
-    - `id`: value of the `id` property of that element
-    - `location`: an object with three properties:
-        - `doc`: whether the source (`source`) or the browser rendition (`dom`) was tested
-        - `type`: the type of location information provided by the tool (`line`, `selector`, `xpath`, or `box`)
-        - `spec`: the location information
-    - `excerpt`: some or all of the code
-
-The most common location types reported by the tools are:
-- `line`: Nu Html Checker
-- `selector`: Axe, QualWeb, WAVE
-- `xpath`: Alfa, ASLint, Equal Access
-- `box`: Editoria11y, Testaro
-- none: HTML CodeSniffer
+A Testaro report can include, for each tool, either or both of these properties:
+- `result`: the result in the native tool format.
+- `standardResult`: the result in a standard format identical for all tools.
 
-The original result of a test act is recorded as the value of a `result` property of the act. The standard-format result is recorded as the value of the `standardResult` property of the act. Its format is shown by this example:
+##### Standard result
 
-``` javascript
-standardResult: {
-  totals: [2, 0, 17, 0],
-  instances: [
-    {
-      ruleID: 'rule01',
-      what: 'Button type invalid',
-      ordinalSeverity: 2,
-      tagName: 'BUTTON'
-      id: '',
-      location: {
-        doc: 'dom',
-        type: 'box',
-        spec: {
-          x: 12,
-          y: 340,
-          width: 46,
-          height: 50
-        }
-      },
-      excerpt: '<button type="link"></button>'
-    },
-    {
-      ruleID: 'rule01',
-      what: 'Button type invalid',
-      ordinalSeverity: 1,
-      tagName: 'BUTTON',
-      id: 'submitbutton',
-      location: {
-        doc: 'dom',
-        type: 'line',
-        spec: 145
-      },
-      excerpt: '<button type="important">Submit</button>'
-    },
-    {
-      ruleID: 'rule02',
-      what: 'Links have empty href attributes',
-      count: 17,
-      ordinalSeverity: 3,
-      tagName: 'A',
-      id: '',
-      location: {
-        doc: '',
-        type: '',
-        spec: ''
-      },
-      excerpt: ''
-    }
-  ]
+###### Properties
+
+The standard result includes three properties:
+- `prevented`: a boolean (`true` or `false`) value, stating whether the page prevented the tool from performing its tests.
+- `totals`: an array of numbers representing how many instances of rule violations at each level of severity the tool reported. There are 4 ordinal severity levels. For example, the array `[3, 0, 14, 10]` would report that there were 3 violations at level 0, 0 at level 1, 14 at level 2, and 10 at level 3.
+- `instances`: an array of objects describing the rule violations. An instance can describe a single violation, usually by one element in the page, or can summarize multiple violations of the same rule.
+
+###### Instances
+
+Here is an example of a standard instance:
+
+```javascript
+{
+  ruleID: 'rule01',
+  what: 'Button type invalid',
+  ordinalSeverity: 2,
+  count: 1,
+  tagName: 'BUTTON'
+  id: '',
+  location: {
+    doc: 'dom',
+    type: 'xpath',
+    spec: '/html[1]/body[1]/section[1]/div[1]/div[1]/ul[1]/li[1]/a[1]'
+  },
+  excerpt: '<button type="link"></button>',
+  boxID: '12:340:46:50',
+  pathID: '/html[1]/body[1]/section[1]/div[1]/div[1]/ul[1]/li[1]/a[1]'
 }
 ```
 
-If a tool has the option to be used without itemization and is being so used, the `instances` array may be empty, or, as shown above, may contain one or more summary instances. Summary instances disclose the numbers of instances that they summarize with a `count` property.
+This instance describes a violation of a rule named `rule01` by a `button` element.
+
+The element has no `id` attribute to distinguish it from other `button` elements, but the tool describes its location. This tool uses an XPath to do that. Tools use various methods for location description, namely:
+- `line` (line number in the code of the page): Nu Html Checker
+- `selector` (CSS selector): Axe, QualWeb, WAVE
+- `xpath`: Alfa, ASLint, Equal Access
+- `box` (coordinates, width, and height of the element box): Editoria11y, Testaro
+- none: HTML CodeSniffer
+The tool also reproduces an excerpt of the element code.
+
+###### Element identification
+
+While the above properties can help you find the offending element, Testaro makes this easier by adding, where practical, two standard element identifiers to each standard instance:
+- `boxID`: a compact representation of the x, y, width, and height of the element bounding box, if the element can be identified and is visible.
+- `pathID`: the XPath of the element, if the element can be identified.
+
+These standard identifiers can help you determine whether violations reported by different tools belong to the same element or different elements. The `boxID` property can also support the making of images of the violating elements.
+
+Some tools limit the efficacy of the current algorithm for standard identifiers:
+- HTML CodeSniffer does not report element locations, and the reported code excerpts exclude all text content.
+- Nu Html Checker reports line and column boundaries of element start tags and truncates element text content in reported code excerpts.
+
+Testing can change the pages being tested, and such changes can cause a particular element to change its physical or logical location. In such cases, an element may appear multiple times in a tool report with different `boxID` or `pathID` values, even though it is, for practical purposes, the same element.
+
+###### Standardization configuration
+
+Each job can specify how Testaro is to handle report standardization. A job can contain a `standard` property, with one of the following values to determine which results the report will include:
+- `'also'`: original and standard.
+- `'only'`: standard only.
+- `'no'`: original only.
+
+If a tool has the option to be used without itemization and is being so used, the `instances` array may be empty, or may contain one or more summary instances. Summary instances disclose the numbers of instances that they summarize with the `count` property. They typically summarize violations by multiple elements, in which case their `id`, `location`, `excerpt`, `boxID`, and `pathID` properties will have empty values.
+
+###### Standardization opinionation
 
 This standard format reflects some judgments. For example:
-- The `ordinalSeverity` property of an instance may have required interpretation. Tools may report severity, certainty, priority, or some combination of those. They may use ordinal or metric quantifications. If they quantify ordinally, their scales may have more or fewer than 4 ranks. Testaro coerces each tool’s severity, certainty, and/or priority classification into a 4-rank ordinal classification. This classification is deemed to express the most common pattern among the tools.
+- The `ordinalSeverity` property of an instance involves interpretation. Tools may report severity, certainty, priority, or some combination of those. They may use ordinal or metric quantifications. If they quantify ordinally, their scales may have more or fewer than 4 ranks. Testaro coerces each tool’s severity, certainty, and/or priority classification into a 4-rank ordinal classification. This classification is deemed to express the most common pattern among the tools.
 - The `tagName` property of an instance may not always be obvious, because in some cases the rule being tested for requires a relationship among more than one element (e.g., “An X element may not have a Y element as its parent”).
 - The `ruleID` property of an instance is a matching rule if the tool issues a message but no rule identifier for each instance. The `nuVal` tool does this. In this case, Testaro is classifying the messages into rules.
 - The `ruleID` property of an instance may reclassify tool rules. For example, if a tool rule covers multiple situations that are dissimilar, that rule may be split into multiple rules with distinct `ruleID` properties.
@@ -905,6 +889,12 @@ To deal with the above problems, you can:
 
 Some measures of these kinds are included in the scoring and reporting features of the Testilo package.
 
+### Tool malfunctions
+
+Tools can become faulty. For example, Alfa stopped reporting any rule violations in mid-April 2024 and resumed doing so at the end of April. In some cases, such as this, the tool maker corrects the fault. In others, the tool changes and forces Testaro to change its handling of the tool.
+
+Testaro would become more reliable if the behavior of its tools were monitored for suspect changes.
+
 ## Repository exclusions
 
 The files in the `temp` directory are presumed ephemeral and are not tracked by `git`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testaro",
-  "version": "38.0.2",
+  "version": "39.0.0",
   "description": "Run 1000 web accessibility tests from 10 tools and get a standardized report",
   "main": "index.js",
   "scripts": {
@@ -37,7 +37,8 @@
     "axe-playwright": "*",
     "dotenv": "*",
     "pixelmatch": "*",
-    "playwright": "*"
+    "playwright": "*",
+    "playwright-dompath": "*"
   },
   "devDependencies": {
     "eslint": "*"

package/procs/identify.js

@@ -0,0 +1,176 @@
+/*
+  © 2024 CVS Health and/or one of its affiliates. All rights reserved.
+
+  Permission is hereby granted, free of charge, to any person obtaining a copy
+  of this software and associated documentation files (the "Software"), to deal
+  in the Software without restriction, including without limitation the rights
+  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+  copies of the Software, and to permit persons to whom the Software is
+  furnished to do so, subject to the following conditions:
+
+  The above copyright notice and this permission notice shall be included in all
+  copies or substantial portions of the Software.
+
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+  SOFTWARE.
+*/
+
+/*
+  identify.js
+  Identifies the element of a standard instance.
+*/
+
+// IMPORTS
+
+// Module to get the XPath of an element.
+const {xPath} = require('playwright-dompath');
+
+// FUNCTIONS
+
+// Returns the bounding box of a locator.
+const boxOf = exports.boxOf = async locator => {
+  try {
+    const isVisible = await locator.isVisible();
+    if (isVisible) {
+      const box = await locator.boundingBox({
+        timeout: 50
+      });
+      if (box) {
+        Object.keys(box).forEach(dim => {
+          box[dim] = Math.round(box[dim], 0);
+        });
+        return box;
+      }
+      else {
+        return null;
+      }
+    }
+    else {
+      return null;
+    }
+  }
+  catch(error) {
+    return null;
+  }
+}
+// Returns a string representation of a bounding box.
+const boxToString = exports.boxToString = box => {
+  if (box) {
+    return ['x', 'y', 'width', 'height'].map(dim => box[dim]).join(':');
+  }
+  else {
+    return '';
+  }
+};
+// Adds a box ID and a path ID to an object.
+const addIDs = async (locators, recipient) => {
+  // If there is exactly 1 of them:
+  const locatorCount = await locators.count();
+  if (locatorCount === 1) {
+    // Add the box ID of the element to the result if none exists yet.
+    if (! recipient.boxID) {
+      const box = await boxOf(locators);
+      recipient.boxID = boxToString(box);
+    }
+    // Add the path ID of the element to the result if none exists yet.
+    if (! recipient.pathID) {
+      recipient.pathID = await xPath(locators);
+    }
+  }
+};
+// Returns the XPath and box ID of the element of a standard instance.
+exports.identify = async (instance, page) => {
+  // If the instance does not yet have both boxID and pathID properties:
+  if (['boxID', 'pathID'].some(key => instance[key] === undefined)) {
+    // Initialize a result.
+    const elementID = {
+      boxID: '',
+      pathID: ''
+    };
+    const {excerpt, id, location, tagName} = instance;
+    const {type, spec} = location;
+    // If the instance specifies a CSS selector or XPath location:
+    if (['selector', 'xpath'].includes(type)) {
+      // Get a locator of the element.
+      let specifier = spec;
+      if (type === 'xpath') {
+        specifier = spec.replace(/\/text\(\)\[\d+\]$/, '');
+      }
+      if (specifier) {
+        if (type === 'xpath') {
+          specifier = `xpath=${specifier}`;
+        }
+        console.log(`Specifier is ${specifier}`);
+        try {
+          const locators = page.locator(specifier);
+          const locatorCount = await locators.count();
+          console.log(`Locator count is ${locatorCount}`);
+          // If the count of matching elements is 1:
+          if (locatorCount === 1) {
+            // Add a box ID and a path ID to the result.
+            await addIDs(locators, elementID);
+          }
+          // Otherwise, if the count is not 1 and the instance specifies an XPath location:
+          else if (type === 'xpath') {
+            // Use the XPath location as the path ID.
+            elementID.pathID = spec;
+          }
+        }
+        catch(error) {
+          console.log(`ERROR locating element by CSS selector or XPath (${error.message})`);
+        }
+      }
+    }
+    // If either ID remains undefined and the instance specifies an element ID:
+    if (id && ! (elementID.boxID && elementID.pathID)) {
+      // Get the first of the locators for elements with the ID.
+      try {
+        let locator = page.locator(`#${id.replace(/([-&;/]|^\d)/g, '\\$1')}`).first();
+        // Add a box ID and a path ID to the result.
+        await addIDs(locator, elementID);
+      }
+      catch(error) {
+        console.log(`ERROR locating element by ID (${error.message})`);
+      }
+    }
+    // If either ID remains undefined and the instance specifies a tag name:
+    if (tagName && ! (elementID.boxID && elementID.pathID)) {
+      // Get locators for elements with the tag name.
+      let locators = page.locator(tagName.toLowerCase());
+      // If there is exactly 1 of them:
+      let locatorCount = await locators.count();
+      if (locatorCount === 1) {
+        // Add a box ID and a path ID to the result.
+        await addIDs(locators, elementID);
+      }
+      // If either ID remains undefined an the instance also specifies an excerpt:
+      if (excerpt && ! (elementID.boxID && elementID.pathID)) {
+        // Get the plain text parts of the excerpt, converting ... to an empty string.
+        const minTagExcerpt = excerpt.replace(/<[^>]+>/g, '<>');
+        const plainParts = (minTagExcerpt.match(/[^<>]+/g) || [])
+        .map(part => part === '...' ? '' : part);
+        // Get the longest of them.
+        const sortedPlainParts = plainParts.sort((a, b) => b.length - a.length);
+        const mainPart = sortedPlainParts.length ? sortedPlainParts[0] : '';
+        // If there is one:
+        if (mainPart.trim().replace(/\s+/g, '').length) {
+          // Get locators for elements with the tag name and the text.
+          const locators = page.locator(tagName.toLowerCase(), {hasText: mainPart});
+          // If there is exactly 1 of them:
+          const locatorCount = await locators.count();
+          if (locatorCount === 1) {
+            // Add a box ID and a path ID to the result.
+            await addIDs(locators, elementID);
+          }
+        }
+      }
+    }
+    // Return the result (not yet getting IDs from Nu Html Checker lines and columns).
+    return elementID;
+  }
+};

package/procs/standardize.js

@@ -30,8 +30,8 @@
 // Limits the length of and unilinearizes a string.
 const cap = rawString => {
   const string = (rawString || '').replace(/[\s\u2028\u2029]+/g, ' ');
-  if (string && string.length > 400) {
-    return `${string.slice(0, 200)} ... ${string.slice(-200)}`;
+  if (string && string.length > 600) {
+    return `${string.slice(0, 300)} … ${string.slice(-300)}`;
   }
   else if (string) {
     return string;
@@ -492,12 +492,13 @@ const convert = (toolName, data, result, standardResult) => {
   else if (
     toolName === 'ed11y'
     && result
-    && ['results', 'errorCount', 'warningCount'].every(key => result[key] !== undefined)
+    && ['imageAlts', 'violations', 'errorCount', 'warningCount']
+    .every(key => result[key] !== undefined)
   ) {
-    // For each result:
-    result.results.forEach(result => {
-      const {test, content, tagName, id, loc, excerpt} = result;
-      if (['test', 'content'].every(key => result[key])) {
+    // For each violation:
+    result.violations.forEach(violation => {
+      const {test, content, tagName, id, loc, excerpt, boxID, pathID} = violation;
+      if (['test', 'content'].every(key => key)) {
         // Standardize the what property.
         let what = '';
         if (content.includes('<p>This')) {
@@ -518,7 +519,9 @@ const convert = (toolName, data, result, standardResult) => {
             type: 'box',
             spec: loc
           },
-          excerpt
+          excerpt,
+          boxID,
+          pathID
         });
       }
     });

package/procs/testaro.js

@@ -31,6 +31,10 @@
 const {getSample} = require('../procs/sample');
 // Module to get locator data.
 const {getLocatorData} = require('../procs/getLocatorData');
+// Module to get element IDs.
+const {boxOf, boxToString} = require('./identify');
+// Module to get the XPath of an element.
+const {xPath} = require('playwright-dompath');
 
 // ########## FUNCTIONS
 
@@ -80,6 +84,9 @@ const report = exports.report = async (withItems, all, ruleID, whats, ordinalSev
     totals[ordinalSeverity] += data.populationRatio;
     // If itemization is required:
     if (withItems) {
+      // Get the bounding box of the element.
+      const {location} = elData;
+      const box = location.type === 'box' ? location.spec : await boxOf(loc);
       // Add a standard instance to the result.
       standardInstances.push({
         ruleID,
@@ -88,7 +95,9 @@ const report = exports.report = async (withItems, all, ruleID, whats, ordinalSev
         tagName: elData.tagName,
         id: elData.id,
         location: elData.location,
-        excerpt: elData.excerpt
+        excerpt: elData.excerpt,
+        boxID: boxToString(box),
+        pathID: await xPath(loc)
       });
     }
   }
@@ -107,7 +116,9 @@ const report = exports.report = async (withItems, all, ruleID, whats, ordinalSev
         type: '',
         spec: ''
       },
-      excerpt: ''
+      excerpt: '',
+      boxID: '',
+      pathID: ''
     });
   }
   // Return the result.

package/run.js

@@ -33,6 +33,8 @@ require('dotenv').config();
 const {actSpecs} = require('./actSpecs');
 // Module to standardize report formats.
 const {standardize} = require('./procs/standardize');
+// Module to identify element bounding boxes.
+const {identify} = require('./procs/identify');
 // Module to send a notice to an observer.
 const {tellServer} = require('./procs/tellServer');
 
@@ -1118,6 +1120,16 @@ const doActs = async (report, actIndex, page) => {
               };
               // Populate it.
               standardize(act);
+              // Add a box ID and a path ID to each of its standard instances if missing.
+              for (const instance of act.standardResult.instances) {
+                const elementID = await identify(instance, page);
+                if (! instance.boxID) {
+                  instance.boxID = elementID ? elementID.boxID : '';
+                }
+                if (! instance.pathID) {
+                  instance.pathID = elementID ? elementID.pathID : '';
+                }
+              };
               // If the original-format result is not to be included in the report:
               if (standard === 'only') {
                 // Remove it.

package/tests/alfa.js

@@ -35,7 +35,8 @@ let alfaRules = require('@siteimprove/alfa-rules').default;
 
 // Conducts and reports the alfa tests.
 exports.reporter = async (page, options) => {
-  const {rules} = options;
+  const {act} = options;
+  const {rules} = act;
   // If only some rules are to be employed:
   if (rules && rules.length) {
     // Remove the other rules.

package/tests/ed11y.js

@@ -29,6 +29,8 @@
 
 // Module to handle files.
 const fs = require('fs/promises');
+// Module to get the XPath of an element.
+const {xPath} = require('playwright-dompath');
 
 // FUNCTIONS
 
@@ -38,21 +40,30 @@ exports.reporter = async (page, options) => {
   const {act, report} = options;
   const {jobData} = report;
   const scriptNonce = jobData && jobData.lastScriptNonce;
-  // Get the test script.
+  // Get the tool script.
   const script = await fs.readFile(`${__dirname}/../ed11y/editoria11y.min.js`, 'utf8');
-  const rawResultJSON = await page.evaluate(args => new Promise(async resolve => {
-    // Impose a timeout on obtaining a result.
+  // Run the tests and get the violating elements and violation facts.
+  const reportJSHandle = await page.evaluateHandle(args => new Promise(async resolve => {
+    // If the report is incomplete after 20 seconds:
     const timer = setTimeout(() => {
-      resolve(JSON.stringify({
-        prevented: true,
-        error: 'ed11y timed out'
-      }));
+      // Return this as the report.
+      resolve({
+        facts: {
+          prevented: true,
+          error: 'ed11y timed out'
+        }
+      });
     }, 20000);
     const {scriptNonce, script, rulesToTest} = args;
-    // When the script is executed:
+    // When the script has been executed, creating data in an Ed11y object:
     document.addEventListener('ed11yResults', () => {
-      // Get the result.
-      const resultObj = {};
+      // Initialize a report containing violating elements and violation facts.
+      const report = {
+        elements: [],
+        facts:  {}
+      };
+      const {elements, facts} = report;
+      // Populate the global facts.
       [
         'version',
         'options',
@@ -64,42 +75,45 @@ exports.reporter = async (page, options) => {
       ]
       .forEach(key => {
         try {
-          resultObj[key] = Ed11y[key];
+          facts[key] = Ed11y[key];
         }
         catch(error) {
           console.log(`ERROR: invalid value of ${key} property of Ed11y (${error.message})`);
         }
       });
-      // Get data on the text alternatives of images from the result.
-      resultObj.imageAlts = Ed11y
+      // Get data on violating text alternatives of images from Ed11y.
+      facts.imageAlts = Ed11y
       .imageAlts
       .filter(item => item[3] !== 'pass')
       .map(item => item.slice(1));
-      // Delete useless properties from the result.
-      delete resultObj.options.sleekTheme;
-      delete resultObj.options.darkTheme;
-      delete resultObj.options.lightTheme;
-      // Initialize the element results.
-      const results = resultObj.results = [];
-      // For each rule violation:
-      Ed11y.results.forEach(elResult => {
-        // If rules were not selected or they were and include this one:
-        if (! rulesToTest || rulesToTest.includes(elResult.test)) {
-          // Create a violation record.
-          const result = {};
-          result.test = elResult.test || '';
-          if (elResult.content) {
-            result.content = elResult.content.replace(/\s+/g, ' ');
+      // Delete useless facts.
+      delete facts.options.sleekTheme;
+      delete facts.options.darkTheme;
+      delete facts.options.lightTheme;
+      // Initialize the violation facts.
+      facts.violations = [];
+      // For each rule violation by an element:
+      Ed11y.results.forEach(violation => {
+        // If rules were not selected or they were and include the violated rule:
+        if (! rulesToTest || rulesToTest.includes(violation.test)) {
+          const violationFacts = {};
+          violationFacts.test = violation.test || '';
+          // If the element is in the page:
+          if (violation.content) {
+            violationFacts.content = violation.content.replace(/\s+/g, ' ');
           }
-          if (elResult.element) {
-            const{element} = elResult;
-            result.tagName = element.tagName || '';
-            result.id = element.id || '';
-            result.loc = {};
+          const {element} = violation;
+          if (element.outerHTML) {
+            // Add the element to the report.
+            elements.push(element);
+            // Add its violation facts to the report.
+            violationFacts.tagName = element.tagName || '';
+            violationFacts.id = element.id || '';
+            violationFacts.loc = {};
             const locRect = element.getBoundingClientRect();
             if (locRect) {
               ['x', 'y', 'width', 'height'].forEach(dim => {
-                result.loc[dim] = Math.round(locRect[dim]);
+                violationFacts.loc[dim] = Math.round(locRect[dim], 0);
               });
             }
             let elText = element.textContent.replace(/\s+/g, ' ').trim();
@@ -109,59 +123,65 @@ exports.reporter = async (page, options) => {
             if (elText.length > 400) {
               elText = `${elText.slice(0, 200)}…${elText.slice(-200)}`;
             }
-            result.excerpt = elText.replace(/\s+/g, ' ');
+            violationFacts.excerpt = elText.replace(/\s+/g, ' ');
+            violationFacts.boxID = ['x', 'y', 'width', 'height']
+            .map(dim => violationFacts.loc[dim])
+            .join(':');
+            facts.violations.push(violationFacts);
           }
-          // Add it to the result.
-          results.push(result);
         }
       });
-      // Return the result.
-      try {
-        const resultJSON = JSON.stringify(resultObj);
-        clearTimeout(timer);
-        resolve(resultJSON);
-      }
-      catch(error) {
-        clearTimeout(timer);
-        resolve(JSON.stringify({
-          prevented: true,
-          error: `Result object not stringified (${error.message})`
-        }));
-      }
+      // Return the report.
+      clearTimeout(timer);
+      resolve(report);
     });
-    // Add the test script to the page.
-    const testScript = document.createElement('script');
+    // Add the tool script to the page.
+    const toolScript = document.createElement('script');
     if (scriptNonce) {
-      testScript.nonce = scriptNonce;
-      console.log(`Added nonce ${scriptNonce} to script`);
+      toolScript.nonce = scriptNonce;
+      console.log(`Added nonce ${scriptNonce} to tool script`);
     }
-    testScript.textContent = script;
-    document.body.insertAdjacentElement('beforeend', testScript);
-    // Run the script.
+    toolScript.textContent = script;
+    document.body.insertAdjacentElement('beforeend', toolScript);
+    // Execute the tool script, creating Ed11y and triggering the event listener.
     try {
       await new Ed11y({
         alertMode: 'headless'
       });
     }
     catch(error) {
-      resolve(JSON.stringify({
-        prevented: true,
-        error: error.message
-      }));
+      resolve({
+        facts: {
+          prevented: true,
+          error: error.message
+        }
+      });
     };
   }), {scriptNonce, script, rulesToTest: act.rules});
-  const result = JSON.parse(rawResultJSON);
-  let data = {};
-  if (result.prevented) {
-    data.success = false;
-    data.prevented = true;
-    data.error = result.error;
-    delete result.prevented;
-    delete result.error;
+  // Add the violation facts to the result.
+  const factsJSHandle = await reportJSHandle.getProperty('facts');
+  const facts = await factsJSHandle.jsonValue();
+  const result = facts;
+  // If there were any violations:
+  const {violations} = facts;
+  if (violations && violations.length) {
+    // Get the violating elements.
+    const elementsJSHandle = await reportJSHandle.getProperty('elements');
+    const elementJSHandles = await elementsJSHandle.getProperties();
+    // For each violation:
+    for (const index in violations) {
+      // Get its path ID.
+      const elementHandle = elementJSHandles.get(index).asElement();
+      const pathID = await xPath(elementHandle);
+      // Add it to the violation facts.
+      violations[index].pathID = pathID;
+    };
   }
-  // Return the act report.
+  // Return the report.
   return {
-    data,
+    data: {
+      prevented: facts.prevented
+    },
     result
   };
 };

package/tests/nuVal.js

@@ -22,7 +22,7 @@
 
 /*
   nuVal
-  This test subjects a page and its source to the Nu Html Checker, thereby testing scripted
+  This tool subjects a page and its source to the Nu Html Checker, thereby testing scripted
   content found only in the loaded page and erroneous content before the browser corrects it.
   The API erratically replaces left and right double quotation marks with invalid UTF-8, which
   appears as 2 or 3 successive instances of the replacement character (U+fffd). Therefore, this