testaro 3.0.0 → 4.0.1

package/README.md

@@ -10,16 +10,6 @@ The purpose of Testaro is to provide programmatic access to over 600 accessibili
 
 Running Testaro requires telling it which operations (including tests) to perform and which URLs to perform them on, and giving Testaro an object to put its output into.
 
-## Origin
-
-Work on the custom tests in this package began in 2017, and work on the multi-package federation that Testaro implements began in early 2018. These two aspects were combined into the [Autotest](https://github.com/jrpool/autotest) package in early 2021 and into this more limited-purpose package, Testaro, in January 2022.
-
-Testaro omits some functionalities of Autotest, such as:
-- tests producing results intended to be human-inspected
-- previous versions of scoring algorithms
-- file operations for score aggregation, report revision, and HTML reports
-- a web user interface
-
 ## System requirements
 
 Version 14 or later of [Node.js](https://nodejs.org/en/).
@@ -35,6 +25,7 @@ Testaro includes some of its own accessibility tests. In addition, it performs t
 - [alfa](https://alfa.siteimprove.com/) (Siteimprove alfa)
 - [Automated Accessibility Testing Tool](https://www.npmjs.com/package/aatt) (Paypal AATT, running HTML CodeSniffer)
 - [axe-playwright](https://www.npmjs.com/package/axe-playwright) (Deque Axe-core)
+- [Tenon](https://tenon.io/documentation/what-tenon-tests.php)
 - [WAVE API](https://wave.webaim.org/api/) (WebAIM WAVE)
 
 As of this version, the counts of tests in the packages referenced above were:
@@ -42,14 +33,11 @@ As of this version, the counts of tests in the packages referenced above were:
 - Alfa: 103
 - Axe-core: 138
 - Equal Access: 163
+- Tenon: 180
 - WAVE: 110
 - subtotal: 612
 - Testaro tests: 16
-- grand total: 628
-
-## Related packages
-
-[Testilo](https://github.com/jrpool/testilo) is an application that facilitates the use of Testaro.
+- grand total: 808
 
 ## Code organization
 
@@ -266,6 +254,24 @@ An example of a **Testaro-defined** test is:
 
 In this case, Testaro runs the `motion` test with the specified parameters.
 
+###### Tenon
+
+The `tenon` test requires two commands:
+- A command of type `tenonRequest`.
+- A command of type `test` with `tenon` as the value of `which`.
+
+The reason for this is that the Tenon API operates asynchronously. You ask it to perform a test, and it puts your request into a queue. To learn whether Tenon has completed your test, you make a status request. You can continue making status requests until Tenon replies that your test has been completed. Then you submit a request for the test result, and Tenon replies with the result. (As of May 2022, status requests were observed to misreport still-running tests as completed. The `tenon` test works around that.)
+
+Tenon says that tests are typically completed in 3 to 6 seconds but that the latency can be longer, depending on demand.
+
+Therefore, you can include a `tenonRequest` command early in your script, and a `tenon` test late in your script. Tenon will move your request through its queue while Testaro is processing your script. When Testaro reaches your `tenon` test command, Tenon will most likely have completed your test. If not, the `tenon` test will wait and then make a second request before giving up.
+
+Thus, a `tenon` test actually does not perform any test; it merely collects the result. The page that was active when the `tenonRequest` command was performed is the one that Tenon tests.
+
+In case you want to perform more than one `tenon` test, you can do so. Just give each pair of commands a distinct `id` property, so each `tenon` test command will request the correct result.
+
+Tenon recommends giving it a public URL rather than giving it the content of a page, if possible. So, it is best to give the `withNewContent` property of the `tenonRequest` command the value `true`, unless the page is not public.
+
 ##### Scoring
 
 An example of a **scoring** command is:
@@ -424,8 +430,14 @@ Another way to run Testaro is to use Testilo, which can handle batches and saves
 
 If a `wave` test is included in the script, an environment variable named `TESTARO_WAVE_KEY` must exist, with your WAVE API key as its value.
 
+If a `tenon` test is included in the script, environment variables named `TESTARO_TENON_USER` and `TESTARO_TENON_PASSWORD` must exist, with your Tenon username and password, respectively, as their values.
+
+The `text` command can interpolate the value of an environment variable into text that it enters on a page, as documented in the `commands.js` file.
+
 Before executing a Testaro script, you can optionally also set the environment variables `TESTARO_DEBUG` (to `'true'` or anything else) and/or `TESTARO_WAITS` (to a non-negative integer). The effects of these variables are described in the `index.js` file.
 
+You may store these environment variables in an untracked `.env` file if you wish, and Testaro will recognize them.
+
 ## Validation
 
 _Executors_ for Testaro validation are located in the `validation` directory.
@@ -450,18 +462,6 @@ You can define additional Testaro commands and functionality. Contributions are
 
 The rationales motivating the Testaro-defined tests and scoring procs can be found in comments within the files of those tests and procs, in the `tests` and `procs/score` directories. Unavoidably, each test is opinionated. Testaro itself, however, can accommodate other tests representing different opinions. Testaro is intended to be neutral with respect to questions such as the criteria for accessibility, the severities of accessibility issues, whether accessibility is binary or graded, and the distinction between usability and accessibility.
 
-### Future work
-
-Further development is contemplated, is taking place, or is welcomed, on:
-- addition of Tenon to the set of packages
-- links with href="#"
-- links and buttons styled non-distinguishably
-- first focused element not first focusable element in DOM
-- never-visible skip links
-- buttons with no text content
-- modal dialogs
-- autocomplete attributes
-
 ## Testing challenges
 
 ### Activation
@@ -478,9 +478,11 @@ Test packages sometimes do redundant testing, in that two or more packages test
 
 The files in the `temp` directory are presumed ephemeral and are not tracked by `git`. When tests require temporary files to be written, Testaro writes them there.
 
-## Origin
+## Related packages
 
-Testaro is derived from [Autotest](https://github.com/jrpool/autotest), which in turn is derived from accessibility test investigations beginning in 2018.
+[Testilo](https://www.npmjs.com/package/testilo) is an application that facilitates the use of Testaro.
+
+Testaro is derived from [Autotest](https://github.com/jrpool/autotest).
 
 Testaro omits some functionalities of Autotest, such as:
 - tests producing results intended to be human-inspected
@@ -488,6 +490,44 @@ Testaro omits some functionalities of Autotest, such as:
 - file operations for score aggregation, report revision, and HTML reports
 - a web user interface
 
+## Origin
+
+Work on the custom tests in this package began in 2017, and work on the multi-package federation that Testaro implements began in early 2018. These two aspects were combined into the [Autotest](https://github.com/jrpool/autotest) package in early 2021 and into this more single-purpose package, Testaro, in January 2022.
+
 ## Etymology
 
 “Testaro” means “collection of tests” in Esperanto.
+
+## Future work
+
+### Improvements
+
+Further development is contemplated, is taking place, or is welcomed, on:
+- addition of Tenon to the set of packages
+- links with href="#"
+- links and buttons styled non-distinguishably
+- first focused element not first focusable element in DOM
+- never-visible skip links
+- buttons with no text content
+- modal dialogs
+- autocomplete attributes
+- inclusion of other test packages, such as:
+   - FAE (https://github.com/opena11y/evaluation-library)
+   - Tenon
+
+## Corrections
+
+Issues found or reported with the current version that need diagnosis and correction include:
+
+### hover
+
+There seem to be a couple of problems with the hover test:
+- The score for unhoverability is documented as 2 times the count of unhoverables, but is reported as 1 time that count.
+- The list of unhoverables in the report is empty.
+Observed after inquiry by Tobias Christian Jensen of Siteimprove on 2022-05-09.
+
+### axe
+
+Configuration to include best practices and experimental tests.
+
+Investigation of tags, including wcag2a, wcag2aa, wcag21a, wcag21aa, best-practice, wcag***, ACT, cat.*.

package/commands.js

@@ -118,6 +118,14 @@ exports.commands = {
         what: [false, 'string', 'hasLength', 'comment']
       }
     ],
+    tenonRequest: [
+      'Request a Tenon test',
+      {
+        id: [true, 'string', 'hasLength', 'ID for this test instance'],
+        withNewContent: [true, 'boolean', '', 'true: use a URL; false: use page content'],
+        what: [false, 'string', 'hasLength', 'comment']
+      }
+    ],
     text: [
       'Enter text into a text input, optionally with 1 placeholder for an all-caps literal environment variable',
       {
@@ -185,7 +193,7 @@ exports.commands = {
       {
         withItems: [true, 'boolean'],
         withNewContent: [
-          false, 'boolean', '', 'true: use a URL; false: use page content; omitted: use both'
+          false, 'boolean', '', 'true: use a URL; false: use page content; omitted: both'
         ]
       }
     ],
@@ -233,6 +241,12 @@ exports.commands = {
         withItems: [true, 'boolean']
       }
     ],
+    tenon: [
+      'Perform a Tenon test',
+      {
+        id: [true, 'string', 'hasLength', 'ID of the requested test instance']
+      }
+    ],
     wave: [
       'Perform a WebAIM WAVE test',
       {

package/index.js

@@ -3,6 +3,8 @@
   testaro main script.
 */
 // ########## IMPORTS
+// Module to keep secrets.
+require('dotenv').config();
 // Requirements for commands.
 const {commands} = require('./commands');
 // ########## CONSTANTS
@@ -40,6 +42,7 @@ const tests = {
   role: 'roles',
   styleDiff: 'style inconsistencies',
   tabNav: 'keyboard navigation between tab elements',
+  tenon: 'Tenon',
   wave: 'WAVE',
   zIndex: 'z indexes'
 };
@@ -55,6 +58,11 @@ const browserTypeNames = {
 };
 // Items that may be waited for.
 const waitables = ['url', 'title', 'body'];
+// Tenon data.
+const tenonData = {
+  accessToken: '',
+  requestIDs: {}
+};
 // ########## VARIABLES
 // Facts about the current session.
 let logCount = 0;
@@ -359,7 +367,7 @@ const textOf = async (page, element) => {
 const matchElement = async (page, selector, matchText, index = 0) => {
   // If the page still exists:
   if (page) {
-    // Wait 3 seconds until the body contains any text to be matched.
+    // Wait 2 seconds until the body contains any text to be matched.
     const slimText = debloat(matchText);
     const bodyText = await page.textContent('body');
     const slimBody = debloat(bodyText);
@@ -706,157 +714,125 @@ const doActs = async (report, actIndex, page) => {
               await require('./procs/test/allVis').allVis(page);
               act.result = 'All elements visible.';
             }
-            // Otherwise, if it is a repetitive keyboard navigation:
-            else if (act.type === 'presses') {
-              const {navKey, what, which, withItems} = act;
-              const matchTexts = which ? which.map(text => debloat(text)) : [];
-              // Initialize the loop variables.
-              let status = 'more';
-              let presses = 0;
-              let amountRead = 0;
-              let items = [];
-              let matchedText;
-              // As long as a matching element has not been reached:
-              while (status === 'more') {
-                // Press the Escape key to dismiss any modal dialog.
-                await page.keyboard.press('Escape');
-                // Press the specified navigation key.
-                await page.keyboard.press(navKey);
-                presses++;
-                // Identify the newly current element or a failure.
-                const currentJSHandle = await page.evaluateHandle(actCount => {
-                  // Initialize it as the focused element.
-                  let currentElement = document.activeElement;
-                  // If it exists in the page:
-                  if (currentElement && currentElement.tagName !== 'BODY') {
-                    // Change it, if necessary, to its active descendant.
-                    if (currentElement.hasAttribute('aria-activedescendant')) {
-                      currentElement = document.getElementById(
-                        currentElement.getAttribute('aria-activedescendant')
-                      );
-                    }
-                    // Or change it, if necessary, to its selected option.
-                    else if (currentElement.tagName === 'SELECT') {
-                      const currentIndex = Math.max(0, currentElement.selectedIndex);
-                      const options = currentElement.querySelectorAll('option');
-                      currentElement = options[currentIndex];
-                    }
-                    // Or change it, if necessary, to its active shadow-DOM element.
-                    else if (currentElement.shadowRoot) {
-                      currentElement = currentElement.shadowRoot.activeElement;
-                    }
-                    // If there is a current element:
-                    if (currentElement) {
-                      // If it was already reached within this command performance:
-                      if (currentElement.dataset.pressesReached === actCount.toString(10)) {
-                        // Report the error.
-                        console.log(`ERROR: ${currentElement.tagName} element reached again`);
-                        status = 'ERROR';
-                        return 'ERROR: locallyExhausted';
-                      }
-                      // Otherwise, i.e. if it is newly reached within this act:
-                      else {
-                        // Mark and return it.
-                        currentElement.dataset.pressesReached = actCount;
-                        return currentElement;
-                      }
-                    }
-                    // Otherwise, i.e. if there is no current element:
-                    else {
-                      // Report the error.
-                      status = 'ERROR';
-                      return 'noActiveElement';
-                    }
-                  }
-                  // Otherwise, i.e. if there is no focus in the page:
-                  else {
-                    // Report the error.
-                    status = 'ERROR';
-                    return 'ERROR: globallyExhausted';
-                  }
-                }, actCount);
-                // If the current element exists:
-                const currentElement = currentJSHandle.asElement();
-                if (currentElement) {
-                  // Update the data.
-                  const tagNameJSHandle = await currentElement.getProperty('tagName');
-                  const tagName = await tagNameJSHandle.jsonValue();
-                  const text = await textOf(page, currentElement);
-                  // If the text of the current element was found:
-                  if (text !== null) {
-                    const textLength = text.length;
-                    // If it is non-empty and there are texts to match:
-                    if (matchTexts.length && textLength) {
-                      // Identify the matching text.
-                      matchedText = matchTexts.find(matchText => text.includes(matchText));
-                    }
-                    // Update the item data if required.
-                    if (withItems) {
-                      const itemData = {
-                        tagName,
-                        text,
-                        textLength
-                      };
-                      if (matchedText) {
-                        itemData.matchedText = matchedText;
+            // Otherwise, if the act is a tenon request:
+            else if (act.type === 'tenonRequest') {
+              const {id, withNewContent} = act;
+              const https = require('https');
+              // If a Tenon access token has not yet been obtained:
+              if (! tenonData.accessToken) {
+                // Authenticate with the Tenon API.
+                const authData = await new Promise(resolve => {
+                  const request = https.request(
+                    {
+                      host: 'tenon.io',
+                      path: '/api/v2/auth',
+                      port: 443,
+                      protocol: 'https:',
+                      method: 'POST',
+                      headers: {
+                        'Content-Type': 'application/json',
+                        'Cache-Control': 'no-cache'
                       }
-                      items.push(itemData);
-                    }
-                    amountRead += textLength;
-                    // If there is no text-match failure:
-                    if (matchedText || ! matchTexts.length) {
-                      // If the element has any specified tag name:
-                      if (! what || tagName === what) {
-                        // Change the status.
-                        status = 'done';
-                        // Perform the action.
-                        const inputText = act.text;
-                        if (inputText) {
-                          await page.keyboard.type(inputText);
-                          presses += inputText.length;
+                    },
+                    response => {
+                      let responseData = '';
+                      response.on('data', chunk => {
+                        responseData += chunk;
+                      });
+                      response.on('end', () => {
+                        try {
+                          const responseJSON = JSON.parse(responseData);
+                          return resolve(responseJSON);
                         }
-                        if (act.action) {
-                          presses++;
-                          await page.keyboard.press(act.action);
-                          await page.waitForLoadState();
+                        catch(error) {
+                          return resolve({
+                            error: 'Tenon did not return JSON authentication data.',
+                            responseData
+                          });
                         }
-                      }
+                      });
                     }
-                  }
-                  else {
-                    status = 'ERROR';
-                  }
+                  );
+                  const tenonUser = process.env.TESTARO_TENON_USER;
+                  const tenonPassword = process.env.TESTARO_TENON_PASSWORD;
+                  const postData = JSON.stringify({
+                    username: tenonUser,
+                    password: tenonPassword
+                  });
+                  request.write(postData);
+                  request.end();
+                });
+                // If the authentication succeeded:
+                if (authData.access_token) {
+                  // Record the access token.
+                  tenonData.accessToken = authData.access_token;
                 }
-                // Otherwise, i.e. if there was a failure:
+                // Otherwise, i.e. if the authentication failed:
                 else {
-                  // Update the status.
-                  status = await currentJSHandle.jsonValue();
+                  console.log('ERROR: tenon authentication failed');
                 }
               }
-              // Add the result to the act.
-              act.result = {
-                status,
-                totals: {
-                  presses,
-                  amountRead
+              // If a Tenon access token exists:
+              if (tenonData.accessToken) {
+                // Request a Tenon test of the page and get a response ID.
+                const option = {};
+                // If Tenon is to be given the URL and not the content of the page:
+                if (withNewContent) {
+                  // Specify this.
+                  option.url = page.url();
                 }
-              };
-              if (status === 'done' && matchedText) {
-                act.result.matchedText = matchedText;
-              }
-              if (withItems) {
-                act.result.items = items;
+                // Otherwise, i.e. if Tenon is to be given the page content:
+                else {
+                  // Specify this.
+                  option.src = await page.content();
+                }
+                // Request a Tenon test and get a response ID.
+                const responseID = await new Promise(resolve => {
+                  const request = https.request(
+                    {
+                      host: 'tenon.io',
+                      path: '/api/v2/',
+                      port: 443,
+                      protocol: 'https:',
+                      method: 'POST',
+                      headers: {
+                        'Content-Type': 'application/json',
+                        'Cache-Control': 'no-cache',
+                        Authorization: `Bearer ${tenonData.accessToken}`
+                      }
+                    },
+                    response => {
+                      let resultJSON = '';
+                      response.on('data', chunk => {
+                        resultJSON += chunk;
+                      });
+                      // When the data arrive, return them as an object.
+                      response.on('end', () => {
+                        try {
+                          const result = JSON.parse(resultJSON);
+                          resolve(result.data.responseID || '');
+                        }
+                        catch (error) {
+                          console.log('ERROR: Tenon did not return JSON.');
+                          resolve('');
+                        }
+                      });
+                    }
+                  );
+                  const postData = JSON.stringify(option);
+                  request.write(postData);
+                  request.end();
+                });
+                // Record the response ID.
+                tenonData.requestIDs[id] = responseID || '';
               }
-              // Add the totals to the report.
-              report.presses += presses;
-              report.amountRead += amountRead;
             }
             // Otherwise, if the act is a test:
             else if (act.type === 'test') {
               // Add a description of the test to the act.
               act.what = tests[act.which];
               // Initialize the arguments.
-              const args = [page];
+              const args = [act.which === 'tenon' ? tenonData : page];
               // Identify the additional validator of the test.
               const testValidator = commands.tests[act.which];
               // If it exists:
@@ -1025,6 +1001,151 @@ const doActs = async (report, actIndex, page) => {
               const qualifier = act.again ? `${1 + act.again} times` : 'once';
               act.result = `pressed ${qualifier}`;
             }
+            // Otherwise, if it is a repetitive keyboard navigation:
+            else if (act.type === 'presses') {
+              const {navKey, what, which, withItems} = act;
+              const matchTexts = which ? which.map(text => debloat(text)) : [];
+              // Initialize the loop variables.
+              let status = 'more';
+              let presses = 0;
+              let amountRead = 0;
+              let items = [];
+              let matchedText;
+              // As long as a matching element has not been reached:
+              while (status === 'more') {
+                // Press the Escape key to dismiss any modal dialog.
+                await page.keyboard.press('Escape');
+                // Press the specified navigation key.
+                await page.keyboard.press(navKey);
+                presses++;
+                // Identify the newly current element or a failure.
+                const currentJSHandle = await page.evaluateHandle(actCount => {
+                  // Initialize it as the focused element.
+                  let currentElement = document.activeElement;
+                  // If it exists in the page:
+                  if (currentElement && currentElement.tagName !== 'BODY') {
+                    // Change it, if necessary, to its active descendant.
+                    if (currentElement.hasAttribute('aria-activedescendant')) {
+                      currentElement = document.getElementById(
+                        currentElement.getAttribute('aria-activedescendant')
+                      );
+                    }
+                    // Or change it, if necessary, to its selected option.
+                    else if (currentElement.tagName === 'SELECT') {
+                      const currentIndex = Math.max(0, currentElement.selectedIndex);
+                      const options = currentElement.querySelectorAll('option');
+                      currentElement = options[currentIndex];
+                    }
+                    // Or change it, if necessary, to its active shadow-DOM element.
+                    else if (currentElement.shadowRoot) {
+                      currentElement = currentElement.shadowRoot.activeElement;
+                    }
+                    // If there is a current element:
+                    if (currentElement) {
+                      // If it was already reached within this command performance:
+                      if (currentElement.dataset.pressesReached === actCount.toString(10)) {
+                        // Report the error.
+                        console.log(`ERROR: ${currentElement.tagName} element reached again`);
+                        status = 'ERROR';
+                        return 'ERROR: locallyExhausted';
+                      }
+                      // Otherwise, i.e. if it is newly reached within this act:
+                      else {
+                        // Mark and return it.
+                        currentElement.dataset.pressesReached = actCount;
+                        return currentElement;
+                      }
+                    }
+                    // Otherwise, i.e. if there is no current element:
+                    else {
+                      // Report the error.
+                      status = 'ERROR';
+                      return 'noActiveElement';
+                    }
+                  }
+                  // Otherwise, i.e. if there is no focus in the page:
+                  else {
+                    // Report the error.
+                    status = 'ERROR';
+                    return 'ERROR: globallyExhausted';
+                  }
+                }, actCount);
+                // If the current element exists:
+                const currentElement = currentJSHandle.asElement();
+                if (currentElement) {
+                  // Update the data.
+                  const tagNameJSHandle = await currentElement.getProperty('tagName');
+                  const tagName = await tagNameJSHandle.jsonValue();
+                  const text = await textOf(page, currentElement);
+                  // If the text of the current element was found:
+                  if (text !== null) {
+                    const textLength = text.length;
+                    // If it is non-empty and there are texts to match:
+                    if (matchTexts.length && textLength) {
+                      // Identify the matching text.
+                      matchedText = matchTexts.find(matchText => text.includes(matchText));
+                    }
+                    // Update the item data if required.
+                    if (withItems) {
+                      const itemData = {
+                        tagName,
+                        text,
+                        textLength
+                      };
+                      if (matchedText) {
+                        itemData.matchedText = matchedText;
+                      }
+                      items.push(itemData);
+                    }
+                    amountRead += textLength;
+                    // If there is no text-match failure:
+                    if (matchedText || ! matchTexts.length) {
+                      // If the element has any specified tag name:
+                      if (! what || tagName === what) {
+                        // Change the status.
+                        status = 'done';
+                        // Perform the action.
+                        const inputText = act.text;
+                        if (inputText) {
+                          await page.keyboard.type(inputText);
+                          presses += inputText.length;
+                        }
+                        if (act.action) {
+                          presses++;
+                          await page.keyboard.press(act.action);
+                          await page.waitForLoadState();
+                        }
+                      }
+                    }
+                  }
+                  else {
+                    status = 'ERROR';
+                  }
+                }
+                // Otherwise, i.e. if there was a failure:
+                else {
+                  // Update the status.
+                  status = await currentJSHandle.jsonValue();
+                }
+              }
+              // Add the result to the act.
+              act.result = {
+                status,
+                totals: {
+                  presses,
+                  amountRead
+                }
+              };
+              if (status === 'done' && matchedText) {
+                act.result.matchedText = matchedText;
+              }
+              if (withItems) {
+                act.result.items = items;
+              }
+              // Add the totals to the report.
+              report.presses += presses;
+              report.amountRead += amountRead;
+            }
             // Otherwise, i.e. if the act type is unknown:
             else {
               // Add the error result to the act.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testaro",
-  "version": "3.0.0",
+  "version": "4.0.1",
   "description": "Automation of accessibility testing",
   "main": "index.js",
   "scripts": {
@@ -30,6 +30,7 @@
     "aatt": "*",
     "accessibility-checker": "*",
     "axe-playwright": "*",
+    "dotenv": "*",
     "pixelmatch": "*",
     "playwright": "*"
   },

package/samples/scripts/simple.json

@@ -1,5 +1,5 @@
 {
-  "what": "Test example.com with alfa",
+  "what": "Test example.com with bulk",
   "strict": true,
   "commands": [
     {
@@ -14,8 +14,7 @@
     },
     {
       "type": "test",
-      "which": "alfa",
-      "what": "Siteimprove alfa package"
+      "which": "bulk"
     }
   ]
 }

package/samples/scripts/tenon.json

@@ -0,0 +1,28 @@
+{
+  "what": "Test Wikipedia with tenon",
+  "strict": true,
+  "commands": [
+    {
+      "type": "launch",
+      "which": "chromium",
+      "what": "Chromium browser"
+    },
+    {
+      "type": "url",
+      "which": "https://en.wikipedia.org/wiki/Main_Page",
+      "what": "Wikipedia English home page"
+    },
+    {
+      "type": "tenonRequest",
+      "withNewContent": true,
+      "id": "a",
+      "what": "tenon request"
+    },
+    {
+      "type": "test",
+      "which": "tenon",
+      "id": "a",
+      "what": "result of prior tenon request"
+    }
+  ]
+}

package/tests/ibm.js

@@ -50,28 +50,54 @@ const report = (result, withItems) => {
   }
   return data;
 };
-const all = {};
-// Returns results of an IBM test.
+// Performs an IBM test.
+const doTest = async (content, withItems, timeLimit) => {
+  // Start a timeout clock.
+  let timeoutID;
+  const wait = new Promise(resolve => {
+    timeoutID = setTimeout(() => {
+      resolve('');
+    }, 1000 * timeLimit);
+  });
+  // Conduct and report the test.
+  const result = run(content);
+  // Wait for the report until the time limit expires.
+  const resultIfFast = await Promise.race([result, wait]);
+  // Delete the report files.
+  try {
+    const reportNames = await fs.readdir('results');
+    for (const reportName of reportNames) {
+      await fs.rm(`results/${reportName}`);
+    }
+  }
+  catch(error) {
+    console.log('ibm test created no result files.');
+  }
+  // Return the result.
+  if (resultIfFast) {
+    clearTimeout(timeoutID);
+    const typeResult = report(result, withItems);
+    return typeResult;
+  }
+  else {
+    console.log('ERROR: getting ibm test report took too long');
+    return 'ERROR: getting ibm test report took too long';
+  }
+};
+// Returns results of one or two IBM tests.
 exports.reporter = async (page, withItems, withNewContent) => {
-  // If the test is to be conducted with existing content:
+  // If a test with existing content is to be performed:
+  const result = {};
   if (! withNewContent) {
-    // Conduct and report it.
-    const content = await page.content();
-    const result = await run(content);
-    all.content = report(result, withItems);
+    const timeLimit = 15;
+    const typeContent = await page.content();
+    result.content = await doTest(typeContent, withItems, timeLimit);
   }
-  // If the test is to be conducted with a URL:
+  // If a test with new content is to be performed:
   if ([true, undefined].includes(withNewContent)) {
-    // Conduct and report it.
-    const content = page.url();
-    const result = await run(content);
-    all.url = report(result, withItems);
+    const timeLimit = 20;
+    const typeContent = page.url();
+    result.url = await doTest(typeContent, withItems, timeLimit);
   }
-  // Delete the report files.
-  const reportNames = await fs.readdir('results');
-  for (const reportName of reportNames) {
-    await fs.rm(`results/${reportName}`);
-  }
-  // Return the result.
-  return {result: all};
+  return {result};
 };

package/tests/tenon.js

@@ -0,0 +1,123 @@
+/*
+  tenon
+  This test processes a previously requested test by the Tenon API.
+*/
+const https = require('https');
+// Wait until a time limit in seconds expires.
+const wait = timeLimit => new Promise(resolve => setTimeout(resolve, 1000 * timeLimit));
+exports.reporter = async (tenonData, id) => {
+  if (tenonData && tenonData.accessToken && tenonData.requestIDs && tenonData.requestIDs[id]) {
+    // Shared request options.
+    const requestOptions = {
+      host: 'tenon.io',
+      path: `/api/v2/${tenonData.requestIDs[id]}`,
+      port: 443,
+      protocol: 'https:',
+      headers: {
+        Authorization: `Bearer ${tenonData.accessToken}`,
+        'Cache-Control': 'no-cache'
+      }
+    };
+    // Gets the test status.
+    const getStatus = async () => {
+      const testStatus = await new Promise((resolve, reject) => {
+        requestOptions.method = 'HEAD';
+        const statusRequest = https.request(requestOptions, statusResponse => {
+          const {statusCode} = statusResponse;
+          resolve(statusCode);
+        });
+        statusRequest.on('error', error => {
+          console.log(`ERROR getting Tenon test status (${error.message})`);
+          reject(`ERROR getting Tenon test status (${error.message})`);
+        });
+        statusRequest.end();
+      });
+      return testStatus;
+    };
+    // Gets the test result.
+    const getResult = async () => {
+      const testResult = await new Promise(resolve => {
+        requestOptions.method = 'GET';
+        const resultRequest = https.request(requestOptions, resultResponse => {
+          let resultJSON = '';
+          resultResponse.on('data', chunk => {
+            resultJSON += chunk;
+          });
+          resultResponse.on('end', () => {
+            try {
+              const result = JSON.parse(resultJSON);
+              resolve(result);
+            }
+            catch(error) {
+              console.log(`ERROR getting Tenon test result (${resultJSON.slice(0, 80)} …)`);
+              resolve({
+                error: 'ERROR getting Tenon test result',
+                resultStart: resultJSON.slice(0, 80)
+              });
+            }
+          });
+        });
+        resultRequest.end();
+      });
+      return testResult;
+    };
+    // Get the test status (not reliable: may say 200 instead of 202).
+    let testStatus = await getStatus();
+    // If the test is still in the Tenon queue:
+    if (testStatus === 202) {
+      // Wait 5 seconds.
+      await wait(5);
+      // Get the test status again.
+      testStatus = await getStatus();
+    }
+    // If the test has allegedly been completed:
+    if (testStatus ===  200) {
+      // Get the test result.
+      let testResult = await getResult();
+      // If the test is still in the Tenon queue:
+      let {status} = testResult;
+      if (status === 202) {
+        // Wait 5 seconds.
+        await wait(5);
+        // Get the test result again.
+        testResult = await getResult();
+        // If the test is still in the Tenon queue:
+        status = testResult.status;
+        if (status === 202) {
+          // Wait 15 more seconds.
+          await wait(15);
+          // Get the test result again.
+          testResult = await getResult();
+          status = testResult.status;
+        }
+      }
+      // If the test has really been completed:
+      if (status === 200) {
+        // Return its result.
+        return {result: testResult};
+      }
+      // Otherwise, i.e. if the test is still running or failed:
+      else {
+        return {result: {
+          error: 'ERROR: Tenon result not retrieved',
+          status
+        }};
+      }
+    }
+    // Otherwise, if the test is still running after a wait for its status:
+    else {
+      // Report the test status.
+      return {result: {
+        error: 'ERROR: test status not completed',
+        testStatus
+      }};
+    }
+  }
+  else {
+    return {
+      result: {
+        error: 'ERROR: tenon authorization and test data incomplete'
+      }
+    };
+  }
+};

package/validation/executors/tenon.js

@@ -0,0 +1,18 @@
+// tenon.js
+// Test executor for tenon sample script.
+
+const fs = require('fs');
+const {handleRequest} = require('../../index');
+const scriptJSON = fs.readFileSync('samples/scripts/tenon.json', 'utf8');
+const script = JSON.parse(scriptJSON);
+const report = {
+  id: '',
+  script,
+  log: [],
+  acts: []
+};
+(async () => {
+  await handleRequest(report);
+  console.log(`Report log:\n${JSON.stringify(report.log, null, 2)}\n`);
+  console.log(`Report acts:\n${JSON.stringify(report.acts, null, 2)}`);
+})();