testaro 5.7.0 → 5.7.4

package/README.md

@@ -93,6 +93,7 @@ A script is a JSON file with the properties:
 
 ```json
 {
+  "id": "string consisting of lower-case ASCII letters and digits",
   "what": "string: description of the script",
   "strict": "boolean: whether redirections should be treated as failures",
   "timeLimit": "number: limit in seconds on the execution of this script",
@@ -100,32 +101,33 @@ A script is a JSON file with the properties:
 }
 ```
 
-The `timeLimit` property is optional. If it is omitted, a default of 300 seconds (5 minutes) is set.
+The `timeLimit` property is optional. If it is omitted, a default of 300 seconds (5 minutes) is set. 
 
 ### Example
 
 Here is an example of a script:
 
-```javascript
+```json
 {
-  what: 'Test example.com with alfa',
+  "id": "samplescript",
+  what: "Test example.com with alfa",
   strict: true,
-  timeLimit: 15,
+  timeLimit: 65,
   commands: [
     {
-      type: 'launch',
-      which: 'chromium',
-      what: 'Chromium browser'
+      type: "launch",
+      which: "chromium",
+      what: "Chromium browser"
     },
     {
-      type: 'url',
-      which: 'https://example.com/',
-      what: 'page with a few accessibility defects'
+      type: "url",
+      which: "https://example.com/",
+      what: "page with a few accessibility defects"
     },
     {
-      type: 'test',
-      which: 'alfa',
-      what: 'Siteimprove alfa package'
+      type: "test",
+      which: "alfa",
+      what: "Siteimprove alfa package"
     }
   ]
 }
@@ -302,6 +304,10 @@ In case you want to perform more than one `tenon` test with the same script, you
 
 Tenon recommends giving it a public URL rather than giving it the content of a page, if possible. So, it is best to give the `withNewContent` property of the `tenonRequest` command the value `true`, unless the page is not public.
 
+###### Continuum
+
+The `continuum` test makes use of the files in the `continuum` directory. The test inserts the contents of all three files into the page as scripts and then uses them to perform the tests of the Continuum package.
+
 ###### HTML CodeSniffer
 
 The `htmlcs` test makes use of the`htmlcs/HTMLCS.js` file. That file was created, and can be recreated if necessary, as follows:
@@ -336,7 +342,7 @@ The changes in `htmlcs/HTMLCS.js` are:
 
 ###### BBC Accessibility Standards Checker
 
-The BBC Accessibility Standards Checker has obsolete dependencies with security vulnerabilities. Therefore, it is not used as a dependency of Testaro. Instead, 6 of its tests were reimplemented, in some case with revisions, as Testaro tests. They were drawn from the 18 automated tests of the Checker. The other 12 tests were found too duplicative of other tests to justify reimplementation.
+The BBC Accessibility Standards Checker has obsolete dependencies with security vulnerabilities. Therefore, it is not used as a dependency of Testaro. Instead, 6 of its tests are reimplemented, in some case with revisions, as Testaro tests. They are drawn from the 18 automated tests of the Checker. The other 12 tests were found too duplicative of other tests to justify reimplementation.
 
 ##### Branching
 
@@ -436,18 +442,18 @@ A typical use for an `expect` property is checking the correctness of a Testaro
 
 You may wish to have Testaro perform the same sequence of tests on multiple web pages. In that case, you can create a _batch_, with the following structure:
 
-```javascript
+```json
 {
-  what: 'Web leaders',
+  what: "Web leaders",
   hosts: {
-    id: 'w3c',
-    which: 'https://www.w3.org/',
-    what: 'W3C'
+    id: "w3c",
+    which: "https://www.w3.org/",
+    what: "W3C"
   },
   {
-    id: 'wikimedia',
-    which: 'https://www.wikimedia.org/',
-    what: 'Wikimedia'
+    id: "wikimedia",
+    which: "https://www.wikimedia.org/",
+    what: "Wikimedia"
   }
 }
 ```
@@ -487,7 +493,7 @@ The argument of `require` is a path relative to the directory of the module in w
 
 Another Node.js package that has Testaro as a dependency can execute the same statements, except changing `'./run'` to `'testaro/run'`.
 
-Testaro will run the script and populate the `log` and `acts` arrays of the `report` object. When Testaro finishes, the `log` and `acts` properties will contain the results. The final statement can further process the `report` object as desired in the `then` callback.
+Testaro will run the script and modify the properties of the `report` object. When Testaro finishes, the `log`, `acts`, and other properties of `report` will contain the results. The final statement can further process the `report` object as desired in the `then` callback.
 
 #### High-level
 
@@ -500,7 +506,7 @@ Relative paths must be relative to the Testaro project directory. For example, i
 
 Also ensure that Testaro can read all those directories and write to `REPORTDIR`.
 
-Place a script into `SCRIPTDIR` and, optionally, a batch into `BATCHDIR`. Each should be named `idvalue.json`, where `idvalue` is replaced with the value of its `id` property. That value must consist of only lower-case ASCII letters and digits.
+Place a script into `SCRIPTDIR` and, optionally, a batch into `BATCHDIR`. Each should be named with a `.json` extension., where `idvalue` is replaced with the value of its `id` property. That value must consist of only lower-case ASCII letters and digits.
 
 Then execute the statement `node high scriptID` or `node high scriptID batchID`, replacing `scriptID` and `batchID` with the `id` values of the script and the batch, respectively.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testaro",
-  "version": "5.7.0",
+  "version": "5.7.4",
   "description": "Automation of accessibility testing",
   "main": "index.js",
   "scripts": {

package/run.js

@@ -1,5 +1,5 @@
 /*
-  index.js
+  run.js
   testaro main script.
 */
 
@@ -103,6 +103,7 @@ let errorLogSize = 0;
 let prohibitedCount = 0;
 let visitTimeoutCount = 0;
 let visitRejectionCount = 0;
+let visitLatency = 0;
 let actCount = 0;
 // Facts about the current browser.
 let browserContext;
@@ -511,6 +512,7 @@ const goto = async (page, url, timeout, waitUntil, isStrict) => {
     url = url.replace('file://', `file://${__dirname}/`);
   }
   // Visit the URL.
+  const startTime = Date.now();
   const response = await page.goto(url, {
     timeout,
     waitUntil
@@ -520,6 +522,7 @@ const goto = async (page, url, timeout, waitUntil, isStrict) => {
     visitTimeoutCount++;
     return 'error';
   });
+  visitLatency += Math.round((Date.now() - startTime) / 1000);
   // If the visit succeeded:
   if (typeof response !== 'string') {
     const httpStatus = response.status();
@@ -733,7 +736,7 @@ const doActs = async (report, actIndex, page) => {
         // If the command is a url:
         if (act.type === 'url') {
           // Visit it and wait until it is stable.
-          page = await visit(act, page, report.strict);
+          page = await visit(act, page, report.isStrict);
         }
         // Otherwise, if the act is a wait for text:
         else if (act.type === 'wait') {
@@ -1335,6 +1338,7 @@ const doScript = async (report) => {
   report.prohibitedCount = prohibitedCount;
   report.visitTimeoutCount = visitTimeoutCount;
   report.visitRejectionCount = visitRejectionCount;
+  report.visitLatency = visitLatency;
   // Add the end time and duration to the report.
   const endTime = new Date();
   report.endTime = endTime.toISOString().slice(0, 19);

package/tests/nuVal.js

@@ -1,46 +1,72 @@
 /*
   nuVal
   This test subjects a page to the Nu Html Checker.
+  That API erratically replaces left and right double quotation marks with invalid UTF-8, which
+  appears as 2 or 3 successive instances of the replacement character (U+fffd). Therefore, this
+  test removes all such quotation marks and the replacement character. That causes
+  'Bad value “” for' to become 'Bad value  for'. Since the corruption of quotation marks is
+  erratic, no better solution is known.
 */
 const https = require('https');
 exports.reporter = async page => {
   const pageContent = await page.content();
   // Get the data from a Nu validation.
-  const data = await new Promise(resolve => {
-    const request = https.request(
-      {
-        host: 'validator.nu',
-        path: '/?parser=html&out=json',
-        method: 'POST',
-        headers: {
-          'User-Agent': 'Mozilla/5.0',
-          'Content-Type': 'text/html; charset=utf-8'
-        }
-      },
-      response => {
-        let report = '';
-        response.on('data', chunk => {
-          report += chunk;
-        });
-        // When the data arrive:
-        response.on('end', async () => {
-          try {
-            // Delete unnecessary properties.
-            const result = JSON.parse(report);
-            return resolve(result);
-          }
-          catch (error) {
-            return resolve({
-              prevented: true,
-              error: error.message,
-              report
-            });
+  const data = await new Promise((resolve, reject) => {
+    try {
+      const request = https.request(
+        {
+          /*
+            Alternatives (more timeout-prone): host=validator.nu; path=/?parser=html@out=json
+            That host crashes instead of ending with a fatal error.
+          */
+          host: 'validator.w3.org',
+          path: '/nu/?parser=html&out=json',
+          method: 'POST',
+          headers: {
+            'User-Agent': 'Mozilla/5.0',
+            'Content-Type': 'text/html; charset=utf-8'
           }
+        },
+        response => {
+          let report = '';
+          response.on('data', chunk => {
+            report += chunk;
+          });
+          // When the data arrive:
+          response.on('end', async () => {
+            try {
+              // Delete left and right quotation marks and their erratic invalid replacements.
+              const result = JSON.parse(report.replace(/[\u{fffd}“”]/ug, ''));
+              return resolve(result);
+            }
+            catch (error) {
+              console.log(`Validation failed (${error.message})`);
+              return resolve({
+                prevented: true,
+                error: error.message,
+                report
+              });
+            }
+          });
+        }
+      );
+      request.write(pageContent);
+      request.end();
+      request.on('error', error => {
+        console.log(error.message);
+        return reject({
+          prevented: true,
+          error: error.message
         });
-      }
-    );
-    request.write(pageContent);
-    request.end();
+      });
+    }
+    catch(error) {
+      console.log(error.message);
+      return reject({
+        prevented: true,
+        error: error.message
+      });
+    }
   });
   return {result: data};
 };