testilo 29.0.1 → 30.1.0

package/README.md

@@ -643,7 +643,7 @@ To test the `digest` module, in the project directory you can execute the statem
 
 ### Summarization
 
-The `summarize` module of Testilo can summarize a scored report. The summary contains, insofar as they exist in the report, its ID, end time, order ID, target data, and total score.
+The `summarize` module of Testilo can summarize a scored report. The summary contains, insofar as they exist in the report, its ID, end time, `sources` property, and total score.
 
 #### Invocation
 
@@ -680,12 +680,14 @@ When a user invokes `summarize` in this example, the `call` module:
 
 If you use Testilo to perform a battery of tests on multiple targets, you may want a single report that compares the total scores received by the targets. Testilo can produce such a _comparison_.
 
-The `compare` module compares the scores in a summary of scored reports. Its `compare()` function takes two arguments:
+The `compare` module compares the scores in a summary report. Its `compare()` function takes two arguments:
 - a comparison function
 - a summary report
 
 The comparison function defines the rules for generating an HTML file comparing the scored reports. The Testilo package contains a `procs/compare` directory, in which there are subdirectories containing modules that export comparison functions. You can use one of those functions, or you can create your own.
 
+Summary reports suitable for comparisons are those that contain one result per target. If a summary report contains results from multiple times per target, tracking (described below) is appropriate, rather than comparison.
+
 #### Invocation
 
 There are two ways to use the `compare` module.
@@ -715,7 +717,7 @@ node call compare 'state legislators' tcp99 240813
 
 When a user invokes `compare` in this example, the `call` module:
 - gets the comparison module from subdirectory `tcp99` of the subdirectory `compare` in the `FUNCTIONDIR` directory.
-- gets the summary report whose file name begins with `'240813'` from the `summarized` subdirectory of the `REPORTDIR` directory.
+- gets the first summary report whose file name begins with `'240813'` from the `summarized` subdirectory of the `REPORTDIR` directory.
 - creates an ID for the comparison.
 - creates the comparison as an HTML document.
 - writes the comparison in the `comparative` subdirectory of the `REPORTDIR` directory, with `state legislators` as a description and the ID as the base of the file name.
@@ -726,6 +728,47 @@ The comparative report created by `compare` is an HTML file, and it expects a `s
 
 To test the `compare` module, in the project directory you can execute the statement `node validation/compare/validate`. If `compare` is valid, all logging statements will begin with “Success” and none will begin with “ERROR”.
 
+### Track
+
+The `track` module of Testilo selects, organizes, and presents data from summaries to show changes over time in total scores. The module produces a web page, showing changes in a table and a line graph. The line graph contains a line for each target (namely, each value of the `sources.target.what` property).
+
+A typical use case for tracking is monitoring, i.e. periodic auditing of one or more web pages.
+
+#### Invocation
+
+##### By a module
+
+A module can invoke `track()` in this way:
+
+```javaScript
+const {track} = require('testilo/track');
+const trackerDir = `${process.env.FUNCTIONDIR}/track/ttp99a`;
+const {tracker} = require(`${trackerDir}/index`);
+const summaryReport = …;
+const [reportID, trackReport] = track(tracker, summaryReport);
+```
+
+The `track()` function returns an ID and an HTML tracking report that shows data for all of the results in the summary report. The invoking module can further dispose of the tracking report as needed.
+
+##### By a user
+
+A user can invoke `track()` in one of these ways:
+
+```javaScript
+node call track ttp99a
+node call track ttp99a 241016
+node call track ttp99a 241016 'ABC Foundation'
+```
+
+When a user invokes `track()` in this example, the `call` module:
+- gets the summary report from the last file in the `summarized` subdirectory of the `REPORTDIR` directory, or if the third argument to `call()` exists the last one whose name begins with `'241016'`.
+- selects the summarized data for all results in the summary report, or if the fourth argument to `call()` exists from all results whose `target.what` property has the value `'ABC Foundation'`.
+- uses tracker `ttp99a` to create a tracking report.
+- assigns an ID to the tracking report.
+- writes the tracking report to the `tracking` subdirectory of the `REPORTDIR` directory, with the ID as the base of its file name.
+
+The tracking reports created by `track()` are HTML files, and they expect a `style.css` file to exist in their directory. The `reports/tracking/style.css` file in Testilo is an appropriate stylesheet to be copied into the directory where tracking reports are written.
+
 ### Tool crediting
 
 If you use Testaro to perform all the tests of all the tools on multiple targets and score the reports with a score proc that maps tool rules onto tool-agnostic issues, you may want to tabulate the comparative efficacy of each tool in discovering instances of issues. Testilo can help you do this by producing a _credit report_.
@@ -772,42 +815,6 @@ When a user invokes `credit` in this example, the `call` module:
 - creates an ID for the credit report.
 - writes the credit report as a JSON file, with the ID as the base of its file name and `legislators` as its description, to the `credit` subdirectory of the `REPORTDIR` directory.
 
-### Track
-
-The `track` module of Testilo selects, organizes, and presents data from summaries to show changes over time in total scores. The module produces a web page, showing changes in a table and (in the future) also in a line graph.
-
-#### Invocation
-
-##### By a module
-
-A module can invoke `track()` in this way:
-
-```javaScript
-const {track} = require('testilo/track');
-const trackerDir = `${process.env.FUNCTIONDIR}/track/ttp99a`;
-const {tracker} = require(`${trackerDir}/index`);
-const summary = …;
-const [reportID, trackReport] = track(tracker, summary);
-```
-
-The `track()` function returns an ID and an HTML tracking report that shows data for all of the results in the summary. The invoking module can further dispose of the report as needed.
-
-##### By a user
-
-A user can invoke `track()` in this way:
-
-```javaScript
-node call track ttp99a 241016T2045-Uf-0 4 'ABC Foundation'
-```
-
-When a user invokes `track()` in this example, the `call` module:
-- gets the summary from the `241016T2045-Uf-0.json` file in the `summarized` subdirectory of the `REPORTDIR` directory.
-- selects the summarized data for all results with the `order` value of `'4'` and the `target.what` value of `'ABC Foundation'`. If the third or fourth argument to `call()` is `null` (or omitted), then `call()` does not select results by `order` or by `target.what`, respectively.
-- uses tracker `ttp99a` to create a tracking report.
-- writes the tracking report to the `tracking` subdirectory of the `REPORTDIR` directory.
-
-The tracking reports created by `track()` are HTML files, and they expect a `style.css` file to exist in their directory. The `reports/tracking/style.css` file in Testilo is an appropriate stylesheet to be copied into the directory where tracking reports are written.
-
 ## Origin
 
 Work on the functionalities of Testaro and Testilo began in 2017. It was named [Autotest](https://github.com/jrpool/autotest) in early 2021 and then partitioned into the more single-purpose packages Testaro and Testilo in January 2022.

package/call.js

@@ -38,7 +38,7 @@ const {compare} = require('./compare');
 const {credit} = require('./credit');
 // Function to summarize reports.
 const {summarize} = require('./summarize');
-// Function to track audits.
+// Function to track results.
 const {track} = require('./track');
 
 // ########## CONSTANTS
@@ -52,11 +52,17 @@ const fnArgs = process.argv.slice(3);
 
 // ########## FUNCTIONS
 
-// Gets a summary report.
+// Gets the last matching summary report.
 const getSummaryReport = async selector => {
   const summaryDir = `${reportDir}/summarized`;
   const summaryReportNames = await fs.readdir(summaryDir);
-  const summaryReportName = summaryReportNames.find(reportName => reportName.startsWith(selector));
+  let summaryReportName;
+  if (summaryReportNames && summaryReportNames.length) {
+    summaryReportName = summaryReportNames.findLast(reportName => reportName.startsWith(selector));
+  }
+  else {
+    summaryReportName = summaryReportNames.pop();
+  }
   if (summaryReportName) {
     const summaryReportJSON = await fs.readFile(`${summaryDir}/${summaryReportName}`, 'utf8');
     const summaryReport = JSON.parse(summaryReportJSON);
@@ -255,7 +261,7 @@ const callSummarize = async (what, selector = '') => {
     const summaryReport = {
       id: getFileID(2),
       what,
-      data: []
+      summaries: []
     };
     // For each report to be summarized:
     for (const reportID of reportIDs) {
@@ -263,7 +269,7 @@ const callSummarize = async (what, selector = '') => {
       const report = await getReport('scored', reportID);
       // Add a summary of it to the summary report.
       const summary = summarize(report);
-      summaryReport.data.push(summary);
+      summaryReport.summaries.push(summary);
     };
     // Save the summary report.
     const summaryDir = `${reportDir}/summarized`;
@@ -302,6 +308,40 @@ const callCompare = async (what, compareProcID, selector) => {
     }
   }
 };
+// Fulfills a tracking request.
+const callTrack = async (trackerID, selector, targetWhat) => {
+  // Get the summary report.
+  try {
+    const summaryReport = await getSummaryReport(selector);
+    // Remove unwanted results from it.
+    summaryReport.summaries = summaryReport.summaries.filter(
+      result => targetWhat
+      ? result.sources
+      && result.sources.target
+      && result.sources.target.what === targetWhat
+      : true
+    );
+    // If any results remain:
+    if (summaryReport.summaries.length) {
+      // Get the tracker.
+      const {tracker} = require(`${functionDir}/track/${trackerID}/index`);
+      // Track the results.
+      const [reportID, trackingReport] = await track(tracker, summaryReport);
+      // Save the tracking report.
+      await fs.mkdir(`${reportDir}/tracking`, {recursive: true});
+      const reportPath = `${reportDir}/tracking/${reportID}.html`;
+      await fs.writeFile(reportPath, trackingReport);
+      console.log(`Tracking report saved in ${reportPath}`);
+    }
+    // Otherwise, i.e. if no results remain:
+    else {
+      console.log('ERROR: No results match the request');
+    }
+  }
+  catch(error) {
+    console.log(`ERROR: Tracking request invalid (${error.message})`);
+  }
+};
 // Fulfills a credit request.
 const callCredit = async (what, selector = '') => {
   // Get the IDs of the scored reports to be credited.
@@ -331,43 +371,6 @@ const callCredit = async (what, selector = '') => {
     console.log('ERROR: No scored reports to be credited');
   }
 };
-// Fulfills a tracking request.
-const callTrack = async (trackerID, summaryID, orderID, targetWhat) => {
-  // Get the summary.
-  try {
-    const summaryJSON = await fs.readFile(`${reportDir}/summarized/${summaryID}.json`, 'utf8');
-    const summary = JSON.parse(summaryJSON);
-    // Remove unwanted audits from it.
-    summary.data = summary.data.filter(result => {
-      if (orderID && result.order !== orderID) {
-        return false;
-      }
-      if (targetWhat && result.target && result.target.what !== targetWhat) {
-        return false;
-      }
-      return true;
-    });
-    // If any results remain:
-    if (summary.data.length) {
-      // Get the tracker.
-      const {tracker} = require(`${functionDir}/track/${trackerID}/index`);
-      // Track the results.
-      const [reportID, trackingReport] = await track(tracker, summary);
-      // Save the tracking report.
-      await fs.mkdir(`${reportDir}/tracking`, {recursive: true});
-      const reportPath = `${reportDir}/tracking/${reportID}.html`;
-      await fs.writeFile(reportPath, trackingReport);
-      console.log(`Tracking report saved in ${reportPath}`);
-    }
-    // Otherwise, i.e. if no audits remain:
-    else {
-      console.log('ERROR: No audits match the request');
-    }
-  }
-  catch(error) {
-    console.log(`ERROR: Tracking request invalid (${error.message})`);
-  }
-};
 
 // ########## OPERATION
 

package/compare.js

@@ -11,6 +11,6 @@
 // Compares the summarized reports and returns a comparison.
 exports.compare = async (id, what, comparer, summaryReport) => {
   // Return the comparison.
-  console.log(`Comparison complete. Report count: ${summaryReport.data.length}`);
+  console.log(`Comparison complete. Report count: ${summaryReport.summaries.length}`);
   return comparer(id, what, summaryReport);
 };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testilo",
-  "version": "29.0.1",
+  "version": "30.1.0",
   "description": "Prepares and processes Testaro reports",
   "main": "call.js",
   "scripts": {

package/procs/compare/tcp40/index.js

@@ -17,17 +17,17 @@ const innestJoiner = '\n          ';
 // ########## FUNCTIONS
 
 // Returns the maximum score.
-const getMaxScore = summaryReport => summaryReport.data.reduce(
+const getMaxScore = summaryReport => summaryReport.summaries.reduce(
   (max, result) => Math.max(max, result.score), 0
 );
 // Converts summary report data to a table body.
 const getTableBody = async summaryReport => {
   const maxScore = getMaxScore(summaryReport);
-  const rows = summaryReport.data
+  const rows = summaryReport.summaries
   .sort((a, b) => a.score - b.score)
   .map(result => {
-    const {id, target, score} = result;
-    const {what, which} = target;
+    const {id, sources, score} = result;
+    const {what, which} = sources.target;
     const pageCell = `<th scope="row"><a href="${which}">${what}</a></th>`;
     const scoreDestination = process.env.DIGEST_URL.replace('__id__', id);
     const numCell = `<td><a href="${scoreDestination}">${score}</a></td>`;
@@ -42,7 +42,7 @@ const getTableBody = async summaryReport => {
 const populateQuery = async (id, what, summaryReport, query) => {
   query.id = id;
   query.what = what;
-  query.pageCount = summaryReport.data.length;
+  query.pageCount = summaryReport.summaries.length;
   query.tableBody = await getTableBody(summaryReport);
   query.dateISO = getNowDate();
   query.dateSlash = getNowDateSlash();

package/procs/difgest/tfp40/index.html

@@ -36,7 +36,7 @@
         </table>
       </header>
       <h2>Introduction</h2>
-      <p>This <q>difgest</q> summarizes the differences between the scores from the two accessibility audits referenced in the synopsis above. A perfect score would be 0.</p>
+      <p>This <q>difgest</q> summarizes the differences between the scores from the two accessibility test results referenced in the synopsis above. A perfect score would be 0.</p>
       <h2>Issue summary</h2>
       <p>Issues are ordered from the one on which B was most superior to the one on which A was most superior.</p>
       <table class="allBorder redBar">

package/procs/track/ttp40/index.html

@@ -28,12 +28,12 @@
       <script src="https://cdn.jsdelivr.net/npm/d3@7"></script>
       <script src="https://cdn.jsdelivr.net/npm/@observablehq/plot@0.6"></script>
       <script type="module">
-        const summaryJSON = '__summaryJSON__';
-        const summary = JSON.parse(summaryJSON);
+        const summaryReportJSON = '__summaryReportJSON__';
+        const summaryReport = JSON.parse(summaryReportJSON);
         const graphData = [];
-        summary.data.forEach(result => {
+        summaryReport.summaries.forEach(result => {
           graphData.push({
-            target: result.target.what,
+            target: result.sources.target.what,
             time: new Date(`20${result.endTime}Z`),
             score: result.score
           });

package/procs/track/ttp40/index.js

@@ -21,29 +21,30 @@ const digestURL = process.env.DIGEST_URL;
 // FUNCTIONS
 
 // Adds parameters to a query for a tracking report.
-const populateQuery = async (id, summary, query) => {
+const populateQuery = async (id, summaryReport, query) => {
   // General parameters.
   query.id = id;
   query.tp = trackerID;
   query.dateISO = getNowDate();
   query.dateSlash = getNowDateSlash();
-  // JSON of pruned summary.
-  summary.data.forEach(result => {
-    delete result.target.id;
+  // JSON of pruned summary report.
+  const {summaries} = summaryReport;
+  summaries.forEach(result => {
+    delete result.sources.target.id;
   });
-  query.summaryJSON = JSON.stringify(summary);
+  query.summaryReportJSON = JSON.stringify(summaryReport);
   // For each score:
   const rows = [];
-  const results = summary.data;
-  const targetWhats = Array.from(new Set(results.map(result => result.target.what))).sort();
-  summary.data.forEach(result => {
+  const targetWhats = Array.from(new Set(summaries.map(result => result.sources.target.what))).sort();
+  summaries.forEach(result => {
     // Create an HTML table row for it.
     const timeCell = `<td>${result.endTime}</td>`;
     const digestLinkDestination = digestURL.replace('__id__', result.id);
     const scoreCell = `<td><a href=${digestLinkDestination}>${result.score}</a></td>`;
     const orderCell = `<td class="center">${result.order}</td>`;
-    const targetID = alphaNumOf(targetWhats.indexOf(result.target.what));
-    const targetLink = `<a href="${result.target.which}">${result.target.what}</a>`;
+    const {target} = result.sources;
+    const targetID = alphaNumOf(targetWhats.indexOf(target.what));
+    const targetLink = `<a href="${target.which}">${target.what}</a>`;
     const targetCell = `<td>${targetID}: ${targetLink}</td>`;
     const row = `<tr>${[timeCell, scoreCell, orderCell, targetCell].join('')}</tr>`;
     // Add the row to the array of rows.
@@ -53,10 +54,10 @@ const populateQuery = async (id, summary, query) => {
   query.scoreRows = rows.join(innerJoiner);
 };
 // Returns a tracking report.
-exports.tracker = async (id, summary) => {
+exports.tracker = async (id, summaryReport) => {
   // Create a query to replace placeholders.
   const query = {};
-  await populateQuery(id, summary, query);
+  await populateQuery(id, summaryReport, query);
   // Get the template.
   let template = await fs.readFile(`${__dirname}/index.html`, 'utf8');
   // Replace its placeholders.

package/summarize.js

@@ -13,13 +13,10 @@ require('dotenv').config();
 // Returns a report summary.
 exports.summarize = report => {
   const {id, jobData, score, sources} = report;
-  const order = sources && sources.order || '';
-  const target = sources && sources.target || '';
   const summary = {
-    id: id || '',
-    endTime: jobData && jobData.endTime || '',
-    order: order || '',
-    target,
+    id: id || null,
+    endTime: jobData && jobData.endTime || null,
+    sources: sources || null,
     score: score && score.summary && score.summary.total || null
   };
   return summary;

package/track.js

@@ -14,10 +14,9 @@ const {getFileID} = require('./procs/util');
 // ########## FUNCTIONS
 
 // Creates and returns a tracking report from a summary.
-exports.track = async (tracker, summary) => {
+exports.track = async (tracker, summaryReport) => {
   // Use the tracker to create a tracking report.
   const id = getFileID(2);
-  const trackingReport = await tracker(id, summary);
-  console.log(`Tracking report ${id} created`);
+  const trackingReport = await tracker(id, summaryReport);
   return [id, trackingReport];
 };