testilo 41.7.0 → 42.0.1

package/README.md

@@ -359,7 +359,7 @@ The second form will create a script that prescribes tests for all the rules of
 
 The third form will create a script that prescribes tests for all the rules classified by the named issue-classification file into any of the specified issues.
 
-In this statement, replace `id` with an ID for the script, such as `headings1`, consisting of ASCII alphanumeric characters, or with `''` if you want Testilo to create an ID. Replace `what` with a string describing the script, or with `''` if you want Testilo to create a generic description. Replace `device` with a valid device ID.
+In this statement, replace `id` with an ID for the script, such as `headings1`, consisting of ASCII alphanumeric characters, or with `''` if you want Testilo to create an ID. Replace `what` with a string describing the script, or with `''` if you want Testilo to create a generic description. Replace `deviceID` with a valid device ID.
 
 The `call` module will retrieve the named classification, if any.
 The `script` module will create a script.
@@ -914,13 +914,15 @@ When a user invokes `track()` in this example, the `call` module:
 
 The tracking reports created by `track()` are HTML files, and they expect a `style.css` file to exist in their directory. The `reports/tracking/style.css` file in Testilo is an appropriate stylesheet to be copied into the directory where tracking reports are written.
 
-### Tool crediting
+### Statistics
 
-If you use Testaro to perform all the tests of all the tools on multiple targets and score the reports with a score proc that maps tool rules onto tool-agnostic issues, you may want to tabulate the comparative efficacy of each tool in discovering instances of issues. Testilo can help you do this by producing a _credit report_.
+If you use Testaro to perform all the tests of all the tools on multiple targets and score the reports with a score proc that maps tool rules onto tool-agnostic issues, you may want to get statistics on tools and issues, aggregated from the scored reports. Testilo has two procs for this purpose.
+
+#### Tool crediting
 
 The `credit` module tabulates the contribution of each tool to the discovery of issue instances in a collection of scored reports. Its `credit()` function takes two arguments: a report description and an array of `score` properties of scored reports.
 
-The credit report contains four sections:
+The function produces a credit report containing four sections:
 - `counts`: for each issue, how many instances each tool reported
 - `onlies`: for each issue of which only 1 tool reported instances, which tool it was
 - `mosts`: for each issue of which at least 2 tools reported instances, which tool(s) reported the maximum instance count
@@ -928,11 +930,11 @@ The credit report contains four sections:
     - `onlies`: a list of the issues that only the tool reported instances of
     - `mosts`: a list of the issues for which the instance count of the tool was not surpassed by that of any other tool
 
-#### Invocation
+##### Invocation
 
 There are two ways to use the `credit` module.
 
-##### By a module
+###### By a module
 
 A module can invoke `credit()` in this way:
 
@@ -944,7 +946,7 @@ const creditReport = credit('June 2025', reportScores);
 
 The first argument to `credit()` is a description to be included in the credit report. The second argument is an array of `score` properties of scored report objects. The `credit()` function returns a credit report. The invoking module can further dispose of the credit report as needed.
 
-##### By a user
+###### By a user
 
 A user can invoke `credit()` in one of these ways:
 
@@ -959,6 +961,43 @@ When a user invokes `credit` in this example, the `call` module:
 - creates an ID for the credit report.
 - writes the credit report as a JSON file, with the ID as the base of its file name and `legislators` as its description, to the `credit` subdirectory of the `REPORTDIR` directory.
 
+### Issue scores
+
+The `issues` module tabulates total issue scores. Its `issues()` function takes two arguments: a report description and an array of `score` properties of scored reports.
+
+The function produces an issue report, an object with issue properties, whose values are the totals of the scores of the respective issues.
+
+##### Invocation
+
+There are two ways to use the `credit` module.
+
+###### By a module
+
+A module can invoke `issues()` in this way:
+
+```javaScript
+const {issues} = require('testilo/issues');
+const reportScores = […];
+const issuesReport = issues('legislators', reportScores);
+```
+
+The arguments to `issues()` are a report description and an array of `score` properties of scored report objects. The `issues()` function returns an issues report. The invoking module can further dispose of the issues report as needed.
+
+###### By a user
+
+A user can invoke `issues()` in one of these ways:
+
+```bash
+node call issues legislators
+node call issues legislators 241106
+```
+
+When a user invokes `issues` in this example, the `call` module:
+- gets all reports, or if the third argument to `call()` exists all reports whose file names begin with `'241106'`, in the `scored` subdirectory of the `REPORTDIR` directory.
+- gets the `score` properties of those reports.
+- creates an ID for the issues report.
+- writes the issues report as a JSON file, with the ID as the base of its file name and `legislators` as its description, to the `issues` subdirectory of the `REPORTDIR` directory.
+
 ## Origin
 
 Work on the functionalities of Testaro and Testilo began in 2017. It was named [Autotest](https://github.com/jrpool/autotest) in early 2021 and then partitioned into the more single-purpose packages Testaro and Testilo in January 2022.

package/call.js

@@ -60,6 +60,8 @@ const {difgest} = require('./difgest');
 const {compare} = require('./compare');
 // Function to credit tools for issues in reports.
 const {credit} = require('./credit');
+// Function to credit tools for issues in reports.
+const {issues} = require('./issues');
 // Function to summarize reports.
 const {summarize} = require('./summarize');
 // Function to track results.
@@ -482,6 +484,34 @@ const callCredit = async (what, selector = '') => {
     console.log('ERROR: No scored reports to be credited');
   }
 };
+// Fulfills an issues request.
+const callIssues = async (what, selector = '') => {
+  // Get the IDs of the scored reports to be analyzed.
+  const reportIDs = await getReportIDs('scored', selector);
+  // If any exist:
+  if (reportIDs.length) {
+    // Get an array of the score properties of the reports to be credited.
+    const reportScores = [];
+    for (const id of reportIDs) {
+      const report = await getReport('scored', id);
+      reportScores.push(report.score);
+    }
+    // Analyze the reports.
+    const reportObj = issues(what, reportScores);
+    // Save the issues report.
+    const issuesDir = `${reportDir}/issues`;
+    await fs.mkdir(issuesDir, {recursive: true});
+    const issuesReportID = getFileID(2);
+    const reportPath = `${issuesDir}/${issuesReportID}.json`;
+    await fs.writeFile(reportPath, `${JSON.stringify(reportObj, null, 2)}\n`);
+    console.log(`Issue scores tallied and issues report saved as ${reportPath}`);
+  }
+  // Otherwise, i.e. if no scored reports are to be analyzed:
+  else {
+    // Report this.
+    console.log('ERROR: No scored reports to be analyzed');
+  }
+};
 
 // ########## OPERATION
 
@@ -558,6 +588,12 @@ else if (fn === 'credit' && fnArgs.length > 0 && fnArgs.length < 3) {
     console.log('Execution completed');
   });
 }
+else if (fn === 'issues' && fnArgs.length > 0 && fnArgs.length < 3) {
+  callIssues(... fnArgs)
+  .then(() => {
+    console.log('Execution completed');
+  });
+}
 else {
   console.log('ERROR: Invalid statement');
 }

package/issues.js

@@ -0,0 +1,46 @@
+/*
+  © 2024 CVS Health and/or one of its affiliates. All rights reserved.
+
+  Permission is hereby granted, free of charge, to any person obtaining a copy
+  of this software and associated documentation files (the "Software"), to deal
+  in the Software without restriction, including without limitation the rights
+  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+  copies of the Software, and to permit persons to whom the Software is
+  furnished to do so, subject to the following conditions:
+
+  The above copyright notice and this permission notice shall be included in all
+  copies or substantial portions of the Software.
+
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+  SOFTWARE.
+*/
+
+/*
+  issues.js
+  Analyzes issue scores in an array of score properties of scored reports.
+*/
+
+// Returns a tabulation of the total scores of issues reported by tools in scored reports.
+exports.issues = (what, reportScores) => {
+  const tally = {};
+  reportScores.forEach(reportScore => {
+    Object.keys(reportScore.details.issue).forEach(issueName => {
+      tally[issueName] ??= 0;
+      tally[issueName] += reportScore.details.issue[issueName].score;
+    });
+  });
+  const scores = [];
+  Object.keys(tally).forEach(issueName => {
+    scores.push([issueName, tally[issueName]]);
+  });
+  scores.sort((a, b) => b[1] - a[1]);
+  return {
+    what,
+    scores
+  };
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testilo",
-  "version": "41.7.0",
+  "version": "42.0.1",
   "description": "Prepares Testaro jobs and processes Testaro reports",
   "main": "call.js",
   "scripts": {