testilo 9.0.3 → 10.0.0

package/README.md

@@ -98,7 +98,7 @@ Suppose you have created this script:
 }
 ```
 
-The `acts` array of this script contains three regular acts and two placeholders.
+The `acts` array of this script contains three acts of type `test` and two acts of type `placeholder`.
 
 #### Batch
 
@@ -265,7 +265,7 @@ A user can invoke `merge` in this way:
     - `node call merge s b e false`
     - `node call merge s b e`
 
-In these statements, replace `s` and `b` with the base names of the script and batch files, respectively. For example, if the script file is named `ts25.json`, then replace `x` with `ts25`. Replace `e` with an email address, or with an empty string if the environment variable `process.env.REQUESTER` exists and you want to use it.
+In these statements, replace `s` and `b` with the base names of the script and batch files, respectively. For example, if the script file is named `ts25.json`, then replace `s` with `ts25`. Replace `e` with an email address, or with an empty string if the environment variable `process.env.REQUESTER` exists and you want to use it.
 
 The first statement will cause a merger with isolation.
 The second and third statements will cause a merger without isolation.
@@ -278,6 +278,16 @@ The `call` module will save the jobs in the `todo` subdirectory of the `process.
 
 To test the `merge` module, in the project directory you can execute the statement `node validation/merge/validate`. All logging statements should begin with “Success” and none should begin with “ERROR”.
 
+### Examples
+
+There are script and batch examples in the `specs` directory.
+
+The script example (with ID `ts21`) performs all of the tests available in Testaro, including the tests of 9 dependent tools. It contains 2 `placeholder` acts, one that specifies the Webkit browser for any `launch` act and one that specifies the Chromium browser for any `launch` act.
+
+The batch example (with ID `orgs`) specifies 2 targets.
+- One of them has the URL `https://example.com` and requires each placeholder to be replaced with 2 acts: a `launch` act and a `url` act.
+- The other has the URL `https://www.w3.org/WAI/ARIA/apg/example-index/accordion/accordion` and requires each placeholder to be replaced with 3 acts: a `launch` act, a `url` act, and a `button` act.
+
 ## Report scoring
 
 ### Introduction
@@ -307,7 +317,7 @@ A scoring function defines scoring rules. The Testilo package contains a `procs/
 
 ### Invocation
 
-There are two ways to use the `score` module.
+There are two ways to invoke the `score` module.
 
 #### By a module
 
@@ -326,13 +336,13 @@ The first argument to `score()` is a scoring function. In this example, it has b
 A user can invoke `score` in this way:
 
 ```bash
-node call score sp25a
-node call score sp25a 75
+node call score tsp25a
+node call score tsp25a 75
 ```
 
 When a user invokes `score` in this example, the `call` module:
-- gets the scoring module `sp25a` from its JSON file `sp25a.json` in the `score` subdirectory of the `process.env.FUNCTIONDIR` directory
-- gets the reports from the r`raw` subdirectory of the `process.env.REPORTDIR` directory
+- gets the scoring module `tsp25a` from its JSON file `tsp25a.json` in the `score` subdirectory of the `process.env.FUNCTIONDIR` directory
+- gets the reports from the `raw` subdirectory of the `process.env.REPORTDIR` directory
 - writes the scored reports to the `scored` subdirectory of the `process.env.REPORTDIR` directory
 
 The optional third argument to call (`75` in this example) is a report selector. Without the argument, `call` gets all the reports in the `raw` subdirectory. With the argument, `call` gets only those reports whose names begin with the argument string.
@@ -341,6 +351,20 @@ The optional third argument to call (`75` in this example) is a report selector.
 
 To test the `score` module, in the project directory you can execute the statement `node validation/score/validate`. All logging statements should begin with “Success” and none should begin with “ERROR”.
 
+### Comprehensive example
+
+The module `procs/score/tsp21.js` is an example of a comprehensive score proc. It is designed to score reports produced by Testaro from jobs that Testilo creates from the `ts21` script. That script causes Testaro to perform 1351 tests belonging to Testaro and nine other tools integrated into Testaro as dependencies.
+
+The `tsp21` score proc combines the results of those tests into a single score for each report. The `score` property that the proc adds to the report shows how the individual test results are aggregated into a score.
+
+The `tsp21` score proc imports the `procs/score/tic21.js` module, which classifies all the tests into 255 “issues”. Any two or more tests classified into the same issue are deemed to be approximately equivalent in intent. Using this classification, the score proc tabulates the test results by issue, so that all instances of the issue, regardless of which tool discovered the instances, are reported together.
+
+The issue classification in `tic21` assigns weights to the issues to reflect their estimated severities,and the scoring algorithm of `tsp21` adopts a compromise between issue-based and tool-based summation. Thus, if 3 tools discover 20 instances of an issue, the score increases (i.e. becomes worse) by more than it would if only 1 tool had discovered them, but less than 3 times as much. The motivations are that discovery of issue instances by multiple tools increases confidence that the issues are real, and passing accessibility tests is per se beneficial because it decreases risks associated with alleged inaccessibility.
+
+The `tsp21` score proc does **not** attempt to identify instances of issues. Thus, if tool A discovers 4 instances and tool B discovers 7 instances of some issue, `tsp21` does not attempt to say which of the 4 discovered by A are a subset of the 7 discovered by B. Instead,`tsp21` naïvely acts as if the 4 are all a subset of the 7, even though in reality the two sets might be entirely disjoint.
+
+In the `tsp21` scoring algorithm, the failure of a tool or of a Testaro test to run on a page increases the score of the page, and messages logged by the browser while tests are being performed on a page, especially error messages, also increase the score.
+
 ## Report digesting
 
 ### Introduction

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testilo",
-  "version": "9.0.3",
+  "version": "10.0.0",
   "description": "Client that scores and digests Testaro reports",
   "main": "aim.js",
   "scripts": {