testilo 7.1.1 → 9.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +341 -149
- package/call.js +80 -154
- package/compare.js +14 -31
- package/digest.js +19 -11
- package/merge.js +109 -34
- package/package.json +1 -1
- package/procs/compare/cp20sqrt/index.html +48 -0
- package/procs/compare/cp20sqrt/index.js +69 -0
- package/procs/score/sp20b.js +1 -1
- package/score.js +18 -11
- package/validation/call/jobs/done/8oqso-script-accordion.json +57 -0
- package/validation/call/jobs/done/8oqso-script-example.json +47 -0
- package/validation/call/specs/batches/batch.json +43 -0
- package/validation/call/specs/scripts/script.json +23 -0
- package/validation/compare/comparer.js +18 -0
- package/validation/compare/comparison.html +19 -0
- package/validation/compare/report0.json +78 -0
- package/validation/compare/report1.json +78 -0
- package/validation/compare/validate.js +51 -0
- package/validation/digest/digest.html +19 -0
- package/validation/digest/digester.js +12 -0
- package/validation/digest/report.json +78 -0
- package/validation/digest/validate.js +64 -0
- package/validation/merge/batch.json +43 -0
- package/validation/merge/script.json +33 -0
- package/validation/merge/validate.js +179 -0
- package/validation/score/report.json +75 -0
- package/validation/score/scorer.js +17 -0
- package/validation/score/validate.js +62 -0
- package/aim.js +0 -45
- package/multiDigest.js +0 -51
- package/multiScore.js +0 -48
package/README.md
CHANGED
|
@@ -5,17 +5,23 @@ Utilities for Testaro
|
|
|
5
5
|
|
|
6
6
|
The Testilo package contains utilities that facilitate the use of the [Testaro](https://www.npmjs.com/package/testaro) package.
|
|
7
7
|
|
|
8
|
-
Testaro performs digital accessibility tests on web artifacts and creates reports in JSON format. The utilities in Testilo
|
|
8
|
+
Testaro performs digital accessibility tests on web artifacts and creates reports in JSON format. The utilities in Testilo fall into two categories:
|
|
9
|
+
- Job preparation
|
|
10
|
+
- Report enhancement
|
|
9
11
|
|
|
10
12
|
Because Testilo supports Testaro, this `README` file presumes that you have access to the Testaro `README` file and therefore does not repeat information provided there.
|
|
11
13
|
|
|
12
14
|
## Branches
|
|
13
15
|
|
|
14
|
-
The `writer` branch contains the state of Testilo as of 2022-11-07. In that branch, Testilo must read and write files
|
|
16
|
+
The `writer` branch contains the state of Testilo as of 2022-11-07. In that branch, Testilo must read and write files. That makes Testilo unusable as a dependency in applications that do not have permission to both read and write files.
|
|
15
17
|
|
|
16
|
-
The `
|
|
18
|
+
The `simple` branch contains the state of Testilo as of 2022-12-23. In that branch, Testilo does not need to read or write files, so applications without write permission can still use Testilo as a dependency.
|
|
17
19
|
|
|
18
|
-
|
|
20
|
+
The `main` branch contains the state of Testilo from 2022-12-24 until now. In that branch, Testilo can prepare jobs that require multi-step operations to reach and pre-process the targets being tested.
|
|
21
|
+
|
|
22
|
+
The `complexmerge` branch refactors `main` and is to be merged into `main` when the refactoring is complete.
|
|
23
|
+
|
|
24
|
+
This `README.md` file documents the `complexmerge` branch and will document the `main` branch when `complexmerge` is merged into `main`.
|
|
19
25
|
|
|
20
26
|
## Dependencies
|
|
21
27
|
|
|
@@ -27,242 +33,428 @@ When Testilo is a dependency of another application, the `.env` file is not impo
|
|
|
27
33
|
|
|
28
34
|
Testilo is written in Node.js. Commands are given to Testilo in a command-line (terminal) interface or programmatically.
|
|
29
35
|
|
|
30
|
-
Shared routines
|
|
36
|
+
Shared routines are _procs_ and are located in the `procs` directory.
|
|
31
37
|
|
|
32
38
|
Testilo can be installed wherever Node.js (version 14 or later) is installed. This can be a server or the same workstation on which Testaro is installed.
|
|
33
39
|
|
|
34
|
-
The reason for Testilo being an independent package, rather than part of Testaro, is that Testilo can be installed on any host, while Testaro can run successfully only on a Windows or Macintosh workstation (and perhaps on some workstations with Ubuntu operating systems). Testaro runs tests similar to those that a human accessibility tester would run, using whatever browsers, input devices, system settings, simulated and attached devices, and assistive technologies tests may require. Thus, Testaro is limited to functionalities that require workstation attributes. For maximum flexibility in the management of Testaro jobs, all other functionalities are located outside of Testaro. You could have software such as Testilo running on a server, communicating with multiple workstations running Testaro
|
|
40
|
+
The reason for Testilo being an independent package, rather than part of Testaro, is that Testilo can be installed on any host, while Testaro can run successfully only on a Windows or Macintosh workstation (and perhaps on some workstations with Ubuntu operating systems). Testaro runs tests similar to those that a human accessibility tester would run, using whatever browsers, input devices, system settings, simulated and attached devices, and assistive technologies tests may require. Thus, Testaro is limited to functionalities that require workstation attributes. For maximum flexibility in the management of Testaro jobs, all other functionalities are located outside of Testaro. You could have software such as Testilo running on a server, communicating with multiple workstations running Testaro. The workstations could receive job orders from the server and return job results to the server for further processing.
|
|
35
41
|
|
|
36
|
-
##
|
|
42
|
+
## Job preparation
|
|
37
43
|
|
|
38
|
-
###
|
|
44
|
+
### Introduction
|
|
39
45
|
|
|
40
|
-
|
|
41
|
-
- Modify the value of the `id` property of the script by:
|
|
42
|
-
- Prefixing the value with a time stamp.
|
|
43
|
-
- Suffixing the value with the value of the `id` property of the host.
|
|
44
|
-
- Modify each `url` command of the script by changing the values of its `which`, `what`, and `id` properties to the values of those properties of the host.
|
|
45
|
-
- Add a `source` property to the script, identifying the script, the host, and the requester.
|
|
46
|
+
Testaro executes _jobs_. In a job, Testaro performs _acts_ (tests and other operations) on _targets_ (typically, web pages). The Testaro `README.md` file specifies the requirements for a job.
|
|
46
47
|
|
|
47
|
-
|
|
48
|
+
You can create a job for Testaro directly, without using Testilo.
|
|
48
49
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
const host = {
|
|
53
|
-
which: https://w3c.org/,
|
|
54
|
-
what: 'World Wide Web Consortium',
|
|
55
|
-
id: w3c
|
|
56
|
-
};
|
|
57
|
-
const {aim} = require('testilo/aim');
|
|
58
|
-
const scriptJSON = await fs.readFile(`${process.env.SCRIPTDIR}/ts25.json`, 'utf8');
|
|
59
|
-
const script = JSON.parse(scriptJSON);
|
|
60
|
-
const aimedScript = aim(script, host, 'developer@w3c.org');
|
|
61
|
-
await fs.writeFile(
|
|
62
|
-
`${process.env.JOBDIR}/${aimedScript.id}.json`, JSON.stringify(aimedScript, null, 2)
|
|
63
|
-
);
|
|
64
|
-
console.log(`Script ${aimedScript.source.script} aimed at ${aimedScript.host.what}`);
|
|
65
|
-
}
|
|
66
|
-
aimScript();
|
|
67
|
-
```
|
|
50
|
+
Testilo can, however, make job preparation more efficient in two scenarios:
|
|
51
|
+
- A common use case is to define a battery of tests and to have those tests performed on multiple targets. In that case, you need to create multiple jobs, one per page. The jobs are identical, except for the target-specific acts (including navigating to targets). If you tell Testilo about the tests and the targets, Testilo can create such collections of jobs for you.
|
|
52
|
+
- Some tests operate on a copy of a target and modify that copy. Usually, one wants test isolation: The results of a test do not depend on any previously performed tests. To ensure test isolation, a job containing such target-modifying tests must follow them with acts that restore the target to its desired pre-test state. Testilo can insert the required target-restoring acts into each job after target-modifying tests.
|
|
68
53
|
|
|
69
|
-
|
|
54
|
+
The `merge` module performs these services.
|
|
70
55
|
|
|
71
|
-
|
|
72
|
-
node call aim ts25 https://w3.org/ 'World Wide Web Consortium' w3c developer@w3c.org
|
|
73
|
-
```
|
|
56
|
+
### Procedure
|
|
74
57
|
|
|
75
|
-
|
|
58
|
+
To use the `merge` module, you need to create a _script_ and a _batch_:
|
|
59
|
+
- The script is an object that specifies a job, except for the targets.
|
|
60
|
+
- The batch is an object that specifies target-specific acts for targets.
|
|
76
61
|
|
|
77
|
-
The
|
|
62
|
+
The script contains an array of acts, with placeholders. For each target in the batch, the `merge` module creates a job, in which the placeholders are replaced with the acts specific to that target in the batch.
|
|
78
63
|
|
|
79
|
-
###
|
|
64
|
+
### Example
|
|
80
65
|
|
|
81
|
-
|
|
66
|
+
Here is an example illustrating how merging works.
|
|
82
67
|
|
|
83
|
-
|
|
68
|
+
#### Script
|
|
84
69
|
|
|
85
|
-
|
|
70
|
+
Suppose you have created this script:
|
|
86
71
|
|
|
87
|
-
```
|
|
72
|
+
```javaScript
|
|
88
73
|
{
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
74
|
+
id: 'ts25',
|
|
75
|
+
what: 'Motion, Axe, and bulk',
|
|
76
|
+
strict: true,
|
|
77
|
+
timeLimit: 60,
|
|
78
|
+
acts: [
|
|
79
|
+
{
|
|
80
|
+
type: 'placeholder',
|
|
81
|
+
which: 'main',
|
|
82
|
+
launch: 'webkit'
|
|
83
|
+
},
|
|
84
|
+
{
|
|
85
|
+
type: 'test',
|
|
86
|
+
which: 'motion',
|
|
87
|
+
what: 'spontaneous change of content; requires webkit',
|
|
88
|
+
delay: 2500,
|
|
89
|
+
interval: 2500,
|
|
90
|
+
count: 5
|
|
91
|
+
},
|
|
92
92
|
{
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
93
|
+
type: 'placeholder',
|
|
94
|
+
which: 'main',
|
|
95
|
+
launch: 'chromium'
|
|
96
96
|
},
|
|
97
97
|
{
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
98
|
+
type: 'test',
|
|
99
|
+
which: 'axe',
|
|
100
|
+
withItems: true,
|
|
101
|
+
rules: [],
|
|
102
|
+
what: 'Axe core, all rules, with details'
|
|
103
|
+
},
|
|
104
|
+
{
|
|
105
|
+
type: 'test',
|
|
106
|
+
which: 'bulk',
|
|
107
|
+
what: 'count of visible elements'
|
|
101
108
|
}
|
|
102
109
|
]
|
|
103
110
|
}
|
|
104
111
|
```
|
|
105
112
|
|
|
106
|
-
|
|
113
|
+
The `acts` array of this script contains tree regular acts and two placeholders.
|
|
107
114
|
|
|
108
|
-
|
|
109
|
-
const {merge} = require('testilo/merge');
|
|
110
|
-
merge('ts25', 'weborgs', 'developer@w3.org')
|
|
111
|
-
.then(() => {
|
|
112
|
-
console.log('Merger complete');
|
|
113
|
-
});
|
|
114
|
-
```
|
|
115
|
+
#### Batch
|
|
115
116
|
|
|
116
|
-
|
|
117
|
+
Suppose you have also created this batch:
|
|
117
118
|
|
|
118
|
-
```
|
|
119
|
-
|
|
119
|
+
```javaScript
|
|
120
|
+
{
|
|
121
|
+
id: 'weborgs',
|
|
122
|
+
what: 'Web standards organizations',
|
|
123
|
+
targets: [
|
|
124
|
+
{
|
|
125
|
+
id: 'mozilla',
|
|
126
|
+
what: 'Mozilla Foundation',
|
|
127
|
+
acts: {
|
|
128
|
+
main: [
|
|
129
|
+
{
|
|
130
|
+
type: 'launch'
|
|
131
|
+
},
|
|
132
|
+
{
|
|
133
|
+
type: 'url',
|
|
134
|
+
which: 'https://foundation.mozilla.org/en/',
|
|
135
|
+
what: 'Mozilla Foundation'
|
|
136
|
+
}
|
|
137
|
+
]
|
|
138
|
+
}
|
|
139
|
+
},
|
|
140
|
+
{
|
|
141
|
+
id: 'w3c',
|
|
142
|
+
what: 'World Wide Web Consortium',
|
|
143
|
+
acts: {
|
|
144
|
+
main: [
|
|
145
|
+
{
|
|
146
|
+
type: 'launch'
|
|
147
|
+
},
|
|
148
|
+
{
|
|
149
|
+
type: 'url',
|
|
150
|
+
which: 'https://www.w3.org/',
|
|
151
|
+
what: 'World Wide Web Consortium'
|
|
152
|
+
}
|
|
153
|
+
]
|
|
154
|
+
}
|
|
155
|
+
}
|
|
156
|
+
]
|
|
157
|
+
}
|
|
120
158
|
```
|
|
121
159
|
|
|
122
|
-
|
|
160
|
+
This batch defines two targets.
|
|
123
161
|
|
|
124
|
-
|
|
162
|
+
#### Isolation option
|
|
125
163
|
|
|
126
|
-
|
|
164
|
+
The `merge` module offers an isolation option. If you exercise it, the `merge` module will act as if the latest placeholder were again inserted after each target-modifying test, except where that test is the last act or the next act after it is a placeholder.
|
|
127
165
|
|
|
128
|
-
|
|
166
|
+
#### Output
|
|
129
167
|
|
|
130
|
-
|
|
168
|
+
Suppose you ask for a merger of the above batch and script, **without** the isolation option. Then the first job produced by `merge` could be:
|
|
131
169
|
|
|
132
170
|
```javaScript
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
171
|
+
{
|
|
172
|
+
id: '7izc1-ts25-mozilla',
|
|
173
|
+
what: 'Motion, Axe, and bulk',
|
|
174
|
+
strict: true,
|
|
175
|
+
timeLimit: 60,
|
|
176
|
+
acts: [
|
|
177
|
+
{
|
|
178
|
+
type: 'launch',
|
|
179
|
+
which: 'webkit'
|
|
180
|
+
},
|
|
181
|
+
{
|
|
182
|
+
type: 'url',
|
|
183
|
+
which: 'https://foundation.mozilla.org/en/',
|
|
184
|
+
what: 'Mozilla Foundation'
|
|
185
|
+
}
|
|
186
|
+
{
|
|
187
|
+
type: 'test',
|
|
188
|
+
which: 'motion',
|
|
189
|
+
what: 'spontaneous change of content; requires webkit',
|
|
190
|
+
delay: 2500,
|
|
191
|
+
interval: 2500,
|
|
192
|
+
count: 5
|
|
193
|
+
},
|
|
194
|
+
{
|
|
195
|
+
type: 'launch',
|
|
196
|
+
which: 'chromium'
|
|
197
|
+
},
|
|
198
|
+
{
|
|
199
|
+
type: 'url',
|
|
200
|
+
which: 'https://foundation.mozilla.org/en/',
|
|
201
|
+
what: 'Mozilla Foundation'
|
|
202
|
+
}
|
|
203
|
+
{
|
|
204
|
+
type: 'test',
|
|
205
|
+
which: 'axe',
|
|
206
|
+
withItems: true,
|
|
207
|
+
rules: [],
|
|
208
|
+
what: 'Axe core, all rules, with details'
|
|
209
|
+
},
|
|
210
|
+
{
|
|
211
|
+
type: 'test',
|
|
212
|
+
which: 'bulk',
|
|
213
|
+
what: 'count of visible elements'
|
|
214
|
+
}
|
|
215
|
+
],
|
|
216
|
+
sources: {
|
|
217
|
+
script: 'ts25',
|
|
218
|
+
batch: 'weborgs',
|
|
219
|
+
target: {
|
|
220
|
+
id: 'mozilla',
|
|
221
|
+
what: 'Mozilla Foundation'
|
|
222
|
+
},
|
|
223
|
+
requester: 'you@yourdomain.tld'
|
|
224
|
+
},
|
|
225
|
+
creationTime: '2023-11-20T15:50:27',
|
|
226
|
+
timeStamp: 'bizc1'
|
|
227
|
+
}
|
|
148
228
|
```
|
|
149
229
|
|
|
150
|
-
|
|
230
|
+
Most of the properties of this job object come from your script and your batch. The acts of type `placeholder` in the script have been replaced with the specified (i.e. `main`) array of acts of the first target of the batch. In this case, that target has only one array of acts, and that array contains two acts, but a target could have multiple act arrays with distinct names, and an act array could include any number of acts or be empty.
|
|
151
231
|
|
|
152
|
-
|
|
153
|
-
node call score sp25a
|
|
154
|
-
node call score sp25a 756mr
|
|
155
|
-
```
|
|
232
|
+
If the named array of acts includes an act of type `launch`, that act gets a `which` property, identical to the value of the `launch` property of the `placeholder` object. In this way, a placeholder can dictate which browser type is to be launched.
|
|
156
233
|
|
|
157
|
-
|
|
234
|
+
The `merge` module has added other properties to the job:
|
|
235
|
+
- a creation time
|
|
236
|
+
- a timestamp, derived from the creation time
|
|
237
|
+
- a job ID, composed of the timestamp, the script ID, and the target ID
|
|
238
|
+
- a `sources` property, identifying the script, the batch, the target within the batch, and an email address obtained from the environment variable `process.env.REQUESTER`.
|
|
158
239
|
|
|
159
|
-
|
|
240
|
+
This job is ready to be executed by Testaro.
|
|
160
241
|
|
|
161
|
-
|
|
242
|
+
If, however, you requested a merger **with** isolation, then `merge` would act as if another instance of
|
|
162
243
|
|
|
163
|
-
|
|
244
|
+
```javaScript
|
|
245
|
+
{
|
|
246
|
+
type: 'placeholder',
|
|
247
|
+
which: 'main',
|
|
248
|
+
launch: 'chromium'
|
|
249
|
+
},
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
were located in the script after the `axe` test, because the `axe` test is target-modifying and could therefore change the result of the `bulk` test that follows it. So, additional acts of type `launch` and `url` would appear after the `axe` test in the job.
|
|
253
|
+
|
|
254
|
+
### Invocation
|
|
164
255
|
|
|
165
|
-
|
|
256
|
+
There are two ways to use the `merge` module.
|
|
166
257
|
|
|
167
|
-
|
|
258
|
+
#### By a module
|
|
259
|
+
|
|
260
|
+
A module can invoke `merge` in this way:
|
|
168
261
|
|
|
169
262
|
```javaScript
|
|
170
|
-
const {
|
|
171
|
-
|
|
172
|
-
.then(hostCount => {
|
|
173
|
-
console.log(`Scoring complete. Count of reports scored: ${hostCount}`);
|
|
174
|
-
});
|
|
263
|
+
const {merge} = require('testilo/merge');
|
|
264
|
+
const jobs = merge(script, batch, requester, true);
|
|
175
265
|
```
|
|
176
266
|
|
|
177
|
-
|
|
267
|
+
This invocation references `script`, `batch`, and `requester` variables that the module must have already defined. The `script` and `batch` variables are a script object and a batch object, respectively. The `requester` variable is an email address. The fourth argument is a boolean, specifying whether to perform isolation; a missing fourth argument is equivalent to `false`. The `merge()` function of the `merge` module generates jobs and returns them in an array. The invoking module can further dispose of the jobs as needed.
|
|
178
268
|
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
269
|
+
#### By a user
|
|
270
|
+
|
|
271
|
+
A user can invoke `merge` in this way:
|
|
272
|
+
|
|
273
|
+
- Create a script and save it as a JSON file in the `scripts` subdirectory of the `process.env.SPECDIR` directory.
|
|
274
|
+
- Create a batch and save it as a JSON file in the `batches` subdirectory of the `process.env.SPECDIR` directory.
|
|
275
|
+
- In the Testilo project directory, execute one of these statements:
|
|
276
|
+
- `node call merge s b e true`
|
|
277
|
+
- `node call merge s b e false`
|
|
278
|
+
- `node call merge s b e`
|
|
279
|
+
|
|
280
|
+
In these statements, replace `s` and `b` with the base names of the script and batch files, respectively. For example, if the script file is named `ts25.json`, then replace `x` with `ts25`. Replace `e` with an email address, or with an empty string if the environment variable `process.env.REQUESTER` exists and you want to use it.
|
|
281
|
+
|
|
282
|
+
The first statement will cause a merger with isolation.
|
|
283
|
+
The second and third statements will cause a merger without isolation.
|
|
182
284
|
|
|
183
|
-
The
|
|
285
|
+
The `call` module will retrieve the named script and batch from their respective directories.
|
|
286
|
+
The `merge` module will create an array of jobs.
|
|
287
|
+
The `call` module will save the jobs in the `todo` subdirectory of the `process.env.JOBDIR` directory.
|
|
184
288
|
|
|
185
|
-
###
|
|
289
|
+
### Validation
|
|
186
290
|
|
|
187
|
-
|
|
291
|
+
To test the `merge` module, in the project directory you can execute the statement `node validation/merge/validate`. All logging statements should begin with “Success” and none should begin with “ERROR”.
|
|
188
292
|
|
|
189
|
-
|
|
293
|
+
## Report scoring
|
|
190
294
|
|
|
191
|
-
|
|
295
|
+
### Introduction
|
|
296
|
+
|
|
297
|
+
Testaro executes jobs and produces reports of test results. A report is identical to a job (see the example above), except that:
|
|
298
|
+
- The acts contain additional data recorded by Testaro to describe the results of the performance of the acts. Acts of type `test` have additional data describing test results (successes, failures, and details).
|
|
299
|
+
- Testaro also adds a `jobData` property, describing information not specific to any particular act.
|
|
300
|
+
|
|
301
|
+
Thus, a report produced by Testaro contains these properties:
|
|
302
|
+
- `id`
|
|
303
|
+
- `what`
|
|
304
|
+
- `strict`
|
|
305
|
+
- `timeLimit`
|
|
306
|
+
- `acts`
|
|
307
|
+
- `sources`
|
|
308
|
+
- `creationTime`
|
|
309
|
+
- `timeStamp`
|
|
310
|
+
- `jobData`
|
|
311
|
+
|
|
312
|
+
Testilo can add scores to a report. In this way, a report can not only detail successes and failures of individual tests but also assign scores to those results and combine the partial scores into total scores. The scores are contained in a new `score` property that Testilo adds to a report.
|
|
313
|
+
|
|
314
|
+
The `score` module scores a report. Its `score()` function takes two arguments:
|
|
315
|
+
- a scoring function
|
|
316
|
+
- an array of report objects
|
|
317
|
+
|
|
318
|
+
A scoring function defines scoring rules. The Testilo package contains a `procs/score` directory, in which there are modules that export scoring functions. You can use one of those scoring functions, or you can create your own.
|
|
319
|
+
|
|
320
|
+
### Invocation
|
|
321
|
+
|
|
322
|
+
There are two ways to use the `score` module.
|
|
323
|
+
|
|
324
|
+
#### By a module
|
|
325
|
+
|
|
326
|
+
A module can invoke `score` in this way:
|
|
192
327
|
|
|
193
328
|
```javaScript
|
|
194
|
-
const
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
const {makeQuery} = require(`${process.env.DIGESTPROCDIR}/${digestProcID}/index`);
|
|
198
|
-
const digestTemplate = await fs.readFile(
|
|
199
|
-
`${process.env.DIGESTPROCDIR}/${digestProcID}/index.html`
|
|
200
|
-
);
|
|
201
|
-
const digestedReport = digest(digestTemplate, makeQuery, scoredReport);
|
|
202
|
-
const digestedReport = digest(makeQuery, scoredReport);
|
|
203
|
-
await fs.writeFile(`${process.env.REPORTDIR_DIGESTED}/${digestedReport.id}.html`, digestedReport);
|
|
204
|
-
console.log(`Report ${digestedReport.id} digested`);
|
|
205
|
-
};
|
|
206
|
-
digestReport('dp25a', '756mr-ts25-w3c');
|
|
329
|
+
const {score} = require('testilo/score');
|
|
330
|
+
const {scorer} = require('testilo/procs/score/sp25a');
|
|
331
|
+
const scoredReports = score(scorer, rawReports);
|
|
207
332
|
```
|
|
208
333
|
|
|
209
|
-
|
|
334
|
+
The first argument to `score()` is a scoring function. In this example, it has been obtained from a JSON file in the Testilo package, but it could be a custom function. The second argument to `score()` is an array of report objects.
|
|
335
|
+
|
|
336
|
+
#### By a user
|
|
337
|
+
|
|
338
|
+
A user can invoke `score` in this way:
|
|
210
339
|
|
|
211
340
|
```bash
|
|
212
|
-
node call
|
|
213
|
-
node call
|
|
341
|
+
node call score sp25a
|
|
342
|
+
node call score sp25a 75
|
|
214
343
|
```
|
|
215
344
|
|
|
216
|
-
|
|
345
|
+
When a user invokes `score` in this example, the `call` module:
|
|
346
|
+
- gets the scoring module `sp25a` from its JSON file `sp25a.json` in the `process.env.SCOREPROCDIR` directory
|
|
347
|
+
- gets the reports from the r`raw` subdirectory of the `process.env.REPORTDIR` directory
|
|
348
|
+
- writes the scored reports to the `scored` subdirectory of the `process.env.REPORTDIR` directory
|
|
349
|
+
|
|
350
|
+
The optional third argument to call (`75` in this example) is a report selector. Without the argument, `call` gets all the reports in the `raw` subdirectory. With the argument, `call` gets only those reports whose names begin with the argument string.
|
|
351
|
+
|
|
352
|
+
### Validation
|
|
353
|
+
|
|
354
|
+
To test the `score` module, in the project directory you can execute the statement `node validation/score/validate`. All logging statements should begin with “Success” and none should begin with “ERROR”.
|
|
355
|
+
|
|
356
|
+
## Report digesting
|
|
357
|
+
|
|
358
|
+
### Introduction
|
|
217
359
|
|
|
218
|
-
|
|
360
|
+
Reports created by Testaro, both originally and after they are scored, are JavaScript objects. When represented as JSON, they are human-readable, but not human-friendly. They are basically designed for machine tractability. Testilo can _digest_ a scored report, converting it to a human-oriented HTML document, or _digest_.
|
|
219
361
|
|
|
220
|
-
The `digest`
|
|
362
|
+
The `digest` module digests a scored report. Its `digest()` function takes three arguments:
|
|
363
|
+
- a digest template
|
|
364
|
+
- a digesting function
|
|
365
|
+
- an array of report objects
|
|
221
366
|
|
|
222
|
-
|
|
367
|
+
The digest template is an HTML document containing placeholders. A copy of the template, with its placeholders replaced by texts, becomes the digest. The digesting function defines the rules for replacing the placeholders with texts. The Testilo package contains a `procs/digest` directory, in which there are subdirectories containing pairs of templates and modules that export digesting functions. You can use one of those pairs, or you can create your own.
|
|
223
368
|
|
|
224
|
-
|
|
369
|
+
### Invocation
|
|
225
370
|
|
|
226
|
-
|
|
371
|
+
There are two ways to use the `digest` module.
|
|
372
|
+
|
|
373
|
+
#### By a module
|
|
374
|
+
|
|
375
|
+
A module can invoke `digest` in this way:
|
|
227
376
|
|
|
228
377
|
```javaScript
|
|
229
|
-
const {
|
|
230
|
-
|
|
231
|
-
.
|
|
232
|
-
|
|
378
|
+
const {digest} = require('testilo/digest');
|
|
379
|
+
const digesterDir = `${process.env.FUNCTIONDIR}/digest/dp25a`;
|
|
380
|
+
fs.readFile(`${digesterDir}/index.html`)
|
|
381
|
+
.then(template => {
|
|
382
|
+
const {digester} = require(`${digesterDir}/index`);
|
|
383
|
+
const digestedReports = digest(template, digester, scoredReports);
|
|
233
384
|
});
|
|
234
385
|
```
|
|
235
386
|
|
|
236
|
-
|
|
387
|
+
The first two arguments to `digest()` are a digest template and a digesting function. In this example, they have been obtained from files in the Testilo package, but they could be custom-made. The third argument to `digest()` is an array of report objects.
|
|
388
|
+
|
|
389
|
+
#### By a user
|
|
390
|
+
|
|
391
|
+
A user can invoke `digest` in this way:
|
|
237
392
|
|
|
238
393
|
```bash
|
|
239
|
-
node call
|
|
394
|
+
node call digest dp25a
|
|
395
|
+
node call digest dp25a 75
|
|
240
396
|
```
|
|
241
397
|
|
|
242
|
-
|
|
398
|
+
When a user invokes `digest` in this example, the `call` module:
|
|
399
|
+
- gets the template and the digesting module from subdirectory `dp25a` in the `digest` subdirectory of the `process.env.FUNCTIONDIR` directory
|
|
400
|
+
- gets the reports from the `scored` subdirectory of the `process.env.REPORTDIR` directory
|
|
401
|
+
- writes the digested reports to the `digested` subdirectory of the `process.env.REPORTDIR` directory
|
|
402
|
+
|
|
403
|
+
The optional third argument to call (`75` in this example) is a report selector. Without the argument, `call` gets all the reports in the `scored` subdirectory of the `process.env.REPORTDIR` directory. With the argument, `call` gets only those reports whose names begin with the argument string.
|
|
243
404
|
|
|
244
|
-
The digests created by `digest` are HTML files, and they expect a `style.css` file to exist in their directory. The `reports/digested/style.css` file in Testilo is an appropriate stylesheet to be copied into the
|
|
405
|
+
The digests created by `digest` are HTML files, and they expect a `style.css` file to exist in their directory. The `reports/digested/style.css` file in Testilo is an appropriate stylesheet to be copied into the directory where digested reports are written.
|
|
245
406
|
|
|
246
|
-
###
|
|
407
|
+
### Validation
|
|
247
408
|
|
|
248
|
-
|
|
409
|
+
To test the `digest` module, in the project directory you can execute the statement `node validation/digest/validate`. All logging statements should begin with “Success” and none should begin with “ERROR”.
|
|
249
410
|
|
|
250
|
-
|
|
411
|
+
### Report comparison
|
|
412
|
+
|
|
413
|
+
If you use Testilo to perform a battery of tests on multiple targets, you may want a single report that compares the total scores received by the targets. Testilo can produce such a _comparative report_.
|
|
414
|
+
|
|
415
|
+
The `compare` module compares the scores in a collection of scored reports. Its `compare()` function takes three arguments:
|
|
416
|
+
- a comparison template
|
|
417
|
+
- a comparison function
|
|
418
|
+
- an array of scored reports
|
|
419
|
+
|
|
420
|
+
The comparison template is an HTML document containing placeholders. A copy of the template, with its placeholders replaced by texts, becomes the comparative report. The comparison function defines the rules for replacing the placeholders with texts. The Testilo package contains a `procs/compare` directory, in which there are subdirectories containing pairs of templates and modules that export comparison functions. You can use one of those pairs, or you can create your own.
|
|
421
|
+
|
|
422
|
+
### Invocation
|
|
423
|
+
|
|
424
|
+
There are two ways to use the `compare` module.
|
|
425
|
+
|
|
426
|
+
#### By a module
|
|
427
|
+
|
|
428
|
+
A module can invoke `compare` in this way:
|
|
251
429
|
|
|
252
430
|
```javaScript
|
|
431
|
+
const fs = require('fs/promises);
|
|
253
432
|
const {compare} = require('testilo/compare');
|
|
254
|
-
compare
|
|
255
|
-
.
|
|
256
|
-
|
|
433
|
+
const comparerDir = `${process.env.FUNCTIONDIR}/compare/cp25a`;
|
|
434
|
+
fs.readFile(`${comparerDir}/index.html`)
|
|
435
|
+
.then(template => {
|
|
436
|
+
const {comparer} = require(`${comparerDir}/index`);
|
|
437
|
+
const comparativeReports = compare(template, comparer, scoredReports);
|
|
257
438
|
});
|
|
258
439
|
```
|
|
259
440
|
|
|
260
|
-
|
|
441
|
+
The first two arguments to `compare()` are a template and a comparison function. In this example, they have been obtained from files in the Testilo package, but they could be custom-made. The third argument to `compare()` is an array of report objects.
|
|
442
|
+
|
|
443
|
+
#### By a user
|
|
444
|
+
|
|
445
|
+
A user can invoke `compare` in this way:
|
|
261
446
|
|
|
262
447
|
```bash
|
|
263
448
|
node call compare cp25a legislators
|
|
264
449
|
```
|
|
265
450
|
|
|
266
|
-
|
|
451
|
+
When a user invokes `compare` in this example, the `call` module:
|
|
452
|
+
- gets the template and the comparison module from subdirectory `cp25a` of the subdirectory `compare` in the `process.env.FUNCTIONDIR` directory
|
|
453
|
+
- gets all the reports in the `scored` subdirectory of the `process.env.REPORTDIR` directory
|
|
454
|
+
- writes the comparative report as an HTML file named `legislators.html` to the `comparative` subdirectory of the `process.env.REPORTDIR` directory
|
|
455
|
+
|
|
456
|
+
The comparative reports created by `compare` are HTML files, and they expect a `style.css` file to exist in their directory. The `reports/comparative/style.css` file in Testilo is an appropriate stylesheet to be copied into the directory where comparative reports are written.
|
|
457
|
+
|
|
458
|
+
### Validation
|
|
267
459
|
|
|
268
|
-
|
|
460
|
+
To test the `compare` module, in the project directory you can execute the statement `node validation/compare/validate`. All logging statements should begin with “Success” and none should begin with “ERROR”.
|