testilo 22.1.0 → 23.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,7 +5,7 @@ Utilities for Testaro
5
5
 
6
6
  The Testilo package contains utilities that facilitate the use of the [Testaro](https://www.npmjs.com/package/testaro) package.
7
7
 
8
- Testaro performs digital accessibility tests on web artifacts and creates reports in JSON format. The utilities in Testilo fall into two categories:
8
+ Testaro performs jobs and creates reports in JSON format. The utilities in Testilo fall into two categories:
9
9
  - Job preparation
10
10
  - Report enhancement
11
11
 
@@ -21,24 +21,34 @@ When Testilo is a dependency of another application, the `.env` file is not impo
21
21
 
22
22
  Testilo is written in Node.js. Commands are given to Testilo in a command-line (terminal) interface or programmatically.
23
23
 
24
- Shared routines are _procs_ and are located in the `procs` directory.
24
+ Shared routines, called _procs_, are located in the `procs` directory.
25
25
 
26
26
  Testilo can be installed wherever Node.js (version 14 or later) is installed. This can be a server or the same workstation on which Testaro is installed.
27
27
 
28
- The reason for Testilo being an independent package, rather than part of Testaro, is that Testilo can be installed on any host, while Testaro can run successfully only on a Windows or Macintosh workstation (and perhaps on some workstations with Ubuntu operating systems). Testaro runs tests similar to those that a human accessibility tester would run, using whatever browsers, input devices, system settings, simulated and attached devices, and assistive technologies tests may require. Thus, Testaro is limited to functionalities that require workstation attributes. For maximum flexibility in the management of Testaro jobs, all other functionalities are located outside of Testaro. You could have software such as Testilo running on a server, communicating with multiple workstations running Testaro. The workstations could receive job orders from the server and return job results to the server for further processing.
28
+ The reason for Testilo being an independent package, rather than part of Testaro, is that Testilo can be installed on any host, while Testaro can run successfully only on a Windows, Macintosh, Ubuntu, or Debian workstation. Testaro runs tests similar to those that a human accessibility tester would run, using whatever browsers, input devices, system settings, simulated and attached devices, and assistive technologies tests may require. Thus, Testaro is limited to functionalities that require workstation attributes. For maximum flexibility in the management of Testaro jobs, all other functionalities are located outside of Testaro. You could have software such as Testilo running on a server, communicating with multiple workstations running Testaro. The workstations could receive jobs from the server and return job reports to the server for further processing.
29
29
 
30
30
  ## Configuration
31
31
 
32
32
  Environment variables for Testilo can be specified in a `.env` file. An example:
33
33
 
34
34
  ```bash
35
- FUNCTIONDIR=../testdir/procs
35
+ FUNCTIONDIR=./procs
36
36
  JOBDIR=../testdir/jobs
37
37
  REPORTDIR=../testdir/reports
38
38
  REQUESTER=a11ymgr@a11yorg.com
39
39
  SPECDIR=../testdir/specs
40
40
  ```
41
41
 
42
+ The `FUNCTIONDIR` environment variable typically references the `procs` directory, but it could reference a different directory in the filesystem where Testilo resides, if you wanted to customize the procs that Testilo uses.
43
+
44
+ `JOBDIR` references a directory in the filesystem where jobs created by the `merge` proc are to be saved.
45
+
46
+ `REPORTDIR` references a directory in the filesystem where reports are saved.
47
+
48
+ `REQUESTER` is an email address that will be used as a job property if no other email address is specified for the `sources.requester` property of the job.
49
+
50
+ `SPECDIR` references a directory in the filesystem where tanrget lists, batches, and scripts can be found. Those are raw materials from which Testaro creates jobs.
51
+
42
52
  ## Job preparation
43
53
 
44
54
  ### Introduction
@@ -49,16 +59,27 @@ You can create a job for Testaro directly, without using Testilo.
49
59
 
50
60
  Testilo can, however, make job preparation more efficient in these scenarios:
51
61
  - You want to perform a battery of tests on multiple targets.
52
- - You want to test targets for particular issues, using whichever tools happen to have tests for those issues.
62
+ - You want to test targets only for particular issues, using whichever tools happen to have tests for those issues.
53
63
 
54
64
  ### Target lists
55
65
 
56
- The simplest version of a list of targets is a _target list_. It is an array of arrays defining 1 or more targets. It is stored as a tab-delimited text file, with one line per target. Each line contains 3 items, with tabs between them:
57
- - An ID for the target
58
- - A description of the target
59
- - The URL of the target
66
+ The simplest version of a list of targets is a _target list_. It is an array of arrays defining 1 or more targets. It can be stored as a tab-delimited text file.
67
+
68
+ A target is defined by 3 items:
69
+ - An ID
70
+ - A description
71
+ - A URL
60
72
 
61
- For example, a stored target list (with “→” representing the Tab character) might be:
73
+ For example, a target list might be:
74
+
75
+ ```javaScript
76
+ [
77
+ ['w3c', 'World Wide Web Consortium', 'https://www.w3.org/'],
78
+ ['moz', 'Mozilla Foundation', 'https://foundation.mozilla.org/en/']
79
+ ]
80
+ ```
81
+
82
+ If this target list were stored as a file, its content would be this (with “→” representing the Tab character):
62
83
 
63
84
  ```text
64
85
  w3c→World Wide Web Consortium→https://www.w3.org/
@@ -128,9 +149,11 @@ Targets can be specified in a more complex way, too. That allows you to create j
128
149
 
129
150
  As shown, a batch, unlike a target list, defines named sequences of acts. They can be plugged into jobs, so various complex operations can be performed on each target.
130
151
 
152
+ A batch is a JavaScript object. It can be converted to JSON and stored in a file.
153
+
131
154
  ### Scripts
132
155
 
133
- The generic, target-independent description of a job is _script_. A script can contain _placeholders_ that Testilo replaces with acts from a batch, creating one job per target. Thus, one script plus one batch can generate an unlimited number of jobs.
156
+ The generic, target-independent description of a job is _script_. A script can contain _placeholders_ that Testilo replaces with acts from a batch, creating one job per target. Thus, one script plus a batch containing _n_ targets will generate _n_ jobs.
134
157
 
135
158
  Here is a script:
136
159
 
@@ -144,6 +167,9 @@ Here is a script:
144
167
  standard: 'also',
145
168
  observe: false,
146
169
  timeStamp: '240115T1200',
170
+ requester: 'you@yourdomain.com',
171
+ urlPrefix: '/file/reports',
172
+ urlSuffix: '.json',
147
173
  acts: [
148
174
  {
149
175
  type: 'placeholder',
@@ -169,13 +195,16 @@ Here is a script:
169
195
  ```
170
196
 
171
197
  A script has several properties that specify facts about the jobs to be created. They include:
172
- - `id`: an ID that uniquely distinguishes the script from other scripts.
198
+ - `id`: an ID. A script can be converted from a JavaScript object to JSON and saved in a file in the `SPECDIR` directory, where it will be named by its ID (e.g., if the ID is `ts99`, the file name will be `ts99.json`). Thus, each script needs an `id` with a unique value.
173
199
  - `what`: a description of the script.
174
- - `strict`: You decide whether Testaro is to throw an error on an attempt to navigate to a URL if the server redirects the request to a URL differing substantially from the specified URL. All differences are considered substantial unless the URLs differ only in the presence and absence of a trailing slash.
175
- - `isolate`: You decide whether to isolate test acts, as needed, from effects of previous test acts. If `true`, the `merge` module will add a copy of the latest placeholder after each target-modifying test act before the immediately following test act.
176
- - `standard`: You choose how the reports of test acts should be standardized. The alternatives are `'no'` (do not standardize), `'also'` (standardize and report both the original and the standardized results), and `'only'` (standardize and report only the standardized results).
177
- - `observe`: You decide how granularly Testaro will allow a server to observe job progress. If `true`, Testaro sends a message to the server to announce each test act (identifying the tool), and the `testaro` tool sends a message to the server to announce each rule when its test is performed. The server can further update the requesting client on the basis of these messages. It is generally user-friendly to make `observe` `true` if the user application makes the user wait while the job is assigned and performed. If the application allows the user to leave and sends the user a message when the job has been completed, `observe` can be set to `false`.
178
- - `timeStamp`: You can specify the date and time that the job is to wait for before it is performed. This specification is a string with the format `yymmddThhMM`.
200
+ - `strict`: `true` if Testaro is to abort jobs when a target redirects a request to a URL differing substantially from the one specified. If `false` Testaro is to allow redirection. All differences are considered substantial unless the URLs differ only in the presence and absence of a trailing slash.
201
+ - `isolate`: If `true`, Testilo, before creating a job, will isolate test acts, as needed, from effects of previous test acts, by inserting a copy of the latest placeholder after each target-modifying test act other than the final act. If `false`, placeholders will not be duplicated.
202
+ - `standard`: If `also`, jobs will tell Testaro to include in its reports both the original results of the tests of tools and the Testaro-standardized results. If `only`, reports are to include only the standardized test results. If `no`, reports are to include only the original results, without standardization.
203
+ - `observe`: If `true`, jobs will tell Testaro to allow granular observation of job progress. If `false`, jobs will tell Testaro not to permit granular observation, but only to send the report to the server when the report is completed. It is generally user-friendly to allow granular observation, and for user applications to implement it, if they make users wait while jobs are assigned and performed, since that process typically takes about 3 minutes.
204
+ - `timeStamp`: This string specifies a UTC date and time when jobs created with the script are to be permitted to be assigned to agents. Thus, jobs can be created from a script for later performance. The value of `timeStamp` is a compact representation in the format `yymmddThhMM`. If the value is an empty string, Testilo will make the date and time equal to the time when jobs are created.
205
+ - `requester`: the email address that any notices of job completion can be sent to, or an empty string if there is a `REQUESTER` environment variable and it is to be used.
206
+ - `urlPrefix`: the start of a URL that the Testaro reports will be retrievable at.
207
+ - `urlSuffix`: the end of that URL. The job ID will be inserted between the `urlPrefix` and the `urlSuffix`.
179
208
  - `acts`: an array of acts.
180
209
 
181
210
  The first act in this example script is a placeholder, whose `which` property is `'private'`. If the above batch were merged with this script, in each job the placeholder would be replaced with the `private` acts of a target. For example, the first act of the first job would launch a Chromium browser, navigate to the Acme login page, complete and submit the login form, wait for the account page to load, run the Axe tests, and then run the QualWeb tests. If the batch contained additional targets, additional jobs would be created, with the login actions for each target specified in the `private` array of the `acts` object of that target.
@@ -207,22 +236,22 @@ The `batch()` function of the `batch` module generates a batch and returns it as
207
236
 
208
237
  A user can invoke `batch` in this way:
209
238
 
210
- - Create a target list and save it as a text file (with tab-delimited items in newline-delimited lines) in the `targetLists` subdirectory of the `process.env.SPECDIR` directory. Name the file `x.tsv`, where `x` is the list ID.
211
- - In the Testilo project directory, execute the statement `node call batch i w`.
239
+ - Create a target list and save it as a text file (with tab-delimited items in newline-delimited lines) in the `targetLists` subdirectory of the `SPECDIR` directory. Name the file `x.tsv`, where `x` is the list ID.
240
+ - In the Testilo project directory, execute the statement `node call batch id what`.
212
241
 
213
- In this statement, replace `i` with the list ID and `w` with a description of the batch.
242
+ In this statement, replace `id` with the list ID and `what` with a string describing the batch.
214
243
 
215
244
  The `call` module will retrieve the named target list.
216
245
  The `batch` module will convert the target list to a batch.
217
- The `call` module will save the batch as a JSON file in the `batches` subdirectory of the `process.env.SPECDIR` directory.
246
+ The `call` module will save the batch as a JSON file in the `batches` subdirectory of the `SPECDIR` directory.
218
247
 
219
248
  ### Issues to script
220
249
 
221
- Testilo classifies tool rules into _issues_. The built-in classifications are located in the `procs/score` directory, in files whose names begin with `tic` (for “Testilo issue classification). You can create additional `tic` files with custom classifications.
250
+ Testilo contains a classification of tool rules into _issues_. It is located in the `procs/score` directory and has a file name starting with `tic` (Testilo issue classification). You can create custom classifications and save them in a `score` subdirectory of the `FUNCTIONDIR` directory.
222
251
 
223
252
  For example, one of the issues in the `tic40.js` file is `mainNot1`. Four rules are classified as belonging to that issue: rule `main_element_only_one` of `aslint` and 3 more rules defined by 3 other tools.
224
253
 
225
- If you want Testaro to test targets for only particular issues, you can name those issues and use the Testilo `script` module to create a script for that purpose. The only tools called by the script will be tools that define rules that are classified as belonging to one or more issues named.
254
+ If you want Testaro to test targets for only particular issues, you can use the `script` module to create a script. Jobs created from that script will make Testaro test for only the issues you specify to the `script` module.
226
255
 
227
256
  If you want Testaro to test targets for **all** the rules of all the available tools, you can use the `script` module to create a script that does not impose any issue restrictions.
228
257
 
@@ -240,9 +269,9 @@ const scriptObj = script(scriptID, issues, issueID0, issueID1, …);
240
269
  ```
241
270
 
242
271
  This invocation references `scriptID`, `issues`, and `issueID` variables.
243
- - The `scriptID` variable is an arbitrary alphanumeric string.
272
+ - The `scriptID` variable specifies the ID that the script will have.
244
273
  - The `issues` variable (if present) is an object that classifies issues, such as the `issues` object in a `tic` file.
245
- - The `issueID` variables (if any) are strings, such as `'regionNoText'`, that name properties of the `issues` object.
274
+ - The `issueID` variables (if any) are strings, such as `'regionNoText'`, that name issues, i.e. properties of the `issues` object, that you want jobs from the script to test for.
246
275
 
247
276
  The `script()` function of the `script` module generates a script and returns it as an object. The invoking module can further modify and use the script as needed.
248
277
 
@@ -255,41 +284,40 @@ const scriptObj = script(scriptID);
255
284
 
256
285
  ##### By a user
257
286
 
258
- A user can invoke `script` in this way: In the Testilo project directory, execute the statement `node call script s c i0 i1 i2 i3 …`.
287
+ A user can invoke `script` in this way: In the Testilo project directory, execute the statement `node call script id ticnn issuea issueb …`.
259
288
 
260
289
  In this statement:
261
- - Replace `s` with an arbitrary ID for the script, such as `headings`.
262
- - Replace `c` with the base name, such as `tic99`, of an issue classification file in the `score` subdirectory of the `process.env.FUNCTIONDIR` directory.
263
- - Replace the remaining arguments (`i0` etc.) with issue IDs from that classification file.
290
+ - Replace `id` with an ID for the script, such as `headings`.
291
+ - Replace `ticnn` with the base, such as `tic99`, of the name of an issue classification file in the `score` subdirectory of the `FUNCTIONDIR` directory.
292
+ - Replace the remaining arguments (`issuea` etc.) with issue names from that classification file.
264
293
 
265
294
  The `call` module will retrieve the named classification.
266
295
  The `script` module will create a script.
267
- The `call` module will save the script as a JSON file in the `scripts` subdirectory of the `process.env.SPECDIR` directory.
296
+ The `call` module will save the script as a JSON file in the `scripts` subdirectory of the `SPECDIR` directory.
268
297
 
269
- To create a script without any issue restrictions, a user can execute the statement `node call script s`.
298
+ To create a script without any issue restrictions, a user can execute the statement `node call script id`.
270
299
 
271
300
  #### Options
272
301
 
273
- When the `script` module creates a script for you, it does not ask you for all of the options that the script may require. Instead, it chooses options. After you invoke `script`, you can edit the script that it creates to revise options.
302
+ When the `script` module creates a script for you, it does not ask you for all of the options that the script may require. Instead, it chooses default options. After you invoke `script`, you can edit the script that it creates to revise options.
274
303
 
275
304
  ### Merge
276
305
 
277
306
  Testilo merges batches with scripts, producing jobs, by means of the `merge` module.
278
307
 
279
- The `merge` module needs to be given a batch and a script.
280
-
281
308
  #### Output
282
309
 
283
- ##### Without isolation
284
-
285
- Suppose you ask for a merger of the above batch and script, **without** the isolation option. Then the first job produced by `merge` will look like this:
310
+ Suppose you ask for a merger of the above batch and script. Then the first job produced by `merge` will look like this:
286
311
 
287
312
  ```javaScript
288
313
  {
289
- id: '231120T1550-ts99-acme',
290
- what: 'Axe on account page',
314
+ id: '240115T1200-4Rw-acme',
315
+ what: 'aside mislocation',
291
316
  strict: true,
292
317
  timeLimit: 60,
318
+ standard: 'also',
319
+ observe: false,
320
+ timeStamp: '240115T1200',
293
321
  acts: [
294
322
  {
295
323
  type: 'launch',
@@ -323,15 +351,45 @@ Suppose you ask for a merger of the above batch and script, **without** the isol
323
351
  {
324
352
  type: 'test',
325
353
  which: 'axe',
326
- detailLevel: 1,
327
- rules: [],
328
- what: 'Axe core, all rules'
354
+ detailLevel: 2,
355
+ rules: ['landmark-complementary-is-top-level'],
356
+ what: 'Axe'
357
+ },
358
+ {
359
+ type: 'launch',
360
+ which: 'chromium'
361
+ },
362
+ {
363
+ type: 'url',
364
+ which: 'https://acmeclothes.com/login.html',
365
+ what: 'Acme Clothes login page'
366
+ },
367
+ {
368
+ type: 'text',
369
+ which: 'User Name',
370
+ what: 'tester34'
371
+ },
372
+ {
373
+ type: 'text',
374
+ which: 'Password',
375
+ what: '34SecretTester'
376
+ },
377
+ {
378
+ type: 'button',
379
+ which: 'Submit',
380
+ what: 'submit the login form'
381
+ },
382
+ {
383
+ type: 'wait',
384
+ which: 'title',
385
+ what: 'account'
329
386
  },
330
387
  {
331
388
  type: 'test',
332
389
  which: 'qualWeb',
333
390
  withNewContent: false,
334
- what: 'QualWeb, all rules'
391
+ rules: ['QW-BP25', 'QW-BP26']
392
+ what: 'QualWeb'
335
393
  }
336
394
  ],
337
395
  sources: {
@@ -341,34 +399,22 @@ Suppose you ask for a merger of the above batch and script, **without** the isol
341
399
  id: 'acme',
342
400
  what: 'Acme Clothes'
343
401
  },
344
- requester: 'you@yourdomain.tld'
402
+ requester: 'you@yourdomain.tld',
403
+ url: '/file/reports/240115T1200-4Rw-acme.json'
345
404
  },
346
- creationTime: '2023-11-20T15:50:27',
347
- timeStamp: '231120T155314'
405
+ creationTime: '2023-11-20T15:50:27'
348
406
  }
349
407
  ```
350
408
 
351
409
  Testilo has substituted the `private` acts from the `acme` target of the batch for the placeholder when creating the job. Testilo also has:
410
+ - inserted a copy of those same acts after the `axe` test act, because `axe` is a target-modifying tool.
352
411
  - let the script determine the browser type of the `launch` act.
353
412
  - added the creation time to the job.
354
- - added a unique timestamp to the job (a more compact representation of the creation time).
355
- - given the job an ID that combines the timestamp with the script ID and the batch ID.
356
- - inserted a `sources` property into the job, recording facts about the script, the batch, the target, and the email address given by the user who requested the merger.
413
+ - given the job an ID that combines the time stamp with a differentiator and the batch ID.
414
+ - inserted a `sources` property into the job, recording facts about the script, the batch, the target, the requester, and the report URL.
357
415
 
358
416
  This is a valid Testaro job.
359
417
 
360
- ##### With isolation
361
-
362
- If, however, you requested a merger **with** isolation, then Testilo would take cognizance of the fact that an `axe` test act is a target-modifying act. Testilo would therefore act as if another instance of the placeholder had been located in the script after the `axe` test act. So, copies of the same 6 acts that precede the `axe` test act would be inserted **after** the `axe` test act, too.
363
-
364
- Of the 9 tools providing tests for Testaro, 6 are target-modifying:
365
- - `alfa`
366
- - `aslint`
367
- - `axe`
368
- - `htmlcs`
369
- - `ibm`
370
- - `testaro`
371
-
372
418
  #### Invocation
373
419
 
374
420
  There are two ways to use the `merge` module.
@@ -379,10 +425,25 @@ A module can invoke `merge` in this way:
379
425
 
380
426
  ```javaScript
381
427
  const {merge} = require('testilo/merge');
382
- const jobs = merge(script, batch, requester, true, 'only', false, '240115T1200', 'jobs/', '.json');
428
+ const jobs = merge(
429
+ script, batch, requester, isolate, standard, isGranular, timeStamp, 'file/reports/', '.json'
430
+ );
383
431
  ```
384
432
 
385
- This invocation references `script`, `batch`, and `requester` variables that the module must have already defined. The `script` and `batch` variables are a script object and a batch object, respectively. The `requester` variable is an email address. The fourth argument is a boolean, specifying whether to perform test isolation. The fifth argument is a string that specifies the Testaro standardization option ('also', 'only', or 'no'). The sixth argument is a boolean, specifying whether Testaro will allow granular network observation of the job. The seventh and eights arguments are strings that contain the parts, before and after the job ID, respectively, of the absolute or relative URL for retrieving the job report. In this case, a job with ID `20240420T1426-R7T-archive` will produce a report that can be retrieved at the relative URL `jobs/20240420T1426-R7T-archive.json`.
433
+ The `merge` module uses these 9 arguments to create jobs from a script and a batch.
434
+
435
+ The arguments are:
436
+ - `script`: a script.
437
+ - `batch`: a batch.
438
+ - `requester`: an email address, or an empty string if there is a `REQUESTER` environment variable to be used.
439
+ - `isolate`: `true` if test isolation is to be performed, or `false` if not.
440
+ - `standard`: one of the standardization options (`'also'`, `'only'`, or `'no'`).
441
+ - `isGranular`: `true` if Testaro is to allow granular observation of the job when performed under a network watch, or `false` if not.
442
+ - `timeStamp`: a time stamp in the format `240115T1200`.
443
+ - `urlPnrefix`: the start of an absolute or relative report URL.
444
+ - `urlSuffix`: the end of a report URL.
445
+
446
+ In this case, a job with ID `240115T1200-4Rw-acme` will produce a report that can be retrieved at the relative URL `/file/reports/240115T1200-4Rw-acme.json`.
386
447
 
387
448
  The `merge()` function of the `merge` module generates jobs and returns them in an array. The invoking module can further dispose of the jobs as needed.
388
449
 
@@ -390,76 +451,29 @@ The `merge()` function of the `merge` module generates jobs and returns them in
390
451
 
391
452
  A user can invoke `merge` in this way:
392
453
 
393
- - Create a script and save it as a JSON file in the `scripts` subdirectory of the `process.env.SPECDIR` directory.
394
- - Create a batch and save it as a JSON file in the `batches` subdirectory of the `process.env.SPECDIR` directory.
395
- - In the Testilo project directory, execute this statement:
396
- - `node call merge scriptName batchName email isolate standard granular todoDir pre post`
454
+ - Create a script and save it as a JSON file in the `scripts` subdirectory of the `SPECDIR` directory.
455
+ - Create a batch and save it as a JSON file in the `batches` subdirectory of the `SPECDIR` directory.
456
+ - In the Testilo project directory, execute the statement `node call merge scriptName batchName requester isolate standard observe todoDir urlPrefix urlSuffix`.
397
457
 
398
- In these statements, replace:
399
- - `scriptName` with the base name of the script file
400
- - `batchName` with the base name of the batch file
401
- - `email` with an email address, or with an empty string if the environment variable `process.env.REQUESTER` exists and you want to use it
402
- - `isolate` with `true` if you want test isolation or `false` if not
458
+ In this statement, replace:
459
+ - `scriptName` with the base name of the script file.
460
+ - `batchName` with the base name of the batch file.
461
+ - `requester` with an email address, or with an empty string if the environment variable `REQUESTER` exists and you want to use it.
462
+ - `isolate` with `true` if you want test isolation or `false` if not.
403
463
  - `standard` with `'also'`, `'only'`, or `'no'` to specify the treatment of standard-format results.
404
- - `granular` with `true` if granular observation is to be permitted, or `false` if not.
405
- - `todoDir` with `true` if the job is to be saved in the `todo` subdirectory or `false` if it is to be saved in the `pending` subdirectory of the `process.env.JOBDIR` directory.
406
- - `pre` with the pre-ID part of the report URL.
407
- - `post` with the post-ID part of the report URL.
464
+ - `observe` with `true` if granular observation is to be permitted, or `false` if not.
465
+ - `todoDir` with `true` if the job is to be saved in the `todo` subdirectory, or `false` if it is to be saved in the `pending` subdirectory, of the `JOBDIR` directory.
466
+ - `urlPrefix` with the pre-ID part of the report URL.
467
+ - `urlSuffix` with the post-ID part of the report URL.
408
468
 
409
469
  The `call` module will retrieve the named script and batch from their respective directories.
410
470
  The `merge` module will create an array of jobs, with or without test isolation.
411
- The `call` module will save the jobs as JSON files in the `todo` or `pending` subdirectory of the `process.env.JOBDIR` directory.
471
+ The `call` module will save the jobs as JSON files in the `todo` or `pending` subdirectory of the `JOBDIR` directory.
412
472
 
413
473
  #### Validation
414
474
 
415
475
  To test the `merge` module, in the project directory you can execute the statement `node validation/merge/validate`. If `merge` is valid, all logging statements will begin with “Success” and none will begin with “ERROR”.
416
476
 
417
- ### Series
418
-
419
- If you want to monitor a web resource by performing identical jobs repeatedly and comparing the results, you can use the `series` module to create a series of identical jobs.
420
-
421
- The jobs in a series differ from one another only in the timestamp segments of their `id` properties. For example, if the first job had the `id` value `240528T1316-mon-mozilla` and the events in the series occurred at intervals of 12 hours, then the second job would have the `id` value `240529T0116-mon-mozilla`.
422
-
423
- The `series` module adds a `sources.series` property to each job in the series. The value of that property is the `id` value of the first job in the series.
424
-
425
- To support monitoring, a server that receives job requests from testing agents can perform a time check on the first job in the queue. If the time specified by the `id` of the first job is in the future, the server can reply that there is no job to do.
426
-
427
- #### Invocation
428
-
429
- There are two ways to use the `series` module.
430
-
431
- ##### By a module
432
-
433
- A module can invoke `series` in this way:
434
-
435
- ```javaScript
436
- const {series} = require('testilo/series');
437
- const jobs = series(job, count, interval);
438
- ```
439
-
440
- This invocation references a `job` variable, whose value is a job object. The `count` variable is an integer, 2 or greater, specifying how many events the series consists of. The `interval` variable is an integer, 1 or greater, specifying how many minutes are to elapse after each event before the next event. The `series()` function of the `series` module generates an array of job objects and returns the array. The invoking module can further dispose of the jobs as needed.
441
-
442
- ##### By a user
443
-
444
- A user can invoke `series` in this way:
445
-
446
- - Create a job and save it as a JSON file in the `todo` subdirectory of the `process.env.JOBDIR` directory.
447
- - In the Testilo project directory, execute this statement:
448
- - `node call series j c i`
449
-
450
- In this statement, replace:
451
- - `j` with a string that the filename of the starting job begins with
452
- - `c` with a count
453
- - `i` with an interval in minutes
454
-
455
- The `call` module will retrieve the first job that matches `j` from the `pending` subdirectory of the `process.env.JOBDIR` directory.
456
- The `series` module will create an array of jobs.
457
- The `call` module will save the jobs as JSON files in the `todo` subdirectory of the `process.env.JOBDIR` directory.
458
-
459
- #### Validation
460
-
461
- To test the `series` module, in the project directory you can execute the statement `node validation/series/validate`. If `series` is valid, all logging statements will begin with “Success” and none will begin with “ERROR”.
462
-
463
477
  ## Report scoring
464
478
 
465
479
  ### Introduction
package/call.js CHANGED
@@ -24,8 +24,6 @@ const {batch} = require('./batch');
24
24
  const {script} = require('./script');
25
25
  // Function to process a merger.
26
26
  const {merge} = require('./merge');
27
- // Function to generate a job series.
28
- const {series} = require('./series');
29
27
  // Function to score reports.
30
28
  const {score} = require('./score');
31
29
  // Function to digest reports.
@@ -99,29 +97,6 @@ const callMerge = async (
99
97
  `Script ${scriptID} and batch ${batchID} merged as ${timeStamp}-… in ${jobDir}/${destination}`
100
98
  );
101
99
  };
102
- // Fulfills a series request.
103
- const callSeries = async (idStart, count, interval) => {
104
- // Get the initial job.
105
- const jobNames = await fs.readdir(`${jobDir}/pending`);
106
- const seriesJobName = jobNames.find(jobName => jobName.startsWith(idStart));
107
- // If it exists:
108
- if (seriesJobName) {
109
- // Generate a job series.
110
- const jobJSON = await fs.readFile(`${jobDir}/todo/${seriesJobName}`, 'utf8');
111
- const job = JSON.parse(jobJSON);
112
- const jobSeries = series(job, Number.parseInt(count), Number.parseInt(interval));
113
- // Save the jobs.
114
- for (const item of jobSeries) {
115
- await fs.writeFile(`${jobDir}/todo/${item.id}.json`, `${JSON.stringify(item, null, 2)}\n`);
116
- }
117
- console.log(`Series of ${jobSeries.length} jobs generated and saved in ${jobDir}/todo`);
118
- }
119
- // Otherwise, i.e. if it does not exist:
120
- else {
121
- // Report this.
122
- console.log('ERROR: No matching to-do job found');
123
- }
124
- };
125
100
  // Gets selected reports.
126
101
  const getReports = async (type, selector = '') => {
127
102
  const allFileNames = await fs.readdir(`${reportDir}/${type}`);
@@ -253,12 +228,6 @@ else if (fn === 'merge' && fnArgs.length === 9) {
253
228
  console.log('Execution completed');
254
229
  });
255
230
  }
256
- else if (fn === 'series' && fnArgs.length === 3) {
257
- callSeries(... fnArgs)
258
- .then(() => {
259
- console.log('Execution completed');
260
- });
261
- }
262
231
  else if (fn === 'score' && fnArgs.length > 0 && fnArgs.length < 3) {
263
232
  callScore(... fnArgs)
264
233
  .then(() => {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testilo",
3
- "version": "22.1.0",
3
+ "version": "23.0.0",
4
4
  "description": "Prepares and processes Testaro reports",
5
5
  "main": "aim.js",
6
6
  "scripts": {
package/series.js DELETED
@@ -1,72 +0,0 @@
1
- /*
2
- series.js
3
- Generates a series of Testaro jobs.
4
- Arguments:
5
- 0. Initial job.
6
- 1. Job count.
7
- 2. Time interval in minutes.
8
- */
9
-
10
- // ########## FUNCTIONS
11
-
12
- // Scores the specified raw reports.
13
- exports.series = (job, count, interval) => {
14
- // If the arguments are valid:
15
- if (
16
- typeof job === 'object'
17
- && count
18
- && typeof count === 'number'
19
- && count === Math.floor(count)
20
- && count > 1
21
- && interval
22
- && typeof interval === 'number'
23
- && interval === Math.floor(interval)
24
- && interval > 0
25
- ) {
26
- // Get a copy of the initial job.
27
- const template = JSON.parse(JSON.stringify(job));
28
- // If it has an ID:
29
- const jobID = template.id;
30
- if (jobID) {
31
- // If the ID specifies a valid time:
32
- const s = jobID.slice(0, 11);
33
- const dateSpec = `20${s[0]}${s[1]}-${s[2]}${s[3]}-${s[4]}${s[5]}`;
34
- const timeSpec = `${s[7]}${s[8]}:${s[9]}${s[10]}`;
35
- const dateTimeSpec = `${dateSpec}T${timeSpec}Z`;
36
- const start = new Date(dateTimeSpec);
37
- const startNum = start.valueOf();
38
- if (startNum) {
39
- // Initialize the series.
40
- const series = [];
41
- // For each job required:
42
- for (let i = 0; i < count; i++) {
43
- // Create it.
44
- const nextJob = JSON.parse(JSON.stringify(template));
45
- nextJob.sources.series = nextJob.id;
46
- // Revise its ID.
47
- const nextDate = new Date(startNum + i * interval * 60000);
48
- const nextTimeStamp = nextDate.toISOString().slice(2, 16).replace(/[-:]/g, '');
49
- nextJob.id = nextJob.id.replace(/^[^-]+/, nextTimeStamp);
50
- // Add the job to the series.
51
- series.push(nextJob);
52
- }
53
- return series;
54
- }
55
- // Otherwise, i.e. if it does not specify a valid time:
56
- else {
57
- // Report this.
58
- console.log('ERROR: Initial job ID starts with an invalid time specification');
59
- }
60
- }
61
- // Otherwise, i.e. if it has no ID:
62
- else {
63
- // Report this.
64
- console.log('ERROR: Initial job has no ID');
65
- }
66
- }
67
- // Otherwise, i.e. if they are invalid:
68
- else {
69
- // Report this.
70
- console.log('ERROR: Arguments invalid');
71
- }
72
- };
@@ -1,37 +0,0 @@
1
- {
2
- "id": "231120T155027-mon-example",
3
- "what": "Job for series validation",
4
- "strict": true,
5
- "timeLimit": 10,
6
- "acts": [
7
- {
8
- "type": "launch",
9
- "which": "chromium",
10
- "url": "https://example.com",
11
- "what": "Example of web page",
12
- "startTime": 1662474496075,
13
- "endTime": 1662474496453
14
- },
15
- {
16
- "type": "test",
17
- "which": "testaro",
18
- "url": "https://example.com",
19
- "withItems": false,
20
- "rules": [
21
- "y",
22
- "bulk"
23
- ]
24
- }
25
- ],
26
- "sources": {
27
- "script": "mon",
28
- "batch": "target",
29
- "target": {
30
- "id": "example",
31
- "what": "Example of web page"
32
- },
33
- "requester": "user@domain.tld"
34
- },
35
- "creationTime": "2023-11-20T15:50:27",
36
- "timeStamp": "231120T155027"
37
- }
@@ -1,78 +0,0 @@
1
- /*
2
- validate.js
3
- Validates series module.
4
- */
5
-
6
- // ########## IMPORTS
7
-
8
- // Function to process files.
9
- const fs = require('fs/promises');
10
- // Function to generate a job series.
11
- const {series} = require('../../series');
12
-
13
- // ########## FUNCTIONS
14
-
15
- // Validates the series module.
16
- const validate = async () => {
17
- // Get the job.
18
- const jobJSON = await fs.readFile(`${__dirname}/job.json`, 'utf8');
19
- const job = JSON.parse(jobJSON);
20
- // Generate the series.
21
- const jobs = series(job, 3, 5);
22
- // Validate the series.
23
- if (Array.isArray(jobs) && jobs.length === 3) {
24
- console.log('Success: The count of jobs is correct');
25
- }
26
- else {
27
- console.log('ERROR: The jobs are not an array of length 3');
28
- return;
29
- }
30
- const job0 = jobs[0];
31
- if (
32
- job.id
33
- && job.id === '240223T0815-mon-example'
34
- && job0.id === job.id
35
- && job0.sources
36
- && job0.sources.series
37
- && job0.sources.series === job.id
38
- ) {
39
- console.log('Success: The first job has the correct id and sources.series');
40
- }
41
- else {
42
- console.log('ERROR: The first job has an incorrect id or sources.series');
43
- return;
44
- }
45
- const job1 = jobs[1];
46
- const job2 = jobs[2];
47
- if (
48
- job2.id
49
- && job2.id === '240223T0825-mon-example'
50
- && job2.sources
51
- && job2.sources.series
52
- && job2.sources.series === '240223T0815-mon-example'
53
- ) {
54
- console.log('Success: The third job has the correct id and sources.series');
55
- }
56
- else {
57
- console.log('ERROR: The first job has an incorrect id or sources.series');
58
- return;
59
- }
60
- if (
61
- job1.acts
62
- && job1.acts.length === 3
63
- && job1.acts[2].rules
64
- && job1.acts[2].rules.length === 2
65
- && job2.acts
66
- && job2.acts.length === 3
67
- && job2.acts[2].rules
68
- && job2.acts[2].rules.length === 2
69
- && job2.acts[2].rules[1] === job1.acts[2].rules[1]
70
- ) {
71
- console.log('Success: The second and third jobs invoke the same rule');
72
- }
73
- else {
74
- console.log('ERROR: The second and third job do not invoke the same rule');
75
- return;
76
- }
77
- };
78
- validate();