testilo 35.1.1 → 36.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -77,11 +77,11 @@ For example, a target list might be:
77
77
  ]
78
78
  ```
79
79
 
80
- A target list can be represented by a text file, in which each target is specified on a line with a Tab character delimiting its description and its URL, which are not quoted. Such a file representing the above target list would have this content, where “➡︎” represents a tab character:
80
+ A target list can be represented by a text file, in which each target is specified on a line with a Vertical Line character (`|`) delimiting its description and its URL, which are not quoted. Such a file representing the above target list would have this content:
81
81
 
82
82
  ```text
83
- World Wide Web Consortium➡︎https://www.w3.org/
84
- Mozilla Foundation➡︎https://foundation.mozilla.org/en/
83
+ World Wide Web Consortium|https://www.w3.org/
84
+ Mozilla Foundation|https://foundation.mozilla.org/en/
85
85
  ```
86
86
 
87
87
  ### Batches
@@ -92,17 +92,15 @@ Targets can be specified in a more complex way, too. That allows you to create j
92
92
  {
93
93
  id: 'clothing-stores',
94
94
  what: 'clothing stores',
95
- targets: [
96
- {
97
- id: 'acme',
95
+ targets: {
96
+ acme: {
98
97
  what: 'Acme Clothes',
99
- which: 'https://acmeclothes.com/',
100
- acts: {
98
+ url: 'https://acmeclothes.com/',
99
+ actGroups: {
101
100
  public: [
102
101
  {
103
102
  type: 'launch',
104
103
  what: 'Acme Clothes home page',
105
- url: 'https://acmeclothes.com/'
106
104
  }
107
105
  ],
108
106
  private: [
@@ -139,10 +137,54 @@ Targets can be specified in a more complex way, too. That allows you to create j
139
137
  }
140
138
  ```
141
139
 
142
- As shown, a batch, unlike a target list, defines named sequences of acts. They can be plugged into jobs, so various complex operations can be performed on each target.
140
+ As shown, a batch, unlike a target list, defines named groups of acts. They can be substituted for script placeholders, so various complex operations can be performed on each target.
141
+
142
+ In this example, the `public` act group contains only 1 act, of type `launch`. A `launch` act in an act group is permitted to have only two properties, `what` and `url`. If either of these is omitted, its value is inherited from the corresponding property of the target. In the `public` act group in this case, the `what` value is specified, but the `url` value will be `https://acmeclothes.com/`, inherited from the target.
143
143
 
144
144
  A batch is a JavaScript object. It can be converted to JSON and stored in a file.
145
145
 
146
+ ### Target list to batch
147
+
148
+ If you have a target list, the `batch` module of Testilo can convert it to a simple batch. The batch will contain, for each target, only one act group, named `main`, containing only a `launch` act. The target’s entry in the target list will determine the `what` and `url` properties of the target, and the `launch` acts will not override the values of those properties.
149
+
150
+ #### Invocation
151
+
152
+ There are two ways to use the `batch` module.
153
+
154
+ ##### By a module
155
+
156
+ A module can invoke `batch()` in this way:
157
+
158
+ ```javaScript
159
+ const {batch} = require('testilo/batch');
160
+ const id = 'divns';
161
+ const what = 'divisions';
162
+ const targets = [
163
+ ['Energy', 'https://abc.com/energy'],
164
+ ['Water', 'https://abc.com/water']
165
+ ];
166
+ const batchObj = batch(id, what, targets);
167
+ ```
168
+
169
+ The `id` argument to `batch()` is an identifier for the target list. The `what` variable describes the target list. The `targets` variable is an array of arrays, with each array containing the 2 items (description and URL) defining one target.
170
+
171
+ The `batch()` function of the `batch` module generates a batch and returns it as an object. Within the batch, each target is given a sequential (base-62 alphanumeric) string as an ID.
172
+
173
+ The invoking module can further dispose of the batch as needed.
174
+
175
+ ##### By a user
176
+
177
+ A user can invoke `batch()` in this way:
178
+
179
+ - Create a target list and save it as a text file (with Vertical-Line-delimited items in Newline-delimited lines) in the `targetLists` subdirectory of the `SPECDIR` directory. Name the file `x.txt`, where `x` is the list ID.
180
+ - In the Testilo project directory, execute the statement `node call batch id what`.
181
+
182
+ In this statement, replace `id` with the list ID and `what` with a string describing the batch. Example: `node call batch divns 'ABC company divisions'`.
183
+
184
+ The `call` module will retrieve the named target list.
185
+ The `batch` module will convert the target list to a batch.
186
+ The `call` module will save the batch as a JSON file named `x.json` (replacing `x` with the list ID) in the `batches` subdirectory of the `SPECDIR` directory.
187
+
146
188
  ### Scripts
147
189
 
148
190
  The generic, target-independent description of a job is _script_. A script can contain _placeholders_ that Testilo replaces with acts from a batch, creating one job per target. Thus, one script plus a batch containing _n_ targets will generate _n_ jobs.
@@ -155,12 +197,32 @@ Here is a script:
155
197
  what: 'aside mislocation',
156
198
  strict: true,
157
199
  isolate: true,
158
- timeLimit: 60,
200
+ standard: 'also',
201
+ observe: false,
202
+ deviceID: 'Kindle Fire HDX',
203
+ browserID: 'webkit',
204
+ lowMotion: false,
205
+ timeLimit: 80,
206
+ creationTimeStamp: ''
207
+ executionTimeStamp: '',
208
+ sources: {
209
+ script: 'ts99',
210
+ batch: '',
211
+ target: {
212
+ what: '',
213
+ url: ''
214
+ },
215
+ lastTarget: false,
216
+ mergeID: '',
217
+ sendReportTo: '',
218
+ requester: ''
219
+ },
159
220
  acts: [
160
221
  {
161
222
  type: 'placeholder',
162
223
  which: 'private',
163
- launch: 'chromium'
224
+ deviceID: 'default',
225
+ browserID: 'chromium'
164
226
  },
165
227
  {
166
228
  type: 'test',
@@ -181,58 +243,22 @@ Here is a script:
181
243
  ```
182
244
 
183
245
  A script has several properties that specify facts about the jobs to be created. They include:
184
- - `id`: an ID. A script can be converted from a JavaScript object to JSON and saved in a file in the `SPECDIR` directory, where it will be named by its ID (e.g., if the ID is `ts99`, the file name will be `ts99.json`). Thus, each script needs an `id` with a unique value.
246
+ - `id`: an ID. A script can be converted from a JavaScript object to JSON and saved in a file in the `SPECDIR` directory, where it will be named by its ID (e.g., if the ID is `ts99`, the file name will be `ts99.json`). Each script needs an `id` with a unique value composed of alphanumeric ASCII characters.
185
247
  - `what`: a description of the script.
186
248
  - `strict`: `true` if Testaro is to abort jobs when a target redirects a request to a URL differing substantially from the one specified. If `false` Testaro is to allow redirection. All differences are considered substantial unless the URLs differ only in the presence and absence of a trailing slash.
187
249
  - `isolate`: If `true`, Testilo, before creating a job, will isolate test acts, as needed, from effects of previous test acts, by inserting a copy of the latest placeholder after each target-modifying test act other than the final act. If `false`, placeholders will not be duplicated.
250
+ - `standard`: When Testaro performs a job, every tool produces its own report. Testaro can convert the test results of each tool report to standard results. The `standard` property specifies how to handle standardization. If `also`, Testaro will include in its reports both the original results of the tests of tools and the Testaro-standardized results. If `only`, reports will include only the standardized test results. If `no`, reports will include only the original results, without standardization.
251
+ - `observe`: Testaro jobs can allow granular observation. If `true`, the job will do so. If `false`, Testaro will not report job progress, but will send a report to the server only when the report is completed. It is generally user-friendly to allow granular observation, and for user applications to implement it, if they make users wait while jobs are assigned and performed, since that process typically takes a few minutes.
188
252
  - `timeLimit`: This specifies the maximum duration, in seconds, of a job. Testaro will abort jobs that are not completed within that time.
253
+ - `deviceID`: This specifies the default device type of the job.
254
+ - `browserID`: This specifies the default browser type (`'chromium'`, `'firefox'`, or `'webkit'`) of the job.
255
+ - `lowMotion`: This is true if the browser is to create tabs with the `reduce-motion` option set to `reduce` instead of `no-preference`. This makes the browser act as if the user has chosen a [motion-reduction option in the settings of the operating system or browser](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-reduced-motion#user_preferences). Motions on pages are, however, often immune to this setting.
256
+ - `creationTimeStamp`, `executionTimeStamp`, and `sources`: These properties will have values assigned to them when jobs are created from the script, except for the `sources.script` property, which will preserve the ID of the script after the `id` property has been replaced with a job ID.
189
257
  - `acts`: an array of acts.
190
258
 
191
- The first act in this example script is a placeholder, whose `which` property is `'private'`. If the above batch were merged with this script, in each job the placeholder would be replaced with the `private` acts of a target. For example, the first act of the first job would launch a Chromium browser, navigate to the Acme login page, complete and submit the login form, wait for the account page to load, run the Axe tests, and then run the QualWeb tests. If the batch contained additional targets, additional jobs would be created, with the login actions for each target specified in the `private` array of the `acts` object of that target.
259
+ In this example, the script contains 3 acts, of which the first is a placeholder. If the above batch were merged with this script, in each job the placeholder would be replaced with the acts in the `private` act group of a target. For example, the first act of the first job would launch a Chromium browser on a default device, navigate to the Acme login page, complete and submit the login form, wait for the account page to load, run the Axe tests, and then run the QualWeb tests. If the batch contained additional targets, additional jobs would be created, with the acts for each target specified by the `private` property of the `actGroups` object of that target.
192
260
 
193
- As shown in this example, when a browser is launched by placeholder substitution, the script can determine the browser type (`chromium`, `firefox`, or `webkit`) by assigning a value to a `launch` property of the placeholder. This allows a script to ensure that the tests it requires are performed with appropriate browser types. Some browser types are incompatible with some tests.
194
-
195
- ### Target list to batch
196
-
197
- If you have a target list, the `batch` module of Testilo can convert it to a simple batch. The batch will contain, for each target, only one array of acts, named `main`, containing only a `launch` act (depending on the script to specify the browser type and depending on the target to specify the URL).
198
-
199
- #### Invocation
200
-
201
- There are two ways to use the `batch` module.
202
-
203
- ##### By a module
204
-
205
- A module can invoke `batch()` in this way:
206
-
207
- ```javaScript
208
- const {batch} = require('testilo/batch');
209
- const id = 'divns';
210
- const what = 'divisions';
211
- const targets = [
212
- ['Energy', 'https://abc.com/energy'],
213
- ['Water', 'https://abc.com/water']
214
- ];
215
- const batchObj = batch(id, what, targets);
216
- ```
217
-
218
- The `id` argument to `batch()` is a unique identifier for the target list. The `what` variable describes the target list. The `targets` variable is an array of arrays, with each array containing the 2 items (description and URL) defining one target.
219
-
220
- The `batch()` function of the `batch` module generates a batch and returns it as an object. Within the batch, each target is given a sequential (base-62 alphanumeric) string as an ID.
221
-
222
- The invoking module can further dispose of the batch as needed.
223
-
224
- ##### By a user
225
-
226
- A user can invoke `batch()` in this way:
227
-
228
- - Create a target list and save it as a text file (with tab-delimited items in newline-delimited lines) in the `targetLists` subdirectory of the `SPECDIR` directory. Name the file `x.txt`, where `x` is the list ID.
229
- - In the Testilo project directory, execute the statement `node call batch id what`.
230
-
231
- In this statement, replace `id` with the list ID and `what` with a string describing the batch. Example: `node call batch divns 'ABC company divisions'`.
232
-
233
- The `call` module will retrieve the named target list.
234
- The `batch` module will convert the target list to a batch.
235
- The `call` module will save the batch as a JSON file named `x.json` (replacing `x` with the list ID) in the `batches` subdirectory of the `SPECDIR` directory.
261
+ As shown in this example, it is possible for any particular placeholder to override the default device type and/or browser type of the script by having its own optional `deviceID` and/or `browserID` property. Some rules are particularly relevant to some device types and/or can be successfully tested only with particular browser types. Overriding the default device and browser types lets you handle such constraints.
236
262
 
237
263
  ### Script creation
238
264
 
@@ -277,9 +303,9 @@ For an issue restriction, it has this structure:
277
303
 
278
304
  If you specify tool options, the script will prescribe the tests for all evaluation rules of the tools that you specify.
279
305
 
280
- If you specify issue options, the script will prescribe the tests for all evaluation rules that are classified in the issues whose IDs you specify. Any tools that do not have any of those rules will be omitted. The value of `specs/issues` is an issue classification object, with a structure like the one in `procs/score/tic40.js`. That one classifies about 1000 rules into about 300 issues.
306
+ If you specify issue options, the script will prescribe the tests for all evaluation rules that are classified into the issues whose IDs you specify. Any tools that do not have any of those rules will be omitted. The value of `specs.issues` is an issue classification object, with a structure like the one in `procs/score/tic43.js`. That one classifies about 1000 rules into about 300 issues.
281
307
 
282
- For example, one issue in the `tic40.js` file is `mainNot1`. Four rules are classified as belonging to that issue: rule `main_element_only_one` of the `aslint` tool and 3 more rules defined by 3 other tools. You can also create custom classifications and save them in a `score` subdirectory of the `FUNCTIONDIR` directory.
308
+ For example, one issue in the `tic43.js` file is `mainNot1`. Four rules are classified as belonging to that issue: rule `main_element_only_one` of the `aslint` tool and 3 more rules defined by 3 other tools. You can also create custom classifications and save them in a `score` subdirectory of the `FUNCTIONDIR` directory.
283
309
 
284
310
  #### Invocation
285
311
 
@@ -318,19 +344,34 @@ The first form will create a script with no restrictions.
318
344
 
319
345
  The second form will create a script that prescribes tests for all the rules of the specified tools.
320
346
 
321
- The third form will create a script that prescribes tests for all the rules classified by the specified issue classification into any of the specified issues.
347
+ The third form will create a script that prescribes tests for all the rules classified by the named issue-classification file into any of the specified issues.
322
348
 
323
- In this statement, replace `id` with an ID for the script, such as `headings1`, consisting of ASCII alphanumeric characters.
349
+ In this statement, replace `id` with an ID for the script, such as `headings1`, consisting of ASCII alphanumeric characters, or with `''` if you want Testilo to create an ID. Replace `what` with a string describing the script, or with `''` if you want Testilo to create a generic description.
324
350
 
325
351
  The `call` module will retrieve the named classification, if any.
326
352
  The `script` module will create a script.
327
353
  The `call` module will save the script as a JSON file in the `scripts` subdirectory of the `SPECDIR` directory, using the `id` value as the base of the file name.
328
354
 
329
- #### Options
355
+ #### Properties
330
356
 
331
- The `script` module will use the value of the `SEND_REPORT_TO` environment variable as the value of the `sendReportTo` property of the script, if that variable exists, and otherwise will leave that property with an empty-string value.
357
+ When the `script` module creates a script for you, it does not ask you for all of the property values that the script may require. Instead, it chooses these default values:
358
+ - `strict`: `false`
359
+ - `isolate`: `true`
360
+ - `standard`: `'only'`
361
+ - `observe`: `false`
362
+ - `deviceID`: `'default'`
363
+ - `browserID`: `'webkit'`
364
+ - `lowMotion`: `false`
365
+ - `timeLimit`: 50 plus 30 per tool
366
+ - `axe` test act: `detailLevel` = 2
367
+ - `ibm` test act: `withItems` = `true`, `withNewContent` = `false`
368
+ - `qualWeb` test act: `withNewContent` = `false`
369
+ - `testaro` test act: `withItems` = true, `stopOnFail` = `false`
370
+ - `wave` test act: `reportType` = 4
332
371
 
333
- When the `script` module creates a script for you, it does not ask you for all of the other options that the script may require. Instead, it chooses default options. For example, it sets the values of `isolate` and `strict` to `true`, and it sets the `withNewContent` property of the `ibm` tool to `false` to prevent occasional infinite loops or crashes when targets cause HTTP2 protocol errors. After you invoke `script`, you can edit the script that it creates to revise options.
372
+ The `webkit` browser type is selected because the other browser types corrupt some tests. The `ibm` test is performed on the existing page content because some targets cause HTTP2 protocol errors when the `ibm` tool tries to visit them.
373
+
374
+ After you invoke `script`, you can edit the script that it creates to revise any of these options.
334
375
 
335
376
  ### Merge
336
377
 
@@ -348,26 +389,13 @@ A module can invoke `merge()` in this way:
348
389
  const {merge} = require('testilo/merge');
349
390
  const script = …;
350
391
  const batch = …;
351
- const standard = 'only';
352
- const observe = false;
353
- const requester = 'me@mydomain.tld';
354
392
  const timeStamp = '241215T1200';
355
- const jobs = merge(script, batch, standard, observe, requester, timeStamp, deviceID);
393
+ const jobs = merge(script, batch, timeStamp);
356
394
  ```
357
395
 
358
396
  The first two arguments are a script and a batch obtained from files or from prior calls to `script()` and `batch()`.
359
397
 
360
- The `standard` argument specifies how to handle standardization. If `also`, jobs will tell Testaro to include in its reports both the original results of the tests of tools and the Testaro-standardized results. If `only`, reports are to include only the standardized test results. If `no`, reports are to include only the original results, without standardization.
361
-
362
- The `observe` argument tells Testaro whether the jobs should allow granular observation. If `true`, it will. If `false`, Testaro will not report job progress, but will send reports to the server only when the reports are completed. It is generally user-friendly to allow granular observation, and for user applications to implement it, if they make users wait while jobs are assigned and performed, since that process typically takes a few minutes.
363
-
364
- The `requester` argument is an email address to which any notices about the job are to be sent.
365
-
366
- The `timeStamp` argument specifies the earliest UTC date and time when the jobs may be assigned, or it may be an empty string if now.
367
-
368
- The `deviceID` argument specifies the ID of the test device. It must be either `'default'` or the ID of one of the test devices recognized by Playwright, published at `https://github.com/microsoft/playwright/blob/main/packages/playwright-core/src/server/deviceDescriptorsSource.json`.
369
-
370
- The `merge()` function returns the jobs in an array. The invoking module can further dispose of the jobs as needed.
398
+ The `merge()` function returns an array of jobs, one job per target in the batch. The invoking module can further dispose of the jobs as needed.
371
399
 
372
400
  ##### By a user
373
401
 
@@ -378,13 +406,13 @@ A user can invoke `merge()` in this way:
378
406
  - In the Testilo project directory, execute the statement:
379
407
 
380
408
  ```javascript
381
- node call merge scriptID batchID standard observe requester timeStamp deviceID todoDir
409
+ node call merge scriptID batchID executionTimeStamp todoDir
382
410
  ```
383
411
 
384
412
  In this statement, replace:
385
413
  - `scriptID` with the ID (which is also the base of the file name) of the script.
386
414
  - `batchID` with the ID (which is also the base of the file name) of the batch.
387
- - `standard`, `observe`, `requester`, `timeStamp`, and `deviceID` as described above.
415
+ - `executionTimeStamp` with a time stamp in format `yymmddThhMM` representing the UTC date and time before which the jobs are not to be executed, or `''` if it is now.
388
416
  - `todoDir`: `true` if the jobs are to be saved in the `todo` subdirectory, or `false` if they are to be saved in the `pending` subdirectory, of the `JOBDIR` directory.
389
417
 
390
418
  The `call` module will retrieve the named script and batch from their respective directories.
@@ -393,61 +421,30 @@ The `call` module will save the jobs as JSON files in the `todo` or `pending` su
393
421
 
394
422
  #### Output
395
423
 
396
- A Testaro job produced by `merge` may look like this:
424
+ A Testaro job produced by `merge` will be identical to the script from which it was derived (see the example above), except that:
425
+ - The `id` property of the job will be revised to uniquely identify the job.
426
+ - The originally empty properties will be populated, as in this example:
397
427
 
398
- ```javaScript
399
- {
400
- id: '240115T1200-4R-0',
401
- what: 'aside mislocation',
402
- strict: true,
403
- timeLimit: 60,
404
- standard: 'also',
405
- observe: false,
406
- sendReportTo: 'https://ourdomain.com/testman/api/report'
407
- timeStamp: '240115T1200',
408
- acts: [
409
- {
410
- type: 'launch',
411
- which: 'webkit',
412
- what: 'Acme Clothes',
413
- deviceID: 'Galaxy S8',
414
- url: 'https://acmeclothes.com/'
415
- },
416
- {
417
- type: 'test',
418
- which: 'axe',
419
- detailLevel: 2,
420
- rules: ['landmark-complementary-is-top-level'],
421
- what: 'Axe'
422
- },
423
- {
424
- type: 'launch',
425
- which: 'webkit',
426
- what: 'Acme Clothes',
427
- deviceID: 'Galaxy S8',
428
- url: 'https://acmeclothes.com/'
429
- },
430
- {
431
- type: 'test',
432
- which: 'qualWeb',
433
- withNewContent: false,
434
- rules: ['QW-BP25', 'QW-BP26']
435
- what: 'QualWeb'
436
- }
437
- ],
438
- sources: {
439
- script: 'ts99',
440
- batch: 'clothing-stores',
441
- lastTarget: false,
442
- target: {
443
- id: 'acme',
444
- what: 'Acme Clothes'
428
+ ```javaScript
429
+
430
+ creationTimeStamp: '241229T0537',
431
+ executionTimeStamp: '250110T1200',
432
+ sources: {
433
+ script: 'ts99',
434
+ batch: 'departments',
435
+ mergeID: '7f',
436
+ sendReportTo: 'https://abccorp.com/api/report',
437
+ requester: 'malavu@abccorp.com'
438
+ target: {
439
+ what: 'Real Estate Management',
440
+ url: 'https://abccorp.com/mgmt/realproperty.html'
441
+ },
442
+ lastTarget: false,
445
443
  },
446
- requester: 'you@yourdomain.tld'
447
- },
448
- creationTimeStamp: '241120T1550'
449
- }
450
- ```
444
+
445
+ ```
446
+
447
+ The `merge()` function will populate the `sources.sendReportTo` property with the value, if any, of the environment variable `SEND_REPORT_TO`, and populate the `sources.requester` property with the value, if any, of the environment variable `REQUESTER`.
451
448
 
452
449
  #### Validation
453
450
 
@@ -472,6 +469,7 @@ Thus, a report produced by Testaro contains these properties:
472
469
  - `timeStamp`
473
470
  - `acts`
474
471
  - `sources`
472
+ - `timeStamp`
475
473
  - `creationTimeStamp`
476
474
  - `jobData`
477
475
 
package/batch.js CHANGED
@@ -25,10 +25,6 @@
25
25
  Converts a target list to a batch.
26
26
  */
27
27
 
28
- // IMPORTS
29
-
30
- const {alphaNumOf} = require('./procs/util');
31
-
32
28
  // FUNCTIONS
33
29
 
34
30
  // Converts a target list to a batch and returns the batch.
@@ -51,25 +47,21 @@ exports.batch = (id, what, targetList) => {
51
47
  const batch = {
52
48
  id,
53
49
  what,
54
- targets: []
50
+ targets: {}
55
51
  };
56
52
  // For each target:
57
- targetList.forEach((target, index) => {
53
+ targetList.forEach(target => {
58
54
  // Add it to the batch.
59
- batch.targets.push({
60
- id: alphaNumOf(index),
61
- what: target[0],
62
- which: target[1],
63
- acts: {
55
+ batch.targets[target[0]] = {
56
+ url: target[1],
57
+ actGroups: {
64
58
  main: [
65
59
  {
66
- type: 'launch',
67
- what: target[0],
68
- url: target[1]
60
+ type: 'launch'
69
61
  }
70
62
  ]
71
63
  }
72
- });
64
+ };
73
65
  });
74
66
  // Return the batch.
75
67
  return batch;
package/call.js CHANGED
@@ -186,8 +186,8 @@ const callScript = async (scriptID, what, optionType, ... optionDetails) => {
186
186
  return;
187
187
  }
188
188
  }
189
- // Create a script.
190
- const scriptObj = script(scriptID, what, optionArg);
189
+ // Create a script with default device and browser IDs.
190
+ const scriptObj = script(scriptID, undefined, undefined, what, optionArg);
191
191
  try {
192
192
  // Save it.
193
193
  const scriptJSON = JSON.stringify(scriptObj, null, 2);
@@ -203,18 +203,14 @@ const callScript = async (scriptID, what, optionType, ... optionDetails) => {
203
203
  const callMerge = async (
204
204
  scriptID,
205
205
  batchID,
206
- standard,
207
- observe,
208
- requester,
209
- timeStamp,
210
- deviceID,
206
+ executionTimeStamp,
211
207
  todoDir
212
208
  ) => {
213
209
  try {
214
210
  // If the todoDir argument is invalid:
215
211
  if (! ['true', 'false'].includes(todoDir)) {
216
212
  // Report this.
217
- throw new Error('Invalid todoDir configuration');
213
+ throw new Error('Invalid todoDir argument to merge');
218
214
  }
219
215
  // Get the script and the batch.
220
216
  const scriptJSON = await fs.readFile(`${specDir}/scripts/${scriptID}.json`, 'utf8');
@@ -222,18 +218,18 @@ const callMerge = async (
222
218
  const batchJSON = await fs.readFile(`${specDir}/batches/${batchID}.json`, 'utf8');
223
219
  const batch = JSON.parse(batchJSON);
224
220
  // Merge them into an array of jobs.
225
- const jobs = merge(script, batch, standard, observe === 'true', requester, timeStamp, deviceID);
221
+ const jobs = merge(script, batch, executionTimeStamp);
226
222
  // Save the jobs.
227
223
  const subdir = `${jobDir}/${todoDir === 'true' ? 'todo' : 'pending'}`;
228
224
  for (const job of jobs) {
229
225
  const jobJSON = JSON.stringify(job, null, 2);
230
226
  await fs.writeFile(`${subdir}/${job.id}.json`, `${jobJSON}\n`);
231
227
  }
232
- const truncatedID = `${jobs[0].timeStamp}-${jobs[0].mergeID}-…`;
228
+ const truncatedID = `${jobs[0].id.replace(/[^-]+$/, '')}…`;
233
229
  console.log(`Script ${scriptID} and batch ${batchID} merged as ${truncatedID} in ${subdir}`);
234
230
  }
235
231
  catch(error) {
236
- console.log(`ERROR merging script and batch (${error.message})`);
232
+ console.log(`ERROR merging script ${scriptID} and batch ${batchID} (${error.message})`);
237
233
  }
238
234
  };
239
235
  // Gets the file base names (equal to the IDs) of the selected reports.
@@ -499,7 +495,7 @@ else if (fn === 'script' && (fnArgs.length === 2 || fnArgs.length > 3)) {
499
495
  console.log('Execution completed');
500
496
  });
501
497
  }
502
- else if (fn === 'merge' && fnArgs.length === 8) {
498
+ else if (fn === 'merge' && fnArgs.length === 4) {
503
499
  callMerge(... fnArgs)
504
500
  .then(() => {
505
501
  console.log('Execution completed');
package/merge.js CHANGED
@@ -30,7 +30,7 @@
30
30
  // Module to keep secrets.
31
31
  require('dotenv').config();
32
32
  // Utility module.
33
- const {alphaNumOf, dateOf, getRandomString, getNowStamp, isValidDeviceID} = require('./procs/util');
33
+ const {alphaNumOf, dateOf, getRandomString, getNowStamp} = require('./procs/util');
34
34
 
35
35
  // ########## CONSTANTS
36
36
 
@@ -49,60 +49,34 @@ const mergeIDLength = 2;
49
49
  // ########## FUNCTIONS
50
50
 
51
51
  // Merges a script and a batch and returns jobs.
52
- exports.merge = (script, batch, standard, observe, requester, timeStamp, deviceID) => {
53
- // If standard is invalid:
54
- if (! ['also', 'only', 'no'].includes(standard)) {
55
- // Report this and quit.
56
- console.log('ERROR: Invalid standard treatment specified');
57
- return [];
58
- }
59
- // If observe is invalid:
60
- if (! [true, false].includes(observe)) {
61
- // Report this and quit.
62
- console.log('ERROR: Invalid observe configuration specified');
63
- return [];
64
- }
52
+ exports.merge = (script, batch, executionTimeStamp) => {
65
53
  // If a time stamp was specified:
66
- if (timeStamp) {
54
+ if (executionTimeStamp) {
67
55
  // If it is invalid:
68
- if (! dateOf(timeStamp)) {
56
+ if (! dateOf(executionTimeStamp)) {
69
57
  // Report this and quit.
70
- console.log('ERROR: Timestamp invalid');
58
+ console.log('ERROR: Execution time stamp invalid');
71
59
  return [];
72
60
  }
73
61
  }
74
62
  // Otherwise, i.e. if no time stamp was specified:
75
63
  else {
76
64
  // Create one for the job.
77
- timeStamp = getNowStamp();
65
+ executionTimeStamp = getNowStamp();
78
66
  }
79
- // If deviceID is invalid:
80
- if (! isValidDeviceID(deviceID)) {
81
- // Report this and quit.
82
- console.log('ERROR: Device ID invalid');
83
- return [];
84
- }
85
- // Initialize a job as a copy of the script.
67
+ // Initialize a job template as a copy of the script.
86
68
  const protoJob = JSON.parse(JSON.stringify(script));
87
- // Add an initialized sources property to it.
88
- protoJob.sources = {
89
- script: script.id,
90
- batch: batch.id,
91
- lastTarget: false,
92
- target: {
93
- id: '',
94
- what: '',
95
- which: ''
96
- },
97
- requester
98
- };
99
- // Add properties to the job.
100
- protoJob.standard = standard;
101
- protoJob.observe = observe;
102
- protoJob.timeStamp = timeStamp;
69
+ // Populate empty properties of the template.
103
70
  protoJob.creationTimeStamp = getNowStamp();
71
+ protoJob.executionTimeStamp = executionTimeStamp;
72
+ const {sources} = protoJob;
73
+ sources.script = script.id;
74
+ sources.batch = batch.id;
75
+ sources.mergeID = getRandomString(mergeIDLength);
76
+ sources.sendReportTo = process.env.SEND_REPORT_TO || '';
77
+ sources.requester = process.env.REQUESTER || '';
104
78
  // If isolation was requested:
105
- if (script.isolate) {
79
+ if (protoJob.isolate) {
106
80
  // For each act:
107
81
  let {acts} = protoJob;
108
82
  let lastPlaceholder = {};
@@ -124,63 +98,55 @@ exports.merge = (script, batch, standard, observe, requester, timeStamp, deviceI
124
98
  acts[actIndex] = JSON.parse(JSON.stringify([act, lastPlaceholder]));
125
99
  }
126
100
  };
127
- // Flatten the acts.
101
+ // Flatten the acts, causing insertion of placeholder copies before acts needing them.
128
102
  protoJob.acts = acts.flat();
129
103
  }
130
- // Delete the no-longer-necessary job property.
131
- delete protoJob.isolate;
132
104
  // Initialize an array of jobs.
133
105
  const jobs = [];
134
- // Get an ID for the merger.
135
- const mergeID = getRandomString(mergeIDLength);
136
106
  // For each target in the batch:
137
107
  const {targets} = batch;
138
- targets.forEach((target, index) => {
108
+ const targetIDs = Object.keys(targets);
109
+ targetIDs.forEach((what, index) => {
139
110
  // If the target has the required identifiers:
140
- const {id, what, which} = target;
141
- if (id && what && which) {
111
+ const {actGroups, url} = targets[what];
112
+ if (actGroups && url) {
142
113
  // Initialize a job.
143
114
  const job = JSON.parse(JSON.stringify(protoJob));
115
+ const {sources} = job;
144
116
  // Make the job ID unique.
145
- const targetID = alphaNumOf(index);
146
- job.id = `${timeStamp}-${mergeID}-${targetID}`;
147
- // Add other properties to the job.
148
- job.mergeID = mergeID;
149
- job.sendReportTo = process.env.SEND_REPORT_TO || '';
117
+ const targetSuffix = alphaNumOf(index);
118
+ job.id = `${executionTimeStamp}-${sources.mergeID}-${targetSuffix}`;
150
119
  // If the target is the last one:
151
- if (index === targets.length - 1) {
120
+ if (index === targetIDs.length - 1) {
152
121
  // Add that fact to the sources property of the job.
153
- job.sources.lastTarget = true;
122
+ sources.lastTarget = true;
154
123
  }
155
- // Add other data to the sources property of the job.
156
- job.sources.target.id = targetID;
157
- job.sources.target.what = target.what;
158
- job.sources.target.which = target.which;
159
- // Replace each placeholder object in the job with the named replacer array of the target.
124
+ // Populate the target-specific properties of the job.
125
+ sources.target.what = what;
126
+ sources.target.url = url;
127
+ // Replace each placeholder object in the job with the named act group of the target.
160
128
  let {acts} = job;
161
129
  for (const actIndex in acts) {
162
130
  const act = acts[actIndex];
163
131
  if (act.type === 'placeholder') {
164
132
  const replacerName = act.which;
165
- if (replacerName && target.acts) {
166
- let replacerActs = target.acts[replacerName];
167
- if (replacerActs) {
168
- // Add properties to any launch act in the replacer.
169
- replacerActs = JSON.parse(JSON.stringify(replacerActs));
170
- for (const replacerAct of replacerActs) {
171
- if (replacerAct.type === 'launch') {
172
- replacerAct.which = act.launch;
173
- replacerAct.deviceID = deviceID;
133
+ const replacerActs = actGroups[replacerName];
134
+ if (replacerName && actGroups && replacerActs) {
135
+ // Add properties to any launch act in the replacer.
136
+ for (const replacerAct of replacerActs) {
137
+ if (replacerAct.type === 'launch') {
138
+ if (act.deviceID) {
139
+ replacerAct.deviceID = act.deviceID;
140
+ }
141
+ if (act.browserID) {
142
+ replacerAct.browserID = act.browserID;
174
143
  }
175
144
  }
176
- acts[actIndex] = replacerActs;
177
- }
178
- else {
179
- console.log(`ERROR: Target ${target.id} has no ${replacerName} replacer`);
180
- }
145
+ };
146
+ acts[actIndex] = replacerActs;
181
147
  }
182
148
  else {
183
- console.log(`ERROR: Placeholder for target ${target.id} not replaceable`);
149
+ console.log(`ERROR: Placeholder for target ${what} not replaceable`);
184
150
  }
185
151
  }
186
152
  }
@@ -190,7 +156,7 @@ exports.merge = (script, batch, standard, observe, requester, timeStamp, deviceI
190
156
  jobs.push(job);
191
157
  }
192
158
  else {
193
- console.log('ERROR: Target in batch missing id, what, or which property');
159
+ console.log(`ERROR: Target ${what} in batch missing actGroups or url property`);
194
160
  }
195
161
  });
196
162
  return jobs;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testilo",
3
- "version": "35.1.1",
3
+ "version": "36.0.0",
4
4
  "description": "Prepares Testaro jobs and processes Testaro reports",
5
5
  "main": "call.js",
6
6
  "scripts": {
@@ -20,10 +20,18 @@
20
20
  <caption>Synopsis</caption>
21
21
  <tr><th>Page</th><td>__org__</td></tr>
22
22
  <tr><th>URL</th><td>__url__</td></tr>
23
- <tr><th>Requester</th><td>__requester__</td></tr>
24
- <tr><th>Device</th><td>__device__</td></tr>
25
23
  <tr><th>Test date</th><td>__dateSlash__</td></tr>
26
24
  <tr><th>Score</th><td>__total__</td></tr>
25
+ </table>
26
+ <table class="allBorder">
27
+ <caption>Configuration</caption>
28
+ <tr><th>Redirections</th><td>__redirections__</td></tr>
29
+ <tr><th>Tool isolation</th><td>__isolation__</td></tr>
30
+ <tr><th>Report format(s)</th><td>__standard__</td></tr>
31
+ <tr><th>Requester</th><td>__requester__</td></tr>
32
+ <tr><th>Device</th><td>__device__</td></tr>
33
+ <tr><th>Browser type</th><td>__browser__</td></tr>
34
+ <tr><th>Reduced motion</th><td>__motion__</td></tr>
27
35
  <tr><th>Tested by</th><td>Testaro__agent__, procedure <code>__ts__</code></td></tr>
28
36
  <tr><th>Scored by</th><td>Testilo, procedure <code>__sp__</code></td></tr>
29
37
  <tr><th>Digested by</th><td>Testilo, procedure <code>__dp__</code></td></tr>
@@ -57,7 +57,7 @@ const getIssueScoreRow = (issueConstants, issueDetails) => {
57
57
  };
58
58
  // Adds parameters to a query for a digest.
59
59
  const populateQuery = (report, query) => {
60
- const {acts, id, sources, score} = report;
60
+ const {browserID, deviceID, id, isolate, lowMotion, score, sources, standard, strict} = report;
61
61
  const {agent, script, target, requester} = sources;
62
62
  const {scoreProcID, summary, details} = score;
63
63
  query.ts = script;
@@ -67,16 +67,16 @@ const populateQuery = (report, query) => {
67
67
  query.dateISO = getNowDate();
68
68
  query.dateSlash = getNowDateSlash();
69
69
  query.org = target.what;
70
- query.url = target.which;
70
+ query.url = target.url;
71
+ query.redirections = strict ? 'prohibited' : 'permitted';
72
+ query.isolation = isolate ? 'yes' : 'no';
73
+ query.standard
74
+ = ['original', 'standard', 'original and standard'][['no', 'only', 'also'].indexOf(standard)];
75
+ query.motion = lowMotion ? 'requested' : 'not requested';
71
76
  query.requester = requester;
72
- const firstLaunch = acts.find(act => act.type === 'launch');
73
- if (firstLaunch) {
74
- query.device = firstLaunch.deviceID || 'unknown';
75
- }
76
- else {
77
- query.device = 'unknown';
78
- }
79
- query.agent = ` on agent ${agent}` || '';
77
+ query.device = deviceID;
78
+ query.browser = browserID;
79
+ query.agent = agent ? ` on agent ${agent}` : '';
80
80
  query.reportURL = process.env.SCORED_REPORT_URL.replace('__id__', id);
81
81
  // Add values for the score-summary table to the query.
82
82
  const rows = {
@@ -134,11 +134,14 @@ const populateQuery = (report, query) => {
134
134
  issueDetailRows.push(`<h5>Rule <code>${ruleID}</code></h5>`);
135
135
  issueDetailRows.push(`<p>Description: ${ruleData.what}</p>`);
136
136
  issueDetailRows.push(`<p>Count of instances: ${ruleData.complaints.countTotal}</p>`);
137
+ /*
138
+ This fails unless the caller handles such URLs and has compatible scored report URLs.
137
139
  const href = `${query.reportURL}?tool=${toolID}&rule=${ruleID}`;
138
140
  const detailLabel = `Issue ${issueSummary} tool ${toolID} rule ${ruleID} instance details`;
139
141
  issueDetailRows.push(
140
142
  `<p><a href="${href}" aria-label="${detailLabel}">Instance details</a></p>`
141
143
  );
144
+ */
142
145
  });
143
146
  });
144
147
  });
@@ -77,7 +77,7 @@ const populateQuery = async (id, what, summaryReport, query) => {
77
77
  const scoreCell = `<td><a href=${digestLinkDestination}>${result.score}</a></td>`;
78
78
  // Create a target cell.
79
79
  const {target} = result.sources;
80
- const targetLink = `<a href="${target.which}">${target.what}</a>`;
80
+ const targetLink = `<a href="${target.url}">${target.what}</a>`;
81
81
  const targetCell = `<td>${targetIDs[target.what]}: ${targetLink}</td>`;
82
82
  const row = `<tr>${[timeCell, scoreCell, targetCell].join('')}</tr>`;
83
83
  // Add the row to the array of rows.
package/script.js CHANGED
@@ -111,18 +111,35 @@ exports.script = (id, what, options = {}) => {
111
111
  });
112
112
  }
113
113
  // Initialize a script.
114
- const timeLimit = Math.round(50 + 30 * Object.keys(toolsRulesData).length);
115
114
  const scriptObj = {
116
115
  id,
117
116
  what,
118
- strict: true,
117
+ strict: false,
119
118
  isolate: true,
120
- timeLimit,
121
- acts: [
119
+ standard: 'only',
120
+ observe: false,
121
+ deviceID: 'default',
122
+ browserID: 'webkit',
123
+ lowMotion: false,
124
+ timeLimit: Math.round(50 + 30 * Object.keys(toolsRulesData).length),
125
+ creationTimeStamp: '',
126
+ executionTimeStamp: '',
127
+ sources: {
128
+ script: id,
129
+ batch: '',
130
+ target: {
131
+ what: '',
132
+ url: ''
133
+ },
134
+ lastTarget: false,
135
+ mergeID: '',
136
+ sendReportTo: '',
137
+ requester: ''
138
+ },
139
+ acts: [
122
140
  {
123
- "type": "placeholder",
124
- "which": "main",
125
- "launch": "webkit"
141
+ type: 'placeholder',
142
+ which: 'main'
126
143
  }
127
144
  ]
128
145
  };