@schwabyio/gta 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -1
- package/dist/SPEC.md +241 -31
- package/dist/chunks/{chunk-VRAFYQNE.js → chunk-OM3ZR56O.js} +1191 -566
- package/dist/gta.js +110 -41
- package/dist/worker.js +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -108,12 +108,14 @@ The steps within a collection always run in order.
|
|
|
108
108
|
|
|
109
109
|
The exit code is `0` when everything passed, `1` when something failed, and `2` when
|
|
110
110
|
`gta` could not run at all, such as with a bad setting or an unknown collection.
|
|
111
|
+
`gta get` exits `1` when a collection is broken, or has a `use:`, an `extends:` or a
|
|
112
|
+
file its body sends that a run would stop at.
|
|
111
113
|
|
|
112
114
|
## Settings
|
|
113
115
|
|
|
114
116
|
Settings come from `settings.yml`, then `GTA_*` environment variables, then flags on the
|
|
115
117
|
command line, each overriding the one before. An unknown setting is an error.
|
|
116
|
-
`gta --help` lists them all:
|
|
118
|
+
`gta --help` lists them all, and `gta get` shows where each one came from:
|
|
117
119
|
|
|
118
120
|
| Setting | Default | What it does |
|
|
119
121
|
| ------------------------ | -------------- | -------------------------------------------------------------- |
|
|
@@ -132,6 +134,11 @@ command line, each overriding the one before. An unknown setting is an error.
|
|
|
132
134
|
The results folder is emptied before every run, so everything in it is from the last one.
|
|
133
135
|
`gta` only ever deletes a folder that holds nothing but its own reports.
|
|
134
136
|
|
|
137
|
+
In a monorepo, projects that share a global project (`uses:` in `project.yml`) share its
|
|
138
|
+
`settings.yml` too. It comes before the project's own, so a project changes a shared
|
|
139
|
+
setting by setting it again in its own file. Each project still needs a `settings.yml`,
|
|
140
|
+
even an empty one.
|
|
141
|
+
|
|
135
142
|
## In CI
|
|
136
143
|
|
|
137
144
|
```yaml
|
package/dist/SPEC.md
CHANGED
|
@@ -152,16 +152,16 @@ tls:
|
|
|
152
152
|
|
|
153
153
|
The file is optional, and so is every key in it. Any other key is an error.
|
|
154
154
|
|
|
155
|
-
| Key | Type | Default | Meaning
|
|
156
|
-
| -------- | ---------------------- | ----------- |
|
|
157
|
-
| `name` | string | folder name | Display name.
|
|
158
|
-
| `uses` | string (relative path) | none | A global project whose
|
|
159
|
-
| `vars` | map of plain values | none | Variables for every collection in the project (§4).
|
|
160
|
-
| `tls.ca` | list of relative paths | none | Certificate files that requests trust (below).
|
|
155
|
+
| Key | Type | Default | Meaning |
|
|
156
|
+
| -------- | ---------------------- | ----------- | ----------------------------------------------------- |
|
|
157
|
+
| `name` | string | folder name | Display name. |
|
|
158
|
+
| `uses` | string (relative path) | none | A global project whose files this one shares (below). |
|
|
159
|
+
| `vars` | map of plain values | none | Variables for every collection in the project (§4). |
|
|
160
|
+
| `tls.ca` | list of relative paths | none | Certificate files that requests trust (below). |
|
|
161
161
|
|
|
162
162
|
**`uses`** names a **global project**: an ordinary project whose `project.yml`
|
|
163
|
-
variables, `environments/`, `requests/`, `endpoints/`, `bases
|
|
164
|
-
project using it shares.
|
|
163
|
+
variables, `environments/`, `requests/`, `endpoints/`, `bases/`, `checks/` and
|
|
164
|
+
`settings.yml` every project using it shares.
|
|
165
165
|
|
|
166
166
|
- It must be a relative path. An absolute path is refused, since it would only work on
|
|
167
167
|
one machine. Write it with `/`; `\` reads the same.
|
|
@@ -170,6 +170,7 @@ project using it shares.
|
|
|
170
170
|
- An environment in the global project merges under the project's environment of the
|
|
171
171
|
same name, key by key, and the project's values win. An environment only the global
|
|
172
172
|
project has is available too.
|
|
173
|
+
- The global project's `settings.yml` lies under the project's the same way (§1.3).
|
|
173
174
|
|
|
174
175
|
**`tls.ca`** lists certificate files that requests trust, for a server whose certificate
|
|
175
176
|
a company or local CA signed, or a server's own self-signed certificate. A request
|
|
@@ -212,8 +213,9 @@ Projects are shared between macOS, Windows and Linux, and read the same on all t
|
|
|
212
213
|
without an extension. A file with such a name, made on macOS or Linux, is reported.
|
|
213
214
|
- Files should use LF line endings. Gravity writes LF, and can add a `.gitattributes`
|
|
214
215
|
scoped to the project's own files so that git keeps them LF on every platform. The same
|
|
215
|
-
block keeps `files/` exactly as committed, so a body sends the same bytes everywhere.
|
|
216
|
-
|
|
216
|
+
block keeps `files/` exactly as committed, so a body sends the same bytes everywhere. A
|
|
217
|
+
global project is a project too: its own block covers its `files/`. For files a body
|
|
218
|
+
sends from another folder, add a `-text` line of your own.
|
|
217
219
|
- The process environment is read by exact name on every platform, although Windows
|
|
218
220
|
itself ignores case (§4, §6).
|
|
219
221
|
|
|
@@ -257,9 +259,22 @@ The results folder is emptied before every run. `gta` refuses to empty one that
|
|
|
257
259
|
files it did not write, or one that holds the project itself. Every report replaces
|
|
258
260
|
each secret's value with `[secret: NAME]` (§6).
|
|
259
261
|
|
|
262
|
+
**From the global project.** A global project (§1.1) can have a `settings.yml` too.
|
|
263
|
+
Every project that `uses` it runs with those settings under its own: a key the
|
|
264
|
+
project's file sets wins, and a key it leaves out comes from the global project's file.
|
|
265
|
+
|
|
266
|
+
- There is no switch to turn this off. A project undoes a shared setting by setting it
|
|
267
|
+
in its own file, back to the default if need be: `environmentType: null`, `tags: []`.
|
|
268
|
+
- A project still needs a `settings.yml` of its own, even an empty one. A global project
|
|
269
|
+
need not have one.
|
|
270
|
+
- A relative `testResultsBasePath` is relative to the project being run, whichever file
|
|
271
|
+
set it.
|
|
272
|
+
|
|
260
273
|
**Overrides.** Each setting can be overridden by an environment variable, then by a
|
|
261
274
|
command-line flag: `GTA_LIMIT_CONCURRENCY=8`, then `--limitConcurrency 8`. A list is
|
|
262
|
-
comma-separated: `--tags smoke,api`.
|
|
275
|
+
comma-separated: `--tags smoke,api`. In all, from lowest to highest: the default, the
|
|
276
|
+
global project's `settings.yml`, the project's, the environment variable, the flag.
|
|
277
|
+
`gta get` lists each setting that is not a default, and where it came from.
|
|
263
278
|
|
|
264
279
|
**Running.** `gta` runs from the project folder:
|
|
265
280
|
|
|
@@ -269,6 +284,12 @@ comma-separated: `--tags smoke,api`.
|
|
|
269
284
|
| `gta all` | Every collection, except those with `exclude: true` (§2.4). |
|
|
270
285
|
| `gta smoke,checkout/` | The collections and directories named, in that order. |
|
|
271
286
|
|
|
287
|
+
`gta get` also reports each `use:` and `extends:` that would stop a run, and each file a
|
|
288
|
+
body names that is in neither the project nor its global project (§2.2), in every
|
|
289
|
+
collection, including those `gta all` leaves out (Appendix A). A file path with
|
|
290
|
+
`{{variables}}` is left to the run. It exits `1` when it finds one, or a broken
|
|
291
|
+
collection, and `0` otherwise.
|
|
292
|
+
|
|
272
293
|
A collection is named by its `id`, or by its place (`checkout/sessions`). A directory is
|
|
273
294
|
named by its name, and `checkout/` names only the directory. `--flag name=value` sets a
|
|
274
295
|
feature flag (§2.9), and `--json` prints the results as JSON. The exit code is `0` when
|
|
@@ -320,6 +341,8 @@ Any key not listed here is an error.
|
|
|
320
341
|
| ---------- | ------------------- | -------- | ------- | ------------------------------------------------------------------------------ |
|
|
321
342
|
| `id` | string | **yes** | | The file name without `.yml` (below). |
|
|
322
343
|
| `steps` | list of steps | no | `[]` | The requests, in run order (§2.1). |
|
|
344
|
+
| `setup` | list of steps | no | | Run once before `steps`; what it sets lasts the run (§2.10). |
|
|
345
|
+
| `teardown` | list of steps | no | | Run once after the rest, even when a step failed (§2.10). |
|
|
323
346
|
| `docs` | string | no | | Markdown. |
|
|
324
347
|
| `tags` | list of tags | no | | Tags that select the whole collection (§2.4). |
|
|
325
348
|
| `stepTags` | boolean | no | `false` | `true` lets steps carry their own tags (§2.4). |
|
|
@@ -376,6 +399,8 @@ Methods: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, `OPTIONS`.
|
|
|
376
399
|
| `tests` | string | no | The checks, as JavaScript: calls on `gta` (§3) and any other code (§5). |
|
|
377
400
|
| `tags` | list of tags | no | The step's own tags. Only with `stepTags: true` on the collection (§2.4). |
|
|
378
401
|
| `flags` | map | no | Feature flags the step needs (§2.9). |
|
|
402
|
+
| `forEach` | string | no | Send the request once for each item of a list (below). |
|
|
403
|
+
| `useTests` | `true` | no | In a request set: the use step's `tests` check this response (§2.5). |
|
|
379
404
|
| `base` | `false` | no | `false` leaves the step's endpoint base out (§2.6). |
|
|
380
405
|
| `docs` | string | no | Markdown. |
|
|
381
406
|
|
|
@@ -387,6 +412,23 @@ Methods: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, `OPTIONS`.
|
|
|
387
412
|
- **Any other key is an error**, so a misspelled key such as `heders:` fails at once
|
|
388
413
|
rather than being ignored. A method key in lower case (`get:`) is reported as such.
|
|
389
414
|
|
|
415
|
+
**`forEach`** sends the request once for each item of a list, each reported as its own
|
|
416
|
+
result, `remove grant (item 2 of 3)`:
|
|
417
|
+
|
|
418
|
+
```yaml
|
|
419
|
+
- name: remove grant
|
|
420
|
+
DELETE: '{{baseUrl}}/grants/{{item}}'
|
|
421
|
+
forEach: '{{grantedRoots}}' # a variable holding ["r1", "r2"], or a list written in place
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
- Its `{{variables}}` resolve when the step runs, so an earlier step can make the list,
|
|
425
|
+
as `gta.set('grantedRoots', ids)` does. The result must be a JSON array.
|
|
426
|
+
- Each item is `{{item}}` in the request and `item` in its scripts. An item that is an
|
|
427
|
+
object or a list is its JSON in the request.
|
|
428
|
+
- An empty list skips the step. Anything that is not a list fails it before anything
|
|
429
|
+
is sent.
|
|
430
|
+
- A use step cannot have `forEach`.
|
|
431
|
+
|
|
390
432
|
### 2.2 `body`
|
|
391
433
|
|
|
392
434
|
A body declares **exactly one** of these keys. Omit `body` for no body.
|
|
@@ -450,6 +492,9 @@ body:
|
|
|
450
492
|
| `{ file, contentType?, filename? }` | A file part. |
|
|
451
493
|
| a list of the above | One part for each item, all with the field name. |
|
|
452
494
|
|
|
495
|
+
`filename: ''` sends an empty name, as a browser does when no file is chosen. Pair it
|
|
496
|
+
with an empty file for the whole of that request.
|
|
497
|
+
|
|
453
498
|
The `Content-Type` is `multipart/form-data` with a boundary chosen for the request. A
|
|
454
499
|
declared multipart type without a boundary, such as `multipart/mixed`, gets one added.
|
|
455
500
|
A declared boundary is used as written.
|
|
@@ -459,13 +504,28 @@ the project the step belongs to. For a step of a request set, that is the set's
|
|
|
459
504
|
project, which may be a global one. The path is the same wherever the collection sits
|
|
460
505
|
inside `collections/`.
|
|
461
506
|
|
|
507
|
+
A file that is not there is looked for in the global project (§1.1), as a request set
|
|
508
|
+
or a base collection is, so projects can share one copy of a file. A project's own file
|
|
509
|
+
of the same path wins. `global:` before the path reads only the global project's:
|
|
510
|
+
|
|
511
|
+
```yaml
|
|
512
|
+
body:
|
|
513
|
+
multipart:
|
|
514
|
+
avatar: { file: files/test-png.png } # the project's, else the global project's
|
|
515
|
+
terms: { file: global:files/terms.pdf } # the global project's only
|
|
516
|
+
```
|
|
517
|
+
|
|
462
518
|
- It must be a relative path, written with `/`. An absolute path is refused.
|
|
519
|
+
- A path out of the project folder, such as `../other/files/a.png`, is read from where
|
|
520
|
+
it points, with no global project to fall back to. A `global:` path stays inside the
|
|
521
|
+
global project folder, and is an error in a project that uses none.
|
|
463
522
|
- `{{variables}}` resolve in the path, so a data file row (§2.8) can choose the file.
|
|
464
523
|
- A file is sent byte for byte. `{{…}}` inside it is not a variable.
|
|
465
524
|
- A file that cannot be read stops the step before anything is sent, in its own `body`
|
|
466
|
-
phase, naming the file.
|
|
525
|
+
phase, naming the file and each place it was looked for.
|
|
467
526
|
- Where a request is shown (results, reports, `req.body`), a file's bytes appear as
|
|
468
|
-
`‹file files/avatar.png, 1234 bytes
|
|
527
|
+
`‹file files/avatar.png, 1234 bytes›`, and a global project's file as
|
|
528
|
+
`‹file global:files/terms.pdf, 5254 bytes›`, however the step named it.
|
|
469
529
|
|
|
470
530
|
### 2.3 `settings` and `headers`
|
|
471
531
|
|
|
@@ -581,6 +641,10 @@ steps:
|
|
|
581
641
|
`{{params.name}}`, and in code as `params.name`, a read-only object holding the real
|
|
582
642
|
values. No variable can take the place of a `params.` name.
|
|
583
643
|
|
|
644
|
+
A default may name variables and other params, as in
|
|
645
|
+
`email: '{{params.accountId}}@example.com'`. Like a `with:` value, it is resolved once
|
|
646
|
+
for each use, so `accountId: '{{$uuid}}'` is one id wherever the set reads it.
|
|
647
|
+
|
|
584
648
|
**A use step** holds only `use`, `with`, `name`, `tags`, `flags`, `docs` and `tests`. A
|
|
585
649
|
method key, `headers`, `body`, `settings` or `before` on it is an error, and `with`
|
|
586
650
|
without `use` is an error too.
|
|
@@ -589,18 +653,59 @@ without `use` is an error too.
|
|
|
589
653
|
global project. `use: auth/login` is one directory down. `use: global:login` looks
|
|
590
654
|
only in the global project.
|
|
591
655
|
- **`with:`** gives plain values; a param left out takes its default. A string may hold
|
|
592
|
-
`{{variables}}`, resolved
|
|
593
|
-
|
|
656
|
+
`{{variables}}`, resolved as the set's first request starts, just after the
|
|
657
|
+
collection's `before.script` has run for it: a value that script sets for each step
|
|
658
|
+
(§4) reaches the set. A missing required value, a name the set does not take, or a
|
|
659
|
+
value or default that cannot be resolved stops the set's steps before anything is
|
|
660
|
+
sent.
|
|
594
661
|
- **Running.** A use step runs each of the set's steps in turn, in the collection's
|
|
595
662
|
variable scope, so what one sets the next can read, and so can the steps after the use
|
|
596
|
-
step. Each request is reported as its own result.
|
|
663
|
+
step. Each request is reported as its own result. When the use step has a `name`,
|
|
664
|
+
reports use it: `sign in` for a set of one step, `sign in › get profile` for a longer
|
|
665
|
+
one.
|
|
597
666
|
- **Layers.** Headers and settings: the collection's, under the set's, under each
|
|
598
667
|
step's own. Scripts run collection, then set, then step: `before.script` before the
|
|
599
|
-
request and `tests` after. The use step's own `tests` run last,
|
|
600
|
-
step.
|
|
668
|
+
request and `tests` after. The use step's own `tests` run last, on the response of
|
|
669
|
+
the set's step marked `useTests: true`, or else its last step. `params` is the set's
|
|
670
|
+
alone: the collection's scripts, and a base collection's or an endpoint's (§2.6,
|
|
671
|
+
§2.7), never see it. In an endpoint's `before.script`, a segment written as
|
|
672
|
+
`{{params.id}}` reads as written.
|
|
601
673
|
- **One level.** A request set must not `use:` another, and has `params`, not `vars`. A
|
|
602
674
|
file in `requests/` without `params` is not a request set.
|
|
603
675
|
|
|
676
|
+
**Saving under the caller's name.** A set can take the name to save a value under as a
|
|
677
|
+
param, save it with `gta.set(params.saveAs, …)`, and read it back in its later steps
|
|
678
|
+
with `{{@params.saveAs}}` (§4). The caller then reads it by the name it chose:
|
|
679
|
+
|
|
680
|
+
```yaml
|
|
681
|
+
# requests/create-user.yml
|
|
682
|
+
params:
|
|
683
|
+
saveTokenAs: { required: true }
|
|
684
|
+
steps:
|
|
685
|
+
- name: create token
|
|
686
|
+
POST: '{{authUrl}}/token'
|
|
687
|
+
tests: gta.set(params.saveTokenAs, res.body.accessToken)
|
|
688
|
+
- name: get profile
|
|
689
|
+
GET: '{{baseUrl}}/profile'
|
|
690
|
+
headers:
|
|
691
|
+
Authorization: Bearer {{@params.saveTokenAs}}
|
|
692
|
+
useTests: true # the use step's tests check this response
|
|
693
|
+
- name: wait for events
|
|
694
|
+
GET: '{{baseUrl}}/wait'
|
|
695
|
+
```
|
|
696
|
+
|
|
697
|
+
```yaml
|
|
698
|
+
# a collection
|
|
699
|
+
- use: create-user
|
|
700
|
+
name: user 1
|
|
701
|
+
with: { saveTokenAs: token1 }
|
|
702
|
+
tests: gta.expectResponseBodyToHaveProperty('email') # on get profile's response
|
|
703
|
+
- GET: '{{baseUrl}}/orders'
|
|
704
|
+
headers: { Authorization: 'Bearer {{token1}}' }
|
|
705
|
+
```
|
|
706
|
+
|
|
707
|
+
Only one step of a set may have `useTests`, and only a set's steps.
|
|
708
|
+
|
|
604
709
|
### 2.6 Endpoint bases
|
|
605
710
|
|
|
606
711
|
What every request to one endpoint gets, wherever the request is written. A file in
|
|
@@ -699,7 +804,8 @@ userId,expectedStatus,iterationLabel
|
|
|
699
804
|
|
|
700
805
|
- **One run per row.** Every step runs for row 1, then every step again for row 2, and
|
|
701
806
|
so on. Each run starts from a fresh variable scope, so nothing one row sets leaks into
|
|
702
|
-
the next. Three steps and two rows report
|
|
807
|
+
the next, except a value set to last the run (§2.10). Three steps and two rows report
|
|
808
|
+
six results.
|
|
703
809
|
- **Precedence.** A row's values sit over the environment and under what code sets
|
|
704
810
|
(§4).
|
|
705
811
|
- **`iterationLabel`**, an optional column, names the row in reports, as in
|
|
@@ -793,6 +899,41 @@ own, what the `command` prints, then overrides.
|
|
|
793
899
|
|
|
794
900
|
Every report records the flag values a run used and where each came from.
|
|
795
901
|
|
|
902
|
+
### 2.10 Setup, teardown and values that last the run
|
|
903
|
+
|
|
904
|
+
`setup` and `teardown` are lists of steps, like `steps`, run once around it:
|
|
905
|
+
|
|
906
|
+
```yaml
|
|
907
|
+
id: approved-domains
|
|
908
|
+
setup: # once, before the first row
|
|
909
|
+
- use: seed-admin # log in, grant an admin role, wait for it to land
|
|
910
|
+
teardown: # once, after the last row, even when a row failed
|
|
911
|
+
- name: remove the grant
|
|
912
|
+
DELETE: '{{baseUrl}}/admins/{{adminId}}'
|
|
913
|
+
steps: # once per row of approved-domains.csv, as before
|
|
914
|
+
- name: approve the domains
|
|
915
|
+
PUT: '{{baseUrl}}/orgs/{{orgId}}/domains'
|
|
916
|
+
```
|
|
917
|
+
|
|
918
|
+
- **Order.** Setup, then the steps once per data row (or once, with no data file), then
|
|
919
|
+
teardown. They work the same with or without a data file.
|
|
920
|
+
- **What setup sets lasts the run.** Every value setup sets or captures is there for
|
|
921
|
+
every row and for teardown. It sits over the environment and under a row's values
|
|
922
|
+
(§4).
|
|
923
|
+
- **A row keeps a value for the rows after it** with
|
|
924
|
+
`gta.set(name, value, { scope: 'run' })`. Anything else a row sets lasts only that
|
|
925
|
+
row, as §2.8 says.
|
|
926
|
+
- **Setup failing stops the rows.** They are not run and count as skipped. Teardown
|
|
927
|
+
still runs.
|
|
928
|
+
- **Teardown always runs**, every step of it, after failures and after `bail`. A run that
|
|
929
|
+
is cancelled, or stopped by `timeoutCollection`, stops where it is.
|
|
930
|
+
- **The collection applies to them**: its headers, settings, `before`, `tests` and
|
|
931
|
+
flags, as to any step. Their steps may be use steps and may use `forEach`, but carry
|
|
932
|
+
no `tags`: they run whenever the collection does.
|
|
933
|
+
- A request set, a base collection and an endpoints file have no setup or teardown.
|
|
934
|
+
- Reports name their results `setup › log in` and `teardown › remove the grant`. Running
|
|
935
|
+
a single step in the desktop app does not run them.
|
|
936
|
+
|
|
796
937
|
---
|
|
797
938
|
|
|
798
939
|
## 3. Checking a response
|
|
@@ -817,6 +958,19 @@ tests: |
|
|
|
817
958
|
A path addresses the body in dot or bracket notation: `user.name`; `groups.0.name` or
|
|
818
959
|
`groups[0].name` for an index; and `sessions[].id` for a property of every item.
|
|
819
960
|
|
|
961
|
+
A key that holds a `.`, `[` or `]`, or is empty, goes in brackets as a JSON string:
|
|
962
|
+
`jwt.payload["https://example.com/id"]`, `modules[""].edition`. Reports show such keys
|
|
963
|
+
the same way.
|
|
964
|
+
|
|
965
|
+
As in xtest, a path can also be a list of keys: `['jwt', 'payload', 'https://example.com/id']`.
|
|
966
|
+
Each item is one key, whatever it holds, and a number is its digits, so
|
|
967
|
+
`['groups', 0, 'name']` reads an index. Every function that takes a path takes a list
|
|
968
|
+
too, and so does `pathToProperty`.
|
|
969
|
+
|
|
970
|
+
A path that runs into a `null` before its end, such as `phoneNumber.number` when
|
|
971
|
+
`phoneNumber` is `null`, reads as that `null` for a check that the value is `null`, as
|
|
972
|
+
xtest read it. For any other check the property is not present.
|
|
973
|
+
|
|
820
974
|
### Body conversion
|
|
821
975
|
|
|
822
976
|
A JSON body is used as it is. An XML body is converted: the root element is the single
|
|
@@ -868,7 +1022,15 @@ gta.expectResponseBodyToHaveUnorderedArray('users', [
|
|
|
868
1022
|
```
|
|
869
1023
|
|
|
870
1024
|
`gta.expectResponseBodyToHaveUnorderedArrayNotThisItem(path, list)` passes when no item
|
|
871
|
-
matches.
|
|
1025
|
+
matches. A `compareValue` may be a `RegExp`.
|
|
1026
|
+
|
|
1027
|
+
- **Each call prefers items an earlier call did not match.** Two calls with the same
|
|
1028
|
+
description find two items when there are two, so each capture and strict validation
|
|
1029
|
+
see a different one. A sort starts this over, since its indexes name other items.
|
|
1030
|
+
- **A list of one `notThisExpectedValue` entry reads as xtest read it.** Without strict
|
|
1031
|
+
validation it means no item has that value, so an empty array passes. With it, it
|
|
1032
|
+
means one item whose value is something else, as any list does. The step's last word
|
|
1033
|
+
on strict validation decides.
|
|
872
1034
|
|
|
873
1035
|
### Strict validation
|
|
874
1036
|
|
|
@@ -881,6 +1043,8 @@ checked, ignored or captured.
|
|
|
881
1043
|
vouch for what is inside.
|
|
882
1044
|
- It is judged after the collection's and the step's `tests` have both run, over what
|
|
883
1045
|
either checked.
|
|
1046
|
+
- A binary or HTML body has no properties to leave unchecked, so strict validation
|
|
1047
|
+
passes. A check on one still fails.
|
|
884
1048
|
|
|
885
1049
|
### Sorting
|
|
886
1050
|
|
|
@@ -922,10 +1086,15 @@ to. Whitespace inside the braces is ignored. In code, read a variable with
|
|
|
922
1086
|
|
|
923
1087
|
- **Values keep their type.** A string that is exactly one reference returns the
|
|
924
1088
|
variable's own value, so a variable written as `true` arrives as a boolean. Anything
|
|
925
|
-
else is text, and `null` becomes the empty string inside it
|
|
1089
|
+
else is text, and `null` becomes the empty string inside it, except in a `json` body,
|
|
1090
|
+
where it is written `null`, so `"website": {{site}}` stays valid JSON.
|
|
926
1091
|
- **An unknown variable is an error**, never literal text. The step fails in its own
|
|
927
1092
|
`interpolate` phase, naming the variable, and nothing is sent. A reference loop, or a
|
|
928
1093
|
chain more than 16 deep, fails the same way.
|
|
1094
|
+
- **`{{@name}}` reads the variable `name` names.** With `saveAs: token1`,
|
|
1095
|
+
`{{@saveAs}}` is the value of `token1`. The name may itself be built from variables.
|
|
1096
|
+
A name that is not text, or names no variable, fails like an unknown variable. A
|
|
1097
|
+
request set uses it to read what it saved under its caller's name (§2.5).
|
|
929
1098
|
- **In YAML, quote a value that starts with `{{`.** Unquoted, YAML reads `{` as a map.
|
|
930
1099
|
|
|
931
1100
|
### Built-in variables
|
|
@@ -945,7 +1114,8 @@ Lowest precedence first:
|
|
|
945
1114
|
|
|
946
1115
|
```
|
|
947
1116
|
global project vars → project vars → base collection vars → collection vars
|
|
948
|
-
→ environment → data file row → gta.set and captures
|
|
1117
|
+
→ environment → what lasts the run → data file row → gta.set and captures
|
|
1118
|
+
→ process environment
|
|
949
1119
|
```
|
|
950
1120
|
|
|
951
1121
|
The environment is the global project's file of that name, if there is one, with the
|
|
@@ -1006,13 +1176,23 @@ There is nothing to load, and no `startXTest` or `endXTest`.
|
|
|
1006
1176
|
| `gta.test(name, fn)`: a named check that passes unless `fn` throws or rejects; `fn` may be async | |
|
|
1007
1177
|
| `gta.get(name)`: a variable's current value | ✓ |
|
|
1008
1178
|
| `gta.set(name, value)`: set a variable for the rest of the run | ✓ |
|
|
1179
|
+
| `gta.set(name, value, { scope: 'run' })`: the same, lasting past this data row too (§2.10) | ✓ |
|
|
1180
|
+
| `gta.skip(reason?)`: send nothing for this step, and report it skipped with the reason | ✓ |
|
|
1181
|
+
| `gta.skipRest(reason?)`: skip the rest of this row's steps; in `before.script`, this one too | ✓ |
|
|
1009
1182
|
| `gta.flag(name)`: a feature flag's value; an unknown flag is an error | ✓ |
|
|
1010
1183
|
| `gta.uuid()`: a random (version 4) UUID | ✓ |
|
|
1011
1184
|
| `gta.uuidv7()`: a time-ordered (version 7) UUID | ✓ |
|
|
1012
1185
|
| `gta.randomInt(min, max)`: a whole number, both ends included | ✓ |
|
|
1013
1186
|
| `gta.date(format, secondsOffset = 0, timeZone = 'local')` | ✓ |
|
|
1014
1187
|
|
|
1015
|
-
|
|
1188
|
+
`gta.set` keeps a string, number, boolean or null as it is, and stores an object or array
|
|
1189
|
+
as JSON text. `undefined` is stored as `null`, so a variable set to nothing still reads
|
|
1190
|
+
`== null`, and `{{name}}` resolves as a null does (§4).
|
|
1191
|
+
|
|
1192
|
+
Calling a response check in `before.script` is an error, and so is `gta.skip` in
|
|
1193
|
+
`tests`, where the request has been sent. After `gta.skip` the script runs to its end,
|
|
1194
|
+
and no later `before.script` runs. `gta.skipRest` skips steps of this row only: the next
|
|
1195
|
+
row, and teardown, still run.
|
|
1016
1196
|
|
|
1017
1197
|
`gta.date` formats with strftime specifiers: `%Y %y %m %d %e %H %I %M %S %L %p %b %B %a
|
|
1018
1198
|
%A %j %Z %z %s %F %T %%`. An unrecognized specifier is left in the output, so a typo is
|
|
@@ -1024,7 +1204,7 @@ xtest's military zone letters (`U` is -08:00, not UTC).
|
|
|
1024
1204
|
| Global | What it is |
|
|
1025
1205
|
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
1026
1206
|
| `res` | `tests` only: `status`, `statusText`, `headers` (lower-cased names), `header(name)`, `body` (parsed JSON, converted XML, or text), `text`, `time` (ms), `size` (bytes). |
|
|
1027
|
-
| `req` | `method`, `url`, `headers`, `body`: as sent in `tests`, as written in `before.script
|
|
1207
|
+
| `req` | `method`, `url`, `headers`, `body`: as sent in `tests`, as written in `before.script`, where a script may change `headers` and `body` (below). |
|
|
1028
1208
|
| `assert` | Node's strict `assert`, for use inside `gta.test`. |
|
|
1029
1209
|
| `console` | Captured into the step's result. |
|
|
1030
1210
|
| `params` | A request set's params (§2.5), in its own scripts and in the tests of the use step running it. |
|
|
@@ -1042,6 +1222,24 @@ run the same on any machine, and a request belongs in a step, where it is record
|
|
|
1042
1222
|
marked as errored. Checks made before it are kept.
|
|
1043
1223
|
- A script that runs for more than 10 seconds is stopped.
|
|
1044
1224
|
|
|
1225
|
+
**Changing the request in `before.script`.** `req` there is the request as written,
|
|
1226
|
+
`{{variables}}` still in it. A script may change it before it is sent, and a later
|
|
1227
|
+
`before.script` sees the change:
|
|
1228
|
+
|
|
1229
|
+
```js
|
|
1230
|
+
const claims = JSON.parse(req.body)
|
|
1231
|
+
if (!params.crmContactId) delete claims.crm_contact_id // leave the member out
|
|
1232
|
+
req.body = JSON.stringify(claims)
|
|
1233
|
+
req.headers['X-Trace'] = '{{traceId}}'
|
|
1234
|
+
delete req.headers['X-Debug']
|
|
1235
|
+
```
|
|
1236
|
+
|
|
1237
|
+
- `req.body` is text, and can be changed for a `json`, `xml`, `text` or `graphql` body.
|
|
1238
|
+
A form, multipart or file body is built from its parts, so it cannot.
|
|
1239
|
+
- `req.headers` is a map of names to values: add, change or delete them.
|
|
1240
|
+
- Variables in what the script writes resolve afterwards, as in the file.
|
|
1241
|
+
- Anything else is a pre-request error, and nothing is sent.
|
|
1242
|
+
|
|
1045
1243
|
### Check files
|
|
1046
1244
|
|
|
1047
1245
|
`checks/*.js` in a project, and in its global project, hold functions every script can
|
|
@@ -1154,7 +1352,8 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1154
1352
|
ignoring case.
|
|
1155
1353
|
- There is no `name` key (use `id`).
|
|
1156
1354
|
- A step has `tags` only if the collection has `stepTags: true`.
|
|
1157
|
-
- With `params`: no step is a use step, and there is no `vars`.
|
|
1355
|
+
- With `params`: no step is a use step, and there is no `vars`, `setup` or `teardown`.
|
|
1356
|
+
- A `setup` or `teardown` step has no `tags`.
|
|
1158
1357
|
- A data file, when present, reads (§2.8).
|
|
1159
1358
|
|
|
1160
1359
|
**Steps**
|
|
@@ -1167,6 +1366,8 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1167
1366
|
- `before` holds only `script`. `before.set` is not supported.
|
|
1168
1367
|
- There is no `expect:` key; checks go in `tests`.
|
|
1169
1368
|
- `base` is only ever `false`.
|
|
1369
|
+
- `forEach` is a string, and a use step has none.
|
|
1370
|
+
- `useTests` is only ever `true`, only on a request set's step, and on one step at most.
|
|
1170
1371
|
|
|
1171
1372
|
**Values**
|
|
1172
1373
|
|
|
@@ -1189,9 +1390,10 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1189
1390
|
**Library files**
|
|
1190
1391
|
|
|
1191
1392
|
- A request set (`requests/`) has `params`, uses no other set, and has no `vars`.
|
|
1192
|
-
- A base collection (`bases/`) has no `steps` or `params`, and no
|
|
1393
|
+
- A base collection (`bases/`) has no `steps`, `setup`, `teardown` or `params`, and no
|
|
1394
|
+
`extends`.
|
|
1193
1395
|
- An endpoint (`endpoints/`) has a URL that is a path starting with `/`, and no use
|
|
1194
|
-
steps.
|
|
1396
|
+
steps. An endpoints file has no `setup` or `teardown`.
|
|
1195
1397
|
- A check file's name is a JavaScript identifier; if it isn't, the file is not loaded.
|
|
1196
1398
|
- `collections/`, `requests/`, `endpoints/` and `bases/` hold files at most one directory
|
|
1197
1399
|
down.
|
|
@@ -1211,8 +1413,8 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1211
1413
|
- `uses` and `tls.ca` entries are relative paths.
|
|
1212
1414
|
- The folder `uses` names has a `project.yml`, and does not itself `uses` another.
|
|
1213
1415
|
- Each `tls.ca` file exists and holds a PEM or DER certificate.
|
|
1214
|
-
- `settings.yml` exists for `gta
|
|
1215
|
-
environment that exists.
|
|
1416
|
+
- `settings.yml` exists for `gta`; a global project's is optional. The `environmentType`
|
|
1417
|
+
a run ends up with, if any, names an environment that exists.
|
|
1216
1418
|
|
|
1217
1419
|
**At run time**
|
|
1218
1420
|
|
|
@@ -1221,8 +1423,16 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1221
1423
|
- Every feature flag named is known to the run.
|
|
1222
1424
|
- The flag `command`, if any, has every quote closed, starts, and exits `0` within 60
|
|
1223
1425
|
seconds, printing a JSON object.
|
|
1224
|
-
- Every file a body names can be read from the project folder
|
|
1225
|
-
|
|
1426
|
+
- Every file a body names can be read from the project folder, or its global
|
|
1427
|
+
project's (§2.2).
|
|
1428
|
+
- Every `use:` and `extends:` names a usable file, and every `with:` suits its set.
|
|
1429
|
+
`gta get` checks these, and the body files named without `{{variables}}`, without
|
|
1430
|
+
running anything (§1.3).
|
|
1431
|
+
- A step's `forEach` resolves to a JSON array.
|
|
1432
|
+
- In `{{@name}}`, `name` holds text naming a variable that exists.
|
|
1433
|
+
- A `before.script` sets `req.body` to text, and only for a `json`, `xml`, `text` or
|
|
1434
|
+
`graphql` body; `req.headers` stays a map.
|
|
1435
|
+
- `gta.set`'s `scope`, when given, is `'run'`.
|
|
1226
1436
|
|
|
1227
1437
|
---
|
|
1228
1438
|
|