@schwabyio/gta 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +140 -1
- package/dist/FUNCTIONS.md +5 -2
- package/dist/SPEC.md +207 -36
- package/dist/THIRD_PARTY_NOTICES.txt +26 -0
- package/dist/chunks/chunk-KVO46B7X.js +234 -0
- package/dist/chunks/chunk-Q3VRXFS5.js +577 -0
- package/dist/chunks/{chunk-ABBBM5UZ.js → chunk-ZDHGCHIV.js} +8802 -1946
- package/dist/gta.js +1036 -998
- package/dist/index.d.ts +3 -0
- package/dist/index.js +14 -0
- package/dist/playwright.d.ts +37 -0
- package/dist/playwright.js +178 -0
- package/dist/types.d.ts +243 -0
- package/dist/worker.js +1 -1
- package/package.json +25 -2
package/README.md
CHANGED
|
@@ -23,7 +23,7 @@ The package has no dependencies to install: everything `gta` uses is bundled int
|
|
|
23
23
|
|
|
24
24
|
## A first project
|
|
25
25
|
|
|
26
|
-
A project is a folder holding a `collections/`
|
|
26
|
+
A project is a folder holding a `collections/` folder, an `environments/` one and a
|
|
27
27
|
`settings.yml`. The smallest project `gta` runs is three files:
|
|
28
28
|
|
|
29
29
|
```yaml
|
|
@@ -139,6 +139,43 @@ In a monorepo, projects that share a global project (`uses:` in `project.yml`) s
|
|
|
139
139
|
setting by setting it again in its own file. Each project still needs a `settings.yml`,
|
|
140
140
|
even an empty one.
|
|
141
141
|
|
|
142
|
+
## Project rules
|
|
143
|
+
|
|
144
|
+
A `rules.yml` beside `project.yml` writes down how a project's files are named, laid out
|
|
145
|
+
and written, so everyone working on it, people and coding agents alike, keeps it the
|
|
146
|
+
same:
|
|
147
|
+
|
|
148
|
+
```yaml
|
|
149
|
+
# rules.yml
|
|
150
|
+
ids:
|
|
151
|
+
collections: kebab-case # or a pattern, such as '{folder}-[a-z0-9-]+'
|
|
152
|
+
layout:
|
|
153
|
+
folders: required # every collection in a folder of collections/
|
|
154
|
+
docs:
|
|
155
|
+
collections: required
|
|
156
|
+
tags:
|
|
157
|
+
allowed: [smoke, regression]
|
|
158
|
+
steps:
|
|
159
|
+
names: required
|
|
160
|
+
tests:
|
|
161
|
+
only: [gta] # tests call the gta.* functions, and nothing else
|
|
162
|
+
statusCode: required
|
|
163
|
+
guide: |
|
|
164
|
+
Name each step for what it proves.
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
```sh
|
|
168
|
+
gta rules # the rules in effect, where each came from, what each means, the guide
|
|
169
|
+
gta lint # check every file against them
|
|
170
|
+
gta lint --json # the findings as JSON, for tools and agents
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
`gta lint` prints each finding at its file and line, with the rule it breaks, and exits
|
|
174
|
+
`1` when it finds one. Rules never change a run: a file that breaks one still runs. A
|
|
175
|
+
global project's `rules.yml` is shared like its `settings.yml`, and a project changes or
|
|
176
|
+
turns off (`null`) any rule in its own file. Every rule is in
|
|
177
|
+
[SPEC.md §1.4](https://github.com/schwabyio/gravity/blob/main/SPEC.md#14-rulesyml).
|
|
178
|
+
|
|
142
179
|
## In CI
|
|
143
180
|
|
|
144
181
|
```yaml
|
|
@@ -147,11 +184,113 @@ even an empty one.
|
|
|
147
184
|
with:
|
|
148
185
|
node-version: 22
|
|
149
186
|
- run: npm install -g @schwabyio/gta
|
|
187
|
+
- run: gta lint
|
|
150
188
|
- run: gta all --environmentType staging --generateJUnitResults
|
|
151
189
|
env:
|
|
152
190
|
apiKey: ${{ secrets.STAGING_API_KEY }}
|
|
153
191
|
```
|
|
154
192
|
|
|
193
|
+
## From Playwright and other code
|
|
194
|
+
|
|
195
|
+
The package is also a library. A test written in code can run a collection or a request
|
|
196
|
+
set, then use the values it saved, such as a token or the id of a user it created, in a
|
|
197
|
+
browser test.
|
|
198
|
+
|
|
199
|
+
### Playwright
|
|
200
|
+
|
|
201
|
+
`@schwabyio/gta/playwright` gives you Playwright's `test` with a `gta` fixture. Point it
|
|
202
|
+
at the project in `playwright.config.ts`:
|
|
203
|
+
|
|
204
|
+
```ts
|
|
205
|
+
// playwright.config.ts
|
|
206
|
+
import { defineConfig } from '@playwright/test'
|
|
207
|
+
import type { GravityConfig } from '@schwabyio/gta/playwright'
|
|
208
|
+
|
|
209
|
+
export default defineConfig<{}, GravityConfig>({
|
|
210
|
+
use: {
|
|
211
|
+
gravity: { project: '../api-tests', environment: 'staging' }
|
|
212
|
+
}
|
|
213
|
+
})
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
```ts
|
|
217
|
+
// tests/dashboard.spec.ts
|
|
218
|
+
import { test, expect } from '@schwabyio/gta/playwright'
|
|
219
|
+
|
|
220
|
+
test('a new user sees their dashboard', async ({ page, gta }) => {
|
|
221
|
+
// requests/create-user.yml, run as a use: step would run it
|
|
222
|
+
const user = await gta.use('create-user', { plan: 'pro' })
|
|
223
|
+
|
|
224
|
+
await page.goto(`/users/${user.values.userId}`)
|
|
225
|
+
await expect(page.getByRole('heading')).toHaveText('Welcome')
|
|
226
|
+
|
|
227
|
+
// collections/billing.yml, starting from what create-user saved
|
|
228
|
+
await gta.run('billing', { vars: user.values })
|
|
229
|
+
})
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
- Each `gta.run` and `gta.use` call is a step in Playwright's report. Each request is a
|
|
233
|
+
step inside it, linked to its step in the YAML, with the request and response attached.
|
|
234
|
+
- A run that fails fails the test at the line that called it, with what failed. With
|
|
235
|
+
`soft: true` the test goes on and fails at the end, as `expect.soft` does.
|
|
236
|
+
- `project` is relative to the folder `playwright.config.ts` is in, and defaults to it.
|
|
237
|
+
`environment`, `flags`, `bail` and `timeoutCollection` go beside it. The project opens
|
|
238
|
+
once per worker, so an environment's flag command runs once per worker.
|
|
239
|
+
- A run counts toward the test's timeout, and one still going when the test ends is
|
|
240
|
+
stopped.
|
|
241
|
+
- To use this `test` with fixtures of your own, combine them with Playwright's
|
|
242
|
+
`mergeTests`.
|
|
243
|
+
|
|
244
|
+
It needs `@playwright/test` 1.51 or later, which installing `gta` does not install.
|
|
245
|
+
|
|
246
|
+
### Any other code
|
|
247
|
+
|
|
248
|
+
```js
|
|
249
|
+
import { openProject } from '@schwabyio/gta'
|
|
250
|
+
|
|
251
|
+
const project = await openProject('api-tests', { environment: 'staging' })
|
|
252
|
+
const login = await project.use('login', { username: 'alice' })
|
|
253
|
+
const checkout = await project.run('checkout', { vars: login.values })
|
|
254
|
+
if (!checkout.passed) throw new Error(checkout.failures)
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
**`openProject(folder, options)`** opens the project in `folder`, the one holding
|
|
258
|
+
`collections/`. The options are `environment`, `flags`, `bail` and `timeoutCollection`.
|
|
259
|
+
Each one you leave out comes from `settings.yml`, the same way `gta` reads it, but here
|
|
260
|
+
the project does not need a `settings.yml`.
|
|
261
|
+
|
|
262
|
+
**`project.run(collection, options)`** runs a collection the way `gta` runs it: setup,
|
|
263
|
+
then the steps once for each data row, then teardown. Name the collection by its id or
|
|
264
|
+
its place in `collections/` (`checkout/sessions`). `tags` and `notTags` don't apply,
|
|
265
|
+
since you name what runs, and a collection with `exclude: true` runs too. The options:
|
|
266
|
+
|
|
267
|
+
| Option | What it does |
|
|
268
|
+
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
|
269
|
+
| `steps` | Run only these steps, named as reports name them or numbered from 1. They run in the file's order, and setup and teardown still run. |
|
|
270
|
+
| `vars` | Values the run starts with. They sit over the environment and under a data row. |
|
|
271
|
+
| `bail` | Stop at the first failing step. |
|
|
272
|
+
| `signal` | An `AbortSignal` that stops the run. |
|
|
273
|
+
| `onResult` | `(result, step) => void`, called as each request finishes. |
|
|
274
|
+
|
|
275
|
+
**`project.use(set, params, options)`** runs a request set as a `use:` step would:
|
|
276
|
+
`login` is `requests/login.yml`, in the project or its global project. `params` are what
|
|
277
|
+
`with:` gives, and a param you leave out takes its default. The options are those of
|
|
278
|
+
`run`, without `steps`.
|
|
279
|
+
|
|
280
|
+
Both resolve to the same outcome, however the steps fare:
|
|
281
|
+
|
|
282
|
+
| Field | What it holds |
|
|
283
|
+
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
284
|
+
| `passed` | `true` when nothing failed or errored and the run finished. |
|
|
285
|
+
| `values` | Every value the run set or captured, by name: `gta.set`, and checks that save what they read, such as `'setAsCollectionVariable'`. Secrets are not hidden here. |
|
|
286
|
+
| `summary` | Totals, and every request's result as gta's JSON report holds it, with secrets shown as `[secret: NAME]`. |
|
|
287
|
+
| `steps` | Where each result's step is: its file and line. |
|
|
288
|
+
| `failures` | What went wrong, as `gta` prints it under Failures. Empty when the run passed. |
|
|
289
|
+
| `error` | Why the run did not start or finish: a file that will not load, `timeoutCollection`, or a cancel. Otherwise `null`. |
|
|
290
|
+
|
|
291
|
+
They reject only when a collection, step or request set doesn't exist, or a value isn't a
|
|
292
|
+
string, number, boolean or null. Types ship with the package.
|
|
293
|
+
|
|
155
294
|
## The file format
|
|
156
295
|
|
|
157
296
|
[SPEC.md](https://github.com/schwabyio/gravity/blob/main/SPEC.md) specifies every file,
|
package/dist/FUNCTIONS.md
CHANGED
|
@@ -216,8 +216,11 @@ gta.expectResponseBodyToHaveUnorderedArray('users', [{ name: 'Ada' }, { name: /^
|
|
|
216
216
|
```
|
|
217
217
|
|
|
218
218
|
An object in the list matches an item holding each of its properties with that value, or
|
|
219
|
-
matching it when the value is a `RegExp`. The item may have others. A
|
|
220
|
-
|
|
219
|
+
matching it when the value is a `RegExp`. The item may have others. A property that is
|
|
220
|
+
itself an object is matched the same way, so `{ data: { status: 'reversed' } }` finds an
|
|
221
|
+
item whose `data` has that status, whatever else `data` holds. An array compares whole.
|
|
222
|
+
A pattern is tested against the value as text, so it never matches an item that is an
|
|
223
|
+
object or an array.
|
|
221
224
|
|
|
222
225
|
**A list of `{ pathToProperty, expectedValue, specialHandling? }` entries.** Together
|
|
223
226
|
they describe **one** item, property by property, and some item must match every entry.
|
package/dist/SPEC.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# The Gravity file format
|
|
2
2
|
|
|
3
|
-
Version 0.
|
|
3
|
+
Version 0.14.0, the version of Gravity and `gta` that reads it: the two are released
|
|
4
|
+
together.
|
|
4
5
|
|
|
5
6
|
This document specifies the YAML files that **Gravity**, the desktop app, and **`gta`**,
|
|
6
7
|
the command-line runner, read and write. Together they make up Gravity Test
|
|
@@ -26,18 +27,19 @@ is a complete, valid project to start from.
|
|
|
26
27
|
|
|
27
28
|
## At a glance
|
|
28
29
|
|
|
29
|
-
| File | Where
|
|
30
|
-
| ----------------------------- |
|
|
31
|
-
| `collections/<id>.yml` | `collections/`, or one
|
|
32
|
-
| `collections/<id>.csv\|.json` | Beside its collection
|
|
33
|
-
| `environments/<name>.yml` | `environments/`
|
|
34
|
-
| `project.yml` | The project folder
|
|
35
|
-
| `settings.yml` | The project folder
|
|
36
|
-
| `
|
|
37
|
-
| `
|
|
38
|
-
| `
|
|
39
|
-
| `
|
|
40
|
-
|
|
|
30
|
+
| File | Where | What it is | § |
|
|
31
|
+
| ----------------------------- | --------------------------------------- | ------------------------------------------- | ---- |
|
|
32
|
+
| `collections/<id>.yml` | `collections/`, or one folder inside it | A collection: requests run in order | §2 |
|
|
33
|
+
| `collections/<id>.csv\|.json` | Beside its collection | A data file: the collection runs once a row | §2.8 |
|
|
34
|
+
| `environments/<name>.yml` | `environments/` | Variables, secrets and flags for one target | §6 |
|
|
35
|
+
| `project.yml` | The project folder | Name, global project, variables, trust | §1.1 |
|
|
36
|
+
| `settings.yml` | The project folder | How `gta` runs the project | §1.3 |
|
|
37
|
+
| `rules.yml` | The project folder | How the project's files are written | §1.4 |
|
|
38
|
+
| `requests/<id>.yml` | `requests/`, or one folder inside it | A request set, run by `use:` | §2.5 |
|
|
39
|
+
| `endpoints/<id>.yml` | `endpoints/`, or one folder inside it | Defaults and checks per method and path | §2.6 |
|
|
40
|
+
| `bases/<id>.yml` | `bases/`, or one folder inside it | A base collection, for `extends:` | §2.7 |
|
|
41
|
+
| `checks/<name>.js` | `checks/` | Shared check functions | §5 |
|
|
42
|
+
| `.env` | The project folder, never committed | Values for secrets | §6 |
|
|
41
43
|
|
|
42
44
|
The smallest project `gta` runs is three files:
|
|
43
45
|
|
|
@@ -89,9 +91,10 @@ A **project** is a folder holding `collections/` and, usually, `environments/`:
|
|
|
89
91
|
payments/ a project
|
|
90
92
|
├── project.yml optional (§1.1)
|
|
91
93
|
├── settings.yml how gta runs it (§1.3)
|
|
94
|
+
├── rules.yml how its files are written (§1.4)
|
|
92
95
|
├── collections/
|
|
93
96
|
│ ├── smoke.yml a collection
|
|
94
|
-
│ └── checkout/ a
|
|
97
|
+
│ └── checkout/ a folder grouping collections: one level only
|
|
95
98
|
│ ├── sessions.yml
|
|
96
99
|
│ ├── sessions.csv its data file (§2.8)
|
|
97
100
|
│ └── refunds.yml
|
|
@@ -106,17 +109,17 @@ payments/ a project
|
|
|
106
109
|
└── .env secret values; not committed (§6)
|
|
107
110
|
```
|
|
108
111
|
|
|
109
|
-
**Every `.yml` file directly in `collections/`, or in a
|
|
112
|
+
**Every `.yml` file directly in `collections/`, or in a folder one level down, is a
|
|
110
113
|
collection.** Nothing else is. Discovery is exact: no other `.yml` in a repository is
|
|
111
114
|
mistaken for a collection. A file that does not parse is reported as a broken
|
|
112
115
|
collection, never skipped in silence.
|
|
113
116
|
|
|
114
|
-
- A
|
|
117
|
+
- A folder inside a folder of `collections/` is reported as a problem and not
|
|
115
118
|
read. `requests/`, `endpoints/` and `bases/` are read to the same depth, and
|
|
116
119
|
`checks/` only at its top level.
|
|
117
|
-
- Names starting with `.` are ignored, as are the
|
|
120
|
+
- Names starting with `.` are ignored, as are the folders `node_modules`, `.git`,
|
|
118
121
|
`reports`, `test-results`, `out` and `dist`.
|
|
119
|
-
-
|
|
122
|
+
- Folders carry no configuration and need no file of their own. They group
|
|
120
123
|
collections for display and for running a group.
|
|
121
124
|
|
|
122
125
|
A project is any folder: the root of a repository, or one service of a monorepo. A
|
|
@@ -160,8 +163,8 @@ The file is optional, and so is every key in it. Any other key is an error.
|
|
|
160
163
|
| `tls.ca` | list of relative paths | none | Certificate files that requests trust (below). |
|
|
161
164
|
|
|
162
165
|
**`uses`** names a **global project**: an ordinary project whose `project.yml`
|
|
163
|
-
variables, `environments/`, `requests/`, `endpoints/`, `bases/`, `checks
|
|
164
|
-
`settings.yml` every project using it shares.
|
|
166
|
+
variables, `environments/`, `requests/`, `endpoints/`, `bases/`, `checks/`,
|
|
167
|
+
`settings.yml` and `rules.yml` every project using it shares.
|
|
165
168
|
|
|
166
169
|
- It must be a relative path. An absolute path is refused, since it would only work on
|
|
167
170
|
one machine. Write it with `/`; `\` reads the same.
|
|
@@ -170,7 +173,15 @@ variables, `environments/`, `requests/`, `endpoints/`, `bases/`, `checks/` and
|
|
|
170
173
|
- An environment in the global project merges under the project's environment of the
|
|
171
174
|
same name, key by key, and the project's values win. An environment only the global
|
|
172
175
|
project has is available too.
|
|
173
|
-
- The global project's `settings.yml` lies under the project's the same way (§1.3)
|
|
176
|
+
- The global project's `settings.yml` lies under the project's the same way (§1.3), and
|
|
177
|
+
so does its `rules.yml` (§1.4).
|
|
178
|
+
- A global project's own `collections/` is **not** shared: a project using it never
|
|
179
|
+
sees or runs those collections. A global project needs none. It may have one for a
|
|
180
|
+
single purpose: **testing what it shares**. A collection there can `use:` its request
|
|
181
|
+
sets, `extends:` its bases and call its checks, against its own environments, so a
|
|
182
|
+
broken shared piece fails in one place, before every project relying on it does.
|
|
183
|
+
Those collections run only when the global project itself does, as a project in
|
|
184
|
+
Gravity or with `gta` in its folder, such as its own CI job.
|
|
174
185
|
|
|
175
186
|
**`tls.ca`** lists certificate files that requests trust, for a server whose certificate
|
|
176
187
|
a company or local CA signed, or a server's own self-signed certificate. A request
|
|
@@ -203,11 +214,11 @@ Projects are shared between macOS, Windows and Linux, and read the same on all t
|
|
|
203
214
|
`use:` or `extends:` name, `uses`, a `tls.ca` file, a file a body sends, and an
|
|
204
215
|
environment's file name. One that differs only in case is an error on every platform,
|
|
205
216
|
naming the spelling on disk. The folders and files of §1 are lower case.
|
|
206
|
-
- **No two names in one
|
|
217
|
+
- **No two names in one folder may differ only in case**, such as `Checkout/` and
|
|
207
218
|
`checkout/`: Linux can hold both, but a macOS or Windows checkout only one. This
|
|
208
219
|
applies in `collections/`, `requests/`, `endpoints/`, `bases/`, `environments/` and
|
|
209
220
|
`checks/`. Collection ids are unique ignoring case too (§2).
|
|
210
|
-
- File,
|
|
221
|
+
- File, folder and environment names must avoid what Windows refuses: the characters
|
|
211
222
|
`< > : " / \ | ? *`, control characters, a trailing dot or space, and the names `CON`,
|
|
212
223
|
`PRN`, `AUX`, `NUL`, `CONIN$`, `CONOUT$`, `COM1`–`COM9` and `LPT1`–`LPT9`, with or
|
|
213
224
|
without an extension. A file with such a name, made on macOS or Linux, is reported.
|
|
@@ -282,7 +293,7 @@ global project's `settings.yml`, the project's, the environment variable, the fl
|
|
|
282
293
|
| --------------------- | ----------------------------------------------------------- |
|
|
283
294
|
| `gta get` | Nothing. It lists what `gta all` would run. |
|
|
284
295
|
| `gta all` | Every collection, except those with `exclude: true` (§2.4). |
|
|
285
|
-
| `gta smoke,checkout/` | The collections and
|
|
296
|
+
| `gta smoke,checkout/` | The collections and folders named, in that order. |
|
|
286
297
|
|
|
287
298
|
`gta get` also reports each `use:` and `extends:` that would stop a run, and each file a
|
|
288
299
|
body names that is in neither the project nor its global project (§2.2), in every
|
|
@@ -290,11 +301,159 @@ collection, including those `gta all` leaves out (Appendix A). A file path with
|
|
|
290
301
|
`{{variables}}` is left to the run. It exits `1` when it finds one, or a broken
|
|
291
302
|
collection, and `0` otherwise.
|
|
292
303
|
|
|
293
|
-
A collection is named by its `id`, or by its place (`checkout/sessions`). A
|
|
294
|
-
named by its name, and `checkout/` names only the
|
|
304
|
+
A collection is named by its `id`, or by its place (`checkout/sessions`). A folder is
|
|
305
|
+
named by its name, and `checkout/` names only the folder. `--flag name=value` sets a
|
|
295
306
|
feature flag (§2.9), and `--json` prints the results as JSON. The exit code is `0` when
|
|
296
307
|
everything passed, `1` when something failed, and `2` when `gta` could not run.
|
|
297
308
|
|
|
309
|
+
### 1.4 `rules.yml`
|
|
310
|
+
|
|
311
|
+
How a project's files are named, laid out and written, so that everyone working on it,
|
|
312
|
+
people and coding agents alike, keeps it consistent. It sits beside `project.yml`, is
|
|
313
|
+
committed, and is optional. `gta lint` checks the project against it. Gravity, the
|
|
314
|
+
desktop app, marks each file, folder and step that breaks a rule, and will not create or
|
|
315
|
+
rename a file or folder that would.
|
|
316
|
+
|
|
317
|
+
```yaml
|
|
318
|
+
ids:
|
|
319
|
+
collections: kebab-case # or a pattern, such as '{folder}-[a-z0-9-]+'
|
|
320
|
+
requests: camelCase
|
|
321
|
+
layout:
|
|
322
|
+
folders: required # every collection in a folder of collections/
|
|
323
|
+
folderNames: [accounts, payments, smoke]
|
|
324
|
+
maxSteps: 30
|
|
325
|
+
steps:
|
|
326
|
+
names: required
|
|
327
|
+
url: ^\{\{baseUrl\}\} # no host written out
|
|
328
|
+
docs:
|
|
329
|
+
collections: required
|
|
330
|
+
tags:
|
|
331
|
+
allowed: [smoke, regression, slow]
|
|
332
|
+
tests:
|
|
333
|
+
only: [gta] # tests call the gta.* functions, and nothing else
|
|
334
|
+
statusCode: required
|
|
335
|
+
guide: |
|
|
336
|
+
Name each step for what it proves. Logins go through `requests/login`.
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
**Rules never change a run.** A file that breaks one opens, sends and runs as before.
|
|
340
|
+
`gta lint` reports it, and a CI job running `gta lint` fails on it. Gravity marks it, and
|
|
341
|
+
flags what `tests.only` does not allow as a script is typed.
|
|
342
|
+
|
|
343
|
+
| Rule | Value | What it checks |
|
|
344
|
+
| -------------------- | ---------------------- | ------------------------------------------------------------------------- |
|
|
345
|
+
| `ids.collections` | style or pattern | Each collection's id, its file name (§2). |
|
|
346
|
+
| `ids.requests` | style or pattern | Each request set's id (§2.5). |
|
|
347
|
+
| `ids.bases` | style or pattern | Each base collection's id (§2.7). |
|
|
348
|
+
| `ids.endpoints` | style or pattern | Each endpoints file's id (§2.6). |
|
|
349
|
+
| `layout.folders` | `required` | Every collection sits in a folder of `collections/`, none at its top. |
|
|
350
|
+
| `layout.folderNames` | list, style or pattern | Each folder of `collections/`: one of the list, or following the format. |
|
|
351
|
+
| `layout.maxSteps` | integer ≥ 1 | The most steps a collection has in `steps`; `setup` and `teardown` aside. |
|
|
352
|
+
| `steps.names` | `required` | Every step has a `name`, and no other step of its file has the same one. |
|
|
353
|
+
| `steps.url` | pattern | Every request step's URL, as written, matches it (below). |
|
|
354
|
+
| `docs.collections` | `required` | Every collection has `docs`. |
|
|
355
|
+
| `docs.requests` | `required` | Every request set has `docs`. |
|
|
356
|
+
| `docs.steps` | `required` | Every step of a collection or request set has `docs`, in every list. |
|
|
357
|
+
| `tags.allowed` | list of tags | Every tag on a collection or a step is one of these. |
|
|
358
|
+
| `tags.collections` | `required` | Every collection has `tags` of its own. |
|
|
359
|
+
| `tests.only` | list (below) | What a `tests` script may call. |
|
|
360
|
+
| `tests.everyStep` | `required` | Every step that sends a request or reads a connection is checked (below). |
|
|
361
|
+
| `tests.statusCode` | `required` | Every request step's checks include its status code (below). |
|
|
362
|
+
|
|
363
|
+
- Any other group or rule is an error, as is a value of the wrong type.
|
|
364
|
+
- A rule that takes `required` also takes `optional`, which turns it off.
|
|
365
|
+
- A **style** is one of `kebab-case` (`create-user`), `snake_case` (`create_user`),
|
|
366
|
+
`camelCase` (`createUser`) and `PascalCase` (`CreateUser`). Lower case, in the first two,
|
|
367
|
+
includes digits: `404-handling` is kebab-case.
|
|
368
|
+
- A **pattern** is a JavaScript regular expression that the whole name must match, as if
|
|
369
|
+
it began with `^` and ended with `$`. `{folder}` in it stands for the folder the file is
|
|
370
|
+
in, so `{folder}-[a-z0-9-]+` asks for `collections/payments/payments-refunds.yml`. A
|
|
371
|
+
value that is not a style and has none of a pattern's characters (`^`, `$`, `.`, `*`,
|
|
372
|
+
`+`, `?`, `(`, `)`, `[`, `]`, `{`, `}`, `|` or a backslash) is an error, so a
|
|
373
|
+
misspelled style is not taken for a pattern.
|
|
374
|
+
- Rules check the project's own files, in `collections/`, `requests/`, `bases/` and
|
|
375
|
+
`endpoints/`. A global project's files follow the global project's `rules.yml`, checked
|
|
376
|
+
when `gta lint` runs in its folder.
|
|
377
|
+
- `steps.*`, `docs.steps`, `tests.everyStep` and `tests.statusCode` are about the steps of
|
|
378
|
+
collections and request sets. An endpoint is a method and a path pattern, not a step
|
|
379
|
+
that runs, so they leave endpoints files alone.
|
|
380
|
+
|
|
381
|
+
**`steps.url`** is a JavaScript regular expression that each request step's URL, as
|
|
382
|
+
written with its `{{variables}}`, must match. Unlike an id pattern it is not anchored:
|
|
383
|
+
`^\{\{baseUrl\}\}` asks that every URL start with `{{baseUrl}}`, so no host is written
|
|
384
|
+
out in a step. A use step and a step reading a connection have no URL of their own.
|
|
385
|
+
|
|
386
|
+
**`tests.everyStep` and `tests.statusCode`** count every script that runs after a step's
|
|
387
|
+
response, wherever it is written: the step's own `tests`, its file's, its base
|
|
388
|
+
collection's (§2.7) and its endpoint's (§2.6), unless the step has `base: false`. With
|
|
389
|
+
`tests.statusCode`, one of them calls `gta.expectResponseStatusCodeToBe`, in any branch,
|
|
390
|
+
or calls a check function (§5) whose own code does. A use step is left to its request
|
|
391
|
+
set, whose steps are checked there. A step reading a connection needs tests under
|
|
392
|
+
`tests.everyStep`, but has no status code of its own to check.
|
|
393
|
+
|
|
394
|
+
**`tests.only`** lists what a `tests` script may call. It must list `gta`:
|
|
395
|
+
|
|
396
|
+
| Entry | Allows |
|
|
397
|
+
| ---------- | --------------------------------------------------------- |
|
|
398
|
+
| `gta` | The `gta.*` functions (§5), except `gta.test`. |
|
|
399
|
+
| `gta.test` | Named checks of your own, with any code inside them (§5). |
|
|
400
|
+
| `checks` | The project's check files, `checks.<file>.<function>()`. |
|
|
401
|
+
| `console` | `console.log()` and the rest. |
|
|
402
|
+
|
|
403
|
+
With `only: [gta]`, a `tests` script holds calls to `gta.*` functions and nothing else:
|
|
404
|
+
|
|
405
|
+
- **Every argument is a value**: a string, number, `true`, `false`, `null` or regular
|
|
406
|
+
expression, a template string, a list or object of values, or `new RegExp(…)` of
|
|
407
|
+
values. It may read `res`, `req`, `params`, `endpoint` and `item` (§5), call another
|
|
408
|
+
`gta.*` function such as `gta.get('id')`, or `JSON.parse` a value, since `gta.set` keeps
|
|
409
|
+
a list or object as JSON text.
|
|
410
|
+
- **An `if` may choose by feature flag** (§2.9): its condition is `gta.flag()` calls
|
|
411
|
+
compared with values, joined by `!`, `&&` and `||`, and what it holds follows the same
|
|
412
|
+
rules.
|
|
413
|
+
- **Nothing else**: no variables, loops, functions of your own, `assert`, `checks.*` or
|
|
414
|
+
`gta.test`. This is checked on the script's syntax, so a call cannot be hidden by
|
|
415
|
+
renaming it (`const c = checks`).
|
|
416
|
+
|
|
417
|
+
```yaml
|
|
418
|
+
tests: |
|
|
419
|
+
gta.expectResponseStatusCodeToBe(200)
|
|
420
|
+
gta.expectResponseBodyToHaveProperty('owner', gta.get('userId'))
|
|
421
|
+
gta.set('nextPage', res.body.links.next)
|
|
422
|
+
if (gta.flag('newCheckout') === true) {
|
|
423
|
+
gta.expectResponseBodyToHaveProperty('version', 2)
|
|
424
|
+
}
|
|
425
|
+
```
|
|
426
|
+
|
|
427
|
+
`tests.only` checks every `tests` script: a collection's own and each of its steps', and
|
|
428
|
+
those of request sets, base collections and endpoints files. `before.script` is not
|
|
429
|
+
checked.
|
|
430
|
+
|
|
431
|
+
**`guide`** is Markdown, for what no rule can check: how steps are named, which request
|
|
432
|
+
set a login goes through, what a collection is for. `gta rules` prints it after the
|
|
433
|
+
rules, and Gravity shows it in Project settings. Each file's guide is read, the global
|
|
434
|
+
project's first, so a project adds to a shared guide rather than replacing it.
|
|
435
|
+
|
|
436
|
+
**From the global project.** A global project (§1.1) can have a `rules.yml` too, and it
|
|
437
|
+
lies under the project's, rule by rule. There is no switch to turn this off: a project
|
|
438
|
+
changes a shared rule by setting it in its own file, and turns it off with `null`, or
|
|
439
|
+
`optional` for a `required` rule. A group set to `null`, such as `tags: null`, turns off
|
|
440
|
+
every rule in it. Neither file is required.
|
|
441
|
+
|
|
442
|
+
**Checking.** From a project folder, or a global project's:
|
|
443
|
+
|
|
444
|
+
| Command | Does |
|
|
445
|
+
| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
446
|
+
| `gta lint` | Checks every file against the rules, and prints each finding at its file and line, with its rule and the file the rule came from. Exits `1` on a finding. |
|
|
447
|
+
| `gta rules` | Lists each rule set, its value, the file it came from and what it means, rules turned off included, then the guide. |
|
|
448
|
+
|
|
449
|
+
- Neither needs `settings.yml` or runs anything, and both take `--json`.
|
|
450
|
+
- A file that will not parse is a finding, since no rule could be checked in it.
|
|
451
|
+
- Both exit `2` when a rule is not valid, or `project.yml` names a global project that
|
|
452
|
+
cannot be used: its rules would quietly go unchecked.
|
|
453
|
+
- A coding agent can run `gta rules` to learn a project's conventions before writing, and
|
|
454
|
+
`gta lint --json` after, to fix what it finds. Gravity offers to add a section saying so
|
|
455
|
+
to the project's `AGENTS.md`, which coding agents read; nothing is written until asked.
|
|
456
|
+
|
|
298
457
|
---
|
|
299
458
|
|
|
300
459
|
## 2. Collection file
|
|
@@ -367,7 +526,7 @@ exactly. `collections/checkout/sessions.yml` starts `id: sessions`.
|
|
|
367
526
|
- **It must match the file name.** A file whose `id` is missing or different is a
|
|
368
527
|
broken collection. Renaming a file means changing its `id` too.
|
|
369
528
|
- **It must be unique in its home, ignoring case.** No two files in a project's
|
|
370
|
-
`collections/` may share an id, including files in different
|
|
529
|
+
`collections/` may share an id, including files in different folders of it; the
|
|
371
530
|
same holds for `requests/`, `bases/` and `endpoints/`. Case is ignored because macOS
|
|
372
531
|
and Windows file systems ignore it. Every file sharing an id is broken.
|
|
373
532
|
- **It is letters, digits and `- _ .`, starting with a letter or digit**:
|
|
@@ -647,13 +806,13 @@ else selects it. With `stepTags: true`, its steps whose tags match are dropped f
|
|
|
647
806
|
was selected.
|
|
648
807
|
|
|
649
808
|
**`exclude: true`** leaves a collection out of group runs: `gta all`, with or without
|
|
650
|
-
tags, and a
|
|
809
|
+
tags, and a folder named to `gta`. Named on its own, it still runs. Use it for work in
|
|
651
810
|
progress, a manual-only collection, or one waiting on a fix. `gta` lists what it left
|
|
652
811
|
out, so a suite never shrinks without saying so.
|
|
653
812
|
|
|
654
813
|
### 2.5 Request sets and `use:`
|
|
655
814
|
|
|
656
|
-
A **request set** is a collection in `requests/`, directly or one
|
|
815
|
+
A **request set** is a collection in `requests/`, directly or one folder down, with a
|
|
657
816
|
`params:` key: the inputs it takes. A step elsewhere runs it with **`use:`**, and passes
|
|
658
817
|
values with **`with:`**.
|
|
659
818
|
|
|
@@ -707,7 +866,7 @@ method key, `headers`, `body`, `settings` or `before` on it is an error, and `wi
|
|
|
707
866
|
without `use` is an error too.
|
|
708
867
|
|
|
709
868
|
- **Finding the set.** `use: login` is `requests/login.yml` in the project, else in its
|
|
710
|
-
global project. `use: auth/login` is one
|
|
869
|
+
global project. `use: auth/login` is one folder down. `use: global:login` looks
|
|
711
870
|
only in the global project.
|
|
712
871
|
- **`with:`** gives plain values; a param left out takes its default. A string may hold
|
|
713
872
|
`{{variables}}`, resolved as the set's first request starts, just after the
|
|
@@ -872,7 +1031,8 @@ userId,expectedStatus,iterationLabel
|
|
|
872
1031
|
string**: a zip code `01234` stays `01234`. A short row leaves its last columns empty;
|
|
873
1032
|
a row with more values than the header is an error.
|
|
874
1033
|
- **JSON** is an array of objects, one per row. Values keep their type, which must be
|
|
875
|
-
string, number, boolean or null.
|
|
1034
|
+
string, number, boolean or null. A row must not name a column twice: JSON readers keep
|
|
1035
|
+
only the last value, so it is an error rather than a value quietly lost.
|
|
876
1036
|
- **`.csv` wins** when both exist. The name must match exactly, case included.
|
|
877
1037
|
- A data file must be at most 10 MB and have at least one row. The header must not have
|
|
878
1038
|
an empty or repeated column name. A data file that will not read makes the collection
|
|
@@ -1160,6 +1320,10 @@ saying so, and the checks after it still run.
|
|
|
1160
1320
|
every item of a simple `list`, in any order. A `RegExp` in the list is a pattern some item
|
|
1161
1321
|
must match as text, and so is one held by an object in the list:
|
|
1162
1322
|
`[/^admin/, { name: /^Grace/ }]`. A pattern never matches an object or array item.
|
|
1323
|
+
An object in the list matches an item holding each of its properties, and the item may
|
|
1324
|
+
have others; a property that is itself an object is matched the same way, so
|
|
1325
|
+
`{ data: { status: 'reversed' } }` finds an item whose `data` has that `status` among
|
|
1326
|
+
other properties. An array compares whole.
|
|
1163
1327
|
|
|
1164
1328
|
A list of `{ pathToProperty, expectedValue, specialHandling? }` objects describes **one**
|
|
1165
1329
|
item, property by property; call it once per item. A property may appear twice, once to
|
|
@@ -1403,7 +1567,10 @@ delete req.headers['X-Debug']
|
|
|
1403
1567
|
|
|
1404
1568
|
- `req.body` is text, and can be changed for a `json`, `xml`, `text` or `graphql` body.
|
|
1405
1569
|
A form, multipart or file body is built from its parts, so it cannot.
|
|
1406
|
-
- `req.headers` is a map of names to values: add, change or delete them.
|
|
1570
|
+
- `req.headers` is a map of names to values: add, change or delete them. A header sent
|
|
1571
|
+
more than once reads as its values joined with `, `, as `res.headers` does, and is
|
|
1572
|
+
still sent once per value unless the script changes it. Set an array to send one
|
|
1573
|
+
header per value.
|
|
1407
1574
|
- Variables in what the script writes resolve afterwards, as in the file.
|
|
1408
1575
|
- Anything else is a pre-request error, and nothing is sent.
|
|
1409
1576
|
|
|
@@ -1465,7 +1632,8 @@ global project's root.
|
|
|
1465
1632
|
- A secret with no value anywhere fails the run, naming it. An empty credential is never
|
|
1466
1633
|
sent.
|
|
1467
1634
|
- Reports replace a secret's value with `[secret: NAME]`, wherever it appears. So does
|
|
1468
|
-
the desktop app where it shows a request as sent
|
|
1635
|
+
the desktop app where it shows a request as sent, or what a script wrote with
|
|
1636
|
+
`console`.
|
|
1469
1637
|
- `.env` holds `NAME=value` lines. `#` starts a comment line, `export ` before a name is
|
|
1470
1638
|
allowed, and a value may be quoted. It must not be committed.
|
|
1471
1639
|
|
|
@@ -1510,7 +1678,7 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1510
1678
|
- It must be YAML that parses, holding a mapping, in a file ending in `.yml`.
|
|
1511
1679
|
- Unknown keys are errors in: a collection's top level, a step, `settings`, `before`,
|
|
1512
1680
|
`body`, `project.yml`, `tls`, an environment file, an environment's `flags`, a param
|
|
1513
|
-
spec and `
|
|
1681
|
+
spec, `settings.yml` and `rules.yml`.
|
|
1514
1682
|
|
|
1515
1683
|
**Collections**
|
|
1516
1684
|
|
|
@@ -1567,7 +1735,7 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1567
1735
|
- An endpoint (`endpoints/`) has a URL that is a path starting with `/`, and no use
|
|
1568
1736
|
steps or connections. An endpoints file has no `setup` or `teardown`.
|
|
1569
1737
|
- A check file's name is a JavaScript identifier; if it isn't, the file is not loaded.
|
|
1570
|
-
- `collections/`, `requests/`, `endpoints/` and `bases/` hold files at most one
|
|
1738
|
+
- `collections/`, `requests/`, `endpoints/` and `bases/` hold files at most one folder
|
|
1571
1739
|
down.
|
|
1572
1740
|
|
|
1573
1741
|
**Portability (§1.2)**
|
|
@@ -1578,7 +1746,7 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1578
1746
|
`Collections/`.
|
|
1579
1747
|
- No two names in `collections/`, `requests/`, `endpoints/`, `bases/`, `environments/`
|
|
1580
1748
|
or `checks/` differ only in case.
|
|
1581
|
-
- No file or
|
|
1749
|
+
- No file or folder name is one Windows refuses.
|
|
1582
1750
|
|
|
1583
1751
|
**Projects**
|
|
1584
1752
|
|
|
@@ -1587,6 +1755,9 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
|
|
|
1587
1755
|
- Each `tls.ca` file exists and holds a PEM or DER certificate.
|
|
1588
1756
|
- `settings.yml` exists for `gta`; a global project's is optional. The `environmentType`
|
|
1589
1757
|
a run ends up with, if any, names an environment that exists.
|
|
1758
|
+
- In `rules.yml`, each rule has a value of the type §1.4 gives it, a pattern compiles,
|
|
1759
|
+
`tests.only` lists `gta`, and `guide` is a string. What breaks a rule is a finding of `gta lint`, never an
|
|
1760
|
+
error: it stops nothing (§1.4).
|
|
1590
1761
|
|
|
1591
1762
|
**At run time**
|
|
1592
1763
|
|
|
@@ -2,6 +2,32 @@ gta bundles the following packages. Each is used under the license that follows
|
|
|
2
2
|
|
|
3
3
|
--------------------------------------------------------------------------------
|
|
4
4
|
|
|
5
|
+
acorn 8.18.0 (MIT)
|
|
6
|
+
|
|
7
|
+
MIT License
|
|
8
|
+
|
|
9
|
+
Copyright (C) 2012-2022 by various contributors (see AUTHORS)
|
|
10
|
+
|
|
11
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
12
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
13
|
+
in the Software without restriction, including without limitation the rights
|
|
14
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
15
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
16
|
+
furnished to do so, subject to the following conditions:
|
|
17
|
+
|
|
18
|
+
The above copyright notice and this permission notice shall be included in
|
|
19
|
+
all copies or substantial portions of the Software.
|
|
20
|
+
|
|
21
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
22
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
23
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
24
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
25
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
26
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
27
|
+
THE SOFTWARE.
|
|
28
|
+
|
|
29
|
+
--------------------------------------------------------------------------------
|
|
30
|
+
|
|
5
31
|
undici 8.10.2 (MIT)
|
|
6
32
|
|
|
7
33
|
MIT License
|