@schwabyio/gta 0.12.0 → 0.14.0

package/dist/SPEC.md

@@ -1,6 +1,7 @@
 # The Gravity file format
 
-Version 0.9
+Version 0.14.0, the version of Gravity and `gta` that reads it: the two are released
+together.
 
 This document specifies the YAML files that **Gravity**, the desktop app, and **`gta`**,
 the command-line runner, read and write. Together they make up Gravity Test
@@ -26,18 +27,19 @@ is a complete, valid project to start from.
 
 ## At a glance
 
-| File                          | Where                                      | What it is                                  | §    |
-| ----------------------------- | ------------------------------------------ | ------------------------------------------- | ---- |
-| `collections/<id>.yml`        | `collections/`, or one directory inside it | A collection: requests run in order         | §2   |
-| `collections/<id>.csv\|.json` | Beside its collection                      | A data file: the collection runs once a row | §2.8 |
-| `environments/<name>.yml`     | `environments/`                            | Variables, secrets and flags for one target | §6   |
-| `project.yml`                 | The project folder                         | Name, global project, variables, trust      | §1.1 |
-| `settings.yml`                | The project folder                         | How `gta` runs the project                  | §1.3 |
-| `requests/<id>.yml`           | `requests/`, or one directory inside it    | A request set, run by `use:`                | §2.5 |
-| `endpoints/<id>.yml`          | `endpoints/`, or one directory inside it   | Defaults and checks per method and path     | §2.6 |
-| `bases/<id>.yml`              | `bases/`, or one directory inside it       | A base collection, for `extends:`           | §2.7 |
-| `checks/<name>.js`            | `checks/`                                  | Shared check functions                      | §5   |
-| `.env`                        | The project folder, never committed        | Values for secrets                          | §6   |
+| File                          | Where                                   | What it is                                  | §    |
+| ----------------------------- | --------------------------------------- | ------------------------------------------- | ---- |
+| `collections/<id>.yml`        | `collections/`, or one folder inside it | A collection: requests run in order         | §2   |
+| `collections/<id>.csv\|.json` | Beside its collection                   | A data file: the collection runs once a row | §2.8 |
+| `environments/<name>.yml`     | `environments/`                         | Variables, secrets and flags for one target | §6   |
+| `project.yml`                 | The project folder                      | Name, global project, variables, trust      | §1.1 |
+| `settings.yml`                | The project folder                      | How `gta` runs the project                  | §1.3 |
+| `rules.yml`                   | The project folder                      | How the project's files are written         | §1.4 |
+| `requests/<id>.yml`           | `requests/`, or one folder inside it    | A request set, run by `use:`                | §2.5 |
+| `endpoints/<id>.yml`          | `endpoints/`, or one folder inside it   | Defaults and checks per method and path     | §2.6 |
+| `bases/<id>.yml`              | `bases/`, or one folder inside it       | A base collection, for `extends:`           | §2.7 |
+| `checks/<name>.js`            | `checks/`                               | Shared check functions                      | §5   |
+| `.env`                        | The project folder, never committed     | Values for secrets                          | §6   |
 
 The smallest project `gta` runs is three files:
 
@@ -89,9 +91,10 @@ A **project** is a folder holding `collections/` and, usually, `environments/`:
 payments/                     a project
 ├── project.yml               optional (§1.1)
 ├── settings.yml              how gta runs it (§1.3)
+├── rules.yml                 how its files are written (§1.4)
 ├── collections/
 │   ├── smoke.yml             a collection
-│   └── checkout/             a directory grouping collections: one level only
+│   └── checkout/             a folder grouping collections: one level only
 │       ├── sessions.yml
 │       ├── sessions.csv      its data file (§2.8)
 │       └── refunds.yml
@@ -106,17 +109,17 @@ payments/                     a project
 └── .env                      secret values; not committed (§6)
 ```
 
-**Every `.yml` file directly in `collections/`, or in a directory one level down, is a
+**Every `.yml` file directly in `collections/`, or in a folder one level down, is a
 collection.** Nothing else is. Discovery is exact: no other `.yml` in a repository is
 mistaken for a collection. A file that does not parse is reported as a broken
 collection, never skipped in silence.
 
-- A directory inside a directory of `collections/` is reported as a problem and not
+- A folder inside a folder of `collections/` is reported as a problem and not
   read. `requests/`, `endpoints/` and `bases/` are read to the same depth, and
   `checks/` only at its top level.
-- Names starting with `.` are ignored, as are the directories `node_modules`, `.git`,
+- Names starting with `.` are ignored, as are the folders `node_modules`, `.git`,
   `reports`, `test-results`, `out` and `dist`.
-- Directories carry no configuration and need no file of their own. They group
+- Folders carry no configuration and need no file of their own. They group
   collections for display and for running a group.
 
 A project is any folder: the root of a repository, or one service of a monorepo. A
@@ -160,8 +163,8 @@ The file is optional, and so is every key in it. Any other key is an error.
 | `tls.ca` | list of relative paths | none        | Certificate files that requests trust (below).        |
 
 **`uses`** names a **global project**: an ordinary project whose `project.yml`
-variables, `environments/`, `requests/`, `endpoints/`, `bases/`, `checks/` and
-`settings.yml` every project using it shares.
+variables, `environments/`, `requests/`, `endpoints/`, `bases/`, `checks/`,
+`settings.yml` and `rules.yml` every project using it shares.
 
 - It must be a relative path. An absolute path is refused, since it would only work on
   one machine. Write it with `/`; `\` reads the same.
@@ -170,7 +173,15 @@ variables, `environments/`, `requests/`, `endpoints/`, `bases/`, `checks/` and
 - An environment in the global project merges under the project's environment of the
   same name, key by key, and the project's values win. An environment only the global
   project has is available too.
-- The global project's `settings.yml` lies under the project's the same way (§1.3).
+- The global project's `settings.yml` lies under the project's the same way (§1.3), and
+  so does its `rules.yml` (§1.4).
+- A global project's own `collections/` is **not** shared: a project using it never
+  sees or runs those collections. A global project needs none. It may have one for a
+  single purpose: **testing what it shares**. A collection there can `use:` its request
+  sets, `extends:` its bases and call its checks, against its own environments, so a
+  broken shared piece fails in one place, before every project relying on it does.
+  Those collections run only when the global project itself does, as a project in
+  Gravity or with `gta` in its folder, such as its own CI job.
 
 **`tls.ca`** lists certificate files that requests trust, for a server whose certificate
 a company or local CA signed, or a server's own self-signed certificate. A request
@@ -203,11 +214,11 @@ Projects are shared between macOS, Windows and Linux, and read the same on all t
   `use:` or `extends:` name, `uses`, a `tls.ca` file, a file a body sends, and an
   environment's file name. One that differs only in case is an error on every platform,
   naming the spelling on disk. The folders and files of §1 are lower case.
-- **No two names in one directory may differ only in case**, such as `Checkout/` and
+- **No two names in one folder may differ only in case**, such as `Checkout/` and
   `checkout/`: Linux can hold both, but a macOS or Windows checkout only one. This
   applies in `collections/`, `requests/`, `endpoints/`, `bases/`, `environments/` and
   `checks/`. Collection ids are unique ignoring case too (§2).
-- File, directory and environment names must avoid what Windows refuses: the characters
+- File, folder and environment names must avoid what Windows refuses: the characters
   `< > : " / \ | ? *`, control characters, a trailing dot or space, and the names `CON`,
   `PRN`, `AUX`, `NUL`, `CONIN$`, `CONOUT$`, `COM1`–`COM9` and `LPT1`–`LPT9`, with or
   without an extension. A file with such a name, made on macOS or Linux, is reported.
@@ -282,7 +293,7 @@ global project's `settings.yml`, the project's, the environment variable, the fl
 | --------------------- | ----------------------------------------------------------- |
 | `gta get`             | Nothing. It lists what `gta all` would run.                 |
 | `gta all`             | Every collection, except those with `exclude: true` (§2.4). |
-| `gta smoke,checkout/` | The collections and directories named, in that order.       |
+| `gta smoke,checkout/` | The collections and folders named, in that order.           |
 
 `gta get` also reports each `use:` and `extends:` that would stop a run, and each file a
 body names that is in neither the project nor its global project (§2.2), in every
@@ -290,11 +301,159 @@ collection, including those `gta all` leaves out (Appendix A). A file path with
 `{{variables}}` is left to the run. It exits `1` when it finds one, or a broken
 collection, and `0` otherwise.
 
-A collection is named by its `id`, or by its place (`checkout/sessions`). A directory is
-named by its name, and `checkout/` names only the directory. `--flag name=value` sets a
+A collection is named by its `id`, or by its place (`checkout/sessions`). A folder is
+named by its name, and `checkout/` names only the folder. `--flag name=value` sets a
 feature flag (§2.9), and `--json` prints the results as JSON. The exit code is `0` when
 everything passed, `1` when something failed, and `2` when `gta` could not run.
 
+### 1.4 `rules.yml`
+
+How a project's files are named, laid out and written, so that everyone working on it,
+people and coding agents alike, keeps it consistent. It sits beside `project.yml`, is
+committed, and is optional. `gta lint` checks the project against it. Gravity, the
+desktop app, marks each file, folder and step that breaks a rule, and will not create or
+rename a file or folder that would.
+
+```yaml
+ids:
+  collections: kebab-case # or a pattern, such as '{folder}-[a-z0-9-]+'
+  requests: camelCase
+layout:
+  folders: required # every collection in a folder of collections/
+  folderNames: [accounts, payments, smoke]
+  maxSteps: 30
+steps:
+  names: required
+  url: ^\{\{baseUrl\}\} # no host written out
+docs:
+  collections: required
+tags:
+  allowed: [smoke, regression, slow]
+tests:
+  only: [gta] # tests call the gta.* functions, and nothing else
+  statusCode: required
+guide: |
+  Name each step for what it proves. Logins go through `requests/login`.
+```
+
+**Rules never change a run.** A file that breaks one opens, sends and runs as before.
+`gta lint` reports it, and a CI job running `gta lint` fails on it. Gravity marks it, and
+flags what `tests.only` does not allow as a script is typed.
+
+| Rule                 | Value                  | What it checks                                                            |
+| -------------------- | ---------------------- | ------------------------------------------------------------------------- |
+| `ids.collections`    | style or pattern       | Each collection's id, its file name (§2).                                 |
+| `ids.requests`       | style or pattern       | Each request set's id (§2.5).                                             |
+| `ids.bases`          | style or pattern       | Each base collection's id (§2.7).                                         |
+| `ids.endpoints`      | style or pattern       | Each endpoints file's id (§2.6).                                          |
+| `layout.folders`     | `required`             | Every collection sits in a folder of `collections/`, none at its top.     |
+| `layout.folderNames` | list, style or pattern | Each folder of `collections/`: one of the list, or following the format.  |
+| `layout.maxSteps`    | integer ≥ 1            | The most steps a collection has in `steps`; `setup` and `teardown` aside. |
+| `steps.names`        | `required`             | Every step has a `name`, and no other step of its file has the same one.  |
+| `steps.url`          | pattern                | Every request step's URL, as written, matches it (below).                 |
+| `docs.collections`   | `required`             | Every collection has `docs`.                                              |
+| `docs.requests`      | `required`             | Every request set has `docs`.                                             |
+| `docs.steps`         | `required`             | Every step of a collection or request set has `docs`, in every list.      |
+| `tags.allowed`       | list of tags           | Every tag on a collection or a step is one of these.                      |
+| `tags.collections`   | `required`             | Every collection has `tags` of its own.                                   |
+| `tests.only`         | list (below)           | What a `tests` script may call.                                           |
+| `tests.everyStep`    | `required`             | Every step that sends a request or reads a connection is checked (below). |
+| `tests.statusCode`   | `required`             | Every request step's checks include its status code (below).              |
+
+- Any other group or rule is an error, as is a value of the wrong type.
+- A rule that takes `required` also takes `optional`, which turns it off.
+- A **style** is one of `kebab-case` (`create-user`), `snake_case` (`create_user`),
+  `camelCase` (`createUser`) and `PascalCase` (`CreateUser`). Lower case, in the first two,
+  includes digits: `404-handling` is kebab-case.
+- A **pattern** is a JavaScript regular expression that the whole name must match, as if
+  it began with `^` and ended with `$`. `{folder}` in it stands for the folder the file is
+  in, so `{folder}-[a-z0-9-]+` asks for `collections/payments/payments-refunds.yml`. A
+  value that is not a style and has none of a pattern's characters (`^`, `$`, `.`, `*`,
+  `+`, `?`, `(`, `)`, `[`, `]`, `{`, `}`, `|` or a backslash) is an error, so a
+  misspelled style is not taken for a pattern.
+- Rules check the project's own files, in `collections/`, `requests/`, `bases/` and
+  `endpoints/`. A global project's files follow the global project's `rules.yml`, checked
+  when `gta lint` runs in its folder.
+- `steps.*`, `docs.steps`, `tests.everyStep` and `tests.statusCode` are about the steps of
+  collections and request sets. An endpoint is a method and a path pattern, not a step
+  that runs, so they leave endpoints files alone.
+
+**`steps.url`** is a JavaScript regular expression that each request step's URL, as
+written with its `{{variables}}`, must match. Unlike an id pattern it is not anchored:
+`^\{\{baseUrl\}\}` asks that every URL start with `{{baseUrl}}`, so no host is written
+out in a step. A use step and a step reading a connection have no URL of their own.
+
+**`tests.everyStep` and `tests.statusCode`** count every script that runs after a step's
+response, wherever it is written: the step's own `tests`, its file's, its base
+collection's (§2.7) and its endpoint's (§2.6), unless the step has `base: false`. With
+`tests.statusCode`, one of them calls `gta.expectResponseStatusCodeToBe`, in any branch,
+or calls a check function (§5) whose own code does. A use step is left to its request
+set, whose steps are checked there. A step reading a connection needs tests under
+`tests.everyStep`, but has no status code of its own to check.
+
+**`tests.only`** lists what a `tests` script may call. It must list `gta`:
+
+| Entry      | Allows                                                    |
+| ---------- | --------------------------------------------------------- |
+| `gta`      | The `gta.*` functions (§5), except `gta.test`.            |
+| `gta.test` | Named checks of your own, with any code inside them (§5). |
+| `checks`   | The project's check files, `checks.<file>.<function>()`.  |
+| `console`  | `console.log()` and the rest.                             |
+
+With `only: [gta]`, a `tests` script holds calls to `gta.*` functions and nothing else:
+
+- **Every argument is a value**: a string, number, `true`, `false`, `null` or regular
+  expression, a template string, a list or object of values, or `new RegExp(…)` of
+  values. It may read `res`, `req`, `params`, `endpoint` and `item` (§5), call another
+  `gta.*` function such as `gta.get('id')`, or `JSON.parse` a value, since `gta.set` keeps
+  a list or object as JSON text.
+- **An `if` may choose by feature flag** (§2.9): its condition is `gta.flag()` calls
+  compared with values, joined by `!`, `&&` and `||`, and what it holds follows the same
+  rules.
+- **Nothing else**: no variables, loops, functions of your own, `assert`, `checks.*` or
+  `gta.test`. This is checked on the script's syntax, so a call cannot be hidden by
+  renaming it (`const c = checks`).
+
+```yaml
+tests: |
+  gta.expectResponseStatusCodeToBe(200)
+  gta.expectResponseBodyToHaveProperty('owner', gta.get('userId'))
+  gta.set('nextPage', res.body.links.next)
+  if (gta.flag('newCheckout') === true) {
+    gta.expectResponseBodyToHaveProperty('version', 2)
+  }
+```
+
+`tests.only` checks every `tests` script: a collection's own and each of its steps', and
+those of request sets, base collections and endpoints files. `before.script` is not
+checked.
+
+**`guide`** is Markdown, for what no rule can check: how steps are named, which request
+set a login goes through, what a collection is for. `gta rules` prints it after the
+rules, and Gravity shows it in Project settings. Each file's guide is read, the global
+project's first, so a project adds to a shared guide rather than replacing it.
+
+**From the global project.** A global project (§1.1) can have a `rules.yml` too, and it
+lies under the project's, rule by rule. There is no switch to turn this off: a project
+changes a shared rule by setting it in its own file, and turns it off with `null`, or
+`optional` for a `required` rule. A group set to `null`, such as `tags: null`, turns off
+every rule in it. Neither file is required.
+
+**Checking.** From a project folder, or a global project's:
+
+| Command     | Does                                                                                                                                                      |
+| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `gta lint`  | Checks every file against the rules, and prints each finding at its file and line, with its rule and the file the rule came from. Exits `1` on a finding. |
+| `gta rules` | Lists each rule set, its value, the file it came from and what it means, rules turned off included, then the guide.                                       |
+
+- Neither needs `settings.yml` or runs anything, and both take `--json`.
+- A file that will not parse is a finding, since no rule could be checked in it.
+- Both exit `2` when a rule is not valid, or `project.yml` names a global project that
+  cannot be used: its rules would quietly go unchecked.
+- A coding agent can run `gta rules` to learn a project's conventions before writing, and
+  `gta lint --json` after, to fix what it finds. Gravity offers to add a section saying so
+  to the project's `AGENTS.md`, which coding agents read; nothing is written until asked.
+
 ---
 
 ## 2. Collection file
@@ -367,7 +526,7 @@ exactly. `collections/checkout/sessions.yml` starts `id: sessions`.
 - **It must match the file name.** A file whose `id` is missing or different is a
   broken collection. Renaming a file means changing its `id` too.
 - **It must be unique in its home, ignoring case.** No two files in a project's
-  `collections/` may share an id, including files in different directories of it; the
+  `collections/` may share an id, including files in different folders of it; the
   same holds for `requests/`, `bases/` and `endpoints/`. Case is ignored because macOS
   and Windows file systems ignore it. Every file sharing an id is broken.
 - **It is letters, digits and `- _ .`, starting with a letter or digit**:
@@ -388,27 +547,31 @@ key**, in capitals, whose value is the URL as a string.
 
 Methods: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, `OPTIONS`.
 
-| Key        | Type         | Required | Meaning                                                                   |
-| ---------- | ------------ | -------- | ------------------------------------------------------------------------- |
-| `<METHOD>` | string       | **yes**  | The URL, query string included.                                           |
-| `name`     | string       | no       | Display name. Defaults to the method and URL.                             |
-| `headers`  | map          | no       | Over the collection's headers (§2.3).                                     |
-| `body`     | map          | no       | Exactly one kind of body (§2.2).                                          |
-| `settings` | map          | no       | Over the collection's settings (§2.3).                                    |
-| `before`   | map          | no       | `script:` run before the request (§5).                                    |
-| `tests`    | string       | no       | The checks, as JavaScript: calls on `gta` (§3) and any other code (§5).   |
-| `tags`     | list of tags | no       | The step's own tags. Only with `stepTags: true` on the collection (§2.4). |
-| `flags`    | map          | no       | Feature flags the step needs (§2.9).                                      |
-| `forEach`  | string       | no       | Send the request once for each item of a list (below).                    |
-| `useTests` | `true`       | no       | In a request set: the use step's `tests` check this response (§2.5).      |
-| `base`     | `false`      | no       | `false` leaves the step's endpoint base out (§2.6).                       |
-| `docs`     | string       | no       | Markdown.                                                                 |
-
-- A step with no method key, or with two, is an error.
+| Key          | Type         | Required | Meaning                                                                   |
+| ------------ | ------------ | -------- | ------------------------------------------------------------------------- |
+| `<METHOD>`   | string       | **yes**  | The URL, query string included.                                           |
+| `name`       | string       | no       | Display name. Defaults to the method and URL.                             |
+| `headers`    | map          | no       | Over the collection's headers (§2.3).                                     |
+| `body`       | map          | no       | Exactly one kind of body (§2.2).                                          |
+| `settings`   | map          | no       | Over the collection's settings (§2.3).                                    |
+| `before`     | map          | no       | `script:` run before the request (§5).                                    |
+| `tests`      | string       | no       | The checks, as JavaScript: calls on `gta` (§3) and any other code (§5).   |
+| `tags`       | list of tags | no       | The step's own tags. Only with `stepTags: true` on the collection (§2.4). |
+| `flags`      | map          | no       | Feature flags the step needs (§2.9).                                      |
+| `forEach`    | string       | no       | Send the request once for each item of a list (below).                    |
+| `useTests`   | `true`       | no       | In a request set: the use step's `tests` check this response (§2.5).      |
+| `connection` | string       | no       | Keep the event stream this request opens as a connection (§2.11).         |
+| `base`       | `false`      | no       | `false` leaves the step's endpoint base out (§2.6).                       |
+| `docs`       | string       | no       | Markdown.                                                                 |
+
+- A step with two method keys is an error, and so is a step with none, unless it is a
+  use step or reads a connection.
 - **The URL is authoritative, query string included.** There is no separate block of
   query parameters. Editors show a parameter table as a view over the URL.
 - A step may instead run a request set with `use:` (§2.5). A use step has no method
   key.
+- A step may also read a connection, an event stream an earlier step keeps open, with
+  `connection:` and no method key (§2.11).
 - **Any other key is an error**, so a misspelled key such as `heders:` fails at once
   rather than being ignored. A method key in lower case (`get:`) is reported as such.
 
@@ -532,12 +695,65 @@ body:
 **`settings`** on a collection apply to every step; a step's own settings merge over
 them. Any other key is an error.
 
-| Key               | Type        | Default | Meaning                                                        |
-| ----------------- | ----------- | ------- | -------------------------------------------------------------- |
-| `timeout`         | number ≥ 0  | `0`     | Milliseconds for the whole request; `0` means no limit.        |
-| `followRedirects` | boolean     | `true`  | Follow 3xx responses.                                          |
-| `maxRedirects`    | integer ≥ 0 | `5`     | The most redirects followed.                                   |
-| `encodeUrl`       | boolean     | `true`  | Percent-encode what a hand-typed URL left raw, before sending. |
+| Key               | Type        | Default | Meaning                                                                                                |
+| ----------------- | ----------- | ------- | ------------------------------------------------------------------------------------------------------ |
+| `timeout`         | number ≥ 0  | `0`     | Milliseconds for the whole request; `0` means no limit. For an event stream, until its headers arrive. |
+| `followRedirects` | boolean     | `true`  | Follow 3xx responses.                                                                                  |
+| `maxRedirects`    | integer ≥ 0 | `5`     | The most redirects followed.                                                                           |
+| `encodeUrl`       | boolean     | `true`  | Percent-encode what a hand-typed URL left raw, before sending.                                         |
+| `maxEvents`       | integer ≥ 0 | `0`     | Stop reading an event stream after this many events; `0` means no limit.                               |
+| `streamTimeout`   | number ≥ 0  | `0`     | Stop reading an event stream this many milliseconds after its headers; `0` means no limit.             |
+| `untilEvent`      | string      | none    | Stop reading an event stream after the first event of this name, its `event:` line.                    |
+
+**Event streams.** A response whose `Content-Type` is `text/event-stream` (Server-Sent
+Events) is read as it arrives, whatever the method, and checked as a list of events
+(§3). Nothing in the step marks it as a stream. Reading stops at whichever comes first:
+
+- the server closes the stream
+- `maxEvents` events have arrived
+- `streamTimeout` milliseconds have passed since its headers arrived
+- the first event named `untilEvent` has arrived
+- 1,000 events or 10 MB, a limit no setting lifts
+
+```yaml
+- name: price stream
+  GET: '{{baseUrl}}/prices/stream?symbol=ACME'
+  headers:
+    Accept: text/event-stream
+  settings:
+    maxEvents: 3
+    streamTimeout: 5000
+  tests: |
+    gta.expectResponseStatusCodeToBe(200)
+    gta.expectResponseBodyToHaveProperty('[0].event', 'subscribed')
+    gta.expectResponseBodyToHaveProperty('[1].data.price', 100)
+```
+
+- Each way of stopping is a normal end, not an error, and the checks run on the events
+  that arrived. A check that wanted more fails with the path it missed.
+- Stopped at a number of events, the body ends with the last one.
+- Cancel, or a connection that drops mid-stream, is an error, as for any request.
+- `maxEvents`, `streamTimeout` and `untilEvent` do nothing to any other response.
+- In the desktop app, the events show as they arrive, and Stop ends the reading as
+  these do, so the checks run on what came.
+- A step that keeps its stream open for later steps is a connection (§2.11).
+
+**Resuming.** A stream is never reconnected: the server closing it ends the step. To
+test that a stream resumes, capture the last event's id and send it as `Last-Event-ID`
+in a later step:
+
+```yaml
+- name: first part
+  GET: '{{baseUrl}}/feed'
+  tests: |
+    gta.set('lastId', res.body.at(-1).id)
+- name: the rest
+  GET: '{{baseUrl}}/feed'
+  headers:
+    Last-Event-ID: '{{lastId}}'
+  tests: |
+    gta.expectResponseBodyToHaveProperty('[0].id', '4')
+```
 
 **`headers`** is a map from header name to one of:
 
@@ -590,13 +806,13 @@ else selects it. With `stepTags: true`, its steps whose tags match are dropped f
 was selected.
 
 **`exclude: true`** leaves a collection out of group runs: `gta all`, with or without
-tags, and a directory named to `gta`. Named on its own, it still runs. Use it for work in
+tags, and a folder named to `gta`. Named on its own, it still runs. Use it for work in
 progress, a manual-only collection, or one waiting on a fix. `gta` lists what it left
 out, so a suite never shrinks without saying so.
 
 ### 2.5 Request sets and `use:`
 
-A **request set** is a collection in `requests/`, directly or one directory down, with a
+A **request set** is a collection in `requests/`, directly or one folder down, with a
 `params:` key: the inputs it takes. A step elsewhere runs it with **`use:`**, and passes
 values with **`with:`**.
 
@@ -650,7 +866,7 @@ method key, `headers`, `body`, `settings` or `before` on it is an error, and `wi
 without `use` is an error too.
 
 - **Finding the set.** `use: login` is `requests/login.yml` in the project, else in its
-  global project. `use: auth/login` is one directory down. `use: global:login` looks
+  global project. `use: auth/login` is one folder down. `use: global:login` looks
   only in the global project.
 - **`with:`** gives plain values; a param left out takes its default. A string may hold
   `{{variables}}`, resolved as the set's first request starts, just after the
@@ -815,7 +1031,8 @@ userId,expectedStatus,iterationLabel
   string**: a zip code `01234` stays `01234`. A short row leaves its last columns empty;
   a row with more values than the header is an error.
 - **JSON** is an array of objects, one per row. Values keep their type, which must be
-  string, number, boolean or null.
+  string, number, boolean or null. A row must not name a column twice: JSON readers keep
+  only the last value, so it is an error rather than a value quietly lost.
 - **`.csv` wins** when both exist. The name must match exactly, case included.
 - A data file must be at most 10 MB and have at least one row. The header must not have
   an empty or repeated column name. A data file that will not read makes the collection
@@ -934,12 +1151,73 @@ steps: # once per row of approved-domains.csv, as before
 - Reports name their results `setup › log in` and `teardown › remove the grant`. Running
   a single step in the desktop app does not run them.
 
+### 2.11 Connections: a stream across steps
+
+A request whose response is an event stream (§2.3) can keep it open for later steps, as
+a **connection** named by `connection:`. A later step with `connection:` and no method
+key sends nothing: it reads the events the connection holds. The steps between can do
+what those events are about:
+
+```yaml
+steps:
+  - name: watch orders
+    GET: '{{baseUrl}}/orders/events'
+    connection: orders # keep the stream open as "orders"
+    settings:
+      untilEvent: subscribed # read until the server confirms, then go on
+  - name: place order
+    POST: '{{baseUrl}}/orders'
+    body:
+      json: '{ "sku": "ACME-1" }'
+    tests: |
+      gta.expectResponseBodyToHaveProperty('id', 'orderId', 'setAsCollectionVariable')
+  - name: order created
+    connection: orders # no method key: reads the connection
+    settings:
+      untilEvent: order.created
+      streamTimeout: 5000
+    tests: |
+      gta.expectResponseBodyToHaveProperty('[0].event', 'order.created')
+      gta.test('the order placed', () => assert.equal(res.body[0].data.id, gta.get('orderId')))
+```
+
+- **Opening.** The step sends its request and reads the stream as any step does, until
+  its `maxEvents`, `streamTimeout` or `untilEvent`. Then, rather than closing it, it
+  keeps reading in the background, holding the events that arrive for the next step
+  that reads the connection. With none of the three set, it reads no events and goes
+  straight on.
+- **Reading.** A reading step takes the events held since the last step read them,
+  then waits for more, until its `maxEvents`, `streamTimeout` or `untilEvent`, or the
+  server closes the stream. With none of the three set, it takes what is held and ends
+  without waiting, as `held`.
+- **What a reading step sees.** Its body is the events it took (§3). Its status and
+  headers are those of the response that opened the connection, `req` is that request,
+  and `res.stream.at` counts from that response's headers.
+- A reading step may have `name`, `settings`, `before`, `tests`, `tags`, `flags`,
+  `useTests` and `docs`, and nothing that would build a request: no `headers`, `body`,
+  `base` or `forEach`. A step opening a connection has no `forEach`, and a use step has
+  no `connection`.
+- **A connection lasts the run.** One opened in `setup` is read by every data row; one
+  opened in a row, by that row's later steps. Every connection closes when the run ends,
+  cancelled or not. A step opening a name already open closes the old connection first,
+  whatever its own response is.
+- A connection the server closes keeps the events it held: the next step reads them,
+  then ends with `close`. A connection holds at most 1,000 events or 10 MB; at that it
+  stops reading, and the step that reads it ends with `limit`.
+- Reading a connection no step has opened is an error, and the step's scripts do not
+  run.
+- **In the desktop app**, a connection opened by sending a step stays open for the steps
+  sent after it, until that step is sent again, the connection is closed from above the
+  step list, Run all starts, or the app quits. Run all, like `gta`, opens its own and
+  closes them when it ends.
+
 ---
 
 ## 3. Checking a response
 
-A step's checks are calls to the [xtest](https://github.com/schwabyio/xtest) functions
-on `gta`, in its `tests` (§5 lists them). This section is what the calls mean.
+A step's checks are calls to functions on `gta`, in its `tests` (§5 lists them).
+[FUNCTIONS.md](./FUNCTIONS.md) documents each one, with examples. This section is the
+rules they share.
 
 ```yaml
 tests: |
@@ -962,14 +1240,14 @@ A key that holds a `.`, `[` or `]`, or is empty, goes in brackets as a JSON stri
 `jwt.payload["https://example.com/id"]`, `modules[""].edition`. Reports show such keys
 the same way.
 
-As in xtest, a path can also be a list of keys: `['jwt', 'payload', 'https://example.com/id']`.
+A path can also be a list of keys: `['jwt', 'payload', 'https://example.com/id']`.
 Each item is one key, whatever it holds, and a number is its digits, so
 `['groups', 0, 'name']` reads an index. Every function that takes a path takes a list
 too, and so does `pathToProperty`.
 
 A path that runs into a `null` before its end, such as `phoneNumber.number` when
-`phoneNumber` is `null`, reads as that `null` for a check that the value is `null`, as
-xtest read it. For any other check the property is not present.
+`phoneNumber` is `null`, reads as that `null` for a check that the value is `null`. For
+any other check the property is not present.
 
 ### Body conversion
 
@@ -978,6 +1256,37 @@ top-level key, namespace prefixes and attributes are dropped, an element holding
 text becomes that string (an empty one `""`), and a repeated element becomes an array.
 A `text/plain` or HTML body is the single property `plaintext`.
 
+An event stream (§2.3) is a list with one item per event:
+
+- **`data`** is the event's `data:` lines joined with a line feed: their JSON value when
+  they parse as JSON, so `[1].data.price` is a number, and otherwise the text, such as
+  `[DONE]`.
+- **`event`** and **`id`** are there only when that event's own lines set them. Neither
+  carries over to the next event.
+- Comments (`: heartbeat`) and a block with no `data:` are not events, `retry:` is not
+  kept, and an event the stream ended in the middle of is dropped.
+
+```text
+event: price
+id: 41
+data: {"symbol":"ACME","price":100}
+
+: heartbeat
+
+data: [DONE]
+
+```
+
+reads as
+
+```json
+[{ "event": "price", "id": "41", "data": { "symbol": "ACME", "price": 100 } }, { "data": "[DONE]" }]
+```
+
+The path `''` is the whole list, so `expectResponseBodyToHaveUnorderedArray('', …)` finds
+events whose order is not guaranteed. Under strict validation every event's properties
+need accounting for, so a small `maxEvents` keeps a busy stream practical.
+
 ### Values and patterns
 
 - A value compares with its type: `'12345'` does not equal `12345`.
@@ -1008,7 +1317,13 @@ saying so, and the checks after it still run.
 ### Unordered arrays
 
 `gta.expectResponseBodyToHaveUnorderedArray(path, list)` passes when the array holds
-every item of a simple `list`, in any order.
+every item of a simple `list`, in any order. A `RegExp` in the list is a pattern some item
+must match as text, and so is one held by an object in the list:
+`[/^admin/, { name: /^Grace/ }]`. A pattern never matches an object or array item.
+An object in the list matches an item holding each of its properties, and the item may
+have others; a property that is itself an object is matched the same way, so
+`{ data: { status: 'reversed' } }` finds an item whose `data` has that `status` among
+other properties. An array compares whole.
 
 A list of `{ pathToProperty, expectedValue, specialHandling? }` objects describes **one**
 item, property by property; call it once per item. A property may appear twice, once to
@@ -1022,15 +1337,16 @@ gta.expectResponseBodyToHaveUnorderedArray('users', [
 ```
 
 `gta.expectResponseBodyToHaveUnorderedArrayNotThisItem(path, list)` passes when no item
-matches. A `compareValue` may be a `RegExp`.
+matches. A simple list may hold patterns here too, and a `compareValue` may be a
+`RegExp`.
 
 - **Each call prefers items an earlier call did not match.** Two calls with the same
   description find two items when there are two, so each capture and strict validation
   see a different one. A sort starts this over, since its indexes name other items.
-- **A list of one `notThisExpectedValue` entry reads as xtest read it.** Without strict
-  validation it means no item has that value, so an empty array passes. With it, it
-  means one item whose value is something else, as any list does. The step's last word
-  on strict validation decides.
+- **A list of one `notThisExpectedValue` entry depends on strict validation.** Without
+  it, the entry means no item has that value, so an empty array passes. With it, the
+  entry means one item whose value is something else, as any list does. The step's last
+  word on strict validation decides.
 
 ### Strict validation
 
@@ -1159,8 +1475,8 @@ request, and `tests` checks the response:
 
 ### The `gta` object
 
-The xtest functions keep their original names, arguments and `specialHandling` strings.
-There is nothing to load, and no `startXTest` or `endXTest`.
+There is nothing to import or load. [FUNCTIONS.md](./FUNCTIONS.md) documents each
+function, with examples.
 
 | Function                                                                                         | In `before.script` |
 | ------------------------------------------------------------------------------------------------ | :----------------: |
@@ -1196,20 +1512,35 @@ row, and teardown, still run.
 
 `gta.date` formats with strftime specifiers: `%Y %y %m %d %e %H %I %M %S %L %p %b %B %a
 %A %j %Z %z %s %F %T %%`. An unrecognized specifier is left in the output, so a typo is
-visible. `timeZone` is `local`, `utc`, an IANA name such as `America/New_York`, or one of
-xtest's military zone letters (`U` is -08:00, not UTC).
+visible. `timeZone` is `local`, `utc`, an IANA name such as `America/New_York`, or a
+military zone letter (`U` is -08:00, not UTC).
 
 ### Other globals
 
-| Global     | What it is                                                                                                                                                              |
-| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `res`      | `tests` only: `status`, `statusText`, `headers` (lower-cased names), `header(name)`, `body` (parsed JSON, converted XML, or text), `text`, `time` (ms), `size` (bytes). |
-| `req`      | `method`, `url`, `headers`, `body`: as sent in `tests`, as written in `before.script`, where a script may change `headers` and `body` (below).                          |
-| `assert`   | Node's strict `assert`, for use inside `gta.test`.                                                                                                                      |
-| `console`  | Captured into the step's result.                                                                                                                                        |
-| `params`   | A request set's params (§2.5), in its own scripts and in the tests of the use step running it.                                                                          |
-| `endpoint` | An endpoint base's `{name}` values (§2.6), in its scripts and in every script of a step under it.                                                                       |
-| `checks`   | The project's check files (below).                                                                                                                                      |
+| Global     | What it is                                                                                                                                                                                                                                  |
+| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `res`      | `tests` only: `status`, `statusText`, `headers` (lower-cased names), `header(name)`, `body` (parsed JSON, converted XML, an event stream's events, or text), `text`, `time` (ms), `size` (bytes), and for an event stream `stream` (below). |
+| `req`      | `method`, `url`, `headers`, `body`: as sent in `tests`, as written in `before.script`, where a script may change `headers` and `body` (below).                                                                                              |
+| `assert`   | Node's strict `assert`, for use inside `gta.test`.                                                                                                                                                                                          |
+| `console`  | Captured into the step's result.                                                                                                                                                                                                            |
+| `params`   | A request set's params (§2.5), in its own scripts and in the tests of the use step running it.                                                                                                                                              |
+| `endpoint` | An endpoint base's `{name}` values (§2.6), in its scripts and in every script of a step under it.                                                                                                                                           |
+| `checks`   | The project's check files (below).                                                                                                                                                                                                          |
+
+**`res` for an event stream** (§2.3): `res.body` is its list of events, as checks see it
+(§3), `res.text` the stream as received, and `res.time` runs until the reading stopped.
+`res.stream` has the rest, and is absent for any other response:
+
+- `endedBy`: what stopped the reading: `close`, `maxEvents`, `streamTimeout`,
+  `untilEvent`, `limit`, `stopped` (the desktop app's Stop button), or `held` (a step on
+  a connection that waited for nothing, §2.11)
+- `at`: for each event, the milliseconds from the response headers to its arrival
+- `connection`: for a step that opens or reads a connection, `{ name, open }`: its name,
+  and whether it was still open when the step ended
+
+```js
+gta.test('first price within 2s', () => assert.ok(res.stream.at[0] < 2000))
+```
 
 Also available are the language itself and the web-standard globals: timers, `URL`,
 `URLSearchParams`, `TextEncoder`, `TextDecoder`, `atob`, `btoa`, `structuredClone` and
@@ -1236,7 +1567,10 @@ delete req.headers['X-Debug']
 
 - `req.body` is text, and can be changed for a `json`, `xml`, `text` or `graphql` body.
   A form, multipart or file body is built from its parts, so it cannot.
-- `req.headers` is a map of names to values: add, change or delete them.
+- `req.headers` is a map of names to values: add, change or delete them. A header sent
+  more than once reads as its values joined with `, `, as `res.headers` does, and is
+  still sent once per value unless the script changes it. Set an array to send one
+  header per value.
 - Variables in what the script writes resolve afterwards, as in the file.
 - Anything else is a pre-request error, and nothing is sent.
 
@@ -1298,7 +1632,8 @@ global project's root.
 - A secret with no value anywhere fails the run, naming it. An empty credential is never
   sent.
 - Reports replace a secret's value with `[secret: NAME]`, wherever it appears. So does
-  the desktop app where it shows a request as sent.
+  the desktop app where it shows a request as sent, or what a script wrote with
+  `console`.
 - `.env` holds `NAME=value` lines. `#` starts a comment line, `export ` before a name is
   allowed, and a value may be quoted. It must not be committed.
 
@@ -1343,7 +1678,7 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
 - It must be YAML that parses, holding a mapping, in a file ending in `.yml`.
 - Unknown keys are errors in: a collection's top level, a step, `settings`, `before`,
   `body`, `project.yml`, `tls`, an environment file, an environment's `flags`, a param
-  spec and `settings.yml`.
+  spec, `settings.yml` and `rules.yml`.
 
 **Collections**
 
@@ -1360,7 +1695,8 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
 **Steps**
 
 - A step has exactly one method key, out of `GET`, `POST`, `PUT`, `PATCH`, `DELETE`,
-  `HEAD` and `OPTIONS`, in capitals, and its value is a string.
+  `HEAD` and `OPTIONS`, in capitals, and its value is a string. A use step and a step
+  reading a connection have none.
 - A step has no key outside the table in §2.1.
 - A use step has no method key, `headers`, `body`, `settings` or `before`. `with` is used
   only with `use`.
@@ -1369,6 +1705,8 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
 - `base` is only ever `false`.
 - `forEach` is a string, and a use step has none.
 - `useTests` is only ever `true`, only on a request set's step, and on one step at most.
+- A step reading a connection has no `headers`, `body`, `base` or `forEach`. A step
+  opening one has no `forEach`, and a use step has no `connection` (§2.11).
 
 **Values**
 
@@ -1377,6 +1715,7 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
 - A tag matches `^[A-Za-z0-9._:-]+$`.
 - A flag name matches `^[A-Za-z0-9_][A-Za-z0-9_.-]*$`, and a flag value is a string,
   number or boolean.
+- A connection name matches `^[A-Za-z0-9_][A-Za-z0-9_.-]*$`.
 - A `settings` value has its type in §2.3.
 
 **Bodies**
@@ -1394,9 +1733,9 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
 - A base collection (`bases/`) has no `steps`, `setup`, `teardown` or `params`, and no
   `extends`.
 - An endpoint (`endpoints/`) has a URL that is a path starting with `/`, and no use
-  steps. An endpoints file has no `setup` or `teardown`.
+  steps or connections. An endpoints file has no `setup` or `teardown`.
 - A check file's name is a JavaScript identifier; if it isn't, the file is not loaded.
-- `collections/`, `requests/`, `endpoints/` and `bases/` hold files at most one directory
+- `collections/`, `requests/`, `endpoints/` and `bases/` hold files at most one folder
   down.
 
 **Portability (§1.2)**
@@ -1407,7 +1746,7 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
   `Collections/`.
 - No two names in `collections/`, `requests/`, `endpoints/`, `bases/`, `environments/`
   or `checks/` differ only in case.
-- No file or directory name is one Windows refuses.
+- No file or folder name is one Windows refuses.
 
 **Projects**
 
@@ -1416,6 +1755,9 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
 - Each `tls.ca` file exists and holds a PEM or DER certificate.
 - `settings.yml` exists for `gta`; a global project's is optional. The `environmentType`
   a run ends up with, if any, names an environment that exists.
+- In `rules.yml`, each rule has a value of the type §1.4 gives it, a pattern compiles,
+  `tests.only` lists `gta`, and `guide` is a string. What breaks a rule is a finding of `gta lint`, never an
+  error: it stops nothing (§1.4).
 
 **At run time**
 
@@ -1430,6 +1772,7 @@ it. Rules checked at run time fail the step, or the run, before anything is sent
   `gta get` checks these, and the body files named without `{{variables}}`, without
   running anything (§1.3).
 - A step's `forEach` resolves to a JSON array.
+- A step reading a connection finds it open: a step before it in the run opened it.
 - In `{{@name}}`, `name` holds text naming a variable that exists.
 - A `before.script` sets `req.body` to text, and only for a `json`, `xml`, `text` or
   `graphql` body; `req.headers` stays a map.