@jterrazz/test 18.2.0 → 18.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -3
- package/dist/browser-shared-index.browser.js +1 -2
- package/dist/checker.js +1 -1
- package/dist/config.d.ts +1 -1
- package/dist/config.js +1 -2
- package/dist/index.d.ts +196 -194
- package/dist/index.js +1742 -1690
- package/dist/oxlint.cjs +5 -8
- package/dist/oxlint.d.cts +1 -1
- package/dist/oxlint.d.ts +1 -1
- package/dist/oxlint.js +1 -3
- package/dist/shared-exchange.js +1 -1
- package/dist/shared-in-place.js +1 -1
- package/dist/shared-manifest.js +4 -4
- package/dist/shared-route-coverage.global-setup.js +1 -1
- package/dist/shared-session-ledger.js +1 -1
- package/dist/shared-specifications.js +0 -1
- package/dist/shared-stub-server.js +13 -0
- package/dist/shared-testcontainers.adapter.js +1 -1
- package/dist/stubs.js +1 -1
- package/dist/vitest.d.ts +1 -6
- package/dist/vitest.js +2 -8
- package/docs/README.md +1 -2
- package/docs/concepts/architecture.md +15 -16
- package/docs/concepts/conventions.md +2 -2
- package/docs/guides/api-specs.md +172 -3
- package/docs/guides/component-specs.md +1 -1
- package/docs/guides/website-specs.md +1 -1
- package/docs/reference/assertions.md +2 -2
- package/docs/reference/glossary.md +1 -1
- package/docs/reference/linting.md +4 -4
- package/docs/reference/tokens.md +1 -1
- package/docs/start/getting-started.md +14 -15
- package/docs/start/set-up-a-project.md +2 -2
- package/package.json +1 -1
- package/docs/guides/deployment-specs.md +0 -265
package/docs/guides/api-specs.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# API specs (`specification.api`)
|
|
2
2
|
|
|
3
|
-
`specification.api()` tests an HTTP API through real requests. Your app runs **in this process**, built by the `server` factory from the real services the runner started, so a request reaches it without a socket, and a stub, a clock and a golden all mean something because the app shares this process's world.
|
|
3
|
+
`specification.api()` tests an HTTP API through real requests. Your app runs **in this process**, built by the `server` factory from the real services the runner started, so a request reaches it without a socket, and a stub, a clock and a golden all mean something because the app shares this process's world. Or it is **already deployed**, named by its `url`, and each request crosses the network to it ([Deployed target (`url`)](#deployed-target-url)): the subject is the same HTTP API, and where it runs is an option, as a website is started by `server` or met at a `url`.
|
|
4
4
|
|
|
5
5
|
Use it when the subject under test is an HTTP surface. For background pipelines use [job](job-specs.md); for binaries use [cli](cli-specs.md).
|
|
6
6
|
|
|
@@ -17,7 +17,7 @@ An api spec answers one question: **given this state and this request, what does
|
|
|
17
17
|
| What it proves | Application logic against real databases, caches and processes |
|
|
18
18
|
| Parallel isolation | Per-worker schema / db-index / file copy (rule `state-isolated-per-worker`) |
|
|
19
19
|
|
|
20
|
-
Because the app shares this process, three things the framework offers are real here and nowhere else: `.intercept()` (msw runs in-process), `.clock()` (it pins the `Date` the app reads), and a golden of the response the app actually built. What the SHIPPED artefact does (its Dockerfile, its wiring, its networking) is
|
|
20
|
+
Because the app shares this process, three things the framework offers are real here and nowhere else: `.intercept()` (msw runs in-process), `.clock()` (it pins the `Date` the app reads), and a golden of the response the app actually built. What the SHIPPED artefact does (its Dockerfile, its wiring, its networking, its certificate) is met at its `url` instead ([Deployed target (`url`)](#deployed-target-url)), where no stub, no clock and no seed reaches it.
|
|
21
21
|
|
|
22
22
|
A `postgres()` service runs `docker/<service>/init.sql` when it starts, under the kebab-case of its record key; `sqlite()` takes its schema from its own `init` or `prismaSchema` option instead ([Services](../reference/services.md#dockerserviceinitsql)).
|
|
23
23
|
|
|
@@ -64,9 +64,19 @@ afterAll(async () => {
|
|
|
64
64
|
| Option | Required | Description |
|
|
65
65
|
| ---------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
66
66
|
| `services` | yes (if the app needs infra) | Named record of service factories (`postgres()`, `redis()`, `sqlite()`). Keys are your test vocabulary; see [services](../reference/services.md) |
|
|
67
|
-
| `server` | yes
|
|
67
|
+
| `server` | yes, or `url` | `(services) => app`: receives the started services record, fully typed (rule `server-sees-declared-services`) |
|
|
68
|
+
| `url` | yes, or `server` | The deployed api's base URL: its own target, exclusive of `server`, `services` and `root` ([Deployed target (`url`)](#deployed-target-url)) |
|
|
68
69
|
| `root` | no | Override for root resolution, reserved for cases where the convention is not enough (rule `no-redundant-root`) |
|
|
69
70
|
|
|
71
|
+
Exactly one of `server` and `url` names the app, as with a website: both, neither, an empty `url` or one that is no `http(s)` URL is refused before anything starts, and so are `services` or `root` beside a `url` (a deployed api is wired to its own services) and an option of a deployed api beside a `server`. Against a `url`, four more options shape each request:
|
|
72
|
+
|
|
73
|
+
| Option | Default | Description |
|
|
74
|
+
| ----------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
75
|
+
| `timeout` | `5000` | The milliseconds one request may take, connection to last byte, before it reads `unreachable: timeout`. A positive number |
|
|
76
|
+
| `insecure` | `false` | Accept a certificate the trust check refuses (self-signed, expired, another host's); `result.tls` still says why |
|
|
77
|
+
| `ca` | none | A certificate authority trusted beside the ones node trusts: its PEM text, a value like any string |
|
|
78
|
+
| `recordedHeaders` | `['content-type', 'location']` | The headers update mode records into a golden that states none yet, those the response carries ([Deployed target (`url`)](#deployed-target-url)) |
|
|
79
|
+
|
|
70
80
|
### Root resolution
|
|
71
81
|
|
|
72
82
|
Without `root`, the framework finds the package being tested by walking up from the specification file. The walk, the tree it reads and when a `root` is redundant are [Services § Root auto-discovery](../reference/services.md#root-auto-discovery)'s.
|
|
@@ -361,6 +371,165 @@ The list is judged whenever the app exposes a route table of its own, an OpenAPI
|
|
|
361
371
|
|
|
362
372
|
The app runs in THIS process, always: there is no second mode, no container binding and no `TEST_MODE`. Why compose mode left, and what was weighed against keeping it: [ADR-007](https://github.com/jterrazz/package-test/blob/main/docs/decisions/007-compose-mode-leaves-the-package.md).
|
|
363
373
|
|
|
374
|
+
## Deployed target (`url`)
|
|
375
|
+
|
|
376
|
+
An api already deployed (a preview your CI just shipped, every host of a platform, a node's address or a closed port) is the same subject met elsewhere: `url` names it, and each request goes over the network, with no browser. The answer is compared as an in-process one is (the same `result.response`, the same `.http` golden, header subset, `{{tokens}}` and `TEST_UPDATE=1`), and two things only a network has join it: an exchange that never answered is a result, and the certificate it came over is read off `result.tls`. The decision is [ADR-022](https://github.com/jterrazz/package-test/blob/main/docs/decisions/022-a-deployed-api-is-an-api-spec-with-a-url.md).
|
|
377
|
+
|
|
378
|
+
```typescript
|
|
379
|
+
// specs/api/deployed.specification.ts
|
|
380
|
+
import { specification } from '@jterrazz/test';
|
|
381
|
+
import { afterAll } from 'vitest';
|
|
382
|
+
|
|
383
|
+
export const { api, cleanup } = await specification.api({
|
|
384
|
+
url: process.env.DEPLOYED_URL ?? 'https://preview.site.example',
|
|
385
|
+
});
|
|
386
|
+
|
|
387
|
+
afterAll(cleanup);
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
The handle destructures to `{ api, cleanup }`, no `docker`: nothing is started, so `cleanup` releases nothing, and it is passed to `afterAll` like every specification's. The transport is node's own `http`, `https`, `tls` and `net`: nothing to install.
|
|
391
|
+
|
|
392
|
+
### What a deployed chain offers
|
|
393
|
+
|
|
394
|
+
| Offered | Not offered, refused by the type and at run time |
|
|
395
|
+
| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
|
|
396
|
+
| `.get(target)`, `.request(<.http text> \| file(...))` | `.post()`, `.put()`, `.delete()`: a request other than a GET is written whole with `.request()` |
|
|
397
|
+
| `.headers()`, `api.with({ headers })`, `.connect(address)` | `.seed()`, `.intercept()`, `.clock()`: a deployed api's database, network and clock are its own |
|
|
398
|
+
| `result.response`, `result.tls` | `result.table()`, `result.file()`, `result.directory()`: what it left behind is its own |
|
|
399
|
+
|
|
400
|
+
### The target
|
|
401
|
+
|
|
402
|
+
Every request names one URL: written whole (`https://grafana.site.example/`), or a path (`/health`) read from `url`, its own path kept (`https://site.example/app` and `/health` are `https://site.example/app/health`). One specification therefore reaches every host of a platform. A target that is neither, and a URL of another scheme, are refused. A request written as `.http` text or in a `__requests__/` file names its URL on the first line, whole (`GET https://site.example/health`, HTTP's absolute form) or as a path; its body is sent as written. A redirect is the answer, never followed:
|
|
403
|
+
|
|
404
|
+
```typescript
|
|
405
|
+
test('a moved page answers its redirect, never followed', async () => {
|
|
406
|
+
// Given - the address of the page before the move
|
|
407
|
+
const result = await api.get('/blog');
|
|
408
|
+
|
|
409
|
+
// Then - the 301 itself, and where it points
|
|
410
|
+
expect(result.response).toMatch(`
|
|
411
|
+
HTTP/1.1 301 Moved Permanently
|
|
412
|
+
location: https://site.example/articles
|
|
413
|
+
`);
|
|
414
|
+
});
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
### Setup: `.connect()`
|
|
418
|
+
|
|
419
|
+
`.connect(address)` opens the socket on another address than the URL's, as curl's `--connect-to`: the `Host` header and the TLS server name stay the URL's, so the certificate is checked against the host the URL names. It is how a node's IP, a tailnet address or a NodePort is probed under the name it serves:
|
|
420
|
+
|
|
421
|
+
```typescript
|
|
422
|
+
test("the node's NodePort serves the public host itself", async () => {
|
|
423
|
+
// Given - the public URL, its socket opened on the node
|
|
424
|
+
const result = await api.connect('100.64.0.7:30443').get('https://site.example/health');
|
|
425
|
+
|
|
426
|
+
// Then - the same answer the edge gives
|
|
427
|
+
expect(result.response).toMatch(`
|
|
428
|
+
HTTP/1.1 200 OK
|
|
429
|
+
content-type: application/json
|
|
430
|
+
|
|
431
|
+
{"status": "ok"}
|
|
432
|
+
`);
|
|
433
|
+
});
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
The address is `host:port`, `[v6]:port`, or a bare host, which keeps the URL's port; a later `.connect()` wins. `.headers()` reads a name case-blind, so a `Host` the test names replaces the URL's, the TLS server name staying the URL's as with curl's `-H Host:`: the probe of a public edge asked for an internal host, `api.headers({ Host: 'grafana.internal.example' }).get('https://site.example/')`, compared to `HTTP/1.1 403 Forbidden`. Every request carries `user-agent: @jterrazz/test` unless a header replaces it.
|
|
437
|
+
|
|
438
|
+
### Unreachable is a result
|
|
439
|
+
|
|
440
|
+
A request that never answered is never thrown. Its response is the one line `unreachable: <reason>`:
|
|
441
|
+
|
|
442
|
+
| Reason | When |
|
|
443
|
+
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
444
|
+
| `refused` | Nothing listens on the port, or a firewall rejects it |
|
|
445
|
+
| `timeout` | No whole response within `timeout`: a firewall dropping the packets, a server that accepts and never answers |
|
|
446
|
+
| `dns` | No resolver knows the name |
|
|
447
|
+
| `reset` | The connection was cut before the response ended |
|
|
448
|
+
| `no route` | The network has no way to the address |
|
|
449
|
+
| `tls <CODE>` | The handshake failed, or the certificate was refused: `tls CERT_HAS_EXPIRED`, `tls DEPTH_ZERO_SELF_SIGNED_CERT`, `tls ERR_TLS_CERT_ALTNAME_INVALID`, `tls UNABLE_TO_VERIFY_LEAF_SIGNATURE` |
|
|
450
|
+
|
|
451
|
+
So a negative probe is one sentence, with no `try`/`catch` and no "it should throw" that passes when anything throws:
|
|
452
|
+
|
|
453
|
+
```typescript
|
|
454
|
+
test('the NodePort is closed to the internet', async () => {
|
|
455
|
+
// Given - the public address, on the port the cluster must not expose
|
|
456
|
+
const result = await api.connect('203.0.113.10:30080').get('http://site.example/');
|
|
457
|
+
|
|
458
|
+
// Then - the firewall drops it
|
|
459
|
+
expect(result.response).toMatch('unreachable: timeout');
|
|
460
|
+
});
|
|
461
|
+
```
|
|
462
|
+
|
|
463
|
+
The line is a golden like any other: a reason takes tokens (`unreachable: tls {{string}}` where the code varies by platform), a golden file may hold it, and `TEST_UPDATE=1` writes it. An answer expected where none came, or the reverse, fails naming both outcomes (`Response outcome mismatch`, [Assertions](../reference/assertions.md#resultresponse-http-response-api)).
|
|
464
|
+
|
|
465
|
+
### `result.tls`: the certificate
|
|
466
|
+
|
|
467
|
+
Over https, `result.tls` holds what the handshake presented, whether or not it was trusted:
|
|
468
|
+
|
|
469
|
+
| Fact | Description |
|
|
470
|
+
| -------------------- | ---------------------------------------------------------------------------------- |
|
|
471
|
+
| `subject` | The subject's common name |
|
|
472
|
+
| `altNames` | The names it is valid for, without their `DNS:` or `IP Address:` prefix |
|
|
473
|
+
| `issuer` | The issuer's common name, or its organisation |
|
|
474
|
+
| `validFrom` | A `Date` |
|
|
475
|
+
| `validTo` | A `Date` |
|
|
476
|
+
| `daysRemaining` | Whole days from the request to `validTo`, negative once expired |
|
|
477
|
+
| `covers(host)` | Whether it is valid for `host`, a wildcard covering one label as a client reads it |
|
|
478
|
+
| `authorized` | Whether it chains to a trusted authority and covers the URL's host |
|
|
479
|
+
| `authorizationError` | The code the trust check refused with, `undefined` when trusted |
|
|
480
|
+
| `protocol` | The TLS version negotiated (`TLSv1.3`) |
|
|
481
|
+
|
|
482
|
+
```typescript
|
|
483
|
+
test('the certificate covers the host with a month to spare', async () => {
|
|
484
|
+
// Given - the public host
|
|
485
|
+
const result = await api.get('https://site.example/');
|
|
486
|
+
|
|
487
|
+
// Then - the answer, and the certificate it came over
|
|
488
|
+
expect(result.response).toMatch('HTTP/1.1 200 OK');
|
|
489
|
+
expect(result.tls?.covers('site.example')).toBe(true);
|
|
490
|
+
expect(result.tls?.daysRemaining).toBeGreaterThanOrEqual(30);
|
|
491
|
+
});
|
|
492
|
+
```
|
|
493
|
+
|
|
494
|
+
A refused certificate reads `unreachable: tls <CODE>` and keeps its facts, so an expired one says by how many days. With `insecure: true` the request goes on over it, and `authorized` and `authorizationError` still say why it would have been refused. A private authority is read by the specification file that names it, `ca: readFileSync(new URL('__fixtures__/ca.pem', import.meta.url), 'utf8')`: the option takes the certificate, not a path.
|
|
495
|
+
|
|
496
|
+
### Update mode against a url
|
|
497
|
+
|
|
498
|
+
`TEST_UPDATE=1` writes a deployed golden as it writes any response golden ([Tokens](../reference/tokens.md#update-mode-tokens-are-preserved)), with two differences that keep a probe's golden about the probe: a golden that states nothing yet records the status, the `recordedHeaders` the response carries, and the body; and a golden that states no body keeps stating none, so a long HTML page is not written into a check of its status. An exchange that never answered is written as its one line.
|
|
499
|
+
|
|
500
|
+
### A platform suite, from the routes it discovers
|
|
501
|
+
|
|
502
|
+
A conformance suite reads its hosts from the platform when its specification file loads, then asks each one, one row of a `test.each` per host:
|
|
503
|
+
|
|
504
|
+
```typescript
|
|
505
|
+
// specs/api/routes/answers.spec.ts
|
|
506
|
+
import { describe, expect, test } from 'vitest';
|
|
507
|
+
|
|
508
|
+
import { api, publicHosts } from '../deployed.specification.js';
|
|
509
|
+
|
|
510
|
+
describe('answers', () => {
|
|
511
|
+
test.each(publicHosts)('%s answers over a certificate that covers it', async (host) => {
|
|
512
|
+
// Given - the host, as the platform routes it
|
|
513
|
+
const result = await api.get(`https://${host}/`);
|
|
514
|
+
|
|
515
|
+
// Then - an answer, and a certificate naming the host with a month to spare
|
|
516
|
+
expect(result.response).toMatch('HTTP/1.1 {{int}}');
|
|
517
|
+
expect(result.tls?.covers(host)).toBe(true);
|
|
518
|
+
expect(result.tls?.daysRemaining).toBeGreaterThanOrEqual(30);
|
|
519
|
+
});
|
|
520
|
+
});
|
|
521
|
+
```
|
|
522
|
+
|
|
523
|
+
`publicHosts` is what the specification file read from the cluster (an `IngressRoute` list, a DNS zone), so a host added to the platform is probed with no line of the suite changed. A value written in place under `test.each` is one value for every row, so it holds tokens where rows differ; a golden per host is `toMatch(file(...))` under `__expected__/`. An application repository points the same specification at its own URL and runs `npx vitest --run --project api` after its deploy step, the URL in the environment. Route coverage is the in-process app's: a deployed api exposes no route table, and records no request.
|
|
524
|
+
|
|
525
|
+
### Pitfalls against a url
|
|
526
|
+
|
|
527
|
+
- **Probing what the in-process spec proves.** A field of the JSON a handler builds is the app's logic, proven where its database and clock are the test's; against a `url`, keep the golden to what the edge decides.
|
|
528
|
+
- **A deadline longer than the test's.** `timeout` bounds one request, vitest's budget (30 s) the test: a suite of firewalled ports waits the whole `timeout` for each.
|
|
529
|
+
- **An endpoint that streams forever** reads `unreachable: timeout`: probe it with a request the server ends.
|
|
530
|
+
- **A reason read as stable across platforms.** A handshake's failure is named by the TLS library node runs on: `unreachable: tls {{string}}` in a golden meant for every machine.
|
|
531
|
+
- **A proxy in the environment.** `HTTPS_PROXY` is not read: a request goes where its URL or its `.connect()` says.
|
|
532
|
+
|
|
364
533
|
## Pitfalls
|
|
365
534
|
|
|
366
535
|
- **Getting `database:` wrong for the number of databases.** See [Services § The `database:` rule](../reference/services.md#the-database-rule).
|
|
@@ -32,7 +32,7 @@ The subject is a unit that draws itself IN A BROWSER, which leaves four neighbou
|
|
|
32
32
|
| A React island inside an `.astro` page | The island is a rendered unit; only the file around it is not | a `.test.tsx` in the `__tests__/` of the island's folder, here |
|
|
33
33
|
| A Next.js server component | It renders on the server and reads what the server reads; there is no tree to mount | a website spec, through the served page |
|
|
34
34
|
|
|
35
|
-
**What runs under node stays there.** In a page, each node-only name the entry publishes (the
|
|
35
|
+
**What runs under node stays there.** In a page, each node-only name the entry publishes (the seven constructors, the services, the docker accessors, the results that walk a disk) refuses where it is called: a page has no filesystem, no child process and no socket.
|
|
36
36
|
|
|
37
37
|
## The constructor: there is none
|
|
38
38
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
`specification.website()` tests a rendered website (its raw HTTP surface: redirects, robots.txt, headers; and its rendered surface: title, head metadata, JSON-LD, console, and full user scenarios) through a real Chromium. It starts the site itself, or targets one already running.
|
|
4
4
|
|
|
5
|
-
Use it when the subject under test is a browser-rendered page. For a JSON/HTTP API surface use [api](api-specs.md)
|
|
5
|
+
Use it when the subject under test is a browser-rendered page. For a JSON/HTTP API surface use [api](api-specs.md), in this process or deployed at a `url` with no browser; for a binary use [cli](cli-specs.md).
|
|
6
6
|
|
|
7
7
|
## What it specifies
|
|
8
8
|
|
|
@@ -79,7 +79,7 @@ Plain values take vitest's native matchers. No framework matcher exists (or is n
|
|
|
79
79
|
| `await result.filesystem.files()` | `string[]` | `expect(await result.filesystem.files()).toContain('shoply.lock')` |
|
|
80
80
|
| `await cli.run(file('./<case>.spec.yaml'))` | `CliResult` | the document asserts itself; the LAST run's result comes back; see [Spec documents](../guides/spec-documents.md#the-three-doors-one-runner) |
|
|
81
81
|
|
|
82
|
-
## `result.response`: HTTP response (api
|
|
82
|
+
## `result.response`: HTTP response (api)
|
|
83
83
|
|
|
84
84
|
| Matcher | Sync/async | Example |
|
|
85
85
|
| -------------------------------- | ---------- | --------------------------------------------------------------------------- |
|
|
@@ -105,7 +105,7 @@ Response status mismatch (./__expected__/user-created.http)
|
|
|
105
105
|
|
|
106
106
|
A header mismatch has the same shape (`Response header mismatch (name)`, with `header:` / `expected:` / `received:` lines; a missing header shows `received: (absent)`). A body mismatch prints a line-by-line structural diff (`Response mismatch (name)`, `- Expected` / `+ Received`) of the expected body (with matchers and tokens rendered as their placeholder text, `{{uuid#order}}`) against the actual body: a JSON value as its pretty-printed lines, and a text body (when both sides are text) as its own lines, so an HTML page or an event stream reads as written. The diff is literal: it does not annotate captured ref values, and a failed `{{uuid#order}}` re-occurrence surfaces only as the two differing lines.
|
|
107
107
|
|
|
108
|
-
|
|
108
|
+
A request to a deployed api (`specification.api({ url })`) that never answered is the one line `unreachable: <reason>` ([API specs § Unreachable is a result](../guides/api-specs.md#unreachable-is-a-result)), compared as the status line is, tokens allowed; against a status line, or another reason, the failure names both outcomes:
|
|
109
109
|
|
|
110
110
|
```text
|
|
111
111
|
Response outcome mismatch (the value in place)
|
|
@@ -15,7 +15,7 @@ The words this manual uses, one meaning each.
|
|
|
15
15
|
| seam | A place where the framework meets an outside tool it loads on demand: a database driver, a browser, a simulator. Each is an optional peer |
|
|
16
16
|
| ratchet | A count of debt kept in `.jterrazz/baseline.json` that may only fall: an unreached route, a convention not yet followed. A run that raises it fails, and `TEST_UPDATE=1` records a fall |
|
|
17
17
|
| specimen | A small sample app a testing tool keeps under `specs/__fixtures__/` to test itself against. Only a repository that tests a testing tool has one |
|
|
18
|
-
| unreachable | What
|
|
18
|
+
| unreachable | What a request to a deployed api (`specification.api({ url })`) reads when no response came (refused, timed out, unresolved, its certificate refused): the one line `unreachable: <reason>`, compared whole like a status line rather than thrown ([API specs](../guides/api-specs.md#unreachable-is-a-result)) |
|
|
19
19
|
| door | Where a test file may sit: in the `__tests__/` of its module's folder (a unit), under `specs/<kind>/` (an assembled product), or in a repository suite (the third door) |
|
|
20
20
|
|
|
21
21
|
The examples use made-up products: `shoply`, a shop CLI; a news app with an events feed; and hosts under `.test` and `.example`, which never resolve on the internet.
|
|
@@ -153,7 +153,7 @@ Every rule a project using the framework meets, across its enforcement channels
|
|
|
153
153
|
| <a id="no-redundant-root"></a>`no-redundant-root` | model | static | all | A `root` pointing at the directory the automatic walk would have found is redundant, and an error. | Drop the option: the walk finds the nearest `package.json` by itself. | Detection by convention has to stay the default; an explicit `root` is only justified where it fails. |
|
|
154
154
|
| <a id="no-runner-alias"></a>`no-runner-alias` | model | static | all | The return is destructured under the constructor's canonical name, unaliased (`{ api, cleanup, docker }`); renaming (`{ api: myApi }`) is an error. | Drop the alias and keep the canonical name. | One instance name per specification makes every spec readable with no local context. |
|
|
155
155
|
| <a id="no-simulated-dom"></a>`no-simulated-dom` | component | static | tests | A test file declares no simulated DOM: the `@vitest-environment happy-dom\|jsdom` pragma is an error. Reach: the roles that RUN a test (`module`, `component`, `specification`); a project config is `no-simulated-dom-in-config`'s. | A rendered thing is a `.test.tsx` in the `__tests__/` of its component's folder, collected by the `component()` project. | A simulated DOM behaves ALMOST like a browser, and what ships is judged by a real one: the component specification renders in the Chromium the website specification already drives. |
|
|
156
|
-
| <a id="no-unknown-constructor"></a>`no-unknown-constructor` | model | static | all |
|
|
156
|
+
| <a id="no-unknown-constructor"></a>`no-unknown-constructor` | model | static | all | Seven constructors and only seven: `specification.api()`, `specification.job()`, `specification.cli(bin)`, `specification.integration()`, `specification.website()`, `specification.mobile()`, `specification.lint()`. Any other member (`.app`, `.http`, `.stack`...) is an error. A rendered component starts nothing: it is a chain, not a member of this record. | Name one of the seven, or reach for the `component` chain: an eighth subject is a decision before it is a constructor. | A closed surface stops parallel constructors being invented and keeps the API memorable. |
|
|
157
157
|
| <a id="runner-in-specification-file"></a>`runner-in-specification-file` | model | static | tests | A runner is created only in a `*.specification.ts(x)` file under `specs/`: calling `specification.*` anywhere else is an error. The `.tsx` spelling is the same file with a `wrap` written in JSX. A `component`-role file is out of reach: a rendered unit starts nothing. | Move the call into `specs/<specification>/<specification>.specification.ts` and export the handle from there. | Anchoring runners to a recognisable filename makes the entry point discoverable and keeps tests declarative. |
|
|
158
158
|
| <a id="database-named-only-when-several"></a>`database-named-only-when-several` | model | checker | tests | With ≥ 2 databases, `database:` is required on every `.seed()`/`.table()`; with exactly one it is forbidden, which is checked by crossing the `services:` record with the calls in the tests. | State `database:` everywhere or nowhere, whichever the record calls for. | The number of databases fixes the call API; the cross-file analysis catches the omission before run time. |
|
|
159
159
|
| <a id="docker-exec-bound-with-await-using"></a>`docker-exec-bound-with-await-using` | model | checker | tests | Docker-aware runners are inferred from the `docker:` option of the imported specification file, and every `.exec()` result bound without `await using` is reported. | Bind the result with `await using`, so the containers the run spawned are disposed with the scope. | Inferring the runners removes the hand-maintained list an oxlint rule would need. |
|
|
@@ -211,7 +211,7 @@ Every rule a project using the framework meets, across its enforcement channels
|
|
|
211
211
|
| <a id="checker-suppression-states-a-reason"></a>`checker-suppression-states-a-reason` | model | checker | specs | Every `checker-disable-next-line` / `checker-disable-line` directive carries `-- <reason>`, wherever a checker pass can be silenced: a test, a document, a fixture under ground. The word inside a string is prose, not a directive. | Write `checker-disable-next-line <name> -- <reason>`. | A suppression with no reason is a rule turned off by someone no longer here: the next reader cannot tell whether the pass was wrong or simply in the way, so the line survives every review. The toolchain's own gate reads `oxlint-disable*` and nothing else. |
|
|
212
212
|
| <a id="document-description-reads-as-a-title"></a>`document-description-reads-as-a-title` | cli | checker | document | A document's `description:` is a title: one line, lowercase initial (an all-caps identifier exempted), no trailing period, under 100 characters. | Write it as a title, not as a sentence. | The description IS the vitest title: a title's rules apply to it, not a paragraph's. |
|
|
213
213
|
| <a id="document-description-unique"></a>`document-description-unique` | cli | checker | document | Two `<case>.spec.yaml` documents in one directory do not share a `description:`. | Name what each case proves; two titles that read alike leave two files to open. | The reporter names the title: two identical titles leave two files to open. |
|
|
214
|
-
| <a id="file-suffix-matches-place"></a>`file-suffix-matches-place` | model | checker | specs | The suffix says the kind, and the tree agrees with it. A `.spec.ts` (the ASSEMBLED product) lives under `specs/<specification>/`: one at a project's own root is an error here, one in any folder `spec-file-in-a-specs-tree`'s. A `.test.ts` (the UNIT) lives in the `__tests__/` of its module's folder; under a specification folder (`api`, `cli`, `
|
|
214
|
+
| <a id="file-suffix-matches-place"></a>`file-suffix-matches-place` | model | checker | specs | The suffix says the kind, and the tree agrees with it. A `.spec.ts` (the ASSEMBLED product) lives under `specs/<specification>/`: one at a project's own root is an error here, one in any folder `spec-file-in-a-specs-tree`'s. A `.test.ts` (the UNIT) lives in the `__tests__/` of its module's folder; under a specification folder (`api`, `cli`, `integration`, `job`, `lint`, `mobile`, `website`) it is an error, fixable by `typescript fix` (a `git mv`). A `.test.ts` that reaches no runner is `specification-test-uses-its-runner`'s, whose fix is the opposite move, and this rule's fix leaves it alone. A first level that is NOT a specification is a repository suite: it covers a tree, keeps `.test.ts`, and `specs-at-declared-depth`'s declared depth is what judges it, unless the tree carries its `*.specification.ts` at its own ROOT, which makes the whole tree one specification's and every `.test.ts` below it a spec. A document is named `<case>.spec.yaml`: `<case>` in kebab-case, without the words `test`/`spec`/`cli` the suffix already carries, and never the bare name of its directory. | Move the `.spec.ts` under `specs/<specification>/` (or rename it `.test.ts` in the `__tests__/` of its module's folder); rename the specification `.test.ts` to `.spec.ts`: `typescript fix` does it with `git mv`, and names every `include`/`exclude` entry of the member the rename broke. | With no agreement between the suffix and the tree, a reader cannot say what a test proves without opening it; and the filename is the first sentence they read: `rm/rm.spec.yaml` says nothing twice. |
|
|
215
215
|
| <a id="ground-beside-its-only-reader"></a>`ground-beside-its-only-reader` | model | checker | ground | Ground in a leaf holding ≥ 2 node tests (`.test.ts`, `.spec.ts`) that a single one of them reads is an error. A `.test.tsx` tree is out of reach, as it is for `specs-at-declared-depth`, and so are the api, cli and job specifications: there a leaf is its subject's standard name (`spec-in-subject-folder`), its aspect files share it by design, and a folder of one spec's own is one `spec-in-subject-folder` moves the spec back out of. | Give that spec its own domain folder, with the ground beside it. | Ground one spec reads, parked where several sit, reads as shared: everyone assumes someone else depends on it and nobody dares touch it. |
|
|
216
216
|
| <a id="ground-shared-or-inline"></a>`ground-shared-or-inline` | model | checker | ground | A ground file exists only if it is shared, and a value lives in a test only while it stays short. The checker counts the TESTS that read each file of the specs trees, and of the ground a module test stands on inside its `__tests__/` (`src/render/__tests__/__expected__/`), a `test()`/`it()` whose body names it with a literal `file()`/`directory()`, a document, and judges its level. Read by ONE test, named by no module test of the member, and read by no document's `fixture:` line, no import and no path joined to `import.meta.dirname`, a file of `__expected__/` or `__requests__/` is an error where the test names it: it belongs in the test, written in place (a golden as the value `toMatch` compares, a request as its text or its body value) unless it is a directory tree, a binary file, a text no literal survives in place (one opening on a line break whose lines share an indentation, which the comparison's dedent strips from the template and the quoted form alike), or longer than its role's cap: 30 lines for `__requests__/`, 150 for `__expected__/` (a JSON file counted as the value it becomes). A golden compared `{ frozen: true }` stays a file, as the bytes are then the contract; one that is a byte copy of a file the repository keeps is reported to be compared with that source, and the fixer leaves it. The caps hold the other way too: a request or a body written in the test past 30 lines, an expectation past 150, is an error where it is written, and belongs in a file beside the test. `__seeds__/` has no form in place: SQL is never written in a test, and SQL tagged in one is an error where the tag stands. A seed is a `.sql` file, beside its test in `./__seeds__/` when one test reads it, never an error for being unshared. `__fixtures__/` is file state a program reads, never a value, and stays a file. Read by several tests, a file lives in the ground of the LOWEST folder its readers share, the law `setup-module-at-lowest-common-folder` holds code to: beside them when they are one leaf's, in `events/__seeds__/` when `events/list/` and `events/get/` read it, in a route's own when the route and the routes under it read it (`map/pois/` and `map/pois/detail/`), and in the specs root's shared ground (`<specs>/__seeds__/`, ...) only when they meet at the root, across specifications. An entry kept above that folder (a pool entry one specification alone reaches among them) is an error where it sits, and belongs in that folder's ground of the same role, and every reader counts where it stands: a test at its folder, a runner at its specification root, a `__setup__/` module at the folder that owns that `__setup__/`. Only a reader outside the specs tree keeps an entry where it is whatever the specs say: a module test of the member, by a relative literal, the literal pieces `named-path-exists` reads or a whole literal spelled from the member root (`'specs/...'`), and the member's Makefile or vitest config, by a path spelled from that root. `no-reach-into-sibling-ground` refuses the path that leaves a leaf for a sibling's, and `no-duplicate-ground` finds copies. A reference outside every test (a `const` of the file, a specification, a helper) counts as shared, and zero references is `no-unused-ground`'s. Fixed by `typescript fix`: SQL a test holds is written to `./__seeds__/<name>.sql` beside it (named by its first `--` comment, kebab-cased, else by the first four meaningful words of its test's title) and named with `file()`; a value past its cap is written to `./__requests__/` or `./__expected__/` beside its test, named by the test's title, and named with `file()`, never one a plain value is compared to (`expect(message).toMatch(...)`, a name or a string literal as the subject): a string has no golden, and the finding names `text()` to wrap it with; a single-use file whose one reference is a literal `file()` handed to `.request(`, `.post(`/`.put(` or `toMatch(` is written in place and deleted (a `file` import no longer used dropped, an emptied ground folder removed); an entry kept above the lowest folder its readers share is moved into that folder's ground (a `git mv` in a work tree) and the literals rewritten from each referrer. | Run `typescript fix`: it writes SQL and a value past its cap to a file beside the test, writes a single-use file into its one test and deletes it, and moves an entry kept above its readers down to the lowest folder they share. A test whose subject IS the file form keeps it, with a reasoned `checker-disable-next-line ground-shared-or-inline`. | A file a test reads is read at a distance: the reader opens a second file to know what the test stands on or expects, and a file one test reads pays that price for nothing, until the value is too long to read in the test, where the file is the clearer page. SQL is the exception at any length: a seed is database state a reader opens as a file, and SQL in a TypeScript string is checked by nothing. A single scenario's file parked in the pool is worse still: it reads as shared, so nobody dares change it. |
|
|
217
217
|
| <a id="named-path-exists"></a>`named-path-exists` | model | checker | ground | Every `file()` and `directory()` a test names with a literal path (a spec under `specs/` and a module test in its `__tests__/` alike), and every path a document's `fixture:` names, names an entry that exists inside a git work tree, one git does not ignore, of the kind the wrapper says, resolved from the test file's directory: a missing or ignored path, a `file()` naming a directory or a `directory()` naming a file is an error, reported where the path is written. So does a path a test resolves from its own directory with literal pieces alone (`resolve(import.meta.dirname, ...)`, `join(...)` alike, `__dirname` alike, `new URL('...', import.meta.url)`), of either kind: several pieces are read joined (`resolve(import.meta.dirname, '..', '__fixtures__', 'gone')`), one piece only when it is spelled `./` or `../`, as a `file()` is. A lone bare piece (`resolve(import.meta.dirname, '__expected__')`) is an output folder a test makes as often as a path it reads, and is not read. A path under the unit's build output (`dist/`, `.artifacts/<tool>/`) is accepted missing, in a spec and in its specification alike: the kind's test builds before it runs. The tree pass reads the `specs/` trees, the member pass every other test of the member. A computed piece leaves its whole path out of reach, as a computed path is, a path spelled into a command a process runs is the author's, and the runtime refuses it with the path it resolved to; a call written inside a string is text, and is not read. | Create the entry (out of `.gitignore`), or fix the path: the message names what it resolved to. | A typo in a path fails a run, a test later, far from the line that made it; statically it is one report line at the path. |
|
|
@@ -225,7 +225,7 @@ Every rule a project using the framework meets, across its enforcement channels
|
|
|
225
225
|
| <a id="setup-module-at-lowest-common-folder"></a>`setup-module-at-lowest-common-folder` | model | checker | specs | A `__setup__/` module sits in the `__setup__/` of the LOWEST folder every module importing it shares, the law ground follows (`ground-shared-or-inline`) read on imports. A reader is any module of the tree importing it by a relative path or a `package.json` `imports` alias, and a reader inside a `__setup__/` counts as the folder that `__setup__/` belongs to. Readers meeting only at the specs root share `<specs>/__setup__/`. A module one spec or test file alone imports is an error at the module: it belongs in that file, written in place, unless the two together would pass the 400 lines `spec-file-within-line-limit` holds a spec to (it then stays a file in its reader's `__setup__/`); a module under `stubs/` is excepted, since a stub is imported through its facade. A module no module of the tree imports is out of this check's reach and knip's to report, as an unused file: a config may name it (a vitest `setupFiles` entry), which no import shows. Fixed by `typescript fix`: a module off its readers' lowest folder moves to that folder's `__setup__/` (a `git mv` in a work tree), with its own relative literals, the `'..'` pieces it climbs from its own folder by (`resolve(import.meta.dirname, '..', '..')`), and every importer's path to it rewritten; one a reader reaches through an `imports` alias is left where it is, since the alias is the package's to rewrite; one a single spec reads is written into that spec by its author. | Run `typescript fix`, which moves it to the `__setup__/` the finding names and rewrites the paths that import it, or write it in its one reader. | Where shared code sits says who shares it: a helper parked above its readers reads as everyone's and one parked beside one of them reads as that leaf's, so the reader cannot trust the folder to tell them what a change reaches. |
|
|
226
226
|
| <a id="spec-code-in-setup-folder"></a>`spec-code-in-setup-folder` | model | checker | specs | Under a specs tree, a module (`.ts`, `.tsx`, `.js`, `.mjs` and their kin) that is no spec (`*.spec.ts`), no test (`*.test.ts`, `*.test.tsx`) and no runner (`*.specification.ts`) sits under a `__setup__/`: a test adapter, a shared value, a helper, a stub facade and its kind folders (`__setup__/stubs/`). Ground is out of reach, and so is a rendered unit (the `.tsx`, or the `.ts` of a hook) beside the `*.test.tsx` of the same name that renders it (`test-in-its-tests-folder`): it is that test's subject, not code that prepares it. A `*.fixtures.ts` is an error anywhere under the tree, `__setup__/` included: a fixture is input data (a file in `__fixtures__/`), so a module that prepares specs is named for what it holds. A file directly at the specs root is `specs-root-holds-specifications-and-shared`'s. | Move the module into the `__setup__/` of the lowest folder its readers share, named for what it holds, and the paths that import it. | A spec folder holds specs, and a reader listing it expects questions asked of the subject: code beside them is a second kind of file mixed in, and a module called `fixtures` promises data it does not hold. |
|
|
227
227
|
| <a id="spec-in-subject-folder"></a>`spec-in-subject-folder` | model | checker | spec | A spec file about one subject sits in the subject's standard folder, derived and never listed: `GET /events` files under `events/list/`, `GET /events/:id` under `events/get/`, `POST /events` under `events/create/`, `PUT\|PATCH /events/:id` under `events/update/`, `DELETE /events/:id` under `events/delete/`, a route under an item (`GET /events/:id/timeline`) its literal segments (`events/timeline/`), a singleton (`GET /health`) `health/get/`, a method-less route, one `jterrazz.config.ts` names so (`test: { methodless: { '/mcp': 'the MCP transport answers GET and POST' } }`), reached by whatever verb a test sends, and never split by it (`one-subject-per-spec`), its segments (`mcp/`); a job its name (`specs/job/<job>/`), a cli command its first word (`specs/cli/<word>/`), or every leading word the `commands` list of the runner its chain's handle came from holds (`specification.cli(bin, { commands: ['db', 'migrate'] })` files `db migrate` under `db/migrate/`; the list written in the call or bound to a `const` of the runner's file). A route of an app mounted under a base path `jterrazz.config.ts` declares (`test: { basePath: '/api' }`) is filed by its path past that base: `GET /api/posts` files under `posts/list/`, never `api/posts/list/`. A GET on one segment lists a plural noun, or a collection `jterrazz.config.ts` names (`test: { collections: ['data'] }`), and gets anything else; a segment is a parameter when the chain writes the route's template (a `:name` segment, or `{ params }` beside the path, `route-written-as-template`), or when it is a template hole or anything but lowercase words. A filled path whose word sits right after a collection with more path after it (`btc` in `/indicators/btc/chart`) reads two ways: it is reported at its chain, naming the template to write (or `{ params: {} }` when the word is the route's own) and never moved; a `.http` request line is read as the route. A page, a journey and the root route have no standard name, and are not judged, and neither is a spec of a specimen (`one-subject-per-spec`). An error at the file's first line; fixed by `typescript fix`, which moves the file whole, rebasing the paths of its `file()`, `directory()`, imports, `vi.mock()` and `new URL(..., import.meta.url)`, and leaving the file to its author when a relative path no rebase carries would break, and removes the folders the move leaves empty. | Run `typescript fix`: it moves the file to its subject's folder, and the ground it stands on with it. | A name derived from the subject is the same in every repository: nobody lists the folders, nobody argues them, and a reader who knows the route knows the path. |
|
|
228
|
-
| <a id="specification-folder-has-its-runner"></a>`specification-folder-has-its-runner` | model | checker | specs | A first-level folder named for one of the
|
|
228
|
+
| <a id="specification-folder-has-its-runner"></a>`specification-folder-has-its-runner` | model | checker | specs | A first-level folder named for one of the seven specifications holds a `*.specification.ts(x)` calling `specification.<specification>(`. Positive-only: any other first-level name is a repository suite, judged by `specs-at-declared-depth`'s declared depth alone. | Create the specification the folder promises, or rename the folder. | Folder = constructor is the law the fork rests on; a specification folder with no runner tells the reader something false before they open a file. |
|
|
229
229
|
| <a id="specification-test-uses-its-runner"></a>`specification-test-uses-its-runner` | model | checker | specs | A test under `specs/<specification>/`, where the specification root holds a `*.specification.ts(x)`, imports that module or constructs a runner itself. Ground is out of reach: nothing under a `__fixtures__/`, `__expected__/`, `__requests__/` or `__seeds__/` folder is a spec, a `.test.ts` under `__expected__/` being the GOLDEN a fixer writes. So is any first-level folder that is not a specification: a specification with no module to sit beside and no runner to reach lives in a repository suite there. | Import `{ <specification> }` from `<specification>.specification.js`, move a module test into the `__tests__/` of its module's folder, or move a repository suite to a first-level folder of its own under `specs/`. A module of the specs tree itself has no module test (its `__tests__/` would be a special folder the ground refuses), so it moves out of `specs/` into a code root (`src/`, or one `typescript.roots` declares) with its test. | A test reaching no runner under a specification is a module test paying the specification's budget and services to prove something that belongs in its module's `__tests__/`. |
|
|
230
230
|
| <a id="golden-path-creates-folders"></a>`golden-path-creates-folders` | model | runtime | ground | A golden's path may name a subfolder (`./__expected__/build/verbose.txt`): the path is the file, and update mode creates the folders it names. | Name the subfolder in the path. | A flat folder stops being readable at a dozen files, and a path that names its subfolder is read where it is written: no second option says where a golden lives. |
|
|
231
231
|
| <a id="folder-follows-assets"></a>`folder-follows-assets` | model | process | specs | The folder follows the assets: a test with asset directories of its own gets its own domain; tests with no local assets group as sibling `<aspect>` files. The static rule only checks the depth. | Read what the assets say: a test that stands on its own ground earns a folder. | It is the assets that settle the grouping: a criterion no channel can decide alone. |
|
|
@@ -251,7 +251,7 @@ Every rule a project using the framework meets, across its enforcement channels
|
|
|
251
251
|
| <a id="result-compared-whole"></a>`result-compared-whole` | model | static | tests | A test that reads a response, a json or a table (`<result>.response`, `<result>.json`, `<result>.table('<name>')` of a result it awaited) compares it WHOLE, once: `toMatch` handed the whole result written in place or its golden, or a table's `toBeEmpty()`. A precise assertion on one of its properties (`expect(result.json.value.total).toBe(1)`) is allowed only IN ADDITION, and the first read of a result no whole comparison reaches is an error. A negation, a RegExp and a read through a helper are no whole comparison; a result read through a helper, a computed table name, or a comparison made in a helper is out of reach, and the rule stays silent. A read whose value is only CARRIED into a later request of the same test (every use of it, through the names and helpers it passes, ends in the arguments of a chain off a handle the test awaits, in place or through the name that holds it: a path, `params`, a body, a header, as the cursor a page hands back) claims nothing and owes no comparison; one `expect(...)` of that value makes it a read again, and the result the test does claim is still compared whole. | Write `expect(<result>.<subject>).toMatch('')` and run `TEST_UPDATE=1`: the whole result is written in place; keep the precise assertion beside it if one property IS the subject. | Probes are a partial oracle: what they do not name is free to regress unseen. The whole result, written where the test reads it, is the default oracle (approval testing) and a precise assertion states what the test is about on top of it. |
|
|
252
252
|
| <a id="no-vitest-snapshot"></a>`no-vitest-snapshot` | model | upstream | tests | No vitest snapshot matcher: `toMatchSnapshot`, `toMatchInlineSnapshot`, `toMatchFileSnapshot`, `toMatchAriaSnapshot`, `toMatchScreenshot`, `toThrowErrorMatchingSnapshot` and `toThrowErrorMatchingInlineSnapshot` are refused by `vitest/no-restricted-matchers`, whose option the `test()` preset sets (ADR-005). | Write the golden: `expect(subject).toMatch(file('./__expected__/<name>.<ext>'))`; under `src/` compare to a literal with `toStrictEqual`. | A snapshot is written by the run that was supposed to be judged by it: nobody reads the diff, and a wrong answer becomes the expectation the moment someone re-records. |
|
|
253
253
|
| <a id="document-follows-its-grammar"></a>`document-follows-its-grammar` | cli | checker | document | A `<case>.spec.yaml` document follows its grammar: closed keys (`description`, `fixture`, `env`, `serve`, `runs`) and closed keys inside a run; `description:` and `runs:` required; `command:` and `exit:` required in every run, `exit:` an integer literal, `waitFor:` only on the last run and never with `stdin:`, a `files:` path relative and under the workdir; every `fixture:` path relative, reaching `__fixtures__/`. | Write the key the grammar names; the message prints the accepted set. | The grammar is read by the runner's own parser: the document lint accepts is exactly the one the runner runs. |
|
|
254
|
-
| <a id="http-first-line-follows-its-grammar"></a>`http-first-line-follows-its-grammar` | api | checker | ground | The first line of a depth-1 `.http` follows its grammar: a request line (`METHOD /path`, or `METHOD https://host/path` for a
|
|
254
|
+
| <a id="http-first-line-follows-its-grammar"></a>`http-first-line-follows-its-grammar` | api | checker | ground | The first line of a depth-1 `.http` follows its grammar: a request line (`METHOD /path`, or `METHOD https://host/path` for a deployed api) under `__requests__/`, a status line (`HTTP/1.1 <status>`, or `unreachable: <reason>` for an exchange that never answered) under `__expected__/`. | Write the opening line the folder calls for, or move the file to the other one. | The opening line is what tells a request from a response: constraining it catches misplaced files. |
|
|
255
255
|
| <a id="no-empty-expected-stream"></a>`no-empty-expected-stream` | cli | checker | document | An expected stream whose whole content is `{{any}}` is an error: the run executes and proves nothing. | Regenerate with `TEST_UPDATE=1` and keep tokens only for the parts that move. | The `vitest/expect-expect` mirror for documents: an assertion that accepts anything is an absent assertion. |
|
|
256
256
|
| <a id="no-malformed-ref"></a>`no-malformed-ref` | model | checker | ground | A malformed ref of a known kind (`{{iso8601#}}`, `{{uuid #id}}`) in a text file under `__expected__/` is an error. | Write `{{kind#name}}` with no space, and a name for the ref. | A malformed capture would fail silently: naming it makes it visible early. |
|
|
257
257
|
| <a id="no-silent-refusal"></a>`no-silent-refusal` | cli | checker | document | A run whose `exit:` is non-zero with neither `stdout:` nor `stderr:` is an error: the document does not say why the command refused. | Write the refusal the command prints: that is what the case proves. | A silent refusal is either a defect in the product, or a golden that forgot the words went to stderr. |
|
package/docs/reference/tokens.md
CHANGED
|
@@ -145,7 +145,7 @@ await expect(result.table('payment_intents', { database: 'db' })).toMatch([
|
|
|
145
145
|
- Segments already covered by a placeholder are **preserved**: a fixture line `Done in {{duration}}` stays `Done in {{duration}}` after an update, no matter what duration the run printed.
|
|
146
146
|
- Values the framework knows to be dynamic are substituted automatically: the spec's cwd becomes `{{workdir}}` in the written fixture without you doing anything.
|
|
147
147
|
- A golden that does not exist yet, or holds nothing but whitespace, is **recorded** from the output: a `.http`, a `.json` and a text golden alike, and a document's `response:` file too: a run that records stands without the goldens it is about to write. A golden is started by naming it, never by stubbing it.
|
|
148
|
-
- A recorded response golden states the status, the `content-type` and the body; a
|
|
148
|
+
- A recorded response golden states the status, the `content-type` and the body; a deployed api's (`specification.api({ url })`) states the headers its specification records (`recordedHeaders`), keeps stating no body where it stated none, and an exchange that never answered is written as its one line, `unreachable: <reason>` ([API specs § Deployed target (`url`)](../guides/api-specs.md#deployed-target-url)).
|
|
149
149
|
|
|
150
150
|
Example, starting from actual output:
|
|
151
151
|
|