@jterrazz/test 16.1.0 → 17.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +45 -171
- package/dist/browser/index.js +2 -2867
- package/dist/browser/services.js +2 -0
- package/dist/browser/stubs.js +2 -0
- package/dist/checker.d.ts +6 -1
- package/dist/checker.js +4551 -2612
- package/dist/config.d.ts +40 -0
- package/dist/config.js +309 -0
- package/dist/in-place.global-setup.d.ts +11 -0
- package/dist/in-place.global-setup.js +21 -0
- package/dist/index.d.ts +1653 -3002
- package/dist/index.js +5226 -3845
- package/dist/mobile-session.global-setup.d.ts +14 -0
- package/dist/mobile-session.global-setup.js +31 -0
- package/dist/oxlint.cjs +6073 -5616
- package/dist/oxlint.d.cts +43 -7212
- package/dist/oxlint.d.ts +43 -7212
- package/dist/oxlint.js +6065 -5609
- package/dist/route-coverage.global-setup.d.ts +42 -0
- package/dist/route-coverage.global-setup.js +2 -0
- package/dist/services.d.ts +3 -0
- package/dist/services.global-setup.d.ts +43 -0
- package/dist/services.global-setup.js +2 -0
- package/dist/services.js +597 -0
- package/dist/shared-appium.adapter.js +428 -0
- package/dist/shared-artifacts.js +32 -0
- package/dist/shared-chapter.js +7 -0
- package/dist/shared-container.port.d.ts +31 -0
- package/dist/shared-derived-projects.js +58 -0
- package/dist/shared-device.port.d.ts +189 -0
- package/dist/shared-exchange.js +447 -0
- package/dist/shared-in-place.js +172 -0
- package/dist/shared-index.browser.js +3191 -0
- package/dist/shared-index.services.d.ts +346 -0
- package/dist/shared-index.stubs.d.ts +287 -0
- package/dist/shared-json-shape.js +17 -0
- package/dist/shared-landmarks.js +71 -0
- package/dist/shared-ledger.js +68 -0
- package/dist/shared-literate-plugin.js +183 -0
- package/dist/shared-manifest.js +1321 -0
- package/dist/shared-pause.js +11 -0
- package/dist/shared-peer.js +88 -0
- package/dist/shared-playwright.adapter.js +330 -0
- package/dist/shared-process.adapter.js +74 -0
- package/dist/shared-refusal.js +35 -0
- package/dist/shared-resolve.js +24 -0
- package/dist/shared-route-coverage.global-setup.js +187 -0
- package/dist/shared-scenario-failure.js +198 -0
- package/dist/shared-serve.js +365 -0
- package/dist/shared-service-images.js +10 -0
- package/dist/shared-services.global-setup.js +179 -0
- package/dist/shared-session-ledger.js +301 -0
- package/dist/shared-setup-entry.js +12 -0
- package/dist/shared-source-scan.js +133 -0
- package/dist/shared-spec-document.js +365 -0
- package/dist/shared-specifications.js +26 -0
- package/dist/shared-stream-body.js +130 -0
- package/dist/shared-stub-server.js +190 -0
- package/dist/shared-temp-root.global-setup.js +80 -0
- package/dist/shared-test-baseline.js +26 -0
- package/dist/shared-test-section.js +58 -0
- package/dist/shared-testcontainers.adapter.js +203 -0
- package/dist/shared-tokens.js +55 -0
- package/dist/shared-yaml-document.js +119 -0
- package/dist/stubs.d.ts +2 -0
- package/dist/stubs.js +657 -0
- package/dist/temp-root.global-setup.d.ts +7 -0
- package/dist/temp-root.global-setup.js +2 -0
- package/dist/vitest.d.ts +97 -163
- package/dist/vitest.js +374 -507
- package/docs/README.md +82 -36
- package/docs/concepts/architecture.md +118 -0
- package/docs/concepts/conventions.md +333 -0
- package/docs/guides/api-specs.md +374 -0
- package/docs/guides/cli-specs.md +564 -0
- package/docs/guides/component-specs.md +252 -0
- package/docs/guides/develop.md +94 -0
- package/docs/guides/integration-specs.md +181 -0
- package/docs/guides/job-specs.md +180 -0
- package/docs/guides/lint-specs.md +174 -0
- package/docs/guides/mobile-specs.md +319 -0
- package/docs/guides/module-tests.md +149 -0
- package/docs/guides/release.md +43 -0
- package/docs/guides/spec-documents.md +189 -0
- package/docs/guides/website-specs.md +413 -0
- package/docs/reference/assertions.md +345 -0
- package/docs/reference/glossary.md +20 -0
- package/docs/reference/linting.md +483 -0
- package/docs/reference/services.md +236 -0
- package/docs/reference/stubs.md +498 -0
- package/docs/reference/time-and-doubles.md +137 -0
- package/docs/reference/tokens.md +258 -0
- package/docs/reference/visitor.md +221 -0
- package/docs/start/getting-started.md +187 -0
- package/docs/start/set-up-a-project.md +214 -0
- package/package.json +50 -30
- package/schema/spec.schema.json +43 -17
- package/dist/appium.adapter.js +0 -489
- package/dist/artifacts.js +0 -32
- package/dist/fixtures.js +0 -110
- package/dist/literate.js +0 -1747
- package/dist/match.js +0 -804
- package/dist/playwright.adapter.js +0 -349
- package/dist/server.js +0 -160
- package/dist/stream.js +0 -321
- package/dist/substring-warning.js +0 -244
- package/docs/01-architecture.md +0 -95
- package/docs/02-developing.md +0 -351
- package/docs/03-testing.md +0 -495
- package/docs/04-operating.md +0 -101
- package/docs/05-module-tests.md +0 -152
- package/docs/06-integration.md +0 -180
- package/docs/07-component.md +0 -246
- package/docs/08-website.md +0 -404
- package/docs/09-mobile.md +0 -270
- package/docs/10-api.md +0 -264
- package/docs/11-jobs.md +0 -157
- package/docs/12-cli.md +0 -677
- package/docs/13-elements.md +0 -242
- package/docs/14-assertions.md +0 -314
- package/docs/15-tokens.md +0 -253
- package/docs/16-contracts.md +0 -438
- package/docs/17-services.md +0 -188
- package/docs/18-conventions.md +0 -305
- package/docs/19-linting.md +0 -510
- package/docs/decisions/001-v9-conventions-on-three-channels.md +0 -50
- package/docs/decisions/002-the-statique-channel-holds-only-what-upstream-does-not.md +0 -49
- package/docs/decisions/003-a-rendered-component-is-a-facet-on-vitest-browser-mode.md +0 -84
- package/docs/decisions/004-a-module-against-real-services-or-a-golden-is-the-integration-facet.md +0 -81
- package/docs/decisions/005-the-doubles-ladder-is-closed.md +0 -86
- package/docs/decisions/006-the-catalogue-reaches-a-file-by-its-role.md +0 -245
- package/docs/decisions/007-compose-mode-leaves-the-package.md +0 -53
- package/docs/decisions/008-vitest-5-node-24-and-the-native-seams-as-optional-peers.md +0 -75
- package/docs/decisions/_template.md +0 -22
- package/docs/reference/classes/Matcher.md +0 -120
- package/docs/reference/classes/ProcessHandle.md +0 -368
- package/docs/reference/functions/defineContract.md +0 -39
- package/docs/reference/functions/defineContracts.md +0 -36
- package/docs/reference/functions/disabled.md +0 -28
- package/docs/reference/functions/enabled.md +0 -20
- package/docs/reference/functions/focused.md +0 -26
- package/docs/reference/functions/mockOf.md +0 -79
- package/docs/reference/functions/postgres.md +0 -31
- package/docs/reference/functions/process.md +0 -33
- package/docs/reference/functions/redis.md +0 -28
- package/docs/reference/functions/required.md +0 -41
- package/docs/reference/functions/selected.md +0 -26
- package/docs/reference/functions/sqlite.md +0 -33
- package/docs/reference/functions/testId.md +0 -20
- package/docs/reference/functions/text.md +0 -40
- package/docs/reference/functions/valued.md +0 -28
- package/docs/reference/functions/waitUntil.md +0 -46
- package/docs/reference/functions/within.md +0 -29
- package/docs/reference/index.md +0 -196
- package/docs/reference/interfaces/BaseResult.md +0 -87
- package/docs/reference/interfaces/CallResult.md +0 -160
- package/docs/reference/interfaces/CaptureScope.md +0 -82
- package/docs/reference/interfaces/CliResult.md +0 -253
- package/docs/reference/interfaces/ContainerAccessor.md +0 -166
- package/docs/reference/interfaces/DirectoryAccessor.md +0 -33
- package/docs/reference/interfaces/FetchResult.md +0 -180
- package/docs/reference/interfaces/FilesystemAccessor.md +0 -46
- package/docs/reference/interfaces/HttpResult.md +0 -126
- package/docs/reference/interfaces/JsonAccessor.md +0 -46
- package/docs/reference/interfaces/PageResult.md +0 -356
- package/docs/reference/interfaces/RenderResult.md +0 -72
- package/docs/reference/interfaces/RenderedText.md +0 -110
- package/docs/reference/interfaces/ResponseAccessor.md +0 -47
- package/docs/reference/interfaces/ScreenResult.md +0 -128
- package/docs/reference/interfaces/TableAccessor.md +0 -26
- package/docs/reference/interfaces/TextAccessor.md +0 -96
- package/docs/reference/type-aliases/ApiHandle.md +0 -68
- package/docs/reference/type-aliases/ApiSpecification.md +0 -213
- package/docs/reference/type-aliases/ApiSpecificationOptions.md +0 -68
- package/docs/reference/type-aliases/BrowserConsoleMessage.md +0 -30
- package/docs/reference/type-aliases/BrowserLinkElement.md +0 -50
- package/docs/reference/type-aliases/BrowserMetaElement.md +0 -40
- package/docs/reference/type-aliases/BrowserOpenOptions.md +0 -87
- package/docs/reference/type-aliases/BrowserPage.md +0 -132
- package/docs/reference/type-aliases/BrowserPort.md +0 -51
- package/docs/reference/type-aliases/CliEnv.md +0 -11
- package/docs/reference/type-aliases/CliHandle.md +0 -80
- package/docs/reference/type-aliases/CliInput.md +0 -36
- package/docs/reference/type-aliases/CliOutput.md +0 -46
- package/docs/reference/type-aliases/CliPort.md +0 -58
- package/docs/reference/type-aliases/CliSpecification.md +0 -144
- package/docs/reference/type-aliases/CliSpecificationOptions.md +0 -134
- package/docs/reference/type-aliases/ComponentChain.md +0 -156
- package/docs/reference/type-aliases/ComponentScenario.md +0 -20
- package/docs/reference/type-aliases/ComponentUi.md +0 -47
- package/docs/reference/type-aliases/ComponentVisitor.md +0 -10
- package/docs/reference/type-aliases/ContainerPort.md +0 -113
- package/docs/reference/type-aliases/Contract.md +0 -68
- package/docs/reference/type-aliases/ContractInput.md +0 -13
- package/docs/reference/type-aliases/ContractRequest.md +0 -95
- package/docs/reference/type-aliases/ContractResponder.md +0 -23
- package/docs/reference/type-aliases/ContractResponse.md +0 -78
- package/docs/reference/type-aliases/ContractResponseValue.md +0 -13
- package/docs/reference/type-aliases/Contracts.md +0 -48
- package/docs/reference/type-aliases/DatabaseKeys.md +0 -16
- package/docs/reference/type-aliases/DatabasePort.md +0 -72
- package/docs/reference/type-aliases/DeviceOpenOptions.md +0 -46
- package/docs/reference/type-aliases/DevicePort.md +0 -51
- package/docs/reference/type-aliases/DeviceScreen.md +0 -32
- package/docs/reference/type-aliases/DeviceTimeouts.md +0 -39
- package/docs/reference/type-aliases/DockerSpecConfig.md +0 -43
- package/docs/reference/type-aliases/DomMount.md +0 -21
- package/docs/reference/type-aliases/ElementKind.md +0 -27
- package/docs/reference/type-aliases/ElementMatch.md +0 -74
- package/docs/reference/type-aliases/ElementOptions.md +0 -29
- package/docs/reference/type-aliases/ElementRef.md +0 -128
- package/docs/reference/type-aliases/ExecOptions.md +0 -37
- package/docs/reference/type-aliases/FileAccessor.md +0 -54
- package/docs/reference/type-aliases/HonoApp.md +0 -31
- package/docs/reference/type-aliases/HttpContractFilter.md +0 -49
- package/docs/reference/type-aliases/HttpResponseInit.md +0 -46
- package/docs/reference/type-aliases/HttpStreamInit.md +0 -58
- package/docs/reference/type-aliases/IntegrationHandle.md +0 -46
- package/docs/reference/type-aliases/IntegrationSpecification.md +0 -111
- package/docs/reference/type-aliases/IntegrationSpecificationOptions.md +0 -42
- package/docs/reference/type-aliases/Intercept.md +0 -19
- package/docs/reference/type-aliases/InterceptOptions.md +0 -28
- package/docs/reference/type-aliases/InterceptScope.md +0 -11
- package/docs/reference/type-aliases/IsolationStrategy.md +0 -75
- package/docs/reference/type-aliases/JobHandle.md +0 -34
- package/docs/reference/type-aliases/JobsHandle.md +0 -57
- package/docs/reference/type-aliases/JobsResult.md +0 -16
- package/docs/reference/type-aliases/JobsSpecification.md +0 -94
- package/docs/reference/type-aliases/JobsSpecificationOptions.md +0 -56
- package/docs/reference/type-aliases/LandmarkKind.md +0 -20
- package/docs/reference/type-aliases/LiterateRunFlags.md +0 -27
- package/docs/reference/type-aliases/LiterateServeRegistration.md +0 -70
- package/docs/reference/type-aliases/MatchFixtureOptions.md +0 -27
- package/docs/reference/type-aliases/MatchableRequest.md +0 -61
- package/docs/reference/type-aliases/MatcherKind.md +0 -34
- package/docs/reference/type-aliases/MobileBackendOptions.md +0 -28
- package/docs/reference/type-aliases/MobileElementKind.md +0 -14
- package/docs/reference/type-aliases/MobileElementMatch.md +0 -60
- package/docs/reference/type-aliases/MobileElementRef.md +0 -20
- package/docs/reference/type-aliases/MobileHandle.md +0 -65
- package/docs/reference/type-aliases/MobileScenario.md +0 -20
- package/docs/reference/type-aliases/MobileSpecification.md +0 -51
- package/docs/reference/type-aliases/MobileSpecificationOptions.md +0 -126
- package/docs/reference/type-aliases/MobileVisitor.md +0 -81
- package/docs/reference/type-aliases/MockOfOptions.md +0 -26
- package/docs/reference/type-aliases/MockPort.md +0 -10
- package/docs/reference/type-aliases/PinnedClock.md +0 -10
- package/docs/reference/type-aliases/PostgresOptions.md +0 -32
- package/docs/reference/type-aliases/ProcessOptions.md +0 -112
- package/docs/reference/type-aliases/RedisOptions.md +0 -20
- package/docs/reference/type-aliases/RenderSubject.md +0 -12
- package/docs/reference/type-aliases/ScreenNode.md +0 -67
- package/docs/reference/type-aliases/ServerPort.md +0 -36
- package/docs/reference/type-aliases/ServerResponse.md +0 -46
- package/docs/reference/type-aliases/ServerSpec.md +0 -21
- package/docs/reference/type-aliases/ServiceHandle.md +0 -216
- package/docs/reference/type-aliases/ServiceRecord.md +0 -12
- package/docs/reference/type-aliases/SpecDocument.md +0 -84
- package/docs/reference/type-aliases/SpecEnvToken.md +0 -21
- package/docs/reference/type-aliases/SpecFileAssertion.md +0 -27
- package/docs/reference/type-aliases/SpecFixture.md +0 -30
- package/docs/reference/type-aliases/SpecKind.md +0 -8
- package/docs/reference/type-aliases/SpecRun.md +0 -108
- package/docs/reference/type-aliases/SpecServeEntry.md +0 -40
- package/docs/reference/type-aliases/SpecStream.md +0 -32
- package/docs/reference/type-aliases/SpecificationConfig.md +0 -290
- package/docs/reference/type-aliases/SqliteOptions.md +0 -36
- package/docs/reference/type-aliases/SseEvent.md +0 -58
- package/docs/reference/type-aliases/StreamBody.md +0 -61
- package/docs/reference/type-aliases/TextFilter.md +0 -22
- package/docs/reference/type-aliases/VisitScenario.md +0 -20
- package/docs/reference/type-aliases/Visitor.md +0 -215
- package/docs/reference/type-aliases/WaitUntilOptions.md +0 -46
- package/docs/reference/type-aliases/WebsiteBackendOptions.md +0 -37
- package/docs/reference/type-aliases/WebsiteHandle.md +0 -51
- package/docs/reference/type-aliases/WebsiteSpecification.md +0 -118
- package/docs/reference/type-aliases/WebsiteSpecificationOptions.md +0 -40
- package/docs/reference/variables/anthropic.md +0 -19
- package/docs/reference/variables/banner.md +0 -10
- package/docs/reference/variables/button.md +0 -10
- package/docs/reference/variables/clock.md +0 -16
- package/docs/reference/variables/complementary.md +0 -10
- package/docs/reference/variables/component.md +0 -10
- package/docs/reference/variables/content.md +0 -10
- package/docs/reference/variables/contentinfo.md +0 -10
- package/docs/reference/variables/dialog.md +0 -10
- package/docs/reference/variables/field.md +0 -10
- package/docs/reference/variables/form.md +0 -10
- package/docs/reference/variables/heading.md +0 -10
- package/docs/reference/variables/http.md +0 -40
- package/docs/reference/variables/intercept.md +0 -11
- package/docs/reference/variables/link.md +0 -10
- package/docs/reference/variables/listitem.md +0 -10
- package/docs/reference/variables/main.md +0 -10
- package/docs/reference/variables/match.md +0 -8
- package/docs/reference/variables/navigation.md +0 -10
- package/docs/reference/variables/openai.md +0 -21
- package/docs/reference/variables/option.md +0 -17
- package/docs/reference/variables/region.md +0 -10
- package/docs/reference/variables/row.md +0 -10
- package/docs/reference/variables/search.md +0 -10
- package/docs/reference/variables/specification.md +0 -8
- package/docs/reference/variables/status.md +0 -10
- package/docs/reference/variables/table.md +0 -10
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
# Spec documents (`<case>.spec.yaml`)
|
|
2
|
+
|
|
3
|
+
A terminal session that is data is written as data: one `<case>.spec.yaml` per test, beside the spec it belongs to, stating what the session stands on, the commands it runs and what each left behind. The format is the cli specification's; an api or a job case is a TypeScript spec, its whole result written in place ([Assertions](../reference/assertions.md#one-verb-compares-a-whole-result-and-its-argument-says-where-the-expectation-lives)). This page owns the format, from the key a document opens with to the door that runs it.
|
|
4
|
+
|
|
5
|
+
- [One file, one test](#one-file-one-test): the unit, and the golden of the whole session
|
|
6
|
+
- [Paths](#paths): `fixture:` names the disk the way `file()` and `directory()` do
|
|
7
|
+
- [A cli document](#a-cli-document): the ground, the runs, what they left behind
|
|
8
|
+
- [The schema and the format](#the-schema-and-the-format)
|
|
9
|
+
- [The three doors, one runner](#the-three-doors-one-runner): the plugin, the bridge, the chain
|
|
10
|
+
- [Failure and update](#failure-and-update) · [When to reach for code](#when-to-reach-for-code)
|
|
11
|
+
|
|
12
|
+
## One file, one test
|
|
13
|
+
|
|
14
|
+
A document is ONE test: its `description:` is the vitest title, and nothing in it lists cases. There is no `cases:` key, no `extends:` and no `kind:`. A case inheriting its given from another file is a given no reader sees, and a document describes a cli session and nothing else. What several cases share is composed visibly instead: `fixture:` takes one path or a list, applied in order, so the pool's base and the leaf's delta both stand on the line (`fixture: [../../__fixtures__/shop/, ./__fixtures__/sold-out/]`).
|
|
15
|
+
|
|
16
|
+
**The assertion is the whole session**: every run's exit code and both its streams, byte for byte, `files:` for what it left on disk. Nothing here is new capability: a document is the chain's semantics in the shape of a data file an editor validates: `fixture:` is `.fixture()`, a run is `.exec()`. The keys, in their canonical order, are `description`, `fixture`, `env`, `serve`, `runs`; `description:` and `runs:` are mandatory.
|
|
17
|
+
|
|
18
|
+
## Paths
|
|
19
|
+
|
|
20
|
+
`fixture:` takes a path relative to the document, spelled in full (`./...`, `../...`), and is judged by the four guards a `file()` call is (chapter [Assertions](../reference/assertions.md#one-verb-compares-a-whole-result-and-its-argument-says-where-the-expectation-lives)):
|
|
21
|
+
|
|
22
|
+
- **Role:** a path whose first ground folder is not `__fixtures__/` is refused by the grammar, on its line, so the checker (`document-follows-its-grammar`) and the runner refuse the same document (`verb-reads-its-ground-folder`'s law, for data).
|
|
23
|
+
- **Locality:** a path leaves its leaf only by climbing: into the ground of a folder that encloses it, or the specs root's shared `<specs>/__fixtures__/` (`no-reach-into-sibling-ground`).
|
|
24
|
+
- **Existence:** every path names an entry git does not ignore, of the kind it names, a directory with its trailing `/` and a file without (`named-path-exists`); at run time a missing one is refused with what it resolved to.
|
|
25
|
+
- **Liveness:** a fixture a document names is referenced, exactly as a `file()` literal references it (`no-unused-ground`, `ground-shared-or-inline`).
|
|
26
|
+
|
|
27
|
+
## A cli document
|
|
28
|
+
|
|
29
|
+
A terminal session is already a specification: a command, its exit code, what it printed, what it left on disk. A cli document writes it as that, never under `__expected__/`, which holds goldens, not scenarios.
|
|
30
|
+
|
|
31
|
+
```yaml
|
|
32
|
+
description: refuses to guess when it is run outside the checkout
|
|
33
|
+
fixture: ../../__fixtures__/repositories-stub/
|
|
34
|
+
env:
|
|
35
|
+
- frozen
|
|
36
|
+
- SHOPLY_ORIGIN=http://127.0.0.1:9
|
|
37
|
+
serve:
|
|
38
|
+
- mcp: { MCP_STUB_WITHHOLD: get-article }
|
|
39
|
+
|
|
40
|
+
runs:
|
|
41
|
+
- command: repositories
|
|
42
|
+
exit: 1
|
|
43
|
+
stderr: |
|
|
44
|
+
Error: no directory with home/ and apps/ above the current directory
|
|
45
|
+
Hint: run inside the shoply checkout
|
|
46
|
+
- command: repositories --json
|
|
47
|
+
exit: 0
|
|
48
|
+
stdout: |
|
|
49
|
+
{
|
|
50
|
+
"data": []
|
|
51
|
+
}
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
### The ground
|
|
55
|
+
|
|
56
|
+
| Key | Shape | Meaning |
|
|
57
|
+
| ---------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
58
|
+
| `fixture:` | string or list | The two forms the chain's `file()` and `directory()` take; the trailing `/` says which: `./__fixtures__/shop/` spreads a directory's contents, `./__fixtures__/config.toml` copies one file under its own name; a path whose `/` disagrees with the disk is refused. Executable bits preserved. A list **layers** in order |
|
|
59
|
+
| `env:` | string or list | Each entry is either a **bare word** naming an env set registered in code, or `KEY=value` inline (`$WORKDIR` expands, as in `.env()`) |
|
|
60
|
+
| `serve:` | string, list of strings, or `- name: { ... }` | Servers registered in code, started once **per file** before the first run and shared by all of them. The mapping form adds env to that server |
|
|
61
|
+
|
|
62
|
+
The words `env:` and `serve:` name are registered once per app, in the options of `specification.cli()` ([CLI specs § Registration](cli-specs.md#registration-once-per-app)).
|
|
63
|
+
|
|
64
|
+
### The runs
|
|
65
|
+
|
|
66
|
+
`runs:` is a list of at least one run, executed **in order**, all in ONE working directory, and **every** one asserted. This is where the format goes past `.exec([...])`, which stops at the first non-zero exit and keeps only the last output: here a non-zero exit does not end the session, because each exit code is part of what the document states.
|
|
67
|
+
|
|
68
|
+
**A run is judged the moment it ends**, before the next command starts: its exit code, its streams and its `files:` all read at that one instant. A run that disagrees throws there and then, and the runs below it do not execute: once the sequence has diverged, what the rest would print is a consequence, not a fact about the program.
|
|
69
|
+
|
|
70
|
+
| Key | Shape | Meaning |
|
|
71
|
+
| ---------- | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
72
|
+
| `command:` | **mandatory** | The argv AFTER the binary, exactly what `.exec('...')` takes. The binary is the runner's (`specification.cli(bin)`), so naming it here would pass it to itself as the first argument. The line goes through the shell, so quoting behaves identically |
|
|
73
|
+
| `stdin:` | optional block scalar | Written to the child, which then reads EOF. Absent keeps the immediately-closed pipe; a TTY is never given |
|
|
74
|
+
| `timeout:` | optional integer | Milliseconds before the run is killed, its whole process group with it (exit code 124) |
|
|
75
|
+
| `waitFor:` | optional, **last run only** | Long-running form: resolve as soon as this text appears, as `.exec(args, { waitFor })` does |
|
|
76
|
+
| `exit:` | **mandatory** literal integer | The exit code the command must return |
|
|
77
|
+
| `stdout:` | optional block scalar | Expected stdout, **byte-exact**. Absent asserts an EMPTY stream |
|
|
78
|
+
| `stderr:` | optional block scalar | Expected stderr, byte-exact. Absent asserts an EMPTY stream |
|
|
79
|
+
| `files:` | optional mapping | On-disk assertions under the working directory, read the moment THIS run ends (below) |
|
|
80
|
+
|
|
81
|
+
**`|` and `|-` are the whole newline story.** A block scalar written `|` keeps the final newline of its text; `|-` drops it. The comparison is byte-exact against that, so a command whose output ends with `\n` is written `|`, and one that ends mid-line is written `|-`.
|
|
82
|
+
|
|
83
|
+
### `files:`: what the run left behind
|
|
84
|
+
|
|
85
|
+
Each key is a path relative to the working directory (never absolute, never escaping it). Each value is one of four forms:
|
|
86
|
+
|
|
87
|
+
```yaml
|
|
88
|
+
files:
|
|
89
|
+
.spwn/agent.yaml: { contains: 'name: bot' } # one needle, or a list of them
|
|
90
|
+
out/report.txt:
|
|
91
|
+
equals: | # exact text, block scalar allowed
|
|
92
|
+
ok
|
|
93
|
+
.spwn/lock: absent # nothing at that path
|
|
94
|
+
out/src: exists # something at that path
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`contains` and `equals` are token-aware: `{ contains: 'wrote {{path}}' }` finds the line whatever the path turned out to be. Each run's mapping is read against the working directory **as that run left it**, so a file may be `absent` for one run and there for the next, which is how a lock, a cache or a build output is described:
|
|
98
|
+
|
|
99
|
+
```yaml
|
|
100
|
+
description: the lock appears with the first apply and is released by the second
|
|
101
|
+
runs:
|
|
102
|
+
- command: status
|
|
103
|
+
exit: 0
|
|
104
|
+
files:
|
|
105
|
+
.spwn/lock: absent
|
|
106
|
+
- command: apply --hold
|
|
107
|
+
exit: 0
|
|
108
|
+
files:
|
|
109
|
+
.spwn/lock: { contains: 'pid: {{int}}' }
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
## The schema and the format
|
|
113
|
+
|
|
114
|
+
The document's JSON Schema ships with the package at `schema/spec.schema.json`, published as the `@jterrazz/test/schema` export, and is generated from the grammar's own constants, so it cannot describe a shape the parser does not read. It closes a document on its keys and states every presence law of the grammar (the keys a mapping cannot do without, the keys that cannot stand together) from `PRESENCE`, the one table the parser refuses with. A meta-test holds the other direction: a JSON Schema validator runs it over every document of this repository, each one the parser reads, it validates, and each one the parser refuses, it refuses (except for what no keyword compares: a `waitFor:` on a run before the last, where the schema holds only that one run at most waits), and it breaks every presence law, for both readers alike. Point an editor at it:
|
|
115
|
+
|
|
116
|
+
```yaml
|
|
117
|
+
# yaml-language-server: $schema=./node_modules/@jterrazz/test/schema/spec.schema.json
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
A schema says which keys exist, not which ORDER they come in, nor how a stream is written. Both are the document's format, and a formatter owns it, as oxfmt owns a source file's: the canonical order (`description, fixture, env, serve, runs` at the top, then `command, stdin, timeout, waitFor, exit, stdout, stderr, files` in a run) and every stream (`stdout`, `stderr`, `stdin`, `files.*.equals`) as a block scalar, `|` keeping the trailing newline and `|-` dropping it. `typescript fix` (`typescript fix`) writes it; a check run names each document out of it and fails, and no baseline holds one: it is never debt, since the formatter always settles it. The document family of rules is in [Linting](../reference/linting.md).
|
|
121
|
+
|
|
122
|
+
The whole [token vocabulary](../reference/tokens.md) works in what a document ASSERTS (a run's streams and `files:` texts) and **never** in `description`, `command` or `stdin`, which are prose and inputs.
|
|
123
|
+
|
|
124
|
+
## The three doors, one runner
|
|
125
|
+
|
|
126
|
+
| Door | Use it when |
|
|
127
|
+
| ---------- | --------------------------------------------------------------------------------------------- |
|
|
128
|
+
| **plugin** | The document IS the test, the common case |
|
|
129
|
+
| **bridge** | The document is the scenario, but one assertion needs code |
|
|
130
|
+
| **chain** | Everything the format cannot say; see [when to reach for code](#when-to-reach-for-code) below |
|
|
131
|
+
|
|
132
|
+
**The plugin.** `literate()` from `@jterrazz/test/vitest` adds the `.spec.yaml` glob to a project's include and transforms each document into a one-test module that runs it through the bound specification's `cli`. The runner shows the document's path as the test file and `description:` as the title, so a failing case opens where it is written. The `cli()` project helper wires it:
|
|
133
|
+
|
|
134
|
+
| `literate` | Collects |
|
|
135
|
+
| -------------------------- | ----------------------------------------------------------------------------- |
|
|
136
|
+
| omitted | every `specs/cli/**/*.spec.yaml`, bound to `./specs/cli/cli.specification.ts` |
|
|
137
|
+
| `{ runner: './other.ts' }` | the same documents, bound to another runner |
|
|
138
|
+
| `{ include: [...] }` | a narrower glob |
|
|
139
|
+
| `false` | no documents at all, a suite written entirely in code |
|
|
140
|
+
|
|
141
|
+
The `runner` is **stated, never guessed** beyond that convention: a repository may declare several cli runners, and nothing could pick the right one without silently binding a case to the wrong app. The door is the `literate` option of the one `cli` project that collects those documents.
|
|
142
|
+
|
|
143
|
+
**The bridge.** `cli.run(file('./<case>.spec.yaml'))` runs a document from a spec and resolves with the LAST run's result, so code adds what the document cannot express.
|
|
144
|
+
|
|
145
|
+
```typescript
|
|
146
|
+
test('scaffolds a shop and leaves the tree we expect', async () => {
|
|
147
|
+
// Given - the whole session, stated in the document
|
|
148
|
+
const result = await cli.run(file('./scaffold.spec.yaml'));
|
|
149
|
+
|
|
150
|
+
// Then - one assertion the format has no vocabulary for
|
|
151
|
+
await expect(result.directory('my-shop')).toMatch(directory('./__expected__/shop-scaffold/'));
|
|
152
|
+
});
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
`.run()` is a DOOR, not a fallback: it stays for a document whose result needs one assertion the grammar has no vocabulary for, or for a document the plugin's glob cannot collect: generated at test time, or an input under `__fixtures__/` whose refusal is the subject of a negative spec. `{ frozen: true }` opts one document out of the update rewrite, the guard `toMatch(file(...), { frozen: true })` is for a golden.
|
|
156
|
+
|
|
157
|
+
## Failure and update
|
|
158
|
+
|
|
159
|
+
On a mismatch the failure renders the description, the command that ran, everything the comparison rejected (the exit code, the stream diff, the `files:` entry), and where to open the document, at the run's own line.
|
|
160
|
+
|
|
161
|
+
A line the comparison ACCEPTED is rendered as equal, with its token, so the marked lines are the mismatch and nothing else. The stack carries one frame: the document's own line.
|
|
162
|
+
|
|
163
|
+
`TEST_UPDATE=1` rewrites what the run produced and nothing else: each run's `exit`, `stdout` and `stderr`. The ground, the commands and `files:` are stated by hand and never touched, and neither are comments or key order: the YAML document is edited, not re-emitted. Placeholders survive by **pattern match**, so a token stays a token wherever the line moved to: see [update mode](../reference/tokens.md#update-mode-tokens-are-preserved).
|
|
164
|
+
|
|
165
|
+
## When to reach for code
|
|
166
|
+
|
|
167
|
+
The format states one subject and one linear story. Reach for the chain when the spec needs:
|
|
168
|
+
|
|
169
|
+
- **logic:** a loop, a computed value, reasoning across two calls the document cannot relate;
|
|
170
|
+
- **containers:** `docker`-aware runs, `await using`, `result.container(name)`;
|
|
171
|
+
- **parallel fan-out:** several commands that must run at once, or an interleaving;
|
|
172
|
+
- **host shell-outs** the binary under test is not: a `git` call, a build step staged by hand;
|
|
173
|
+
- **structural JSON of a cli run:** `result.json` compared whole to its value;
|
|
174
|
+
- **a refusal of the chain itself:** a directory handed as a body, a database the record lacks.
|
|
175
|
+
|
|
176
|
+
A document that starts wanting a conditional is a chain wearing a header. Write it in code.
|
|
177
|
+
|
|
178
|
+
## Pitfalls
|
|
179
|
+
|
|
180
|
+
- **Putting a `<case>.spec.yaml` under `__expected__/`.** A document is the CASE, not a golden; it lives beside the spec it belongs to (`no-document-in-ground`).
|
|
181
|
+
- **Writing a token in a `description:`, a `command:` or a `stdin:`.** Those are prose and inputs; placeholders work in what is asserted.
|
|
182
|
+
- **Quoting a stream instead of writing a block scalar.** `stdout: "a\nb\n"` is a golden no diff can show. It is out of [the format](#the-schema-and-the-format), and `typescript fix` rewrites it.
|
|
183
|
+
- **A `fixture:` naming a golden's folder.** It reads `__fixtures__/`; `./__expected__/shop/` is refused on its line.
|
|
184
|
+
- **Listing cases in one file, or inheriting a given.** One file is one test, and the given is visible: a list of paths layers, `extends:` does not exist.
|
|
185
|
+
- **Writing an api or a job case as a document.** The format is a cli session's; an api or job case is a spec, its whole result in place; `kind:` is an unknown key.
|
|
186
|
+
|
|
187
|
+
## Related
|
|
188
|
+
|
|
189
|
+
[CLI specs](cli-specs.md) · [Assertions](../reference/assertions.md) · [Tokens](../reference/tokens.md) · [Linting](../reference/linting.md)
|
|
@@ -0,0 +1,413 @@
|
|
|
1
|
+
# Website specs (`specification.website`)
|
|
2
|
+
|
|
3
|
+
`specification.website()` tests a rendered website (its raw HTTP surface: redirects, robots.txt, headers; and its rendered surface: title, head metadata, JSON-LD, console, and full user scenarios) through a real Chromium. It starts the site itself, or targets one already running.
|
|
4
|
+
|
|
5
|
+
Use it when the subject under test is a browser-rendered page. For a JSON/HTTP API surface use [api](api-specs.md); for a binary use [cli](cli-specs.md).
|
|
6
|
+
|
|
7
|
+
## What it specifies
|
|
8
|
+
|
|
9
|
+
A website spec answers one question: **given this backend and this visitor, what does the served site put on the screen, and what does it report to the console?** The subject is the assembled product met through an address (a running server the runner started, or a deployment already up), and everything it reads is what a visitor can see: the rendered text, the accessibility tree, the document head, the console. A single rendered unit inside that page is a component spec ([Component specs](component-specs.md)); a route module or a loader behind it is a module test ([Module tests](module-tests.md)).
|
|
10
|
+
|
|
11
|
+
## The constructor
|
|
12
|
+
|
|
13
|
+
Exactly one of `server` (start the site locally) or `url` (target a running site) is required; passing both, or neither, throws immediately.
|
|
14
|
+
|
|
15
|
+
```typescript
|
|
16
|
+
// specs/website/website.specification.ts
|
|
17
|
+
import { specification } from '@jterrazz/test';
|
|
18
|
+
import { afterAll } from 'vitest';
|
|
19
|
+
|
|
20
|
+
export const { cleanup, website } = await specification.website({
|
|
21
|
+
server: { command: 'npm run start', ready: '/' },
|
|
22
|
+
});
|
|
23
|
+
|
|
24
|
+
afterAll(cleanup);
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
`command` is how you start the site in production or in preview (`npm run start`, `next start`, `astro preview`), run from the project root, and `ready` is a path the site answers once it is up, usually `/`. The command reads its port from `PORT` ([below](#options)).
|
|
28
|
+
|
|
29
|
+
```typescript
|
|
30
|
+
// targeting a deployed preview instead of starting one
|
|
31
|
+
export const { cleanup, website } = await specification.website({
|
|
32
|
+
url: 'https://preview-1234.my-site.pages.dev',
|
|
33
|
+
});
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
### Options
|
|
37
|
+
|
|
38
|
+
| Option | Description |
|
|
39
|
+
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
40
|
+
| `server` | How the site is started locally: a `process()`, the `ProcessOptions` object itself, or `(services) => ...` returning either. Exactly one of `server` / `url` |
|
|
41
|
+
| `url` | Target an already-running site (deployed, preview, dev server). Exactly one of `server` / `url` |
|
|
42
|
+
| `services` | A named record started BEFORE the site and stopped with it: a database it reads, a `process()` backend it calls. See [Services beside the site](#services-beside-the-site) |
|
|
43
|
+
| `backend` | `{ env, port? }`: start a declared stub backend and inject its URL into the server child. Requires `server` mode; see [Declared backend](#declared-backend) |
|
|
44
|
+
| `external` | `'allow' \| 'block'`: cross-origin policy for `.visit()`. Default `'block'` with `server`, `'allow'` with `url`; see [Cross-origin policy](#cross-origin-policy-external) |
|
|
45
|
+
| `root` | **Project-root override** (rule `no-redundant-root`): the cwd of the `server` command. Auto-discovered from the calling file when absent. Not a fixtures root |
|
|
46
|
+
| `transform` | **Escape hatch only** (rule `no-transform-duplicating-tokens`): a normaliser applied to every compared reading (the head projection and the json-ld among them) before the comparison, for framework noise no token can name. It never mutates what the page said |
|
|
47
|
+
|
|
48
|
+
`server` is a `ProcessOptions`, whatever of the three forms states it: `command`, `ready`, `port`, `cwd`, `env`, `before`, `timeout`. That shape has one owner, and it is not this page: [Services § `process()`](../reference/services.md#process-the-one-shape-an-external-process-takes) holds every field, because a website's server is an external process like any other the framework owns.
|
|
49
|
+
|
|
50
|
+
The chosen port is injected as `PORT`; the command reads it the same way it would in production. If the process never answers on `ready` within `timeout`, or exits first, `specification.website()` fails with the command's captured output attached. On teardown the child is ended the way every `process()` is ([Services § `process()`](../reference/services.md#process-the-one-shape-an-external-process-takes)), its own children with it.
|
|
51
|
+
|
|
52
|
+
The handle destructures to `{ website, cleanup, url }` (rule `no-runner-alias`), no `docker`: a browser is not a container. `url` is the resolved base URL: the one the server started on, or the `url` option with its trailing slash trimmed.
|
|
53
|
+
|
|
54
|
+
### Services beside the site
|
|
55
|
+
|
|
56
|
+
A site under test is rarely alone: it reads a database, or it calls an API that must be up before the first page is requested. `services` is that record (the same one every other specification takes, see [Services](../reference/services.md)), started before the site, in declaration order, and stopped with the specification.
|
|
57
|
+
|
|
58
|
+
That is why `server` may be a FUNCTION of the record: the site is handed the URL of the thing it was started beside, resolved once that thing is listening rather than guessed at config time. The example (a `next dev` beside the API it calls and that API's database) is [Services § `process()`](../reference/services.md#process-the-one-shape-an-external-process-takes)'s.
|
|
59
|
+
|
|
60
|
+
A declared database is not seeded by the chain; a website chain has no `.seed()`. It is seeded by whatever owns it: the service's own `docker/<service>/init.sql`, or a `before:` command on the `process()` that migrates it.
|
|
61
|
+
|
|
62
|
+
## The chain
|
|
63
|
+
|
|
64
|
+
### Two terminal actions: `.fetch()` and `.visit()`
|
|
65
|
+
|
|
66
|
+
#### `.fetch(path, { method })`: one raw HTTP exchange
|
|
67
|
+
|
|
68
|
+
`.fetch(path)` performs a single GET, and `.fetch(path, { method: 'POST' })` another method (`DELETE`, `HEAD`, `OPTIONS`, `PATCH`, `PUT`), and never follows redirects: the redirect itself is the result, not something to chase:
|
|
69
|
+
|
|
70
|
+
```typescript
|
|
71
|
+
test('surfaces a permanent redirect without following it', async () => {
|
|
72
|
+
// Given - the legacy path
|
|
73
|
+
const result = await website.fetch('/old');
|
|
74
|
+
|
|
75
|
+
// Then - the 308 IS the result, with its target readable
|
|
76
|
+
expect(result.status).toBe(308);
|
|
77
|
+
expect(result.location).toBe('/');
|
|
78
|
+
});
|
|
79
|
+
|
|
80
|
+
test('serves robots.txt as plain text', async () => {
|
|
81
|
+
// Given - the robots surface
|
|
82
|
+
const result = await website.fetch('/robots.txt');
|
|
83
|
+
|
|
84
|
+
// Then - the whole file matches one golden
|
|
85
|
+
expect(result.headers['content-type']).toBe('text/plain');
|
|
86
|
+
expect(result.body).toMatch(file('./__expected__/robots.txt'));
|
|
87
|
+
});
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
`FetchResult` accessors:
|
|
91
|
+
|
|
92
|
+
| Member | Type | Description |
|
|
93
|
+
| ----------------- | --------------------- | -------------------------------------------------------------------------- |
|
|
94
|
+
| `result.status` | `number` | HTTP status; 3xx surfaces exactly as sent, never followed |
|
|
95
|
+
| `result.location` | `string \| undefined` | The `location` header, or undefined |
|
|
96
|
+
| `result.headers` | flat map | Response headers, lower-cased keys |
|
|
97
|
+
| `result.body` | `TextAccessor` | Raw response body: `toMatch(file('./__expected__/robots.txt'))`, `.grep()` |
|
|
98
|
+
| `result.json` | `JsonAccessor` | Response body parsed as JSON |
|
|
99
|
+
|
|
100
|
+
#### `.visit(path, scenario?)`: a rendered page
|
|
101
|
+
|
|
102
|
+
`.visit()` renders the page in a real chromium and resolves with the captured document. There is **one browser process per vitest worker**, shared by every runner the worker hosts and launched lazily on the first `.visit()`, so a spec file that only calls `.fetch()` never pays the browser-launch cost. It is Playwright's headless shell, named explicitly, the lighter of its two Chromium builds. Each visit gets a fresh, isolated browser context, and records no video and no trace: a scenario that fails leaves one screenshot ([Evidence on failure](#evidence-on-failure)).
|
|
103
|
+
|
|
104
|
+
```typescript
|
|
105
|
+
test('captures the full head surface of a rendered page', async () => {
|
|
106
|
+
// Given - the fixture homepage
|
|
107
|
+
const result = await website.visit('/');
|
|
108
|
+
|
|
109
|
+
// Then - one golden covers title, canonical, alternates, and metas
|
|
110
|
+
expect(result.status).toBe(200);
|
|
111
|
+
expect(result.head).toMatch(file('./__expected__/home.head.json'));
|
|
112
|
+
});
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
### Visit scenarios: the When
|
|
116
|
+
|
|
117
|
+
A scenario is the interaction that happens **before** the capture, the visit's When. The capture always reflects the **final** page state, after the scenario ran:
|
|
118
|
+
|
|
119
|
+
```typescript
|
|
120
|
+
test('subscribes through the form and captures the final state', async () => {
|
|
121
|
+
// Given - a visitor on the homepage
|
|
122
|
+
const result = await website.visit('/', async (visitor) => {
|
|
123
|
+
// When - they fill the form and subscribe
|
|
124
|
+
await visitor.fill(field('Email'), 'visitor@site.test');
|
|
125
|
+
await visitor.click(button('Subscribe'));
|
|
126
|
+
await visitor.see(content('Thanks for subscribing'));
|
|
127
|
+
});
|
|
128
|
+
|
|
129
|
+
// Then - the capture reflects the page after the interaction
|
|
130
|
+
expect(result.content.text).toContain('Thanks for subscribing');
|
|
131
|
+
expect(result.errors.text).toBe('');
|
|
132
|
+
});
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
**No `expect()` inside a scenario (rule `no-expect-in-scenario`).** A scenario is pure interaction; assertions live in the Then, on the returned result. Splitting interaction from assertion keeps the grammar of setup, then action, then result intact, and keeps scenarios replayable independent of what they're checked against.
|
|
136
|
+
|
|
137
|
+
The visitor's verbs and the words it names elements with are [The visitor](../reference/visitor.md)'s, whole: `click` `fill` `press` `select` `check` `hover` `goto` `see` `gone`, the targets and the landmarks, the `focused` / `disabled` / `selected` / `valued` modifiers, `within()`, exact names and `testId()`. One vocabulary, three surfaces: this page adds none of its own and redefines none of them. Two things are website-specific and stated there: `goto(path)` navigates within the site under test, and the ARIA landmarks are this specification's and the component specification's, never the device's.
|
|
138
|
+
|
|
139
|
+
Designating exactly one element is rule `target-matches-one-element`, and the refusal it prints (the candidates, the accessible name each matched on, and what each suggested fix LEAVES) is [The visitor § Designating exactly one element](../reference/visitor.md#designating-exactly-one-element).
|
|
140
|
+
|
|
141
|
+
Navigating within a scenario changes what the capture describes:
|
|
142
|
+
|
|
143
|
+
```typescript
|
|
144
|
+
test('navigates to another page and captures where it landed', async () => {
|
|
145
|
+
// Given - a visitor on the homepage
|
|
146
|
+
const result = await website.visit('/', async (visitor) => {
|
|
147
|
+
// When - they follow the articles link
|
|
148
|
+
await visitor.click(link('Articles'));
|
|
149
|
+
});
|
|
150
|
+
|
|
151
|
+
// Then - the result is the destination page
|
|
152
|
+
expect(result.url).toContain('/articles');
|
|
153
|
+
expect(result.content.text).toContain('All articles');
|
|
154
|
+
});
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
### Setup: `.headers()`
|
|
158
|
+
|
|
159
|
+
`.headers({...})` sets HTTP headers for both terminal actions: the raw `.fetch()` exchange and the browser context behind `.visit()`. Repeated calls merge. The main use case is a User-Agent override, e.g. asserting on what an AI crawler sees:
|
|
160
|
+
|
|
161
|
+
```typescript
|
|
162
|
+
test('sends chain headers on the raw exchange', async () => {
|
|
163
|
+
// Given - an AI crawler user agent
|
|
164
|
+
const result = await website.headers({ 'User-Agent': 'GPTBot/1.0' }).fetch('/robots.txt');
|
|
165
|
+
|
|
166
|
+
// Then - the exchange succeeds like any other client
|
|
167
|
+
expect(result.status).toBe(200);
|
|
168
|
+
expect(result.body.text).toContain('Allow: /');
|
|
169
|
+
});
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
### Setup: `website.with({ clock, headers })`
|
|
173
|
+
|
|
174
|
+
The clock and the headers a whole FILE shares are stated once, on a view of the specification whose every chain starts with them (`const french = website.with({ headers: { 'Accept-Language': 'fr' } })`), and a chain's own setup merges over it ([Conventions](../concepts/conventions.md#what-a-file-shares)).
|
|
175
|
+
|
|
176
|
+
### Setup: `.clock()`
|
|
177
|
+
|
|
178
|
+
`.clock('2026-03-04T09:30:00Z')` pins the calendar of the PAGE (what the site's own scripts read when they call `new Date()`) before the first byte is parsed, so a stamp rendered on load is the stated instant and not the moment the navigation happened to start. It is released with the visit.
|
|
179
|
+
|
|
180
|
+
```typescript
|
|
181
|
+
test('stamps the moment the page was opened', async () => {
|
|
182
|
+
// Given - the page's calendar pinned for this visit
|
|
183
|
+
const result = await website.clock('2026-03-04T09:30:00Z').visit('/clock');
|
|
184
|
+
|
|
185
|
+
// Then - the rendered stamp is the stated instant
|
|
186
|
+
expect(result.content.text).toContain('2026-03-04T09:30:00.000Z');
|
|
187
|
+
});
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
A `.fetch()` opens no page, so it has no clock to pin: the chain refuses the pairing rather than ignoring it. Assert the moment of a raw exchange with a `{{iso8601}}` token in its golden. The primitive behind the setup is [Time and doubles § The clock](../reference/time-and-doubles.md#the-clock).
|
|
191
|
+
|
|
192
|
+
`.clock()` pins the page, never the SERVER: a window the server computes with its own clock (a feed of the last hour, a token's expiry) is read off the real one. Declare the stub relative to now, so the window holds whenever the spec runs, `http.json({ publishedAt: new Date(Date.now() - 60_000).toISOString() })`, and assert the stamp the page renders by shape, capturing it with a token, `{{iso8601#published}}`, which every later occurrence in the spec must equal; the instant itself is never written.
|
|
193
|
+
|
|
194
|
+
## The result
|
|
195
|
+
|
|
196
|
+
| Member | Type | Description |
|
|
197
|
+
| ------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
|
198
|
+
| `result.status` | `number` | HTTP status of the main document response |
|
|
199
|
+
| `result.url` | `string` | Final URL, after redirects and any scenario navigation |
|
|
200
|
+
| `result.title` | `TextAccessor` | `document.title` |
|
|
201
|
+
| `result.head` | `JsonAccessor` | `{ title, canonical, alternates, metas }`: the one-golden-per-page SEO surface |
|
|
202
|
+
| `result.jsonLd` | `JsonAccessor` | Every `application/ld+json` block, parsed, as one array |
|
|
203
|
+
| `result.meta(name)` | `string \| undefined` | Content of a named meta: `.meta('description')`, `.meta('og:image')` |
|
|
204
|
+
| `result.canonical` | `string \| null` | The canonical `<link>` href, or null |
|
|
205
|
+
| `result.alternates` | flat map | Hreflang alternates, keyed by language code |
|
|
206
|
+
| `result.links` | array | `<link>` elements of the head, in DOM order |
|
|
207
|
+
| `result.content` | `TextAccessor` | Rendered body text (`document.body.innerText`) |
|
|
208
|
+
| `result.html` | `TextAccessor` | Full serialized DOM (`document.documentElement.outerHTML`) |
|
|
209
|
+
| `result.console` | `TextAccessor` | Every console message, one `[type] text` line per message, and every uncaught exception as an `[error] Uncaught ...` line |
|
|
210
|
+
| `result.errors` | `TextAccessor` | Console errors only, plus any uncaught page error (`Uncaught Error: ...`) |
|
|
211
|
+
| `result.tree` | `TextAccessor` | The ARIA snapshot of the rendered `<body>`: the page's outline, as a golden |
|
|
212
|
+
|
|
213
|
+
### `result.tree`: the outline, as the accessibility tree draws it
|
|
214
|
+
|
|
215
|
+
```typescript
|
|
216
|
+
test('lays the article out as a reader walks it', async () => {
|
|
217
|
+
// Given - the fixture article page
|
|
218
|
+
const result = await website.visit('/articles/hexagonal');
|
|
219
|
+
|
|
220
|
+
// Then - the outline is what a screen reader would walk
|
|
221
|
+
expect(result.tree).toMatch(file('./__expected__/article.aria.yaml'));
|
|
222
|
+
});
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
It is produced by the same Playwright `ariaSnapshot()` the component specification calls, so a page's outline and a component's outline are the same kind of golden and comparable to each other. Deterministic where a screenshot is not, and it says what the markup MEANS rather than what it is made of.
|
|
226
|
+
|
|
227
|
+
### The `head` golden: one per page
|
|
228
|
+
|
|
229
|
+
`result.head` is the **stable, assertion-friendly projection** of the document head: title, canonical, hreflang alternates, and named metas collapsed into one object. It is the one golden a page needs for its SEO surface:
|
|
230
|
+
|
|
231
|
+
```typescript
|
|
232
|
+
test('exposes canonical, alternates, and named metas directly', async () => {
|
|
233
|
+
// Given - the fixture homepage
|
|
234
|
+
const result = await website.visit('/');
|
|
235
|
+
|
|
236
|
+
// Then - the accessors read the head without a golden...
|
|
237
|
+
expect(result.canonical).toBe('https://site.test/');
|
|
238
|
+
expect(result.alternates['x-default']).toBe('https://site.test/');
|
|
239
|
+
expect(result.meta('og:title')).toBe('Fixture: Home');
|
|
240
|
+
|
|
241
|
+
// ...or snapshot the whole surface at once
|
|
242
|
+
expect(result.head).toMatch(file('./__expected__/home.head.json'));
|
|
243
|
+
});
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
**A meta the FRAMEWORK owns is dropped by the runner, not absorbed by the golden.** A view-transitions router writes its own metas into every head; a golden that swallows them says the site declares something it never did, and every page's golden grows the same two lines. `transform` is the door: the same option `specification.cli()` has, for the same reason (rule `no-transform-duplicating-tokens`):
|
|
247
|
+
|
|
248
|
+
```typescript
|
|
249
|
+
export const { cleanup, website } = await specification.website({
|
|
250
|
+
server: { command: 'npm run preview', ready: '/' },
|
|
251
|
+
// the router's own metas, which no `{{token}}` can name
|
|
252
|
+
transform: (text) => text.replaceAll(/,?"astro-view-transitions-[a-z]+":\s*"[^"]*"/gu, ''),
|
|
253
|
+
});
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
It applies to every COMPARED reading of a result (the head projection, the json-ld, the content, the console), so it answers for whatever it is handed; what the page said is still read whole through `meta()`, `canonical`, `alternates` and the rest. Prefer a `{{token}}` wherever the noise has a shape the grammar already names.
|
|
257
|
+
|
|
258
|
+
Structured data gets the same treatment: every `ld+json` block on the page, in one golden:
|
|
259
|
+
|
|
260
|
+
```typescript
|
|
261
|
+
test('parses every json-ld block into one array', async () => {
|
|
262
|
+
const result = await website.visit('/');
|
|
263
|
+
|
|
264
|
+
expect(result.jsonLd).toMatch(file('./__expected__/home.jsonld.json'));
|
|
265
|
+
});
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
### Console assertions
|
|
269
|
+
|
|
270
|
+
The console splits into the full stream and the error-only stream, the same shape as `stdout`/`stderr` on a cli result:
|
|
271
|
+
|
|
272
|
+
```typescript
|
|
273
|
+
test('keeps a clean page silent on both streams', async () => {
|
|
274
|
+
// Given - the healthy homepage
|
|
275
|
+
const result = await website.visit('/');
|
|
276
|
+
|
|
277
|
+
// Then - no console output at all
|
|
278
|
+
expect(result.console.text).toBe('');
|
|
279
|
+
expect(result.errors.text).toBe('');
|
|
280
|
+
});
|
|
281
|
+
|
|
282
|
+
test('separates console errors from the full stream', async () => {
|
|
283
|
+
// Given - a page that logs and errors
|
|
284
|
+
const result = await website.visit('/noisy');
|
|
285
|
+
|
|
286
|
+
// Then - the full stream carries both, the error stream only the error
|
|
287
|
+
expect(result.console).toMatch(file('./__expected__/noisy.console.txt'));
|
|
288
|
+
expect(result.errors.text).toContain('boom');
|
|
289
|
+
});
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
## Unique here
|
|
293
|
+
|
|
294
|
+
### Declared backend
|
|
295
|
+
|
|
296
|
+
A site under test usually talks to an API. The `backend` option starts a small **stub backend** (plain `node:http`, no extra dependency) BEFORE the server command and injects its URL into the server child's environment under `backend.env`; the site reads it the same way it would in production:
|
|
297
|
+
|
|
298
|
+
```typescript
|
|
299
|
+
// specs/website/website.specification.ts
|
|
300
|
+
export const { cleanup, website } = await specification.website({
|
|
301
|
+
server: { command: 'npm run start', ready: '/' },
|
|
302
|
+
backend: { env: 'API_URL' },
|
|
303
|
+
});
|
|
304
|
+
|
|
305
|
+
afterAll(cleanup);
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
| `backend` field | Description |
|
|
309
|
+
| --------------- | -------------------------------------------------------------------------------- |
|
|
310
|
+
| `env` | Env var receiving the stub's URL in the server child (e.g. `'API_URL'`) |
|
|
311
|
+
| `port` | Fixed port; pins a stable stub URL across runs. Default: a free OS-assigned port |
|
|
312
|
+
|
|
313
|
+
`backend` requires `server` mode: with `url` it refuses (the type already forbids the combination): a deployed site cannot be pointed at a local stub. The **ownership boundary**: the framework owns the server child, so it injects the env var itself, and that is the whole wiring.
|
|
314
|
+
|
|
315
|
+
What the stub serves is declared per chain, as [stubs](../reference/stubs.md): the feature's `__setup__/stubs/` facade, exactly the form `api`/`job` use:
|
|
316
|
+
|
|
317
|
+
```typescript
|
|
318
|
+
import newsroom from './__setup__/stubs/newsroom.stubs.js';
|
|
319
|
+
|
|
320
|
+
describe('declared backend', () => {
|
|
321
|
+
test('renders the events feed from the declared backend', async () => {
|
|
322
|
+
// Given - the backend under stub for this chain
|
|
323
|
+
const result = await website.intercept(newsroom).visit('/events');
|
|
324
|
+
|
|
325
|
+
// Then - the page rendered what the stub declared
|
|
326
|
+
expect(result.content.text).toContain('Jazz night at the harbour');
|
|
327
|
+
expect(result.errors.text).toBe('');
|
|
328
|
+
});
|
|
329
|
+
});
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
The stub backend **resets between chains** the way databases do: one chain = one terminal action, and its stubs replace the previous chain's wholesale. Selection is the shared one: the first declared stub that matches and is not exhausted wins, and a stub with no `times` is unlimited, so a page re-fetching the same endpoint (re-render, retry) replays it.
|
|
333
|
+
|
|
334
|
+
Strictness is the analog of `external: 'block'`: a request matching no declared stub is answered **501** (a JSON body naming the path and listing the declared routes) and **recorded**; when the `.visit()` completes, the action **throws** an error enumerating every unmatched request (method, path, count). The failure evidence (screenshot on a scenario error) is captured first, as always. A chain with **zero** intercepts leaves the stub unguarded, the same boundary as MSW never mounting without an `.intercept()`.
|
|
335
|
+
|
|
336
|
+
Two pieces of plumbing are handled for you:
|
|
337
|
+
|
|
338
|
+
- **CORS:** the stub answers the `OPTIONS` preflight and stamps permissive `Access-Control-Allow-*` headers on every response, so client-side fetches from the site's origin just work.
|
|
339
|
+
- **`external: 'block'`:** the stub's origin is allow-listed automatically; declared-backend fetches are never aborted as third-party noise.
|
|
340
|
+
|
|
341
|
+
### Storage
|
|
342
|
+
|
|
343
|
+
`.storage({ local, session })` fills `localStorage` and `sessionStorage` before the page's first script runs, so a page that reads a preference on load sees it; `.storage('throws')` makes every read of either store throw the `SecurityError` a browser that blocks site data throws. A later call replaces the earlier, and a key the page already holds is kept, since the script runs again on each navigation. A chain ending in `.fetch()` is refused: no page opens, so nothing could read the storage.
|
|
344
|
+
|
|
345
|
+
```ts
|
|
346
|
+
const result = await website.storage({ local: { theme: 'dark' } }).visit('/settings');
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
### Cross-origin policy (`external`)
|
|
350
|
+
|
|
351
|
+
`external: 'block'` aborts every request leaving the site under test during a `.visit()`: analytics beacons, third-party CDNs, ad scripts never fire, so a visit stays deterministic. `'allow'` lets them through.
|
|
352
|
+
|
|
353
|
+
The default follows the mode: `'block'` with `server` (you own the deployment, third-party noise is not the point), `'allow'` with `url` (a deployed site legitimately loads third-party assets). Override with the top-level `external` option when a spec needs the opposite of its mode's default.
|
|
354
|
+
|
|
355
|
+
A request the policy aborts is the runner's doing, not the site's: the browser's console error for it (`Failed to load resource`) stays out of `.errors`, and one for a request the policy let through stays in.
|
|
356
|
+
|
|
357
|
+
### Evidence on failure
|
|
358
|
+
|
|
359
|
+
When a scenario throws (an element never becomes visible, a `see()` times out), the error carries a full-page screenshot of the state the scenario died in, referenced by its path in the error message. It is written under `.artifacts/vitest/evidence/`, in the folder of the run that took it, so it outlives the run and its temp root, until the next run starts, which sweeps every run's evidence but a living one's. The original error is never masked; the screenshot is attached evidence, not a replacement.
|
|
360
|
+
|
|
361
|
+
### Playwright: the optional peer
|
|
362
|
+
|
|
363
|
+
Playwright is not a hard dependency: `.fetch()`-only spec files never need it. `.visit()` imports it lazily, and needs it installed:
|
|
364
|
+
|
|
365
|
+
```bash
|
|
366
|
+
npm install -D playwright && npx playwright install chromium
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
Calling `.visit()` without playwright installed throws exactly that guidance; there is nothing else to search for.
|
|
370
|
+
|
|
371
|
+
Provisioning the environment is not this package's job. In CI, the shared validate workflow does it: `browsers: true` (jterrazz-actions) installs cached chromium, versioned from the caller's lockfile, between Build and Test. On a workstation, `j install` provisions `playwright-browsers` once.
|
|
372
|
+
|
|
373
|
+
### Folder layout
|
|
374
|
+
|
|
375
|
+
```text
|
|
376
|
+
specs/website/
|
|
377
|
+
├── website.specification.ts # runner at the specification ROOT (rule `specs-at-declared-depth`)
|
|
378
|
+
└── <domain>/
|
|
379
|
+
├── <aspect>.spec.ts
|
|
380
|
+
├── __expected__/ # ALL expected fixtures, FLAT (*.head.json, *.jsonld.json, *.console.txt, ...)
|
|
381
|
+
└── __setup__/stubs/ # the stubs declaring what the backend serves, with the `backend` option
|
|
382
|
+
├── newsroom.stubs.ts
|
|
383
|
+
└── http/...
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
No `__seeds__/` or `__requests__/`: a website chain has no `.seed()` setup and no request-file format, whatever its runner declares under `services`; `.fetch()`/`.visit()` calls are inline, and the golden is always `__expected__/<name>`.
|
|
387
|
+
|
|
388
|
+
**Where a page's golden goes.** A website spec reads its subject page by page, one spec file per page, and a golden read by one file is that file's: written in place while it is short (`ground-shared-or-inline`), else beside its one reader. A domain folder may not nest (`specs-at-declared-depth`), so a domain that would hold one page's golden among many pages' specs becomes a domain of its own (jterrazz-web's `metadata/`, `sitemap/`, `feed/` and `not-found/` sit beside `journeys/` and `redirects/`), and a domain of one spec per surface keeps each golden with its one reader, in place up to the cap and in the domain's `__expected__/` past it.
|
|
389
|
+
|
|
390
|
+
**A subject and its oracle are derived together, or pinned together.** A spec that visits a path a helper computes ("the newest article", "a school project") and compares it with a golden pinning that page's literal values fails against a page nobody chose the day the corpus moves the helper's pick. Either derive both (the path and the expectation from the same subject) or pin both: a literal path and its golden. A derived subject with a pinned golden is the one pairing to refuse in review.
|
|
391
|
+
|
|
392
|
+
**A table of pages is the one sanctioned exception, and the checker cannot see it.** `one-subject-per-spec` refuses a spec visiting several pages by literal paths; a `test.each` whose rows feed `visit()` computed paths passes both it and `no-unused-ground` unseen: the checker reads literals, not tables. A site that keeps one such table (every redirect rule, every legacy slug) declares it in its own test guide, prints each row as a line (`status source -> destination`) and compares the whole table with a value written in place, never a mapping of the source's own table into the expectation, which restates the code and lets a wrong row pass.
|
|
393
|
+
|
|
394
|
+
**The production build belongs to the site's gate, not to each spec.** `server.command` starts what the build left (`astro preview`, `next start`); the Makefile's `test: build` (or a stamp file the build touches, rebuilt when a source is newer) owns rebuilding it, so no website spec walks the source tree for mtimes before it starts.
|
|
395
|
+
|
|
396
|
+
## Pitfalls
|
|
397
|
+
|
|
398
|
+
- **Passing both `server` and `url`, or neither.** The options type makes the invalid combinations inexpressible; the compiler rejects them before anything runs.
|
|
399
|
+
- **Using `expect()` inside a scenario callback.** Forbidden (rule `no-expect-in-scenario`): the scenario is the When; assertions belong on the result the `.visit()` promise resolves to.
|
|
400
|
+
- **Reaching for `testId()` first, or to escape an ambiguity.** It is the last rung, and `test-id-states-what-is-missing` refuses it without the line naming what the element lacks; an ambiguity is scoped with `within()`, `target-matches-one-element`'s fix (see [The visitor § `testId()`](../reference/visitor.md#testid-the-one-escape-hatch)).
|
|
401
|
+
- **Using `see()` to prove something went away.** It cannot: an element that never appeared and one that disappeared read the same to it. `gone(element)` is the absence primitive.
|
|
402
|
+
- **Snapshotting the tree to prove focus.** The tree carries none; [The visitor § Modifiers](../reference/visitor.md#modifiers) says what does, and what a golden may pin instead.
|
|
403
|
+
- **Calling `.visit()` without playwright installed.** The error names the exact fix (`npm install -D playwright && npx playwright install chromium`); there is no silent fallback.
|
|
404
|
+
- **Expecting `.fetch()` to follow redirects.** It never does: the 3xx status and `location` header ARE the result; chase the target with a second `.fetch()` if the spec needs to.
|
|
405
|
+
- **Assuming `external` defaults the same way in both modes.** It flips with the constructor mode: `'block'` with `server`, `'allow'` with `url`. Pass it explicitly to override.
|
|
406
|
+
- **Reading `result.head` field-by-field instead of snapshotting it.** It is designed as the one golden per page (`toMatch(file('./__expected__/home.head.json'))`); use the direct accessors (`canonical`, `alternates`, `meta()`) only for a single targeted probe.
|
|
407
|
+
- **Calling `.intercept()` without the `backend` option.** It throws immediately, naming the fix: add `backend: { env: '...' }` to the `specification.website()` options; a chain's `.http` exchanges need a stub to serve them.
|
|
408
|
+
- **Combining `backend` with `url` mode.** The type forbids it and the runtime refuses: a deployed site reads its API URL at its own deploy time; there is nothing a local stub could inject into.
|
|
409
|
+
- **A site that loads `next/font/google`.** Its build downloads the fonts: the suite becomes network-bound, and fails offline or behind a proxy. Load the family from a file in the repository (`next/font/local`) so the suite needs no network.
|
|
410
|
+
|
|
411
|
+
## Related
|
|
412
|
+
|
|
413
|
+
[Set up a project](../start/set-up-a-project.md) · [Assertions](../reference/assertions.md) · [Tokens](../reference/tokens.md) · [Mobile specs](mobile-specs.md) · [Component specs](component-specs.md)
|