@crvouga/atlas 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/README.md +30 -11
- package/dist/core/src/cli.d.ts +1 -1
- package/dist/core/src/cli.js +8 -3
- package/dist/core/src/config.d.ts +7 -0
- package/dist/core/src/config.js +48 -14
- package/dist/core/src/run/execute.d.ts +6 -0
- package/dist/core/src/run/execute.js +4 -2
- package/dist/reports/src/doctor.d.ts +1 -1
- package/dist/reports/src/doctor.js +3 -1
- package/dist/reports/src/http-store.d.ts +4 -0
- package/dist/reports/src/http-store.js +6 -3
- package/dist/reports/src/init.js +6 -1
- package/dist/reports/src/live-reporter.d.ts +34 -0
- package/dist/reports/src/live-reporter.js +122 -0
- package/dist/reports/src/report-cli.d.ts +1 -1
- package/dist/reports/src/report-cli.js +3 -2
- package/dist/reports/src/store-url.js +1 -1
- package/dist/runners/src/codecept.d.ts +14 -0
- package/dist/runners/src/codecept.js +16 -0
- package/dist/runners/src/jest.d.ts +50 -0
- package/dist/runners/src/jest.js +52 -0
- package/dist/runners/src/mocha.d.ts +40 -0
- package/dist/runners/src/mocha.js +39 -0
- package/dist/runners/src/node-test.d.ts +28 -0
- package/dist/runners/src/node-test.js +44 -0
- package/dist/runners/src/publish.d.ts +30 -0
- package/dist/runners/src/publish.js +68 -0
- package/dist/schema/src/api.d.ts +1 -0
- package/dist/schema/src/api.js +1 -1
- package/dist/schema/src/index.d.ts +2 -0
- package/dist/schema/src/index.js +2 -0
- package/dist/schema/src/live-run.d.ts +123 -0
- package/dist/schema/src/live-run.js +30 -0
- package/dist/schema/src/manifest.d.ts +2 -2
- package/dist/schema/src/project.d.ts +20 -0
- package/dist/schema/src/project.js +10 -0
- package/dist/schema/src/report-sources.d.ts +1 -0
- package/dist/schema/src/report-sources.js +2 -1
- package/dist/schema/src/runs-index.d.ts +1 -1
- package/dist/schema/src/server-config.d.ts +84 -0
- package/dist/schema/src/server-config.js +12 -0
- package/dist/server/src/app.d.ts +10 -0
- package/dist/server/src/app.js +148 -10
- package/dist/server/src/config.d.ts +1 -1
- package/dist/server/src/config.js +20 -3
- package/dist/server/src/dev.d.ts +3 -3
- package/dist/server/src/dev.js +22 -13
- package/dist/server/src/events.d.ts +3 -3
- package/dist/server/src/events.js +1 -1
- package/dist/server/src/index.d.ts +2 -0
- package/dist/server/src/index.js +2 -0
- package/dist/server/src/live-runs.d.ts +52 -0
- package/dist/server/src/live-runs.js +229 -0
- package/dist/server/src/main.d.ts +1 -1
- package/dist/server/src/main.js +7 -5
- package/dist/server/src/openapi.d.ts +2 -5
- package/dist/server/src/openapi.js +71 -7
- package/dist/server/src/projects.d.ts +3 -0
- package/dist/server/src/projects.js +154 -0
- package/dist/server/src/settings-page.d.ts +9 -0
- package/dist/server/src/settings-page.js +93 -0
- package/dist/server/src/settings.d.ts +23 -0
- package/dist/server/src/settings.js +72 -0
- package/dist/server/src/site.d.ts +8 -6
- package/dist/server/src/site.js +45 -10
- package/dist/server/src/store-api.js +43 -22
- package/docs/deploying.md +19 -19
- package/docs/dogfooding.md +2 -2
- package/docs/extending.md +3 -3
- package/docs/integrating.md +9 -9
- package/docs/reference/cli.md +9 -7
- package/docs/reference/core.md +6 -0
- package/docs/reference/http-api.md +498 -2
- package/docs/reference/reports.md +55 -1
- package/docs/reference/runners.md +169 -0
- package/docs/reference/schema.md +111 -0
- package/docs/reference/server.md +154 -14
- package/docs/report-backends.md +8 -8
- package/docs/report-contract.md +7 -7
- package/docs/reporting-issues.md +3 -3
- package/docs/runners.md +69 -0
- package/docs/server.md +61 -25
- package/docs/statecharts.md +22 -20
- package/llms-full.txt +713 -105
- package/llms.txt +3 -1
- package/package.json +31 -9
- package/packages/runners/README.md +6 -0
- package/packages/runners/compat/codecept.cjs +2 -0
- package/packages/runners/compat/jest.cjs +2 -0
- package/packages/runners/compat/mocha.cjs +2 -0
- package/packages/schema/schemas/atlas-change.schema.json +4 -0
- package/packages/schema/schemas/hello-event.schema.json +4 -0
- package/packages/schema/schemas/live-run-start.schema.json +42 -0
- package/packages/schema/schemas/live-run-update.schema.json +93 -0
- package/packages/schema/schemas/live-run.schema.json +225 -0
- package/packages/schema/schemas/project-slug.schema.json +8 -0
- package/packages/schema/schemas/project.schema.json +22 -0
- package/packages/schema/schemas/projects.schema.json +33 -0
- package/packages/schema/schemas/server-config.schema.json +14 -0
- package/packages/schema/schemas/server-editable-settings.schema.json +132 -0
- package/packages/schema/schemas/server-settings-snapshot.schema.json +285 -0
- package/packages/server/openapi.json +5101 -779
- package/packages/visualizer/README.md +13 -13
- package/packages/visualizer/bin/atlas-client.mjs +176 -0
- package/packages/visualizer/bin/atlas-visualizer.mjs +2 -175
- package/packages/visualizer/src/app/AppShell.tsx +26 -17
- package/packages/visualizer/src/app/report-sources.tsx +2 -1
- package/packages/visualizer/src/components/ActivityIndicator.module.css +4 -0
- package/packages/visualizer/src/components/ActivityIndicator.tsx +6 -0
- package/packages/visualizer/src/components/Header.module.css +101 -227
- package/packages/visualizer/src/components/Header.tsx +79 -41
- package/packages/visualizer/src/components/ProjectPicker.tsx +64 -0
- package/packages/visualizer/src/data/adapters/report-backend.test.ts +3 -2
- package/packages/visualizer/src/data/adapters/report-backend.ts +1 -1
- package/packages/visualizer/src/data/queries.ts +14 -7
- package/packages/visualizer/src/evidence/api.ts +2 -1
- package/packages/visualizer/src/evidence/check-detail.tsx +19 -4
- package/packages/visualizer/src/evidence/document-page.tsx +6 -2
- package/packages/visualizer/src/evidence/evidence-context.tsx +17 -12
- package/packages/visualizer/src/evidence/evidence.module.css +11 -1
- package/packages/visualizer/src/evidence/feature-page.tsx +7 -3
- package/packages/visualizer/src/evidence/model-page.tsx +6 -2
- package/packages/visualizer/src/evidence/project-home.tsx +45 -0
- package/packages/visualizer/src/evidence/recording-page.tsx +4 -1
- package/packages/visualizer/src/evidence/spec-index.tsx +2 -1
- package/packages/visualizer/src/evidence/test-filter.tsx +21 -0
- package/packages/visualizer/src/live/live-query.ts +21 -0
- package/packages/visualizer/src/live/live-runs.tsx +69 -0
- package/packages/visualizer/src/live/live-stream.ts +30 -9
- package/packages/visualizer/src/live/report-progress.ts +62 -0
- package/packages/visualizer/src/live/test-progress.tsx +34 -0
- package/packages/visualizer/src/runtime.ts +17 -1
- package/packages/visualizer/src/state/ui-store.ts +3 -2
- package/packages/visualizer/src/views/ChartPage.tsx +2 -0
- package/packages/visualizer/src/views/ChartTests.tsx +43 -0
- package/packages/visualizer/src/views/SidePanel.tsx +2 -0
- package/packages/visualizer/vite/atlas-files.test.ts +4 -4
- package/packages/visualizer/vite/atlas-files.ts +1 -5
- package/packages/visualizer/vite/atlas-server-proxy.ts +2 -1
package/docs/report-backends.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
> [!NOTE]
|
|
4
4
|
> Atlas has two kinds of backend, and this page is about the first.
|
|
5
5
|
>
|
|
6
|
-
> - **Run sources** (this page): where the
|
|
6
|
+
> - **Run sources** (this page): where the client finds *statechart runs* (`manifest.json` per run)
|
|
7
7
|
> and live snapshots of a run in progress: a directory, HTTP files, or an HTTP API.
|
|
8
8
|
> - **Layer stores**: where *layers of evidence for any kind of spec* are kept, an `ObjectStore`
|
|
9
9
|
> such as a directory or a bucket. Any number of them are one report. See
|
|
@@ -14,7 +14,7 @@
|
|
|
14
14
|
|
|
15
15
|
Atlas maps the product's areas, screens, events and journeys from a spec, then overlays evidence
|
|
16
16
|
of what the product did. Specs and reports are independent. Local development, CI, device labs
|
|
17
|
-
and hosted runners can supply reports to the same product atlas simultaneously. The
|
|
17
|
+
and hosted runners can supply reports to the same product atlas simultaneously. The client works
|
|
18
18
|
before the first report exists.
|
|
19
19
|
|
|
20
20
|
## Connect sources
|
|
@@ -33,9 +33,9 @@ The default source is the directory passed to `--runs`. Add sources with a JSON
|
|
|
33
33
|
```
|
|
34
34
|
|
|
35
35
|
```sh
|
|
36
|
-
atlas
|
|
37
|
-
atlas
|
|
38
|
-
atlas
|
|
36
|
+
atlas client dev --specs ./specs --runs ./atlas-runs --sources ./atlas.sources.json
|
|
37
|
+
atlas client build --specs ./specs --runs ./atlas-runs --sources ./atlas.sources.json --out ./atlas-site
|
|
38
|
+
atlas client publish-data --specs ./specs --runs ./atlas-runs --sources ./atlas.sources.json --out ./atlas-data
|
|
39
39
|
```
|
|
40
40
|
|
|
41
41
|
Directory paths resolve against the configuration file. Source ids are unique, stable names
|
|
@@ -45,7 +45,7 @@ Credentials must not appear in this file or URLs: descriptors are sent to the br
|
|
|
45
45
|
included in static output.
|
|
46
46
|
|
|
47
47
|
Remote services must allow requests
|
|
48
|
-
from the
|
|
48
|
+
from the client's origin. For private services, use an authenticated same-origin gateway. Atlas
|
|
49
49
|
does not download private GitHub artifacts or sign object-storage requests itself: expose their
|
|
50
50
|
published data over HTTP or implement a gateway using the contract below.
|
|
51
51
|
|
|
@@ -113,7 +113,7 @@ updates to keep a snapshot steady; manual Refresh still works. Resuming refreshe
|
|
|
113
113
|
|
|
114
114
|
## Identity, failures and publishing
|
|
115
115
|
|
|
116
|
-
|
|
116
|
+
Client identities combine source ids and native run ids. Cache keys include the backend location
|
|
117
117
|
and transport. Matching run names cannot mix results or media. Shared links retain the qualified
|
|
118
118
|
identity; old links containing only a run id still resolve to workspace reports.
|
|
119
119
|
|
|
@@ -136,7 +136,7 @@ URLs and never expose the directory paths.
|
|
|
136
136
|
|
|
137
137
|
`src/data/adapters/report-backend.ts` defines `ReportBackend`: stable identity and cache key,
|
|
138
138
|
report listing, snapshot loading, media resolution and optional change subscription. The query
|
|
139
|
-
layer consumes this separately from the spec adapter. An embedded
|
|
139
|
+
layer consumes this separately from the spec adapter. An embedded client can supply a backend
|
|
140
140
|
through `ReportSourcesContext`; a new built-in transport can add a factory alongside
|
|
141
141
|
`createReportBackend`. Screens, maps and journeys require no transport-specific code.
|
|
142
142
|
|
package/docs/report-contract.md
CHANGED
|
@@ -6,7 +6,7 @@ unit tests, linters, anything that ends in "this check passed or failed". If thi
|
|
|
6
6
|
disagree, that is a bug: [file it](reporting-issues.md).
|
|
7
7
|
|
|
8
8
|
The contract is small on purpose. It is one document (the **layer**), one rule (how layers compose),
|
|
9
|
-
and one interface (the **backend** that stores them). Everything else, the
|
|
9
|
+
and one interface (the **backend** that stores them). Everything else, the client, the gate, the CLI,
|
|
10
10
|
is built on those three and can be replaced.
|
|
11
11
|
|
|
12
12
|
- [Concepts](#concepts)
|
|
@@ -289,7 +289,7 @@ import { verifyObjectStore } from '@crvouga/atlas/reports/object-store-conforman
|
|
|
289
289
|
const { ok, failures } = await verifyObjectStore(myS3Store); // writes under a scratch prefix, cleans up
|
|
290
290
|
```
|
|
291
291
|
|
|
292
|
-
Several backends are one report: pass them all to `combineBackends`, to the
|
|
292
|
+
Several backends are one report: pass them all to `combineBackends`, to the client
|
|
293
293
|
(`reportsHost`), or to the CLI (`--store` for directories, `--backends <module>` for the rest). A
|
|
294
294
|
backend that cannot be reached is named in the notices and leaves the others usable; asking again
|
|
295
295
|
retries it.
|
|
@@ -302,7 +302,7 @@ entry whose content has not arrived:
|
|
|
302
302
|
A layer's checks are the contract. Next to them the store holds the two other things a whole project
|
|
303
303
|
needs, written the same way (content first, catalog entry last, nothing overwritten):
|
|
304
304
|
|
|
305
|
-
- **Spec sets** (`layers/specs/…`): the specs of one commit, so a
|
|
305
|
+
- **Spec sets** (`layers/specs/…`): the specs of one commit, so a client needs no checkout. A spec set is
|
|
306
306
|
`{ schemaVersion, id, branch, sha, recordedAtMs, freshness?, files: [{ path, digest, fingerprint, kind,
|
|
307
307
|
title, domain, bytes }] }` ([`spec-set.schema.json`](../packages/schema/schemas/spec-set.schema.json)). Its
|
|
308
308
|
id is the digest of the branch, commit and file digests, so one commit's specs are one set however many
|
|
@@ -310,7 +310,7 @@ needs, written the same way (content first, catalog entry last, nothing overwrit
|
|
|
310
310
|
The outline a report is laid over is the **newest set from `main`** (then `master`, then the newest);
|
|
311
311
|
a set from a modified checkout is used only if no committed set exists.
|
|
312
312
|
- **Run bundles** (`layers/runs/<run>/…` and `layers/runs/index/…`): a finished statechart run with its
|
|
313
|
-
screenshots and clips, for the
|
|
313
|
+
screenshots and clips, for the client. Uploaded only when the run's privacy scan passed; the manifest is
|
|
314
314
|
written last, then the catalog entry.
|
|
315
315
|
|
|
316
316
|
The keys of a layer, bottom up:
|
|
@@ -376,13 +376,13 @@ and composed with layers; Atlas never rewrites them. `copyStore` and `atlas repo
|
|
|
376
376
|
the number of checks (plus a few bytes for each layer ever read), not the age of the project, and a refresh
|
|
377
377
|
with nothing new is one request for the newest page of the catalog. Folding is exact in any order because the winner
|
|
378
378
|
rule is a total order; a test proves the incremental report equals combining every layer from scratch.
|
|
379
|
-
The node sends the
|
|
379
|
+
The node sends the client only that: one request, only winning rows, with an ETag.
|
|
380
380
|
- Measured on a laptop with a directory backend: 300 layers of 50 checks in about 0.2 s; 1,000 layers
|
|
381
381
|
of 100 checks, each recorded against 1,000 specs (100,000 results), in about 0.9 s. Composing two
|
|
382
382
|
thousand layers of 100 checks (200,000 results) already in memory takes about 0.3 s.
|
|
383
383
|
`performance.test.ts` holds budgets ten times looser, to catch an accidental quadratic step on a
|
|
384
384
|
slow machine.
|
|
385
|
-
- A directory backend lists only the directories a prefix can reach. The
|
|
385
|
+
- A directory backend lists only the directories a prefix can reach. The client polls a small index,
|
|
386
386
|
reads an immutable layer once and caches it forever, and loads recordings only when opened.
|
|
387
387
|
|
|
388
388
|
## Other documents of the contract
|
|
@@ -394,7 +394,7 @@ and composed with layers; Atlas never rewrites them. `copyStore` and `atlas repo
|
|
|
394
394
|
| Report status | `report-status.schema.json` | The output of `atlas report status --format json`. |
|
|
395
395
|
| Spec set | `spec-set.schema.json` | The specs of one commit, as published beside the layers. |
|
|
396
396
|
| Run manifest | `manifest.schema.json` | What `atlas run` writes for a statechart run. Publish it as a layer with `--manifest`. |
|
|
397
|
-
| Runs index, sources, change notices | `runs-index`, `report-sources`, `atlas-change` | How the
|
|
397
|
+
| Runs index, sources, change notices | `runs-index`, `report-sources`, `atlas-change` | How the client finds runs. See [report backends](report-backends.md). |
|
|
398
398
|
| CloudEvents | `cloud-event.schema.json` | Business events a run observed. |
|
|
399
399
|
|
|
400
400
|
All of them are JSON Schema 2020-12 and ship in the package at `@crvouga/atlas/schemas/<name>.schema.json`.
|
package/docs/reporting-issues.md
CHANGED
|
@@ -16,7 +16,7 @@ This guide is written for coding agents. Read it top to bottom once, then use
|
|
|
16
16
|
|
|
17
17
|
`<part>` is the part of `@crvouga/atlas` the issue is about: `core` (the main import and the
|
|
18
18
|
`atlas` command), or the import path after the package name: `playwright`, `bun`, `webdriver`,
|
|
19
|
-
`detox`, `vitest`, `schema`, `reports`; `visualizer` for the `atlas
|
|
19
|
+
`detox`, `vitest`, `schema`, `reports`; `visualizer` for the `atlas client` command. Use `core`
|
|
20
20
|
when you cannot tell.
|
|
21
21
|
|
|
22
22
|
1. [Search for an existing issue](#1-search-first). Comment on it instead of opening a duplicate.
|
|
@@ -43,10 +43,10 @@ File when:
|
|
|
43
43
|
|
|
44
44
|
- **bug**: Atlas throws or hangs; lint, plan or a run gives a result the spec does not support;
|
|
45
45
|
a driver taps, types, waits or records differently from what its README says; the manifest or a
|
|
46
|
-
report does not match its JSON Schema; the
|
|
46
|
+
report does not match its JSON Schema; the Atlas client shows something other than what the run
|
|
47
47
|
recorded; the package fails to install, import, build or type-check.
|
|
48
48
|
- **feature**: you need something Atlas does not have: a spec construct, a config option, a driver
|
|
49
|
-
capability, a CLI flag, a field in the manifest, a view in the
|
|
49
|
+
capability, a CLI flag, a field in the manifest, a view in the Atlas client, an export format. Check
|
|
50
50
|
that part's README and `docs/` first.
|
|
51
51
|
|
|
52
52
|
Do not file when:
|
package/docs/runners.md
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
# Native test runner reporters
|
|
2
|
+
|
|
3
|
+
Projects upload results to a shared Atlas node under their own project slug. CodeceptJS, Mocha, Jest and Node's test runner have native reporters; other runners can publish their JUnit or CTRF output with `atlas report publish`. All reporters are entry points of the single `@crvouga/atlas` package.
|
|
4
|
+
|
|
5
|
+
## Configure the project once
|
|
6
|
+
|
|
7
|
+
Set these in the test process, locally or in CI:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
export ATLAS_SERVER_URL=https://atlas.example.com
|
|
11
|
+
export ATLAS_PROJECT=checkout-service
|
|
12
|
+
export ATLAS_PUBLISH_TOKEN=your-publish-token
|
|
13
|
+
export ATLAS_ENVIRONMENT=linux-chromium
|
|
14
|
+
export ATLAS_BRANCH=main
|
|
15
|
+
export ATLAS_SHA=your-commit-sha
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
The server owns its storage configuration and credentials. Consumers only need the server URL and a publish token. Upload failures fail the test invocation; reporters wait for publication before finishing. Each invocation streams a live run, then publishes final evidence, with stable check IDs relative to the project directory, durations, failures and skipped outcomes. Unknown outcomes, TODOs and skipped tests are `not-run`, never passed. Parallel processes upload independent layers; use distinct producer/environment values for independent suites.
|
|
19
|
+
|
|
20
|
+
A test file is part of a check's identity, not automatically a specification: link checks to product specifications through the existing report mapping workflow when importing JUnit/CTRF, or publish explicit `specPath` checks through the API.
|
|
21
|
+
|
|
22
|
+
## CodeceptJS
|
|
23
|
+
|
|
24
|
+
Use its [Mocha reporter interface](https://codecept.io/reports/):
|
|
25
|
+
|
|
26
|
+
```sh
|
|
27
|
+
bunx codeceptjs run --reporter @crvouga/atlas/codecept
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Streams scenario starts and outcomes, then records durations and scenario steps when supplied by CodeceptJS. Use the standard single-process runner; parallel workers should each publish their own layer. Runner-native screenshot/video attachments can also be imported from CTRF/JUnit; this reporter does not read arbitrary files from runner objects.
|
|
31
|
+
|
|
32
|
+
## Mocha
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
bunx mocha --reporter @crvouga/atlas/mocha
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Or configure `reporter` and `reporterOptions` in Mocha. Options include `url`, `token`, `producer`, `environment`, `branch`, `sha`, `rootDir` and `timeoutMs` (30 seconds by default); explicit options override the environment. Final failures and hook failures are failed checks. Passing retries include a flaky label. Mocha parallel mode is not supported by this reporter's event model; run each shard separately.
|
|
39
|
+
|
|
40
|
+
## Jest
|
|
41
|
+
|
|
42
|
+
Keep Jest's console reporter alongside Atlas:
|
|
43
|
+
|
|
44
|
+
```js
|
|
45
|
+
export default {
|
|
46
|
+
reporters: ['default', ['@crvouga/atlas/jest', { producer: 'unit' }]],
|
|
47
|
+
};
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
The reporter streams test-case starts and results from workers and uploads full aggregated evidence after workers finish. File-level exceptions become failed checks even when no test ran. Watch mode resets publication errors on each invocation. It accepts the same options as Mocha.
|
|
51
|
+
|
|
52
|
+
## Node's test runner
|
|
53
|
+
|
|
54
|
+
```sh
|
|
55
|
+
node --test --test-reporter=spec --test-reporter-destination=stdout \
|
|
56
|
+
--test-reporter=@crvouga/atlas/node-test --test-reporter-destination=stdout
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Uses the native async event stream, with environment configuration. Reports leaf tests; successful suite containers are excluded, while suite hook failures remain failed checks. Test names include enclosing suites so identical names in different suites keep distinct IDs. Skips and TODOs remain not-run, including TODO failures.
|
|
60
|
+
|
|
61
|
+
## Other runners and frameworks
|
|
62
|
+
|
|
63
|
+
Playwright, Vitest, Bun, WebDriver/Appium and Detox already have Atlas statechart adapters. Pytest, JUnit/TestNG, NUnit, xUnit, Go, Rust, Cypress, WebdriverIO, Cucumber and Robot Framework can publish JUnit XML or CTRF JSON through the existing importer. See [integration commands](integrating.md) and the [report contract](report-contract.md). These report importers complement native reporters; they do not require a storage backend in the consumer project.
|
|
64
|
+
|
|
65
|
+
## Operate the singleton
|
|
66
|
+
|
|
67
|
+
Run `atlas server` once for the projects the node hosts; point local and CI publishers at that URL and set each project's `ATLAS_PROJECT`. Configure backends using environment variables or open **Settings** in the client (`/admin`). Authentication and listening addresses remain deployment configuration. Settings changes are validated and saved atomically for the next server restart. Environment variables and flags override saved values. A backend change does not migrate historical data: retain the old backend as a read source when switching the upload destination.
|
|
68
|
+
|
|
69
|
+
Native reporters send a heartbeat every 5 seconds. The server interrupts invocations after 30 seconds of silence, or after the configurable progress-stall timeout. It preserves finished results and never marks missing tests as passed. See [live test runs](server.md#live-test-runs-and-heartbeats).
|
package/docs/server.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
# The Atlas server
|
|
2
2
|
|
|
3
|
-
One
|
|
4
|
-
combined report, live**. It is a Bun API that serves a Vite-built single-page
|
|
3
|
+
One node for multiple projects that **takes in results from every environment, keeps them, and shows everyone the
|
|
4
|
+
combined report, live**. It is a Bun API that serves a Vite-built single-page client: one host, one port, in
|
|
5
5
|
development and in production.
|
|
6
6
|
|
|
7
7
|
```
|
|
8
|
-
laptop ──────┐ ┌── browsers (the
|
|
8
|
+
laptop ──────┐ ┌── browsers (the client: reads /api, listens on /api/events)
|
|
9
9
|
CI job ──────┼── PUT /api/store/… POST /api/checks ─▶│
|
|
10
10
|
device lab ──┘ atlas report publish | stream │ atlas report status (the gate: asks /api/status)
|
|
11
11
|
│
|
|
@@ -18,8 +18,8 @@ development and in production.
|
|
|
18
18
|
- **Its default backend is its own filesystem** (`./atlas-data`). That folder is the whole state: back it up,
|
|
19
19
|
mount it on a volume, or put the state in a bucket instead ([backends](#backends)). Several backends are one
|
|
20
20
|
report, as ever.
|
|
21
|
-
- **The report is live.** Every
|
|
22
|
-
says so and each
|
|
21
|
+
- **The report is live.** Every client holds one server-sent-events connection; when results arrive the server
|
|
22
|
+
says so and each client reads only what changed ([streaming](#streaming)).
|
|
23
23
|
- **Layered, never partitioned**, as everywhere in Atlas: the server only stores immutable, content-addressed
|
|
24
24
|
layers and works out the report from them. Replace it, restore it, or run a second one on a copy of the data:
|
|
25
25
|
the same layers give the same report ([the contract](report-contract.md)).
|
|
@@ -36,13 +36,13 @@ export ATLAS_TOKEN=$(openssl rand -hex 24) # admin token; see "Tokens and role
|
|
|
36
36
|
bunx atlas serve # http://127.0.0.1:8080, data in ./atlas-data
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
-
It runs on Bun and on Node 22.18 or later. The first start builds the
|
|
39
|
+
It runs on Bun and on Node 22.18 or later. The first start builds the client once into `~/.cache/atlas/site`
|
|
40
40
|
(or serves `./atlas-site`, or `--site <dir>`); the build is the same for every project, because the server writes
|
|
41
41
|
its own address book into the page (`window.__ATLAS_CONFIG__`).
|
|
42
42
|
|
|
43
|
-
**Working on Atlas or on the
|
|
43
|
+
**Working on Atlas or on the client:** `atlas dev` runs the server on a port of its own and the client's Vite dev
|
|
44
44
|
server in front of it. Vite proxies `/api` (server-sent events included) to the server, so the browser, the
|
|
45
|
-
uploaders and the
|
|
45
|
+
uploaders and the client all use **one host**, exactly as in production, and a change to the client shows at once.
|
|
46
46
|
|
|
47
47
|
```sh
|
|
48
48
|
bunx atlas dev --data .atlas-data # http://127.0.0.1:5180
|
|
@@ -90,7 +90,7 @@ purposes.
|
|
|
90
90
|
|
|
91
91
|
A backend is anything that implements `ObjectStore`: a folder, an S3-compatible bucket (`s3://`, `r2://`,
|
|
92
92
|
`gs://`), or **another Atlas server** (`https://…`, with `ATLAS_TOKEN` for it). The server reads every backend
|
|
93
|
-
and shows one report; one that is down is named in the
|
|
93
|
+
and shows one report; one that is down is named in the client and skipped, and the server recovers by itself.
|
|
94
94
|
|
|
95
95
|
- **Uploads go to one backend**: the `data` folder, or the one named by `write` / `ATLAS_WRITE`. Name a bucket
|
|
96
96
|
to make it the system of record, so the server holds nothing a replacement would lose:
|
|
@@ -107,7 +107,7 @@ and shows one report; one that is down is named in the viewer and skipped, and t
|
|
|
107
107
|
|
|
108
108
|
| Role | May |
|
|
109
109
|
| --- | --- |
|
|
110
|
-
| `read` | look: the
|
|
110
|
+
| `read` | look: the client, the report, `/api/status` |
|
|
111
111
|
| `publish` | also upload layers, specs and runs, and remove the scratch objects `atlas report doctor` writes |
|
|
112
112
|
| `admin` | all server operations |
|
|
113
113
|
|
|
@@ -191,24 +191,26 @@ there (`409`), so nothing half-uploaded is ever shown.
|
|
|
191
191
|
|
|
192
192
|
| Event | Data | Meaning |
|
|
193
193
|
| --- | --- | --- |
|
|
194
|
-
| `hello` | `{ version, ready, time }` | Sent first. The report's current version. |
|
|
194
|
+
| `hello` | `{ version, ready, time, heartbeatMs? }` | Sent first. The report's current version. |
|
|
195
195
|
| `report` | `{ version, totalLayers, added: [{ id, producer, environment, branch, sha, recordedAtMs, counts }], addedTotal }` | Layers arrived. Enough to say who sent what, without asking. |
|
|
196
196
|
| `atlas:change` | `{ scope: "specs" \| "runs", files, runIds }` | New specs or runs. |
|
|
197
|
-
| `
|
|
197
|
+
| `run` | A bounded `LiveRun` snapshot | Start, progress logs, heartbeat, completion, and interruption; no producer credentials. |
|
|
198
|
+
| `projects` | `{}` | Another project was discovered; refresh the project picker. |
|
|
199
|
+
| `ping` | `{}` | Every 15 s, so proxies keep the connection and the client can tell a quiet server from a dead connection. |
|
|
198
200
|
|
|
199
|
-
|
|
200
|
-
connection, a laptop that slept) loses nothing: it reads again when it reconnects, and the
|
|
201
|
+
Run events update progress directly in the client cache. Report and specification events trigger coalesced reads of the affected data. Events arriving during a read cause a trailing refresh rather than cancelling that read. A client that missed some (a dropped
|
|
202
|
+
connection, a laptop that slept) loses nothing: it reads again when it reconnects, and the client also reads on a
|
|
201
203
|
slow timer behind the stream. Past `limits.maxStreams` viewers the server answers `503` with `Retry-After` and the
|
|
202
|
-
|
|
204
|
+
client polls instead; a client that stops reading is let go rather than held in memory.
|
|
203
205
|
|
|
204
206
|
### Reading only what changed
|
|
205
207
|
|
|
206
208
|
`GET /api/reports/evidence/effective` is the whole report: only the layers that win a check, each with only the
|
|
207
|
-
rows that win, with an ETag. A
|
|
209
|
+
rows that win, with an ETag. A client that holds a version asks `?since=<version>` and is sent **only the layers
|
|
208
210
|
that changed since, and the ids that no longer win anything** (`"delta": true`). Applying that gives exactly the
|
|
209
211
|
report the server would have sent whole (`applyEffectiveDelta`, tested against every earlier version of random
|
|
210
|
-
runs); a version the server does not remember, or an older server, answers with the whole report, which a
|
|
211
|
-
takes as it always did. A result streamed into a 100 000-check report costs a
|
|
212
|
+
runs); a version the server does not remember, or an older server, answers with the whole report, which a client
|
|
213
|
+
takes as it always did. A result streamed into a 100 000-check report costs a client a few kilobytes.
|
|
212
214
|
|
|
213
215
|
## API reference
|
|
214
216
|
|
|
@@ -250,7 +252,7 @@ laptop with 2 000 layers of 100 checks (200 000 results, 10 000 slots) and with
|
|
|
250
252
|
| --- | --- | --- |
|
|
251
253
|
| Start: read and fold every layer | 0.4 s | 1.7 s |
|
|
252
254
|
| A refresh with nothing new | 4 ms | 4 ms |
|
|
253
|
-
| Upload one layer → every
|
|
255
|
+
| Upload one layer → every client told | 4 ms | 5 ms |
|
|
254
256
|
| `GET effective`, cached / `304` | 1 ms / 0.2 ms | 13 ms / 0.5 ms |
|
|
255
257
|
| `GET status`, cached for the version | 12 ms first | 70 ms first |
|
|
256
258
|
|
|
@@ -264,22 +266,56 @@ What makes it so, none of it at the cost of the contract:
|
|
|
264
266
|
- **An upload is read once.** A layer's manifest is parsed and verified when it arrives and handed to the report,
|
|
265
267
|
not read again; blob existence is asked 16 at a time.
|
|
266
268
|
- **The newest end of the catalog is all a refresh looks at**, with a full look now and then.
|
|
267
|
-
- **Compressed once.** The
|
|
268
|
-
report version is gzipped once however many viewers fetch it, and events are encoded once for every
|
|
269
|
-
- **A small
|
|
269
|
+
- **Compressed once.** The client's files are brotli-compressed once (gzip for clients that cannot read it), a
|
|
270
|
+
report version is gzipped once however many viewers fetch it, and events are encoded once for every client.
|
|
271
|
+
- **A small client.** The graph, the Gherkin parser, the recording player and the finder load when used: the
|
|
270
272
|
first page is about 250 KB over the wire.
|
|
271
273
|
|
|
272
274
|
## Operating it
|
|
273
275
|
|
|
274
|
-
- **One
|
|
275
|
-
every
|
|
276
|
+
- **One node for multiple projects.** Local environments and CI select their project slug and talk to the same node, which tells
|
|
277
|
+
every client at once. Two servers on one backend both work and show the same report, but a client hears about
|
|
276
278
|
an upload only from the server that took it and about the other's at the next refresh, so give a team one.
|
|
277
279
|
- **State is the write backend.** Back up `./atlas-data` (or the bucket). Restoring it, or starting a new server
|
|
278
280
|
on it, shows the same report.
|
|
279
281
|
- **Stopping.** `SIGTERM` stops accepting, lets requests in flight finish, ends the event streams and exits 0.
|
|
280
282
|
- **Logs.** One JSON line per request (method, path, status, ms) to stdout, none for the probes. The first lines
|
|
281
283
|
say what it reads and which backend takes uploads, and warn when there are no tokens.
|
|
282
|
-
- **A backend goes down.** The server keeps serving what it has, names the backend in the
|
|
284
|
+
- **A backend goes down.** The server keeps serving what it has, names the backend in the client and in
|
|
283
285
|
`/readyz`, and recovers by itself. The gate fails closed: unknown is not a pass.
|
|
284
286
|
- **Upgrades.** The layer format is a stable, versioned contract; upgrade the server before the clients that
|
|
285
287
|
publish when a release says the format moved.
|
|
288
|
+
|
|
289
|
+
## Configure from the client
|
|
290
|
+
|
|
291
|
+
Open **Settings** (`/admin`) and enter an admin token. The token stays in page memory. Read and publish tokens cannot edit settings; the browser sign-in box only grants reading. The API uses `Authorization: Bearer` and refuses cross-origin browser writes.
|
|
292
|
+
|
|
293
|
+
Use the Storage, Live updates, and Report rules forms to edit saved settings. Advanced includes request limits, logging, a JSON editor, and a read-only view of the active configuration. Save changes and restart Atlas to apply them. Atlas validates the configuration and atomically replaces `atlas.server.json`, preserving authentication and deployment settings. Changes apply at the next restart; environment variables and flags retain precedence. Concurrent edits require reloading the current revision. Data is never moved or removed by changing settings. Keep an old backend as a read source when moving future uploads to a new one. Embedded servers can provide their own settings persistence or manage configuration outside this API.
|
|
294
|
+
|
|
295
|
+
Native test reporters for CodeceptJS, Mocha, Jest and Node upload directly to this server; see [runner setup](runners.md). Every publisher uses the same server address and only a publish token. Storage credentials belong in the server environment.
|
|
296
|
+
|
|
297
|
+
## Live test runs and heartbeats
|
|
298
|
+
|
|
299
|
+
The default native reporters and statechart runner stream tests and steps through `/api/live`, before the invocation finishes. Running tests appear inside their report with an animated status mark and the latest step. Open a test to read its progress log; its previous recorded result stays available separately. The **Running** filter narrows both the spec list and the test rows, and counts update as tests finish or time out. Statechart tests appear inside the chart and screen details. Selecting an archived run leaves live progress out of that historical view. Finished checks immediately enter normal immutable report layers. Operational progress is separate from historical winners, so a previous passed result never implies that the current invocation passed.
|
|
300
|
+
|
|
301
|
+
Start with `POST /api/live/<uuid>`, send ordered check deltas with `PATCH /api/live/<uuid>`, and renew the producer lease with `POST /api/live/<uuid>/heartbeat` every 5 seconds. Finish with a terminal update. Only the publisher token that registered the invocation can update it. Replayed sequences must have identical content; late writes cannot revive an expired invocation. `createLiveReporter` handles batching, retries, heartbeat timers and the final flush.
|
|
302
|
+
|
|
303
|
+
The server uses its own receipt clock. By default it interrupts a run after 30 seconds without a heartbeat (`ATLAS_RUN_TIMEOUT_SECONDS`). Heartbeats do not reset progress: after 15 minutes with no changed tests/steps, a separate stall timeout interrupts it (`ATLAS_RUN_STALL_SECONDS`, 0 disables). Both settings are editable in the admin UI. Healthy long tests continue renewing heartbeats; increase or disable the progress timeout for tests that legitimately make no observable progress for longer.
|
|
304
|
+
|
|
305
|
+
Interrupted runs retain completed results and show why the remaining work stopped. Missing outcomes never become passes. The server persists bounded summaries under `server/live/` in its write backend (an HTTP relay keeps operational leases in its own data folder; the remote server only accepts immutable report objects) and restores outstanding leases as interrupted after a server restart. S3 server credentials need GetObject/PutObject for that prefix; generate the updated `atlas report policy --role both` policy. Producers still receive no backend credentials. Live `run` events carry bounded snapshots directly into test rows, including step logs and interruption reasons. The legacy `atlas:change` announcement remains available for older clients; its optional `liveRunId` identifies lease updates. Reconnecting viewers re-fetch progress, reports, and specifications. With SSE connected, live progress uses a 60-second safety refresh; without SSE it polls every two seconds. Returning to the tab or coming back online also revalidates reports and progress.
|
|
306
|
+
|
|
307
|
+
In this repository, use `bun atlas` and `bun example:run` in two terminals. Every example Gherkin scenario executes, including sharing and renaming; `@unrecorded` no longer excludes tests.
|
|
308
|
+
|
|
309
|
+
## Multiple projects on one node
|
|
310
|
+
|
|
311
|
+
Start the node with `atlas server` (or `bun atlas` in this repository). The server builds or finds the Atlas client and serves it with the API from one process. It never imports a consumer or an example. In this repository the node uses port 5180 and `./atlas-data`; `bun example:run` publishes the Todo app to project `todo-app`.
|
|
312
|
+
|
|
313
|
+
Set `ATLAS_PROJECT=todo-app` alongside `ATLAS_SERVER_URL` for native reporters and statechart runs. Report CLI commands accept `--project todo-app` (or `ATLAS_PROJECT`) with an HTTP store. Programmatic HTTP stores, live reporters and native reporter options accept `project: 'todo-app'`. Keep the same slug for the invocation's live progress, final results, spec sets and recordings. Slugs use 1–64 lowercase letters, digits or hyphens, starting and ending with a letter or digit.
|
|
314
|
+
|
|
315
|
+
Every project API is available as `/api/projects/<slug>/…`, for example `/api/projects/todo-app/checks`, `/api/projects/todo-app/events` and `/api/projects/todo-app/reports/evidence/effective`. The unprefixed API accepts optional `?project=todo-app` or `X-Atlas-Project: todo-app`. Conflicting selectors are refused. `GET /api/projects` discovers projects. A successful authenticated upload creates a project; reads of an unknown project return 404.
|
|
316
|
+
|
|
317
|
+
Projects isolate check identities, specs, run IDs, recordings, lease summaries, event streams and report caches. Identical layer bytes are preserved as uploaded. The node stores each project's objects under `projects/<slug>/` and discovery markers under `server/projects/`. Unqualified requests continue reading the legacy default namespace, advertised as **Default** when it has reports. Existing data is never implicitly moved. For S3, use the combined server policy (`atlas report policy --role both`) with access to both project namespaces and the catalog. An HTTP relay selects the same project on its upstream node and keeps discovery and leases locally.
|
|
318
|
+
|
|
319
|
+
The project picker sits at the top right; the header also provides spec dropdown breadcrumbs. Project selection lives in the page URL, so links and refreshes preserve it. Switching projects reloads the client, clearing report queries, media and event connections. Custom report-source preferences are keyed by the project's data URL. Project namespaces share the node's authentication roles: they are organizational partitions, not separate tenant authorization policies. Settings and server identity remain node-wide.
|
|
320
|
+
|
|
321
|
+
Project indexes are opened lazily; at most 64 are active in memory. An index with no active requests or event subscribers is closed after five idle minutes. Event subscribers share the node-wide stream limit. Project discovery lists only small catalog markers, never every report layer.
|
package/docs/statecharts.md
CHANGED
|
@@ -5,7 +5,7 @@ paths that cover every transition, run those paths against the real product thro
|
|
|
5
5
|
browser, a phone, a plain function), and write a manifest of what happened, with screenshots, video
|
|
6
6
|
clips, captions, business events and standard test reports. Each run publishes as a layer
|
|
7
7
|
(`atlas report publish --manifest`), so it combines with every other suite's evidence. This page covers
|
|
8
|
-
running them, from a runnable example to the
|
|
8
|
+
running them, from a runnable example to the Atlas client.
|
|
9
9
|
|
|
10
10
|
```
|
|
11
11
|
specs/ atlas lint the charts are pure, reachable, free of silent dead ends,
|
|
@@ -35,7 +35,7 @@ served by a small Node server) specified as two charts:
|
|
|
35
35
|
- `specs/getting-started.feature` and `specs/working-the-list.feature`, the same stories in Gherkin.
|
|
36
36
|
- `specs/todo.qnt`, the list, its sync, and its actions as a Quint model.
|
|
37
37
|
|
|
38
|
-
The
|
|
38
|
+
The Atlas client lists the features under Gherkin and the model under Quint.
|
|
39
39
|
|
|
40
40
|
It has every kind of event: user events (taps and typing), a **system** event ("Sync fails", which
|
|
41
41
|
the test triggers through a real HTTP endpoint on the example server), a **time** event ("Retry
|
|
@@ -65,24 +65,26 @@ visualizer updates when a run finishes.
|
|
|
65
65
|
|
|
66
66
|
```sh
|
|
67
67
|
bun run atlas:run # example:clients, then example:run: every chart of the example, in showcase mode
|
|
68
|
-
bun
|
|
68
|
+
bun atlas # the example's specs, chart runs, and layered reports
|
|
69
69
|
```
|
|
70
70
|
|
|
71
|
-
Each config writes its own run and the
|
|
71
|
+
Each config writes its own run and the Atlas client shows one run at a time: the latest is the
|
|
72
72
|
single-browser run, and the run across clients is the one before it in the run picker.
|
|
73
73
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
74
|
+
Gherkin, Quint, DMN, Alloy and OpenSLO documents sit beside the charts. Results compose across
|
|
75
|
+
environments, branches and backends within each project. **Sources** switches report sources on or off.
|
|
76
|
+
|
|
77
|
+
`bun atlas` runs the singleton Atlas server and the client together at `http://localhost:5180`.
|
|
78
|
+
It starts a project-agnostic node with no example data. Select a project using the top-right picker.
|
|
79
|
+
In another terminal run `bun run example:run`: statecharts, Gherkin scenarios and Quint traces stream running checks, completed results and current steps
|
|
80
|
+
to project `todo-app` on that server while testing. Producers renew a heartbeat every 5 seconds; 30 seconds of silence
|
|
81
|
+
marks an invocation interrupted. Final specs and recordings upload when their privacy scan completes.
|
|
82
|
+
Tests show inline running indicators and logs without waiting for the whole suite to finish. Report history
|
|
83
|
+
is retained in `atlas-data/projects/todo-app`; local run artifacts remain in `atlas-runs` and
|
|
84
|
+
`atlas-reports`. To upload a completed run again without rerunning tests,
|
|
85
|
+
use `bun run example:publish`. Set `ATLAS_SERVER_URL` for a different upload destination (and
|
|
86
|
+
`ATLAS_PUBLISH_TOKEN` when it requires one). Restart `bun atlas` when switching from the older
|
|
87
|
+
file-based example client; a busy port is an error so publishers cannot silently target another server.
|
|
86
88
|
|
|
87
89
|
A showcase run ends with:
|
|
88
90
|
|
|
@@ -92,7 +94,7 @@ Transitions 14/14 passed, 0 failed, 0 not reached
|
|
|
92
94
|
Manifest: examples/todo/atlas-runs/<run-id>/manifest.json
|
|
93
95
|
```
|
|
94
96
|
|
|
95
|
-
`example:run` then runs the Gherkin features in a browser and the Quint model, and
|
|
97
|
+
`example:run` then runs the Gherkin features in a browser and the Quint model, and streams every scenario and trace as it runs, including sharing and renaming. The final recordings are uploaded alongside the completed statechart run.
|
|
96
98
|
|
|
97
99
|
`bun test examples` runs [`model.test.ts`](https://github.com/crvouga/atlas/blob/main/examples/todo/model.test.ts): the same charts and journeys
|
|
98
100
|
against `TodoModel`, a plain class, through `functionDriver` and `describeAtlas`. Only the driver
|
|
@@ -517,7 +519,7 @@ atlas report publish --producer statecharts --environment desktop-web \
|
|
|
517
519
|
--manifest atlas-runs/<run>/manifest.json --specs ./specs --upload-run
|
|
518
520
|
```
|
|
519
521
|
|
|
520
|
-
`--upload-run` also sends the run's screenshots and clips to the store (or the server) so the
|
|
522
|
+
`--upload-run` also sends the run's screenshots and clips to the store (or the server) so the client can show them
|
|
521
523
|
(only when the run's privacy scan passed).
|
|
522
524
|
|
|
523
525
|
## Config reference
|
|
@@ -627,7 +629,7 @@ Runs produce media and manifests that get shared, so Atlas refuses to leak real
|
|
|
627
629
|
|
|
628
630
|
## Visualizer
|
|
629
631
|
|
|
630
|
-
The [`atlas
|
|
632
|
+
The [`atlas client`](../packages/visualizer) command renders the charts from the specs directory and
|
|
631
633
|
overlays runs from the runs directory: screenshots on states, clips on transitions, statuses and
|
|
632
634
|
history. See its README for how to point it at a project.
|
|
633
635
|
|
|
@@ -638,4 +640,4 @@ and completed steps while paths are still running.
|
|
|
638
640
|
|
|
639
641
|
Layered reports for any spec appear beside the charts: each Gherkin, Quint, DMN, Alloy and OpenSLO
|
|
640
642
|
document shows the combined result of its checks per environment, with the layer behind each. Point
|
|
641
|
-
the
|
|
643
|
+
the client at your stores with `reportsHost` ([integrating Atlas](integrating.md#6-browse-it)).
|