@crvouga/atlas 1.13.0 → 1.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +7 -1
- package/dist/playwright/src/index.d.ts +4 -0
- package/dist/playwright/src/index.js +4 -0
- package/dist/playwright/src/ui-catalog.d.ts +51 -0
- package/dist/playwright/src/ui-catalog.js +44 -0
- package/dist/reports/src/evidence-types.d.ts +3 -0
- package/dist/reports/src/layer-merge.d.ts +5 -1
- package/dist/reports/src/layer-merge.js +9 -2
- package/dist/reports/src/report-index.js +9 -5
- package/dist/reports/src/ui-catalog-publish.d.ts +92 -0
- package/dist/reports/src/ui-catalog-publish.js +68 -0
- package/dist/reports/src/ui-catalog.d.ts +229 -0
- package/dist/reports/src/ui-catalog.js +273 -0
- package/dist/reports/src/ui-state.d.ts +70 -0
- package/dist/reports/src/ui-state.js +66 -0
- package/dist/schema/src/api.d.ts +350 -0
- package/dist/schema/src/api.js +71 -0
- package/dist/server/src/app.js +15 -0
- package/dist/server/src/openapi.js +22 -2
- package/docs/reference/http-api.md +46 -0
- package/docs/reference/playwright.md +63 -0
- package/docs/reference/reports.md +588 -1
- package/docs/reference/schema.md +33 -0
- package/docs/report-contract.md +2 -2
- package/docs/rich-evidence.md +5 -0
- package/docs/server.md +2 -0
- package/docs/ui-catalog.md +173 -0
- package/docs/ui.md +15 -0
- package/llms-full.txt +246 -2
- package/llms.txt +1 -0
- package/package.json +2 -1
- package/packages/schema/schemas/effective-delta.schema.json +73 -0
- package/packages/schema/schemas/effective-evidence.schema.json +73 -0
- package/packages/schema/schemas/layer-rows.schema.json +73 -0
- package/packages/server/openapi.json +1441 -79
- package/packages/visualizer/src/app/AppShell.tsx +36 -30
- package/packages/visualizer/src/app/router.tsx +10 -1
- package/packages/visualizer/src/catalog/UiCatalogFilters.tsx +65 -0
- package/packages/visualizer/src/catalog/UiCatalogPage.tsx +342 -0
- package/packages/visualizer/src/catalog/provider.tsx +25 -0
- package/packages/visualizer/src/catalog/search.ts +68 -0
- package/packages/visualizer/src/catalog/words.ts +26 -0
- package/packages/visualizer/src/components/Header.tsx +17 -4
package/docs/reference/schema.md
CHANGED
|
@@ -498,6 +498,39 @@ export const RunDetailSchema = …
|
|
|
498
498
|
|
|
499
499
|
`GET /api/reports/runs/detail`: one run and its last result for every test, environment and producer it recorded.
|
|
500
500
|
|
|
501
|
+
#### `UiProfileSchema`
|
|
502
|
+
|
|
503
|
+
```ts
|
|
504
|
+
export const UiProfileSchema = …
|
|
505
|
+
```
|
|
506
|
+
|
|
507
|
+
A state's profile: the client and the choices it was captured under.
|
|
508
|
+
|
|
509
|
+
#### `UiImageSchema`
|
|
510
|
+
|
|
511
|
+
```ts
|
|
512
|
+
export const UiImageSchema = …
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
The picture of a capture.
|
|
516
|
+
|
|
517
|
+
#### `UiCellSchema`
|
|
518
|
+
|
|
519
|
+
```ts
|
|
520
|
+
export const UiCellSchema = …
|
|
521
|
+
```
|
|
522
|
+
|
|
523
|
+
One state in one profile.
|
|
524
|
+
|
|
525
|
+
#### `UiCatalogSchema`
|
|
526
|
+
|
|
527
|
+
```ts
|
|
528
|
+
export const UiCatalogSchema = …
|
|
529
|
+
```
|
|
530
|
+
|
|
531
|
+
`GET /api/reports/ui-catalog`: the UI catalog, computed from the winning results of the `ui-state` and `ui-plan` parts
|
|
532
|
+
of every layer. States are in id order and cells in profile order, so the same layers always give the same document.
|
|
533
|
+
|
|
501
534
|
### Command results (`--format json`)
|
|
502
535
|
|
|
503
536
|
Source: `packages/schema/src/cli-results.ts`
|
package/docs/report-contract.md
CHANGED
|
@@ -84,7 +84,7 @@ Zod schema as `LayerDocumentSchema` from `@crvouga/atlas/schema`.
|
|
|
84
84
|
| `freshness` | `modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Advisory only; see [Statuses](#statuses-and-the-gate). |
|
|
85
85
|
| `run` | Optional `{ id, title?, startedAtMs?, endedAtMs? }`: the execution a local or manual producer names. Layers with the same `id` in one project are one run in the run list, however many shards or batches uploaded them; a later invocation at the same branch and commit gets another `id`. Without it a run is its `runUrl`, else `branch@sha`, as before. `title` is for people. Grouping and display only: it is **never part of the slot**, never decides a winner or a status, and never hides `sha`, `branch` or `freshness`. A `run` a reader cannot read is treated as absent. |
|
|
86
86
|
| `specs` | The specs the layer was recorded against, with a `fingerprint` of their text, so a reader can warn when a spec changed since. Optional. |
|
|
87
|
-
| `parts[].kind` | An open category (`behavior`, `performance`, `accessibility`, …). A new kind needs no change to any reader. Part of the slot. |
|
|
87
|
+
| `parts[].kind` | An open category (`behavior`, `performance`, `accessibility`, …). A new kind needs no change to any reader. Part of the slot. `ui-state` and `ui-plan` are the [UI catalog](ui-catalog.md)'s: a captured UI state, and the states that are expected. |
|
|
88
88
|
| `parts[].checks[]` | `id` (required, stable), `title`, `status`, `detail`, `specPath`, `measurements`, `durationMs`. |
|
|
89
89
|
| `tags`, `labels`, `links`, `attachments`, `annotations`, `extensions` | Evidence, on the layer, a part or a check: what a result carries beyond its outcome. All optional and open. See [evidence](#evidence) and [the guide](extending.md). |
|
|
90
90
|
| `checks[].steps`, `checks[].attempts` | A check's timeline, and every try when it was retried. Optional. |
|
|
@@ -227,7 +227,7 @@ are part of the contract:
|
|
|
227
227
|
Tools map to these four. CTRF `skipped`, `pending` and `other`, and JUnit `<skipped>`, are `not-run`.
|
|
228
228
|
An Atlas path that passed on retry (`flaky`) is `passed`, with the detail "Passed on retry (flaky)".
|
|
229
229
|
|
|
230
|
-
Timings that only qualify a check (parts of kind `performance`) are not counted as checks when
|
|
230
|
+
Timings that only qualify a check (parts of kind `performance`), and what a UI catalog expects (`ui-plan`), are not counted as checks when
|
|
231
231
|
anything else is recorded. A measurement over its allowance is reported and never fails a check.
|
|
232
232
|
|
|
233
233
|
Opinions (parts of kind `critique`, the verdicts of [automated UI reviewers](critics.md)) are never counted as checks,
|
package/docs/rich-evidence.md
CHANGED
|
@@ -191,6 +191,11 @@ time. Every field after `unit` is optional and none of them decides a status, a
|
|
|
191
191
|
- **Reading it from the API.** `GET /api/reports/tests/history` entries carry `latency` (their product-latency
|
|
192
192
|
measurements) next to `durationMs`. The pure functions are in `@crvouga/atlas/reports/latency`.
|
|
193
193
|
|
|
194
|
+
## A catalog of UI states
|
|
195
|
+
|
|
196
|
+
A screenshot per screen, per viewport and color scheme, is a *UI catalog*: check ids are the states, the viewport and scheme are
|
|
197
|
+
`parameters`, and the viewer's `/catalog` shows them with what is still to be captured. See [the UI catalog](ui-catalog.md).
|
|
198
|
+
|
|
194
199
|
## Finding runs and artifacts
|
|
195
200
|
|
|
196
201
|
The viewer's **Runs** and **Artifacts** tabs and the API answer one query, so a link means the same thing everywhere.
|
package/docs/server.md
CHANGED
|
@@ -522,6 +522,8 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
|
|
|
522
522
|
| `GET /api/projects/{project}/reports/runs` | read | The runs that recorded the report. |
|
|
523
523
|
| `GET /api/reports/artifacts` | read | Recorded results that carry evidence, across the whole history. |
|
|
524
524
|
| `GET /api/projects/{project}/reports/artifacts` | read | Recorded results that carry evidence, across the whole history. |
|
|
525
|
+
| `GET /api/reports/ui-catalog` | read | The UI catalog: captured UI states per profile, and what is still expected. |
|
|
526
|
+
| `GET /api/projects/{project}/reports/ui-catalog` | read | The UI catalog: captured UI states per profile, and what is still expected. |
|
|
525
527
|
| `GET /api/reports/runs/detail` | read | Every test one run recorded. |
|
|
526
528
|
| `GET /api/projects/{project}/reports/runs/detail` | read | Every test one run recorded. |
|
|
527
529
|
| `GET /api/reports/outline` | read | The specs a report is laid over. |
|
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
# The UI catalog
|
|
2
|
+
|
|
3
|
+
A browsable inventory of a product's user interface: each **state** (a screen, or one state of a screen) captured once per
|
|
4
|
+
**profile** (a viewport, a color scheme, a device), with the facts needed to find and trust it, and what should exist but has not
|
|
5
|
+
been captured yet. Any program that can write a layer can feed it, and the catalog fills in as results stream in, so a run of
|
|
6
|
+
thousands of captures can be watched while it happens and split across as many machines as needed.
|
|
7
|
+
|
|
8
|
+
Nothing about it is new to the report contract. A state is an ordinary check with a screenshot, the expected set is an ordinary
|
|
9
|
+
layer, and the stream is the one every viewer already holds. This page is the convention that makes those usable as a catalog, the
|
|
10
|
+
decisions behind it, and what is not built yet.
|
|
11
|
+
|
|
12
|
+
- [How a state is written](#how-a-state-is-written)
|
|
13
|
+
- [What is expected](#what-is-expected)
|
|
14
|
+
- [Publishing from any collector](#publishing-from-any-collector)
|
|
15
|
+
- [Sharding](#sharding)
|
|
16
|
+
- [Streaming and loading](#streaming-and-loading)
|
|
17
|
+
- [Exploring it](#exploring-it)
|
|
18
|
+
- [Decisions](#decisions)
|
|
19
|
+
- [What stays in your project, and what is not built yet](#what-stays-in-your-project-and-what-is-not-built-yet)
|
|
20
|
+
|
|
21
|
+
## How a state is written
|
|
22
|
+
|
|
23
|
+
A state is one **check** in a layer whose part has `kind` **`ui-state`**.
|
|
24
|
+
|
|
25
|
+
| Part of the idea | Where it goes |
|
|
26
|
+
| --- | --- |
|
|
27
|
+
| The state's identity | The check `id`. Make it stable (`checkout/payment`, `home?cohort=new`): a new capture replaces an old one only if the id is the same. |
|
|
28
|
+
| How the capture went | `status`: `passed` is **captured**, `failed` is attempted and failed (say why in `detail`), `running` is being taken, `not-run` is skipped. |
|
|
29
|
+
| The picture | An attachment with `role: "screenshot"` (else the first image), stored in the layer by content or given as an `https` `uri`. `width` and `height` help the viewer lay it out. |
|
|
30
|
+
| The profile | The layer's `environment` (`desktop-web`, `mobile-web`, `ios`, `android`) and `parameters` (`{ "viewport": "390x844", "scheme": "dark" }`). Parameters split a slot, so the same state at another viewport is **another result, side by side**, never a replacement. |
|
|
31
|
+
| Facts the producer declares | The check's `extensions["atlas.ui-state"]`: `route`, `fixture`, `cohort`, `area`, `flags` (a list), `covers` (the product surfaces the state shows). All optional. |
|
|
32
|
+
| Facts that describe without splitting | The check's `labels` (`shard`, `browser`) and `tags`. Labels never make a different slot. |
|
|
33
|
+
| Where it came from | The layer's `producer`, `sha`, `branch`, `runUrl`, `freshness`, `recordedAtMs`: shown with every capture, with the layer id and the backend that holds it. |
|
|
34
|
+
|
|
35
|
+
```json
|
|
36
|
+
{
|
|
37
|
+
"producer": "device-farm", "kind": "ui-state", "environment": "mobile-web",
|
|
38
|
+
"parameters": { "viewport": "390x844", "scheme": "dark" }, "recordedAtMs": 1760000000000,
|
|
39
|
+
"checks": [{
|
|
40
|
+
"id": "checkout/payment", "title": "Payment, card declined", "status": "passed",
|
|
41
|
+
"labels": { "shard": "2/8" },
|
|
42
|
+
"extensions": { "atlas.ui-state": { "route": "/checkout", "fixture": "declined-card", "cohort": "new-customer", "flags": ["new-nav"], "covers": ["payment-form"] } },
|
|
43
|
+
"attachments": [{ "name": "payment.png", "role": "screenshot", "contentType": "image/png", "path": "payment.png" }]
|
|
44
|
+
}]
|
|
45
|
+
}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
`atlas report publish --checks file.json --store …` publishes that file, reading `payment.png` from beside it. Nothing in it
|
|
49
|
+
mentions how the page was driven. Because a state is a check, the gate counts it (a `failed` capture fails `atlas report status`),
|
|
50
|
+
`atlas report status` lists it, and a reader that does not know the catalog still shows a check with a screenshot. Evidence never
|
|
51
|
+
decides a result: a picture that cannot be read leaves the check, its status and its slot as they were.
|
|
52
|
+
|
|
53
|
+
## What is expected
|
|
54
|
+
|
|
55
|
+
What should exist is a **layer too**, of kind **`ui-plan`**: the same ids, the same profiles (environment and parameters), every
|
|
56
|
+
check `not-run`, published by a planner, by each shard for the states it was given, or both. A plan fills a slot of its own (the
|
|
57
|
+
kind is part of the slot), so a capture never erases it and it never hides a capture; and `not-run` is never a pass.
|
|
58
|
+
|
|
59
|
+
| A cell (one state in one profile) is | When |
|
|
60
|
+
| --- | --- |
|
|
61
|
+
| **Captured** | the winning capture passed |
|
|
62
|
+
| **Capture failed** | the winning capture failed (its `detail` says why, and a diagnostic picture may be attached) |
|
|
63
|
+
| **Capturing** | the winning result is `running` |
|
|
64
|
+
| **Skipped** | a producer recorded it as `not-run` |
|
|
65
|
+
| **Not captured yet** | a plan expects it and nothing has been recorded |
|
|
66
|
+
|
|
67
|
+
A captured cell that no plan names is shown as **not in the expected set**. **Without any plan, nothing is "not captured yet" and
|
|
68
|
+
nothing is "not in the expected set"**: what should exist is then unknown, and the catalog says so instead of guessing. "Not yet
|
|
69
|
+
captured" is therefore different from "never declared".
|
|
70
|
+
|
|
71
|
+
Coverage against an **inventory of the product's surfaces** is a plan entry with `"inventory": true` (the id is the surface;
|
|
72
|
+
nothing is captured for it). A state says which surfaces it shows in `covers`. The catalog lists every surface as *covered* (a
|
|
73
|
+
state that shows it is captured), *states cover it, none captured yet*, or *no state covers it*.
|
|
74
|
+
|
|
75
|
+
## Publishing from any collector
|
|
76
|
+
|
|
77
|
+
Any program that writes the layer format feeds the catalog: a shell script, a Go test, a Storybook crawler, a device farm.
|
|
78
|
+
|
|
79
|
+
```ts
|
|
80
|
+
import { publishChecks } from '@crvouga/atlas/reports/publish-checks';
|
|
81
|
+
import { uiPlanLayer, uiStateLayer } from '@crvouga/atlas/reports/ui-catalog-publish';
|
|
82
|
+
|
|
83
|
+
// The planner, once: what the product should have, in every profile.
|
|
84
|
+
await publishChecks(store, uiPlanLayer({
|
|
85
|
+
producer: 'catalog-plan', recordedAtMs: Date.now(), environment: 'mobile-web', profile: { viewport: '390x844', scheme: 'dark' },
|
|
86
|
+
states: [{ id: 'checkout/payment', covers: ['payment-form'] }, { id: 'settings', inventory: true }]
|
|
87
|
+
}));
|
|
88
|
+
|
|
89
|
+
// Each collector or shard, with its own captures and no knowledge of the others.
|
|
90
|
+
await publishChecks(store, uiStateLayer({
|
|
91
|
+
producer: 'web-capture', recordedAtMs: Date.now(), environment: 'mobile-web', profile: { viewport: '390x844', scheme: 'dark' }, shard: '2/8',
|
|
92
|
+
states: [{ id: 'checkout/payment', route: '/checkout', covers: ['payment-form'], screenshot: { data: png, width: 390, height: 844 } }]
|
|
93
|
+
}));
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
`uiStateLayer` and `uiPlanLayer` only build the input of `publishChecks`; they use no browser and no test framework. `store` can be
|
|
97
|
+
a directory, a bucket or the server (`httpStore`), exactly as for any layer.
|
|
98
|
+
|
|
99
|
+
**Playwright is one adapter and not a dependency.** `captureUiState(page, { id, ready })` from `@crvouga/atlas/playwright` waits
|
|
100
|
+
for a readiness check (bounded), photographs the page and returns the capture for `uiStateLayer`. A page that never became ready is
|
|
101
|
+
a `failed` capture that says why (with a picture of what it showed), never an exception that ends the run. Nothing that reads,
|
|
102
|
+
composes or shows the catalog imports a collection framework, and a test keeps it that way.
|
|
103
|
+
|
|
104
|
+
## Sharding
|
|
105
|
+
|
|
106
|
+
A shard is **any publisher that does not know the others**: by case, by profile, by area or by machine; Atlas does not care which.
|
|
107
|
+
Each publishes its own layers (a `shard` label records which, and is shown with each capture). The guarantees are those of every
|
|
108
|
+
layer ([the contract](report-contract.md#guarantees)), which is what the tests prove for the catalog:
|
|
109
|
+
|
|
110
|
+
- The captures of N shards (N is 1 or more), combined, equal the catalog one unsharded run produces.
|
|
111
|
+
- Running a shard again replaces only the cells it produces (a newer `recordedAtMs` wins its slot); every other cell is untouched.
|
|
112
|
+
- A shard that never publishes leaves its cells **not captured yet** when a plan names them, and absent otherwise. They are never
|
|
113
|
+
shown as captured, and the other shards' cells are unaffected.
|
|
114
|
+
- Atlas does not need the shard count. Completeness is the plan: a cell is expected because a plan says so.
|
|
115
|
+
|
|
116
|
+
## Streaming and loading
|
|
117
|
+
|
|
118
|
+
- **Streaming.** There is no second stream. A viewer holds the report over `/api/events` and reads what changed with
|
|
119
|
+
`/api/reports/evidence/effective?since=<version>`; the catalog is computed from the report it holds, so a capture appears when
|
|
120
|
+
its layer arrives, with no reload and no file rebuilt. Events carry metadata only; the picture is fetched when a card is on screen.
|
|
121
|
+
- **Reconnecting.** A viewer that was offline or whose stream dropped reads the delta since the version it holds (or the whole report
|
|
122
|
+
when the server no longer remembers it). Cells are keyed by state and profile, so nothing is missing and nothing is duplicated.
|
|
123
|
+
- **No stream.** A viewer without the stream (or a static export) polls, as the rest of the viewer does, and shows everything that exists.
|
|
124
|
+
- **Slow and failing backends.** Each backend is listed and read on its own, so a slow one never holds back the others: their
|
|
125
|
+
captures are shown, and the slow backend's appear when it answers. A backend that fails, or a layer that cannot be read, is named
|
|
126
|
+
on the page and in the document (`sources`); a capture that depended on it is not shown as captured.
|
|
127
|
+
- **Images.** A picture is a file in the layer, addressed by content (`/layers/<layer>/blobs/attachments/<sha256>`), loaded lazily
|
|
128
|
+
for the cards on screen, and served under the same untrusted-bytes rules as every attachment. A picture given as a `uri` is
|
|
129
|
+
followed only if it is `http` or `https`. A producer may leave the bytes out (`omitted`), which is told apart from never taken.
|
|
130
|
+
|
|
131
|
+
## Exploring it
|
|
132
|
+
|
|
133
|
+
The viewer's **UI catalog** (`/catalog`, in the menu beside the spec types once the report holds a state or a plan) shows a card per
|
|
134
|
+
state with its picture and a coverage badge per profile, and filters by words, coverage, profile, area, route, cohort, feature flag,
|
|
135
|
+
surface and producer. The address holds the filters and the open state, so a link reproduces the view. Opening a state shows its
|
|
136
|
+
profiles side by side with the declared facts, the failure reason, and for each picture the layer, backend, producer, shard, commit
|
|
137
|
+
and time. *Download the manifest* saves the filtered catalog as JSON.
|
|
138
|
+
|
|
139
|
+
```sh
|
|
140
|
+
curl "$ATLAS_SERVER_URL/api/reports/ui-catalog?coverage=failed,missing&profile=…" # JSON; add format=markdown for text
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
`GET /api/reports/ui-catalog` answers the same document the viewer computes (`uiCatalog` from
|
|
144
|
+
`@crvouga/atlas/reports/ui-catalog`), filtered by `q`, `coverage`, `profile`, `area`, `route`, `cohort`, `flag`, `surface` and
|
|
145
|
+
`producer`, paged with `limit` and `offset`, with an ETag. Its picture `path` is relative to `/api/reports`.
|
|
146
|
+
|
|
147
|
+
## Decisions
|
|
148
|
+
|
|
149
|
+
The questions the original proposal left open, and how they are answered here. Each is the simplest answer that stays inside the
|
|
150
|
+
[stable contracts](report-contract.md#versioning-and-stability): optional fields only, no change to a slot or a winner.
|
|
151
|
+
|
|
152
|
+
| Question | Decision |
|
|
153
|
+
| --- | --- |
|
|
154
|
+
| Layer and event schema | An existing check in a `ui-state` part, a `screenshot` attachment, facts under the `atlas.ui-state` extension. No new part type, no new schema version. The report row gained one optional field, `state`, only for `ui-state` and `ui-plan` rows. |
|
|
155
|
+
| Identity and slot | The check id is the state; the slot is the usual (producer, kind, spec, id, environment, parameters). Profile dimensions are `parameters` because they must split the slot; labels never do. |
|
|
156
|
+
| Expected versus captured | A `ui-plan` layer. `ui-plan` is a diagnostic kind (like `performance`): it qualifies a check and is not counted as one when the check has any other result. |
|
|
157
|
+
| Coverage against an inventory | `inventory: true` plan entries plus each state's `covers`. |
|
|
158
|
+
| Carrying images | By content in the layer (`bundleKey`) or by `uri`; loaded lazily; the existing size limits of an upload apply. |
|
|
159
|
+
| The event stream | No new stream or events: the existing `report` event and `?since=` deltas. |
|
|
160
|
+
| Sharding | No shard concept in the format; a shard is any publisher. A `shard` label is provenance. Completeness comes from the plan, so Atlas needs no shard count. |
|
|
161
|
+
| What moves | Reading, composing, the viewer, the document and the API, the publish builders, a Playwright capture adapter. Staying with a product: its state declarations, its inventory, its fixtures and its runner. |
|
|
162
|
+
| Where it lives | `@crvouga/atlas/reports/ui-catalog` (model), `…/ui-catalog-publish` (builders), `…/ui-state` (convention); the viewer's `/catalog`; the server's `/api/reports/ui-catalog`; `@crvouga/atlas/playwright`'s `captureUiState`. |
|
|
163
|
+
| Exploration capabilities | Search, filters, profile comparison, coverage and surface coverage, failure diagnostics, deep links, manifest export. |
|
|
164
|
+
| Static export | The catalog is computed from the layers a viewer holds, so a static export (`atlas client build`) shows it; it just does not stream. |
|
|
165
|
+
|
|
166
|
+
## What stays in your project, and what is not built yet
|
|
167
|
+
|
|
168
|
+
**Stays in a project** (it is specific to the product): the list of states and profiles to capture, the inventory of surfaces,
|
|
169
|
+
fixtures and test data, the runner that drives the product to each state, and the readiness conditions of each screen.
|
|
170
|
+
|
|
171
|
+
**Not built yet:** a command line to list and publish a catalog without a script (use `atlas report publish --checks` today), a
|
|
172
|
+
dedicated static single-file export, diff views that compare two commits of a state, and a hosting-platform publish step beyond
|
|
173
|
+
`atlas serve`. The first slice makes the catalog explorable, streaming, shardable and independent of how it was collected.
|
package/docs/ui.md
CHANGED
|
@@ -96,6 +96,20 @@ when Watch is on screen.
|
|
|
96
96
|
- The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
|
|
97
97
|
none are listed under "No domain declared"; the Tests view reports domain problems).
|
|
98
98
|
|
|
99
|
+
## UI catalog
|
|
100
|
+
|
|
101
|
+
`/catalog` (`catalog/`) is the viewer's inventory of captured UI states ([the guide](ui-catalog.md)). It is computed in the page from
|
|
102
|
+
the layers the viewer already holds (`catalog/provider.tsx`, only while the page is open), so states appear as results stream in and
|
|
103
|
+
it works without a server. Cards are Mantine `Card`s in a responsive `SimpleGrid`; the filters live in the shell's navigation column
|
|
104
|
+
(`UiCatalogFilters`) and the open state in a `Drawer`, and both are the address (`catalog/search.ts`).
|
|
105
|
+
|
|
106
|
+
- Only `captured` is called captured. Every other coverage (`Capture failed`, `Capturing`, `Skipped`, `Not captured yet`) has its own
|
|
107
|
+
words and mark, and a cell without a picture says so in words instead of showing an empty frame. Never show a plan as a result.
|
|
108
|
+
- A backend that is not available, a notice, and layers that could not be read are shown above the grid; the catalog says it may be
|
|
109
|
+
incomplete. Waiting follows the loading policy below, and layers still arriving are counted.
|
|
110
|
+
- Pictures are `loading="lazy"` and only the cards on screen are fetched. The grid is bounded with *Show more*.
|
|
111
|
+
- The open state shows every profile side by side with the layer, backend, producer, shard, commit and time of each picture.
|
|
112
|
+
|
|
99
113
|
## Reliability
|
|
100
114
|
|
|
101
115
|
An open test shows its reliability (`explorer/Reliability.tsx`, worked out by `reports/reliability` from the history the page
|
|
@@ -276,6 +290,7 @@ The narrowly scoped exceptions are:
|
|
|
276
290
|
| Area | Screens/interactions | Mantine composition |
|
|
277
291
|
| --- | --- | --- |
|
|
278
292
|
| Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
|
|
293
|
+
| UI catalog | Captured UI states per profile as cards with coverage badges, filters in the index column, surface coverage, an open state with its profiles side by side and where each picture came from, manifest download | AppShell navbar, SimpleGrid, Card, Image, MultiSelect, TextInput, Drawer, Accordion, Table, Badge, Alert |
|
|
279
294
|
| Project home | Tests/runs/artifacts tabs, a run's Artifacts (files of its layers, shared recordings, direct links), Watch playlist of a filtered set, metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion, Modal, Grid, Switch |
|
|
280
295
|
| Test detail | Current outcomes, statistics, duration graph, product latency with budgets and baselines, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
|
|
281
296
|
| Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
|
package/llms-full.txt
CHANGED
|
@@ -1179,6 +1179,11 @@ time. Every field after `unit` is optional and none of them decides a status, a
|
|
|
1179
1179
|
- **Reading it from the API.** `GET /api/reports/tests/history` entries carry `latency` (their product-latency
|
|
1180
1180
|
measurements) next to `durationMs`. The pure functions are in `@crvouga/atlas/reports/latency`.
|
|
1181
1181
|
|
|
1182
|
+
## A catalog of UI states
|
|
1183
|
+
|
|
1184
|
+
A screenshot per screen, per viewport and color scheme, is a *UI catalog*: check ids are the states, the viewport and scheme are
|
|
1185
|
+
`parameters`, and the viewer's `/catalog` shows them with what is still to be captured. See [the UI catalog](ui-catalog.md).
|
|
1186
|
+
|
|
1182
1187
|
## Finding runs and artifacts
|
|
1183
1188
|
|
|
1184
1189
|
The viewer's **Runs** and **Artifacts** tabs and the API answer one query, so a link means the same thing everywhere.
|
|
@@ -1464,6 +1469,182 @@ This is a statement about publication. Atlas does not check that a stored video
|
|
|
1464
1469
|
re-check later that a stored file is still readable.
|
|
1465
1470
|
</doc>
|
|
1466
1471
|
|
|
1472
|
+
<doc path="docs/ui-catalog.md">
|
|
1473
|
+
# The UI catalog
|
|
1474
|
+
|
|
1475
|
+
A browsable inventory of a product's user interface: each **state** (a screen, or one state of a screen) captured once per
|
|
1476
|
+
**profile** (a viewport, a color scheme, a device), with the facts needed to find and trust it, and what should exist but has not
|
|
1477
|
+
been captured yet. Any program that can write a layer can feed it, and the catalog fills in as results stream in, so a run of
|
|
1478
|
+
thousands of captures can be watched while it happens and split across as many machines as needed.
|
|
1479
|
+
|
|
1480
|
+
Nothing about it is new to the report contract. A state is an ordinary check with a screenshot, the expected set is an ordinary
|
|
1481
|
+
layer, and the stream is the one every viewer already holds. This page is the convention that makes those usable as a catalog, the
|
|
1482
|
+
decisions behind it, and what is not built yet.
|
|
1483
|
+
|
|
1484
|
+
- [How a state is written](#how-a-state-is-written)
|
|
1485
|
+
- [What is expected](#what-is-expected)
|
|
1486
|
+
- [Publishing from any collector](#publishing-from-any-collector)
|
|
1487
|
+
- [Sharding](#sharding)
|
|
1488
|
+
- [Streaming and loading](#streaming-and-loading)
|
|
1489
|
+
- [Exploring it](#exploring-it)
|
|
1490
|
+
- [Decisions](#decisions)
|
|
1491
|
+
- [What stays in your project, and what is not built yet](#what-stays-in-your-project-and-what-is-not-built-yet)
|
|
1492
|
+
|
|
1493
|
+
## How a state is written
|
|
1494
|
+
|
|
1495
|
+
A state is one **check** in a layer whose part has `kind` **`ui-state`**.
|
|
1496
|
+
|
|
1497
|
+
| Part of the idea | Where it goes |
|
|
1498
|
+
| --- | --- |
|
|
1499
|
+
| The state's identity | The check `id`. Make it stable (`checkout/payment`, `home?cohort=new`): a new capture replaces an old one only if the id is the same. |
|
|
1500
|
+
| How the capture went | `status`: `passed` is **captured**, `failed` is attempted and failed (say why in `detail`), `running` is being taken, `not-run` is skipped. |
|
|
1501
|
+
| The picture | An attachment with `role: "screenshot"` (else the first image), stored in the layer by content or given as an `https` `uri`. `width` and `height` help the viewer lay it out. |
|
|
1502
|
+
| The profile | The layer's `environment` (`desktop-web`, `mobile-web`, `ios`, `android`) and `parameters` (`{ "viewport": "390x844", "scheme": "dark" }`). Parameters split a slot, so the same state at another viewport is **another result, side by side**, never a replacement. |
|
|
1503
|
+
| Facts the producer declares | The check's `extensions["atlas.ui-state"]`: `route`, `fixture`, `cohort`, `area`, `flags` (a list), `covers` (the product surfaces the state shows). All optional. |
|
|
1504
|
+
| Facts that describe without splitting | The check's `labels` (`shard`, `browser`) and `tags`. Labels never make a different slot. |
|
|
1505
|
+
| Where it came from | The layer's `producer`, `sha`, `branch`, `runUrl`, `freshness`, `recordedAtMs`: shown with every capture, with the layer id and the backend that holds it. |
|
|
1506
|
+
|
|
1507
|
+
```json
|
|
1508
|
+
{
|
|
1509
|
+
"producer": "device-farm", "kind": "ui-state", "environment": "mobile-web",
|
|
1510
|
+
"parameters": { "viewport": "390x844", "scheme": "dark" }, "recordedAtMs": 1760000000000,
|
|
1511
|
+
"checks": [{
|
|
1512
|
+
"id": "checkout/payment", "title": "Payment, card declined", "status": "passed",
|
|
1513
|
+
"labels": { "shard": "2/8" },
|
|
1514
|
+
"extensions": { "atlas.ui-state": { "route": "/checkout", "fixture": "declined-card", "cohort": "new-customer", "flags": ["new-nav"], "covers": ["payment-form"] } },
|
|
1515
|
+
"attachments": [{ "name": "payment.png", "role": "screenshot", "contentType": "image/png", "path": "payment.png" }]
|
|
1516
|
+
}]
|
|
1517
|
+
}
|
|
1518
|
+
```
|
|
1519
|
+
|
|
1520
|
+
`atlas report publish --checks file.json --store …` publishes that file, reading `payment.png` from beside it. Nothing in it
|
|
1521
|
+
mentions how the page was driven. Because a state is a check, the gate counts it (a `failed` capture fails `atlas report status`),
|
|
1522
|
+
`atlas report status` lists it, and a reader that does not know the catalog still shows a check with a screenshot. Evidence never
|
|
1523
|
+
decides a result: a picture that cannot be read leaves the check, its status and its slot as they were.
|
|
1524
|
+
|
|
1525
|
+
## What is expected
|
|
1526
|
+
|
|
1527
|
+
What should exist is a **layer too**, of kind **`ui-plan`**: the same ids, the same profiles (environment and parameters), every
|
|
1528
|
+
check `not-run`, published by a planner, by each shard for the states it was given, or both. A plan fills a slot of its own (the
|
|
1529
|
+
kind is part of the slot), so a capture never erases it and it never hides a capture; and `not-run` is never a pass.
|
|
1530
|
+
|
|
1531
|
+
| A cell (one state in one profile) is | When |
|
|
1532
|
+
| --- | --- |
|
|
1533
|
+
| **Captured** | the winning capture passed |
|
|
1534
|
+
| **Capture failed** | the winning capture failed (its `detail` says why, and a diagnostic picture may be attached) |
|
|
1535
|
+
| **Capturing** | the winning result is `running` |
|
|
1536
|
+
| **Skipped** | a producer recorded it as `not-run` |
|
|
1537
|
+
| **Not captured yet** | a plan expects it and nothing has been recorded |
|
|
1538
|
+
|
|
1539
|
+
A captured cell that no plan names is shown as **not in the expected set**. **Without any plan, nothing is "not captured yet" and
|
|
1540
|
+
nothing is "not in the expected set"**: what should exist is then unknown, and the catalog says so instead of guessing. "Not yet
|
|
1541
|
+
captured" is therefore different from "never declared".
|
|
1542
|
+
|
|
1543
|
+
Coverage against an **inventory of the product's surfaces** is a plan entry with `"inventory": true` (the id is the surface;
|
|
1544
|
+
nothing is captured for it). A state says which surfaces it shows in `covers`. The catalog lists every surface as *covered* (a
|
|
1545
|
+
state that shows it is captured), *states cover it, none captured yet*, or *no state covers it*.
|
|
1546
|
+
|
|
1547
|
+
## Publishing from any collector
|
|
1548
|
+
|
|
1549
|
+
Any program that writes the layer format feeds the catalog: a shell script, a Go test, a Storybook crawler, a device farm.
|
|
1550
|
+
|
|
1551
|
+
```ts
|
|
1552
|
+
import { publishChecks } from '@crvouga/atlas/reports/publish-checks';
|
|
1553
|
+
import { uiPlanLayer, uiStateLayer } from '@crvouga/atlas/reports/ui-catalog-publish';
|
|
1554
|
+
|
|
1555
|
+
// The planner, once: what the product should have, in every profile.
|
|
1556
|
+
await publishChecks(store, uiPlanLayer({
|
|
1557
|
+
producer: 'catalog-plan', recordedAtMs: Date.now(), environment: 'mobile-web', profile: { viewport: '390x844', scheme: 'dark' },
|
|
1558
|
+
states: [{ id: 'checkout/payment', covers: ['payment-form'] }, { id: 'settings', inventory: true }]
|
|
1559
|
+
}));
|
|
1560
|
+
|
|
1561
|
+
// Each collector or shard, with its own captures and no knowledge of the others.
|
|
1562
|
+
await publishChecks(store, uiStateLayer({
|
|
1563
|
+
producer: 'web-capture', recordedAtMs: Date.now(), environment: 'mobile-web', profile: { viewport: '390x844', scheme: 'dark' }, shard: '2/8',
|
|
1564
|
+
states: [{ id: 'checkout/payment', route: '/checkout', covers: ['payment-form'], screenshot: { data: png, width: 390, height: 844 } }]
|
|
1565
|
+
}));
|
|
1566
|
+
```
|
|
1567
|
+
|
|
1568
|
+
`uiStateLayer` and `uiPlanLayer` only build the input of `publishChecks`; they use no browser and no test framework. `store` can be
|
|
1569
|
+
a directory, a bucket or the server (`httpStore`), exactly as for any layer.
|
|
1570
|
+
|
|
1571
|
+
**Playwright is one adapter and not a dependency.** `captureUiState(page, { id, ready })` from `@crvouga/atlas/playwright` waits
|
|
1572
|
+
for a readiness check (bounded), photographs the page and returns the capture for `uiStateLayer`. A page that never became ready is
|
|
1573
|
+
a `failed` capture that says why (with a picture of what it showed), never an exception that ends the run. Nothing that reads,
|
|
1574
|
+
composes or shows the catalog imports a collection framework, and a test keeps it that way.
|
|
1575
|
+
|
|
1576
|
+
## Sharding
|
|
1577
|
+
|
|
1578
|
+
A shard is **any publisher that does not know the others**: by case, by profile, by area or by machine; Atlas does not care which.
|
|
1579
|
+
Each publishes its own layers (a `shard` label records which, and is shown with each capture). The guarantees are those of every
|
|
1580
|
+
layer ([the contract](report-contract.md#guarantees)), which is what the tests prove for the catalog:
|
|
1581
|
+
|
|
1582
|
+
- The captures of N shards (N is 1 or more), combined, equal the catalog one unsharded run produces.
|
|
1583
|
+
- Running a shard again replaces only the cells it produces (a newer `recordedAtMs` wins its slot); every other cell is untouched.
|
|
1584
|
+
- A shard that never publishes leaves its cells **not captured yet** when a plan names them, and absent otherwise. They are never
|
|
1585
|
+
shown as captured, and the other shards' cells are unaffected.
|
|
1586
|
+
- Atlas does not need the shard count. Completeness is the plan: a cell is expected because a plan says so.
|
|
1587
|
+
|
|
1588
|
+
## Streaming and loading
|
|
1589
|
+
|
|
1590
|
+
- **Streaming.** There is no second stream. A viewer holds the report over `/api/events` and reads what changed with
|
|
1591
|
+
`/api/reports/evidence/effective?since=<version>`; the catalog is computed from the report it holds, so a capture appears when
|
|
1592
|
+
its layer arrives, with no reload and no file rebuilt. Events carry metadata only; the picture is fetched when a card is on screen.
|
|
1593
|
+
- **Reconnecting.** A viewer that was offline or whose stream dropped reads the delta since the version it holds (or the whole report
|
|
1594
|
+
when the server no longer remembers it). Cells are keyed by state and profile, so nothing is missing and nothing is duplicated.
|
|
1595
|
+
- **No stream.** A viewer without the stream (or a static export) polls, as the rest of the viewer does, and shows everything that exists.
|
|
1596
|
+
- **Slow and failing backends.** Each backend is listed and read on its own, so a slow one never holds back the others: their
|
|
1597
|
+
captures are shown, and the slow backend's appear when it answers. A backend that fails, or a layer that cannot be read, is named
|
|
1598
|
+
on the page and in the document (`sources`); a capture that depended on it is not shown as captured.
|
|
1599
|
+
- **Images.** A picture is a file in the layer, addressed by content (`/layers/<layer>/blobs/attachments/<sha256>`), loaded lazily
|
|
1600
|
+
for the cards on screen, and served under the same untrusted-bytes rules as every attachment. A picture given as a `uri` is
|
|
1601
|
+
followed only if it is `http` or `https`. A producer may leave the bytes out (`omitted`), which is told apart from never taken.
|
|
1602
|
+
|
|
1603
|
+
## Exploring it
|
|
1604
|
+
|
|
1605
|
+
The viewer's **UI catalog** (`/catalog`, in the menu beside the spec types once the report holds a state or a plan) shows a card per
|
|
1606
|
+
state with its picture and a coverage badge per profile, and filters by words, coverage, profile, area, route, cohort, feature flag,
|
|
1607
|
+
surface and producer. The address holds the filters and the open state, so a link reproduces the view. Opening a state shows its
|
|
1608
|
+
profiles side by side with the declared facts, the failure reason, and for each picture the layer, backend, producer, shard, commit
|
|
1609
|
+
and time. *Download the manifest* saves the filtered catalog as JSON.
|
|
1610
|
+
|
|
1611
|
+
```sh
|
|
1612
|
+
curl "$ATLAS_SERVER_URL/api/reports/ui-catalog?coverage=failed,missing&profile=…" # JSON; add format=markdown for text
|
|
1613
|
+
```
|
|
1614
|
+
|
|
1615
|
+
`GET /api/reports/ui-catalog` answers the same document the viewer computes (`uiCatalog` from
|
|
1616
|
+
`@crvouga/atlas/reports/ui-catalog`), filtered by `q`, `coverage`, `profile`, `area`, `route`, `cohort`, `flag`, `surface` and
|
|
1617
|
+
`producer`, paged with `limit` and `offset`, with an ETag. Its picture `path` is relative to `/api/reports`.
|
|
1618
|
+
|
|
1619
|
+
## Decisions
|
|
1620
|
+
|
|
1621
|
+
The questions the original proposal left open, and how they are answered here. Each is the simplest answer that stays inside the
|
|
1622
|
+
[stable contracts](report-contract.md#versioning-and-stability): optional fields only, no change to a slot or a winner.
|
|
1623
|
+
|
|
1624
|
+
| Question | Decision |
|
|
1625
|
+
| --- | --- |
|
|
1626
|
+
| Layer and event schema | An existing check in a `ui-state` part, a `screenshot` attachment, facts under the `atlas.ui-state` extension. No new part type, no new schema version. The report row gained one optional field, `state`, only for `ui-state` and `ui-plan` rows. |
|
|
1627
|
+
| Identity and slot | The check id is the state; the slot is the usual (producer, kind, spec, id, environment, parameters). Profile dimensions are `parameters` because they must split the slot; labels never do. |
|
|
1628
|
+
| Expected versus captured | A `ui-plan` layer. `ui-plan` is a diagnostic kind (like `performance`): it qualifies a check and is not counted as one when the check has any other result. |
|
|
1629
|
+
| Coverage against an inventory | `inventory: true` plan entries plus each state's `covers`. |
|
|
1630
|
+
| Carrying images | By content in the layer (`bundleKey`) or by `uri`; loaded lazily; the existing size limits of an upload apply. |
|
|
1631
|
+
| The event stream | No new stream or events: the existing `report` event and `?since=` deltas. |
|
|
1632
|
+
| Sharding | No shard concept in the format; a shard is any publisher. A `shard` label is provenance. Completeness comes from the plan, so Atlas needs no shard count. |
|
|
1633
|
+
| What moves | Reading, composing, the viewer, the document and the API, the publish builders, a Playwright capture adapter. Staying with a product: its state declarations, its inventory, its fixtures and its runner. |
|
|
1634
|
+
| Where it lives | `@crvouga/atlas/reports/ui-catalog` (model), `…/ui-catalog-publish` (builders), `…/ui-state` (convention); the viewer's `/catalog`; the server's `/api/reports/ui-catalog`; `@crvouga/atlas/playwright`'s `captureUiState`. |
|
|
1635
|
+
| Exploration capabilities | Search, filters, profile comparison, coverage and surface coverage, failure diagnostics, deep links, manifest export. |
|
|
1636
|
+
| Static export | The catalog is computed from the layers a viewer holds, so a static export (`atlas client build`) shows it; it just does not stream. |
|
|
1637
|
+
|
|
1638
|
+
## What stays in your project, and what is not built yet
|
|
1639
|
+
|
|
1640
|
+
**Stays in a project** (it is specific to the product): the list of states and profiles to capture, the inventory of surfaces,
|
|
1641
|
+
fixtures and test data, the runner that drives the product to each state, and the readiness conditions of each screen.
|
|
1642
|
+
|
|
1643
|
+
**Not built yet:** a command line to list and publish a catalog without a script (use `atlas report publish --checks` today), a
|
|
1644
|
+
dedicated static single-file export, diff views that compare two commits of a state, and a hosting-platform publish step beyond
|
|
1645
|
+
`atlas serve`. The first slice makes the catalog explorable, streaming, shardable and independent of how it was collected.
|
|
1646
|
+
</doc>
|
|
1647
|
+
|
|
1467
1648
|
<doc path="docs/server.md">
|
|
1468
1649
|
# The Atlas server
|
|
1469
1650
|
|
|
@@ -1989,6 +2170,8 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
|
|
|
1989
2170
|
| `GET /api/projects/{project}/reports/runs` | read | The runs that recorded the report. |
|
|
1990
2171
|
| `GET /api/reports/artifacts` | read | Recorded results that carry evidence, across the whole history. |
|
|
1991
2172
|
| `GET /api/projects/{project}/reports/artifacts` | read | Recorded results that carry evidence, across the whole history. |
|
|
2173
|
+
| `GET /api/reports/ui-catalog` | read | The UI catalog: captured UI states per profile, and what is still expected. |
|
|
2174
|
+
| `GET /api/projects/{project}/reports/ui-catalog` | read | The UI catalog: captured UI states per profile, and what is still expected. |
|
|
1992
2175
|
| `GET /api/reports/runs/detail` | read | Every test one run recorded. |
|
|
1993
2176
|
| `GET /api/projects/{project}/reports/runs/detail` | read | Every test one run recorded. |
|
|
1994
2177
|
| `GET /api/reports/outline` | read | The specs a report is laid over. |
|
|
@@ -2655,7 +2838,7 @@ Zod schema as `LayerDocumentSchema` from `@crvouga/atlas/schema`.
|
|
|
2655
2838
|
| `freshness` | `modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Advisory only; see [Statuses](#statuses-and-the-gate). |
|
|
2656
2839
|
| `run` | Optional `{ id, title?, startedAtMs?, endedAtMs? }`: the execution a local or manual producer names. Layers with the same `id` in one project are one run in the run list, however many shards or batches uploaded them; a later invocation at the same branch and commit gets another `id`. Without it a run is its `runUrl`, else `branch@sha`, as before. `title` is for people. Grouping and display only: it is **never part of the slot**, never decides a winner or a status, and never hides `sha`, `branch` or `freshness`. A `run` a reader cannot read is treated as absent. |
|
|
2657
2840
|
| `specs` | The specs the layer was recorded against, with a `fingerprint` of their text, so a reader can warn when a spec changed since. Optional. |
|
|
2658
|
-
| `parts[].kind` | An open category (`behavior`, `performance`, `accessibility`, …). A new kind needs no change to any reader. Part of the slot. |
|
|
2841
|
+
| `parts[].kind` | An open category (`behavior`, `performance`, `accessibility`, …). A new kind needs no change to any reader. Part of the slot. `ui-state` and `ui-plan` are the [UI catalog](ui-catalog.md)'s: a captured UI state, and the states that are expected. |
|
|
2659
2842
|
| `parts[].checks[]` | `id` (required, stable), `title`, `status`, `detail`, `specPath`, `measurements`, `durationMs`. |
|
|
2660
2843
|
| `tags`, `labels`, `links`, `attachments`, `annotations`, `extensions` | Evidence, on the layer, a part or a check: what a result carries beyond its outcome. All optional and open. See [evidence](#evidence) and [the guide](extending.md). |
|
|
2661
2844
|
| `checks[].steps`, `checks[].attempts` | A check's timeline, and every try when it was retried. Optional. |
|
|
@@ -2798,7 +2981,7 @@ are part of the contract:
|
|
|
2798
2981
|
Tools map to these four. CTRF `skipped`, `pending` and `other`, and JUnit `<skipped>`, are `not-run`.
|
|
2799
2982
|
An Atlas path that passed on retry (`flaky`) is `passed`, with the detail "Passed on retry (flaky)".
|
|
2800
2983
|
|
|
2801
|
-
Timings that only qualify a check (parts of kind `performance`) are not counted as checks when
|
|
2984
|
+
Timings that only qualify a check (parts of kind `performance`), and what a UI catalog expects (`ui-plan`), are not counted as checks when
|
|
2802
2985
|
anything else is recorded. A measurement over its allowance is reported and never fails a check.
|
|
2803
2986
|
|
|
2804
2987
|
Opinions (parts of kind `critique`, the verdicts of [automated UI reviewers](critics.md)) are never counted as checks,
|
|
@@ -4332,6 +4515,20 @@ when Watch is on screen.
|
|
|
4332
4515
|
- The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
|
|
4333
4516
|
none are listed under "No domain declared"; the Tests view reports domain problems).
|
|
4334
4517
|
|
|
4518
|
+
## UI catalog
|
|
4519
|
+
|
|
4520
|
+
`/catalog` (`catalog/`) is the viewer's inventory of captured UI states ([the guide](ui-catalog.md)). It is computed in the page from
|
|
4521
|
+
the layers the viewer already holds (`catalog/provider.tsx`, only while the page is open), so states appear as results stream in and
|
|
4522
|
+
it works without a server. Cards are Mantine `Card`s in a responsive `SimpleGrid`; the filters live in the shell's navigation column
|
|
4523
|
+
(`UiCatalogFilters`) and the open state in a `Drawer`, and both are the address (`catalog/search.ts`).
|
|
4524
|
+
|
|
4525
|
+
- Only `captured` is called captured. Every other coverage (`Capture failed`, `Capturing`, `Skipped`, `Not captured yet`) has its own
|
|
4526
|
+
words and mark, and a cell without a picture says so in words instead of showing an empty frame. Never show a plan as a result.
|
|
4527
|
+
- A backend that is not available, a notice, and layers that could not be read are shown above the grid; the catalog says it may be
|
|
4528
|
+
incomplete. Waiting follows the loading policy below, and layers still arriving are counted.
|
|
4529
|
+
- Pictures are `loading="lazy"` and only the cards on screen are fetched. The grid is bounded with *Show more*.
|
|
4530
|
+
- The open state shows every profile side by side with the layer, backend, producer, shard, commit and time of each picture.
|
|
4531
|
+
|
|
4335
4532
|
## Reliability
|
|
4336
4533
|
|
|
4337
4534
|
An open test shows its reliability (`explorer/Reliability.tsx`, worked out by `reports/reliability` from the history the page
|
|
@@ -4512,6 +4709,7 @@ The narrowly scoped exceptions are:
|
|
|
4512
4709
|
| Area | Screens/interactions | Mantine composition |
|
|
4513
4710
|
| --- | --- | --- |
|
|
4514
4711
|
| Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
|
|
4712
|
+
| UI catalog | Captured UI states per profile as cards with coverage badges, filters in the index column, surface coverage, an open state with its profiles side by side and where each picture came from, manifest download | AppShell navbar, SimpleGrid, Card, Image, MultiSelect, TextInput, Drawer, Accordion, Table, Badge, Alert |
|
|
4515
4713
|
| Project home | Tests/runs/artifacts tabs, a run's Artifacts (files of its layers, shared recordings, direct links), Watch playlist of a filtered set, metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion, Modal, Grid, Switch |
|
|
4516
4714
|
| Test detail | Current outcomes, statistics, duration graph, product latency with budgets and baselines, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
|
|
4517
4715
|
| Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
|
|
@@ -5978,6 +6176,52 @@ Scoped to one project. The same project may be selected on the unprefixed API us
|
|
|
5978
6176
|
|
|
5979
6177
|
Responses: `200` The matching results and the facets.; `304` Unchanged since the `If-None-Match` ETag.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `503` The server is still reading its backends. Retry after the number of seconds in `Retry-After`.
|
|
5980
6178
|
|
|
6179
|
+
### The UI catalog: captured UI states per profile, and what is still expected
|
|
6180
|
+
|
|
6181
|
+
`GET /api/reports/ui-catalog` · role: read
|
|
6182
|
+
|
|
6183
|
+
A browsable inventory of the product's user interface, computed from the winning results of the `ui-state` and `ui-plan` parts of every layer (see the UI catalog guide). Each state is a check id; each cell is one profile of it (the layer's environment and parameters) with its coverage: `captured` only when the winning capture passed, `failed`, `capturing`, `skipped`, or `missing` when a plan expects it and nothing has been captured. A picture is read from the cell's `capture.image.path` under the reports API root, or `uri`. What is expected is itself a layer, so a shard that never published leaves its states `missing`, never captured. The document equals what the viewer computes from the same report. Narrow it by `q` (words), `coverage`, `profile` (a profile `key`), `area`, `route`, `cohort`, `flag`, `surface` and `producer`: AND across parameters, OR within one; a state keeps only the cells that match `profile` and `coverage`. Page the states with `limit` and `offset`. Viewers need not poll: they hold the report over `/api/events` and read changes with `/api/reports/evidence/effective?since=`.
|
|
6184
|
+
|
|
6185
|
+
- `limit` (query): At most this many states (default 200).
|
|
6186
|
+
- `offset` (query): Skip this many states (default 0).
|
|
6187
|
+
- `q` (query): Words that must all appear in a state's id, title, route, fixture, cohort, area, flags, surfaces or producers.
|
|
6188
|
+
- `coverage` (query): Coverage values: `captured`, `failed`, `capturing`, `skipped`, `missing`. Comma-separated.
|
|
6189
|
+
- `profile` (query): Profile keys, as listed in `profiles[].key`.
|
|
6190
|
+
- `area` (query): Areas.
|
|
6191
|
+
- `route` (query): Routes.
|
|
6192
|
+
- `cohort` (query): Cohorts.
|
|
6193
|
+
- `flag` (query): Feature flags a state was captured with.
|
|
6194
|
+
- `surface` (query): Surfaces a state covers.
|
|
6195
|
+
- `producer` (query): Producers that captured or planned a state.
|
|
6196
|
+
- `format` (query): `markdown` for Markdown instead of JSON.
|
|
6197
|
+
- `project` (query): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
|
|
6198
|
+
|
|
6199
|
+
Responses: `200` The catalog.; `304` Unchanged since the `If-None-Match` ETag.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `503` The server is still reading its backends. Retry after the number of seconds in `Retry-After`.
|
|
6200
|
+
|
|
6201
|
+
### The UI catalog: captured UI states per profile, and what is still expected
|
|
6202
|
+
|
|
6203
|
+
`GET /api/projects/{project}/reports/ui-catalog` · role: read
|
|
6204
|
+
|
|
6205
|
+
A browsable inventory of the product's user interface, computed from the winning results of the `ui-state` and `ui-plan` parts of every layer (see the UI catalog guide). Each state is a check id; each cell is one profile of it (the layer's environment and parameters) with its coverage: `captured` only when the winning capture passed, `failed`, `capturing`, `skipped`, or `missing` when a plan expects it and nothing has been captured. A picture is read from the cell's `capture.image.path` under the reports API root, or `uri`. What is expected is itself a layer, so a shard that never published leaves its states `missing`, never captured. The document equals what the viewer computes from the same report. Narrow it by `q` (words), `coverage`, `profile` (a profile `key`), `area`, `route`, `cohort`, `flag`, `surface` and `producer`: AND across parameters, OR within one; a state keeps only the cells that match `profile` and `coverage`. Page the states with `limit` and `offset`. Viewers need not poll: they hold the report over `/api/events` and read changes with `/api/reports/evidence/effective?since=`.
|
|
6206
|
+
|
|
6207
|
+
Scoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.
|
|
6208
|
+
|
|
6209
|
+
- `project` (path, required): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
|
|
6210
|
+
- `limit` (query): At most this many states (default 200).
|
|
6211
|
+
- `offset` (query): Skip this many states (default 0).
|
|
6212
|
+
- `q` (query): Words that must all appear in a state's id, title, route, fixture, cohort, area, flags, surfaces or producers.
|
|
6213
|
+
- `coverage` (query): Coverage values: `captured`, `failed`, `capturing`, `skipped`, `missing`. Comma-separated.
|
|
6214
|
+
- `profile` (query): Profile keys, as listed in `profiles[].key`.
|
|
6215
|
+
- `area` (query): Areas.
|
|
6216
|
+
- `route` (query): Routes.
|
|
6217
|
+
- `cohort` (query): Cohorts.
|
|
6218
|
+
- `flag` (query): Feature flags a state was captured with.
|
|
6219
|
+
- `surface` (query): Surfaces a state covers.
|
|
6220
|
+
- `producer` (query): Producers that captured or planned a state.
|
|
6221
|
+
- `format` (query): `markdown` for Markdown instead of JSON.
|
|
6222
|
+
|
|
6223
|
+
Responses: `200` The catalog.; `304` Unchanged since the `If-None-Match` ETag.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `503` The server is still reading its backends. Retry after the number of seconds in `Retry-After`.
|
|
6224
|
+
|
|
5981
6225
|
### Every test one run recorded
|
|
5982
6226
|
|
|
5983
6227
|
`GET /api/reports/runs/detail` · role: read
|
package/llms.txt
CHANGED
|
@@ -11,6 +11,7 @@ Given only the repository link, start at docs/agents.md: install `@crvouga/atlas
|
|
|
11
11
|
- [Integrating Atlas into a project](https://raw.githubusercontent.com/crvouga/atlas/main/docs/integrating.md): This guide takes a project from nothing to a layered, gated report of its specs.
|
|
12
12
|
- [Native test runner reporters](https://raw.githubusercontent.com/crvouga/atlas/main/docs/runners.md): Projects upload results to a shared Atlas node under their own project slug.
|
|
13
13
|
- [Send rich evidence](https://raw.githubusercontent.com/crvouga/atlas/main/docs/rich-evidence.md): A check that says only its title and passed answers one question:
|
|
14
|
+
- [The UI catalog](https://raw.githubusercontent.com/crvouga/atlas/main/docs/ui-catalog.md): A browsable inventory of a product's user interface:
|
|
14
15
|
|
|
15
16
|
## Run and use the server
|
|
16
17
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@crvouga/atlas",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.14.0",
|
|
4
4
|
"description": "Atlas: specs of any kind, reported as layered evidence. Publish checks from any runner (JUnit, CTRF, a script), combine them across nodes, branches, environments and backends, gate CI on the result, and run statecharts against the real app.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -150,6 +150,7 @@
|
|
|
150
150
|
"docs/server.md",
|
|
151
151
|
"docs/extending.md",
|
|
152
152
|
"docs/rich-evidence.md",
|
|
153
|
+
"docs/ui-catalog.md",
|
|
153
154
|
"docs/statecharts.md",
|
|
154
155
|
"docs/report-contract.md",
|
|
155
156
|
"docs/report-backends.md",
|