@crvouga/atlas 1.15.0 → 1.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/CHANGELOG.md +13 -1
  2. package/dist/reports/src/catalog-health.d.ts +15 -2
  3. package/dist/reports/src/catalog-health.js +23 -2
  4. package/dist/reports/src/test-catalog.d.ts +13 -0
  5. package/dist/reports/src/test-catalog.js +21 -5
  6. package/dist/schema/src/api.d.ts +35 -0
  7. package/dist/schema/src/api.js +14 -3
  8. package/dist/schema/src/server-config.d.ts +3 -0
  9. package/dist/schema/src/server-config.js +5 -0
  10. package/dist/server/src/app.d.ts +2 -0
  11. package/dist/server/src/app.js +6 -2
  12. package/dist/server/src/config.js +4 -0
  13. package/dist/server/src/main.js +1 -0
  14. package/dist/server/src/openapi.js +2 -2
  15. package/docs/extending.md +1 -1
  16. package/docs/reference/http-api.md +4 -4
  17. package/docs/reference/reports.md +41 -2
  18. package/docs/reference/server.md +2 -0
  19. package/docs/server.md +11 -2
  20. package/docs/ui.md +28 -1
  21. package/llms-full.txt +44 -8
  22. package/package.json +1 -1
  23. package/packages/schema/schemas/catalog-health.schema.json +20 -0
  24. package/packages/schema/schemas/server-config.schema.json +17 -0
  25. package/packages/schema/schemas/test-catalog.schema.json +26 -0
  26. package/packages/schema/schemas/test-history.schema.json +40 -0
  27. package/packages/server/openapi.json +90 -4
  28. package/packages/visualizer/src/evidence/applicability-view.tsx +46 -20
  29. package/packages/visualizer/src/evidence/applicability.test.ts +68 -1
  30. package/packages/visualizer/src/evidence/applicability.ts +121 -26
  31. package/packages/visualizer/src/explorer/Explorer.tsx +105 -10
  32. package/packages/visualizer/src/explorer/HealthView.tsx +402 -0
  33. package/packages/visualizer/src/explorer/TestDetail.tsx +14 -1
  34. package/packages/visualizer/src/explorer/applicability.ts +43 -0
  35. package/packages/visualizer/src/explorer/catalog.tsx +33 -0
  36. package/packages/visualizer/src/explorer/search.ts +20 -7
package/docs/server.md CHANGED
@@ -202,6 +202,7 @@ is said in words and the server does not start. The file has a published
202
202
  | Whose specs the report is laid over | `outlineBranch` | `--outline-branch <branch>` | `ATLAS_OUTLINE_BRANCH` | `main`, then `master`, then the newest |
203
203
  | Only layers recorded on these branches take part | `include.branches` | `--branch <name>` (repeatable) | `ATLAS_ONLY_BRANCH` | all |
204
204
  | Leave out layers from modified checkouts and unknown origins | `include.ignoreModified` | `--ignore-modified` | `ATLAS_IGNORE_MODIFIED` | no |
205
+ | Root mappings `from>to` that join spec paths a producer publishes to the paths the specs are recorded under, for the catalog health report (comma-separated; an empty `from` matches every path) | `catalog.roots` | | `ATLAS_CATALOG_ROOTS` | none |
205
206
  | How often (seconds) the backends are looked at for layers written by something else | `refreshSeconds` | `--refresh <seconds>` | `ATLAS_REFRESH_SECONDS` | `10` |
206
207
  | Let anyone look, not upload | `auth.anonymousRead` | `--anonymous-read` | `ATLAS_ANONYMOUS_READ` | no |
207
208
  | Listen publicly with no tokens (something in front signs people in) | `auth.insecure` | `--insecure` | `ATLAS_INSECURE` | no |
@@ -442,8 +443,16 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
442
443
  the Tests total. Two spec paths are one logical spec only when a mapping says so
443
444
  (`?root=<from>><to>`, for example `?root=%3Especs/` to put `checkout/cart.feature` under `specs/`); similar paths
444
445
  are reported as ambiguous and never merged, and a similar title is never an identity. Give a check the label
445
- `outline` to say which scenario outline an example belongs to. A domain shared by nearly every spec is flagged, but
446
- explicit domain metadata always stays authoritative. The document only reads: it changes no status.
446
+ `outline` to say which scenario outline an example belongs to, and `behavior-id` / `example-id` to give a behavior
447
+ or one example a stable id of your own: Atlas counts them and names an id that sits on more than one check (a
448
+ rename or a move), but a check is still its spec path and id, so nothing is joined by them. A domain shared by nearly
449
+ every spec is flagged, but explicit domain metadata always stays authoritative. The document only reads: it changes
450
+ no status.
451
+ - Mappings that should always apply are a setting, not a query parameter: `catalog.roots` in `atlas.server.json` (a list
452
+ of `from>to`) or `ATLAS_CATALOG_ROOTS` (comma-separated; it replaces the file's list). The health document applies
453
+ them, then adds the request's `?root=`, and says in `roots[].source` which is which. A mapping with no `>` stops the
454
+ server starting. The viewer's **Catalog health** tab shows the same document, and its mapping form (the `root`
455
+ address parameter) tries further mappings without changing the server.
447
456
  - The server keeps the newest 100 results of every test (progress reported as `running` is not history) and every run's last result per test, in memory, as layers
448
457
  arrive; it is rebuilt from the layers on restart. History never changes a result.
449
458
  - Without a published spec set, tests are grouped under the specs their layers recorded (`specs[]`: path, domain,
package/docs/ui.md CHANGED
@@ -91,11 +91,38 @@ when Watch is on screen.
91
91
  what they know with labels: `build`, `platform`, and `applies-to` with `applies-to-by` (the policy that asserts the
92
92
  evidence still holds for those builds; without it the assertion is refused). The "Stale or unverified" filter
93
93
  (`evidence=stale`) lists every behavior whose evidence does not apply, including those with none.
94
+ - The test explorer makes the same reading of every test from the catalog (`explorer/applicability.ts`): the catalog
95
+ carries the spec's fingerprint now (`spec.fingerprint`) and, on each current result, `specRevision`, `freshness` and
96
+ `provenance` (the labels above), all optional. The same address names (`release`, `releaseSha`, `releasePlatform`,
97
+ `evidence=stale`) give the "Stale or unverified" button, the "Stale evidence first" sort, a verdict badge on rows
98
+ (stale always; unknown or none when a release is named or evidence is filtered or sorted), and the summary in test
99
+ detail. Layers carry no publication time, because a layer's id is the digest of its content and a time of upload in
100
+ it would make a retried publish a different layer: a producer that knows states `published-at` and `enriched-at`
101
+ labels, shown apart from "Checks ran", which alone is the age.
94
102
  - A recording is always labelled illustrative: it shows one run and does not approve a behavior for production.
95
103
  `demo`/`representative` tags only add a "Representative demo" label.
96
104
  - The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
97
105
  none are listed under "No domain declared"; the Tests view reports domain problems).
98
106
 
107
+ ## Catalog health
108
+
109
+ The **Catalog health** tab (`explorer/HealthView.tsx`, `?tab=health`; `@crvouga/atlas/reports/catalog-health`) explains why the Specs
110
+ and the Tests disagree. The Specs list links to it ("Why do these counts differ from the Tests?"). On a server it reads
111
+ `GET /api/reports/tests/health`; elsewhere the same document is worked out in the page from the specs and results it holds, and an
112
+ older server without the route is told so in words.
113
+
114
+ - Denominators are never one number: spec files, defined specs, specs with tests, specs executed and defined but not executed
115
+ (the coverage gaps, which add to no passed count), tests, outlines, examples and observations (an outline, its examples and
116
+ their per-environment results are told apart). Each figure has its meaning in its `title`.
117
+ - A table walks from the Specs total to the Tests total, step by step, with the sentences below it. Findings are an `Accordion`
118
+ (warnings open), each with its severity as text, its code, how many it affects and the paths. Paths that might be one promise
119
+ are shown as "may be the same promise as ...; not merged": a similar title never joins two specs.
120
+ - Root mappings (`from>to`) join spec paths. Those the server is configured with (`catalog.roots`, `ATLAS_CATALOG_ROOTS`) are
121
+ labelled "Configured on the server"; others are tried in the form and kept in the address (`root`), labelled "Tried here", and
122
+ change nothing on the server. A mapping that matched no path is a warning. Joined specs list every original path.
123
+ - Domains are listed as declared; a domain shared by nearly every spec is a finding, never rewritten. Identity states how many
124
+ checks are known only by title, how many declare `behavior-id` / `example-id`, and the limits of both.
125
+
99
126
  ## UI catalog
100
127
 
101
128
  `/catalog` (`catalog/`) is the viewer's inventory of captured UI states ([the guide](ui-catalog.md)). It is computed in the page from
@@ -320,7 +347,7 @@ The narrowly scoped exceptions are:
320
347
  | --- | --- | --- |
321
348
  | Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
322
349
  | UI catalog | Captured UI states per profile as cards with coverage badges, filters in the index column, surface coverage, an open state with its profiles side by side and where each picture came from, manifest download | AppShell navbar, SimpleGrid, Card, Image, MultiSelect, TextInput, Drawer, Accordion, Table, Badge, Alert |
323
- | Project home | Tests/runs/artifacts/compare tabs, a two-run comparison with promise, outcome, recording and figure tabs per behavior, a run's Artifacts (files of its layers, shared recordings, direct links), Watch playlist of a filtered set, metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion, Modal, Grid, Switch |
350
+ | Project home | Tests/runs/artifacts/compare/catalog health tabs, a two-run comparison with promise, outcome, recording and figure tabs per behavior, a run's Artifacts (files of its layers, shared recordings, direct links), Watch playlist of a filtered set, metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion, Modal, Grid, Switch |
324
351
  | Test detail | Current outcomes, statistics, duration graph, product latency with budgets and baselines, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
325
352
  | Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
326
353
  | Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
package/llms-full.txt CHANGED
@@ -1850,6 +1850,7 @@ is said in words and the server does not start. The file has a published
1850
1850
  | Whose specs the report is laid over | `outlineBranch` | `--outline-branch <branch>` | `ATLAS_OUTLINE_BRANCH` | `main`, then `master`, then the newest |
1851
1851
  | Only layers recorded on these branches take part | `include.branches` | `--branch <name>` (repeatable) | `ATLAS_ONLY_BRANCH` | all |
1852
1852
  | Leave out layers from modified checkouts and unknown origins | `include.ignoreModified` | `--ignore-modified` | `ATLAS_IGNORE_MODIFIED` | no |
1853
+ | Root mappings `from>to` that join spec paths a producer publishes to the paths the specs are recorded under, for the catalog health report (comma-separated; an empty `from` matches every path) | `catalog.roots` | | `ATLAS_CATALOG_ROOTS` | none |
1853
1854
  | How often (seconds) the backends are looked at for layers written by something else | `refreshSeconds` | `--refresh <seconds>` | `ATLAS_REFRESH_SECONDS` | `10` |
1854
1855
  | Let anyone look, not upload | `auth.anonymousRead` | `--anonymous-read` | `ATLAS_ANONYMOUS_READ` | no |
1855
1856
  | Listen publicly with no tokens (something in front signs people in) | `auth.insecure` | `--insecure` | `ATLAS_INSECURE` | no |
@@ -2090,8 +2091,16 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
2090
2091
  the Tests total. Two spec paths are one logical spec only when a mapping says so
2091
2092
  (`?root=<from>><to>`, for example `?root=%3Especs/` to put `checkout/cart.feature` under `specs/`); similar paths
2092
2093
  are reported as ambiguous and never merged, and a similar title is never an identity. Give a check the label
2093
- `outline` to say which scenario outline an example belongs to. A domain shared by nearly every spec is flagged, but
2094
- explicit domain metadata always stays authoritative. The document only reads: it changes no status.
2094
+ `outline` to say which scenario outline an example belongs to, and `behavior-id` / `example-id` to give a behavior
2095
+ or one example a stable id of your own: Atlas counts them and names an id that sits on more than one check (a
2096
+ rename or a move), but a check is still its spec path and id, so nothing is joined by them. A domain shared by nearly
2097
+ every spec is flagged, but explicit domain metadata always stays authoritative. The document only reads: it changes
2098
+ no status.
2099
+ - Mappings that should always apply are a setting, not a query parameter: `catalog.roots` in `atlas.server.json` (a list
2100
+ of `from>to`) or `ATLAS_CATALOG_ROOTS` (comma-separated; it replaces the file's list). The health document applies
2101
+ them, then adds the request's `?root=`, and says in `roots[].source` which is which. A mapping with no `>` stops the
2102
+ server starting. The viewer's **Catalog health** tab shows the same document, and its mapping form (the `root`
2103
+ address parameter) tries further mappings without changing the server.
2095
2104
  - The server keeps the newest 100 results of every test (progress reported as `running` is not history) and every run's last result per test, in memory, as layers
2096
2105
  arrive; it is rebuilt from the layers on restart. History never changes a result.
2097
2106
  - Without a published spec set, tests are grouped under the specs their layers recorded (`specs[]`: path, domain,
@@ -3195,7 +3204,7 @@ carry any of these. Every one is optional.
3195
3204
  | `annotations[]` | Notes about the result: a warning, a deprecation, an accessibility finding, a coverage figure. |
3196
3205
  | `attempts[]` | Every try, when the check was retried. |
3197
3206
  | `tags[]` | Free labels to filter by: `smoke`, `payments`. |
3198
- | `labels{}` | Facts about the result that are not its identity: `owner`, `severity`, `browser`, `build`. The viewer reads `build`, `platform`, `applies-to` and `applies-to-by` to tell whether evidence applies to a release (see [the viewer conventions](ui.md#behaviors-read-watch-inspect)). |
3207
+ | `labels{}` | Facts about the result that are not its identity: `owner`, `severity`, `browser`, `build`. The viewer reads `build`, `platform`, `applies-to` and `applies-to-by` to tell whether evidence applies to a release, and `published-at` and `enriched-at` (epoch milliseconds or an ISO date) to show when it was published or enriched apart from when the checks ran (see [the viewer conventions](ui.md#behaviors-read-watch-inspect)). `behavior-id` and `example-id` declare a stable id for the behavior and for one example of an outline, and `outline` names the outline; the [catalog health](server.md#reading-the-report-as-an-agent) report counts them and names an id shared by several checks. They never change which check a result belongs to. |
3199
3208
  | `scope{}` | What the evidence does and does not show: surface, device kind, synthetic data, scripted dependencies, audio, pacing, what was asserted and its limits. Unknown when absent. See [Evidence scope](rich-evidence.md#evidence-scope). |
3200
3209
  | `links[]` | Where to read more: the CI run, the issue, the trace client. |
3201
3210
  | `extensions{}` | Structured data of your own or a tool's, under a name you own. |
@@ -4510,11 +4519,38 @@ when Watch is on screen.
4510
4519
  what they know with labels: `build`, `platform`, and `applies-to` with `applies-to-by` (the policy that asserts the
4511
4520
  evidence still holds for those builds; without it the assertion is refused). The "Stale or unverified" filter
4512
4521
  (`evidence=stale`) lists every behavior whose evidence does not apply, including those with none.
4522
+ - The test explorer makes the same reading of every test from the catalog (`explorer/applicability.ts`): the catalog
4523
+ carries the spec's fingerprint now (`spec.fingerprint`) and, on each current result, `specRevision`, `freshness` and
4524
+ `provenance` (the labels above), all optional. The same address names (`release`, `releaseSha`, `releasePlatform`,
4525
+ `evidence=stale`) give the "Stale or unverified" button, the "Stale evidence first" sort, a verdict badge on rows
4526
+ (stale always; unknown or none when a release is named or evidence is filtered or sorted), and the summary in test
4527
+ detail. Layers carry no publication time, because a layer's id is the digest of its content and a time of upload in
4528
+ it would make a retried publish a different layer: a producer that knows states `published-at` and `enriched-at`
4529
+ labels, shown apart from "Checks ran", which alone is the age.
4513
4530
  - A recording is always labelled illustrative: it shows one run and does not approve a behavior for production.
4514
4531
  `demo`/`representative` tags only add a "Representative demo" label.
4515
4532
  - The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
4516
4533
  none are listed under "No domain declared"; the Tests view reports domain problems).
4517
4534
 
4535
+ ## Catalog health
4536
+
4537
+ The **Catalog health** tab (`explorer/HealthView.tsx`, `?tab=health`; `@crvouga/atlas/reports/catalog-health`) explains why the Specs
4538
+ and the Tests disagree. The Specs list links to it ("Why do these counts differ from the Tests?"). On a server it reads
4539
+ `GET /api/reports/tests/health`; elsewhere the same document is worked out in the page from the specs and results it holds, and an
4540
+ older server without the route is told so in words.
4541
+
4542
+ - Denominators are never one number: spec files, defined specs, specs with tests, specs executed and defined but not executed
4543
+ (the coverage gaps, which add to no passed count), tests, outlines, examples and observations (an outline, its examples and
4544
+ their per-environment results are told apart). Each figure has its meaning in its `title`.
4545
+ - A table walks from the Specs total to the Tests total, step by step, with the sentences below it. Findings are an `Accordion`
4546
+ (warnings open), each with its severity as text, its code, how many it affects and the paths. Paths that might be one promise
4547
+ are shown as "may be the same promise as ...; not merged": a similar title never joins two specs.
4548
+ - Root mappings (`from>to`) join spec paths. Those the server is configured with (`catalog.roots`, `ATLAS_CATALOG_ROOTS`) are
4549
+ labelled "Configured on the server"; others are tried in the form and kept in the address (`root`), labelled "Tried here", and
4550
+ change nothing on the server. A mapping that matched no path is a warning. Joined specs list every original path.
4551
+ - Domains are listed as declared; a domain shared by nearly every spec is a finding, never rewritten. Identity states how many
4552
+ checks are known only by title, how many declare `behavior-id` / `example-id`, and the limits of both.
4553
+
4518
4554
  ## UI catalog
4519
4555
 
4520
4556
  `/catalog` (`catalog/`) is the viewer's inventory of captured UI states ([the guide](ui-catalog.md)). It is computed in the page from
@@ -4739,7 +4775,7 @@ The narrowly scoped exceptions are:
4739
4775
  | --- | --- | --- |
4740
4776
  | Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
4741
4777
  | UI catalog | Captured UI states per profile as cards with coverage badges, filters in the index column, surface coverage, an open state with its profiles side by side and where each picture came from, manifest download | AppShell navbar, SimpleGrid, Card, Image, MultiSelect, TextInput, Drawer, Accordion, Table, Badge, Alert |
4742
- | Project home | Tests/runs/artifacts/compare tabs, a two-run comparison with promise, outcome, recording and figure tabs per behavior, a run's Artifacts (files of its layers, shared recordings, direct links), Watch playlist of a filtered set, metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion, Modal, Grid, Switch |
4778
+ | Project home | Tests/runs/artifacts/compare/catalog health tabs, a two-run comparison with promise, outcome, recording and figure tabs per behavior, a run's Artifacts (files of its layers, shared recordings, direct links), Watch playlist of a filtered set, metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion, Modal, Grid, Switch |
4743
4779
  | Test detail | Current outcomes, statistics, duration graph, product latency with budgets and baselines, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
4744
4780
  | Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
4745
4781
  | Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
@@ -5829,9 +5865,9 @@ Diagnoses the report's metadata, so a domain filter or a coverage percentage can
5829
5865
 
5830
5866
  `counts` keeps the denominators apart: spec files, defined specs, specs with an executed test (and those without: the defined coverage gaps), tests, outlines, examples and per-environment observations. `reconciliation` walks from the Specs count to the Tests count. `findings` name ambiguous and unmatched spec paths, identical or same-titled specs, a domain shared by nearly every spec, tests with no spec, checks that fall back to their title as identity and root mappings that matched nothing.
5831
5867
 
5832
- Two paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to. Send `format=markdown` for Markdown.
5868
+ Two paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to; the labels `behavior-id` and `example-id` declare stable ids, which are counted and checked for ids shared by several checks. Send `format=markdown` for Markdown.
5833
5869
 
5834
- - `root` (query): A root mapping `from>to`, URL-encoded. Repeatable or comma-separated.
5870
+ - `root` (query): A root mapping `from>to`, URL-encoded. Repeatable or comma-separated. It is added to the mappings the server is configured with (`catalog.roots`, `ATLAS_CATALOG_ROOTS`); `roots[].source` says which is which.
5835
5871
  - `format` (query): `markdown` for a Markdown report instead of JSON.
5836
5872
  - `project` (query): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5837
5873
 
@@ -5845,12 +5881,12 @@ Diagnoses the report's metadata, so a domain filter or a coverage percentage can
5845
5881
 
5846
5882
  `counts` keeps the denominators apart: spec files, defined specs, specs with an executed test (and those without: the defined coverage gaps), tests, outlines, examples and per-environment observations. `reconciliation` walks from the Specs count to the Tests count. `findings` name ambiguous and unmatched spec paths, identical or same-titled specs, a domain shared by nearly every spec, tests with no spec, checks that fall back to their title as identity and root mappings that matched nothing.
5847
5883
 
5848
- Two paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to. Send `format=markdown` for Markdown.
5884
+ Two paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to; the labels `behavior-id` and `example-id` declare stable ids, which are counted and checked for ids shared by several checks. Send `format=markdown` for Markdown.
5849
5885
 
5850
5886
  Scoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.
5851
5887
 
5852
5888
  - `project` (path, required): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5853
- - `root` (query): A root mapping `from>to`, URL-encoded. Repeatable or comma-separated.
5889
+ - `root` (query): A root mapping `from>to`, URL-encoded. Repeatable or comma-separated. It is added to the mappings the server is configured with (`catalog.roots`, `ATLAS_CATALOG_ROOTS`); `roots[].source` says which is which.
5854
5890
  - `format` (query): `markdown` for a Markdown report instead of JSON.
5855
5891
 
5856
5892
  Responses: `200` The diagnostics.; `304` Unchanged since the `If-None-Match` ETag.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `503` The server is still reading its backends. Retry after the number of seconds in `Retry-After`.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@crvouga/atlas",
3
- "version": "1.15.0",
3
+ "version": "1.17.0",
4
4
  "description": "Atlas: specs of any kind, reported as layered evidence. Publish checks from any runner (JUnit, CTRF, a script), combine them across nodes, branches, environments and backends, gate CI on the result, and run statecharts against the real app.",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -229,6 +229,18 @@
229
229
  "maximum": 9007199254740991,
230
230
  "description": "Checks whose id equals their title"
231
231
  },
232
+ "declaredBehaviorIds": {
233
+ "description": "Checks that declare a `behavior-id` label",
234
+ "type": "integer",
235
+ "minimum": 0,
236
+ "maximum": 9007199254740991
237
+ },
238
+ "declaredExampleIds": {
239
+ "description": "Checks that declare an `example-id` label",
240
+ "type": "integer",
241
+ "minimum": 0,
242
+ "maximum": 9007199254740991
243
+ },
232
244
  "basis": {
233
245
  "type": "string"
234
246
  }
@@ -278,6 +290,14 @@
278
290
  "minimum": 0,
279
291
  "maximum": 9007199254740991,
280
292
  "description": "Distinct paths the mapping changed"
293
+ },
294
+ "source": {
295
+ "description": "Where it was declared: the server's `catalog.roots` setting or this request's `root` parameter",
296
+ "type": "string",
297
+ "enum": [
298
+ "configuration",
299
+ "request"
300
+ ]
281
301
  }
282
302
  },
283
303
  "required": [
@@ -102,6 +102,23 @@
102
102
  },
103
103
  "additionalProperties": false
104
104
  },
105
+ "catalog": {
106
+ "default": {
107
+ "roots": []
108
+ },
109
+ "type": "object",
110
+ "properties": {
111
+ "roots": {
112
+ "default": [],
113
+ "description": "Root mappings `from>to` that relate spec paths a producer publishes to the paths the specs are recorded under (an empty `from` matches every path). The catalog health report applies them to every request, beside any `root` the request adds",
114
+ "type": "array",
115
+ "items": {
116
+ "type": "string"
117
+ }
118
+ }
119
+ },
120
+ "additionalProperties": false
121
+ },
105
122
  "refreshSeconds": {
106
123
  "default": 10,
107
124
  "description": "How often the backends are looked at for layers written by something other than this server",
@@ -529,6 +529,10 @@
529
529
  "kind": {
530
530
  "type": "string"
531
531
  },
532
+ "fingerprint": {
533
+ "description": "The fingerprint of the spec text now (the promise). Compare it with a result's `specRevision`: different means the promise changed after the result was recorded. Absent: unknown, never \"the same\". Absent from an older server",
534
+ "type": "string"
535
+ },
532
536
  "tests": {
533
537
  "type": "integer",
534
538
  "minimum": 0,
@@ -747,6 +751,10 @@
747
751
  },
748
752
  "kind": {
749
753
  "type": "string"
754
+ },
755
+ "fingerprint": {
756
+ "description": "The fingerprint of the spec text now (the promise). Compare it with a result's `specRevision`: different means the promise changed after the result was recorded. Absent: unknown, never \"the same\". Absent from an older server",
757
+ "type": "string"
750
758
  }
751
759
  },
752
760
  "required": [
@@ -869,6 +877,24 @@
869
877
  "description": "The fingerprint of its spec text that the layer recorded, when it recorded one. Two results with different revisions were recorded against different promises. Absent: not recorded, never assumed to match",
870
878
  "type": "string"
871
879
  },
880
+ "freshness": {
881
+ "description": "`modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Absent for a clean checkout, on results other than a test's current ones, and from an older server. Advisory; never changes a status",
882
+ "type": "string",
883
+ "enum": [
884
+ "modified",
885
+ "unknown"
886
+ ]
887
+ },
888
+ "provenance": {
889
+ "description": "The labels of its layer and check that say what it ran against and who vouches for it: `build`, `platform`, `applies-to`, `applies-to-by`, `published-at`, `enriched-at`. Present on a test's current results when it has any, and absent from an older server. Never decides a result",
890
+ "type": "object",
891
+ "propertyNames": {
892
+ "type": "string"
893
+ },
894
+ "additionalProperties": {
895
+ "type": "string"
896
+ }
897
+ },
872
898
  "evidence": {
873
899
  "type": "object",
874
900
  "properties": {
@@ -45,6 +45,10 @@
45
45
  },
46
46
  "kind": {
47
47
  "type": "string"
48
+ },
49
+ "fingerprint": {
50
+ "description": "The fingerprint of the spec text now (the promise). Compare it with a result's `specRevision`: different means the promise changed after the result was recorded. Absent: unknown, never \"the same\". Absent from an older server",
51
+ "type": "string"
48
52
  }
49
53
  },
50
54
  "required": [
@@ -167,6 +171,24 @@
167
171
  "description": "The fingerprint of its spec text that the layer recorded, when it recorded one. Two results with different revisions were recorded against different promises. Absent: not recorded, never assumed to match",
168
172
  "type": "string"
169
173
  },
174
+ "freshness": {
175
+ "description": "`modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Absent for a clean checkout, on results other than a test's current ones, and from an older server. Advisory; never changes a status",
176
+ "type": "string",
177
+ "enum": [
178
+ "modified",
179
+ "unknown"
180
+ ]
181
+ },
182
+ "provenance": {
183
+ "description": "The labels of its layer and check that say what it ran against and who vouches for it: `build`, `platform`, `applies-to`, `applies-to-by`, `published-at`, `enriched-at`. Present on a test's current results when it has any, and absent from an older server. Never decides a result",
184
+ "type": "object",
185
+ "propertyNames": {
186
+ "type": "string"
187
+ },
188
+ "additionalProperties": {
189
+ "type": "string"
190
+ }
191
+ },
170
192
  "evidence": {
171
193
  "type": "object",
172
194
  "properties": {
@@ -820,6 +842,24 @@
820
842
  "description": "The fingerprint of its spec text that the layer recorded, when it recorded one. Two results with different revisions were recorded against different promises. Absent: not recorded, never assumed to match",
821
843
  "type": "string"
822
844
  },
845
+ "freshness": {
846
+ "description": "`modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Absent for a clean checkout, on results other than a test's current ones, and from an older server. Advisory; never changes a status",
847
+ "type": "string",
848
+ "enum": [
849
+ "modified",
850
+ "unknown"
851
+ ]
852
+ },
853
+ "provenance": {
854
+ "description": "The labels of its layer and check that say what it ran against and who vouches for it: `build`, `platform`, `applies-to`, `applies-to-by`, `published-at`, `enriched-at`. Present on a test's current results when it has any, and absent from an older server. Never decides a result",
855
+ "type": "object",
856
+ "propertyNames": {
857
+ "type": "string"
858
+ },
859
+ "additionalProperties": {
860
+ "type": "string"
861
+ }
862
+ },
823
863
  "evidence": {
824
864
  "type": "object",
825
865
  "properties": {
@@ -3699,7 +3699,7 @@
3699
3699
  "Report"
3700
3700
  ],
3701
3701
  "summary": "Why specs, domains and counts disagree",
3702
- "description": "Diagnoses the report's metadata, so a domain filter or a coverage percentage can be trusted or fixed at its source. It reads; it never changes a status, a slot or a winner.\n\n`counts` keeps the denominators apart: spec files, defined specs, specs with an executed test (and those without: the defined coverage gaps), tests, outlines, examples and per-environment observations. `reconciliation` walks from the Specs count to the Tests count. `findings` name ambiguous and unmatched spec paths, identical or same-titled specs, a domain shared by nearly every spec, tests with no spec, checks that fall back to their title as identity and root mappings that matched nothing.\n\nTwo paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to. Send `format=markdown` for Markdown.\n\nRole needed: `read`.",
3702
+ "description": "Diagnoses the report's metadata, so a domain filter or a coverage percentage can be trusted or fixed at its source. It reads; it never changes a status, a slot or a winner.\n\n`counts` keeps the denominators apart: spec files, defined specs, specs with an executed test (and those without: the defined coverage gaps), tests, outlines, examples and per-environment observations. `reconciliation` walks from the Specs count to the Tests count. `findings` name ambiguous and unmatched spec paths, identical or same-titled specs, a domain shared by nearly every spec, tests with no spec, checks that fall back to their title as identity and root mappings that matched nothing.\n\nTwo paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to; the labels `behavior-id` and `example-id` declare stable ids, which are counted and checked for ids shared by several checks. Send `format=markdown` for Markdown.\n\nRole needed: `read`.",
3703
3703
  "x-atlas-role": "read",
3704
3704
  "security": [
3705
3705
  {
@@ -3714,7 +3714,7 @@
3714
3714
  "required": false,
3715
3715
  "name": "root",
3716
3716
  "in": "query",
3717
- "description": "A root mapping `from>to`, URL-encoded. Repeatable or comma-separated.",
3717
+ "description": "A root mapping `from>to`, URL-encoded. Repeatable or comma-separated. It is added to the mappings the server is configured with (`catalog.roots`, `ATLAS_CATALOG_ROOTS`); `roots[].source` says which is which.",
3718
3718
  "example": ">specs/",
3719
3719
  "schema": {
3720
3720
  "type": "string"
@@ -3798,7 +3798,7 @@
3798
3798
  "Report"
3799
3799
  ],
3800
3800
  "summary": "Why specs, domains and counts disagree",
3801
- "description": "Diagnoses the report's metadata, so a domain filter or a coverage percentage can be trusted or fixed at its source. It reads; it never changes a status, a slot or a winner.\n\n`counts` keeps the denominators apart: spec files, defined specs, specs with an executed test (and those without: the defined coverage gaps), tests, outlines, examples and per-environment observations. `reconciliation` walks from the Specs count to the Tests count. `findings` name ambiguous and unmatched spec paths, identical or same-titled specs, a domain shared by nearly every spec, tests with no spec, checks that fall back to their title as identity and root mappings that matched nothing.\n\nTwo paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to. Send `format=markdown` for Markdown.\n\nScoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.\n\nRole needed: `read`.",
3801
+ "description": "Diagnoses the report's metadata, so a domain filter or a coverage percentage can be trusted or fixed at its source. It reads; it never changes a status, a slot or a winner.\n\n`counts` keeps the denominators apart: spec files, defined specs, specs with an executed test (and those without: the defined coverage gaps), tests, outlines, examples and per-environment observations. `reconciliation` walks from the Specs count to the Tests count. `findings` name ambiguous and unmatched spec paths, identical or same-titled specs, a domain shared by nearly every spec, tests with no spec, checks that fall back to their title as identity and root mappings that matched nothing.\n\nTwo paths are one logical spec only when a `root` mapping relates them (`from>to`: paths under `from` are the same specs as paths under `to`; an empty `from` matches every path). Without one, similar paths are reported as ambiguous and never merged. A check's label `outline` names the outline its example belongs to; the labels `behavior-id` and `example-id` declare stable ids, which are counted and checked for ids shared by several checks. Send `format=markdown` for Markdown.\n\nScoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.\n\nRole needed: `read`.",
3802
3802
  "x-atlas-role": "read",
3803
3803
  "security": [
3804
3804
  {
@@ -3825,7 +3825,7 @@
3825
3825
  "required": false,
3826
3826
  "name": "root",
3827
3827
  "in": "query",
3828
- "description": "A root mapping `from>to`, URL-encoded. Repeatable or comma-separated.",
3828
+ "description": "A root mapping `from>to`, URL-encoded. Repeatable or comma-separated. It is added to the mappings the server is configured with (`catalog.roots`, `ATLAS_CATALOG_ROOTS`); `roots[].source` says which is which.",
3829
3829
  "example": ">specs/",
3830
3830
  "schema": {
3831
3831
  "type": "string"
@@ -20619,6 +20619,10 @@
20619
20619
  "kind": {
20620
20620
  "type": "string"
20621
20621
  },
20622
+ "fingerprint": {
20623
+ "description": "The fingerprint of the spec text now (the promise). Compare it with a result's `specRevision`: different means the promise changed after the result was recorded. Absent: unknown, never \"the same\". Absent from an older server",
20624
+ "type": "string"
20625
+ },
20622
20626
  "tests": {
20623
20627
  "type": "integer",
20624
20628
  "minimum": 0,
@@ -20837,6 +20841,10 @@
20837
20841
  },
20838
20842
  "kind": {
20839
20843
  "type": "string"
20844
+ },
20845
+ "fingerprint": {
20846
+ "description": "The fingerprint of the spec text now (the promise). Compare it with a result's `specRevision`: different means the promise changed after the result was recorded. Absent: unknown, never \"the same\". Absent from an older server",
20847
+ "type": "string"
20840
20848
  }
20841
20849
  },
20842
20850
  "required": [
@@ -20959,6 +20967,24 @@
20959
20967
  "description": "The fingerprint of its spec text that the layer recorded, when it recorded one. Two results with different revisions were recorded against different promises. Absent: not recorded, never assumed to match",
20960
20968
  "type": "string"
20961
20969
  },
20970
+ "freshness": {
20971
+ "description": "`modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Absent for a clean checkout, on results other than a test's current ones, and from an older server. Advisory; never changes a status",
20972
+ "type": "string",
20973
+ "enum": [
20974
+ "modified",
20975
+ "unknown"
20976
+ ]
20977
+ },
20978
+ "provenance": {
20979
+ "description": "The labels of its layer and check that say what it ran against and who vouches for it: `build`, `platform`, `applies-to`, `applies-to-by`, `published-at`, `enriched-at`. Present on a test's current results when it has any, and absent from an older server. Never decides a result",
20980
+ "type": "object",
20981
+ "propertyNames": {
20982
+ "type": "string"
20983
+ },
20984
+ "additionalProperties": {
20985
+ "type": "string"
20986
+ }
20987
+ },
20962
20988
  "evidence": {
20963
20989
  "type": "object",
20964
20990
  "properties": {
@@ -21772,6 +21798,18 @@
21772
21798
  "maximum": 9007199254740991,
21773
21799
  "description": "Checks whose id equals their title"
21774
21800
  },
21801
+ "declaredBehaviorIds": {
21802
+ "description": "Checks that declare a `behavior-id` label",
21803
+ "type": "integer",
21804
+ "minimum": 0,
21805
+ "maximum": 9007199254740991
21806
+ },
21807
+ "declaredExampleIds": {
21808
+ "description": "Checks that declare an `example-id` label",
21809
+ "type": "integer",
21810
+ "minimum": 0,
21811
+ "maximum": 9007199254740991
21812
+ },
21775
21813
  "basis": {
21776
21814
  "type": "string"
21777
21815
  }
@@ -21821,6 +21859,14 @@
21821
21859
  "minimum": 0,
21822
21860
  "maximum": 9007199254740991,
21823
21861
  "description": "Distinct paths the mapping changed"
21862
+ },
21863
+ "source": {
21864
+ "description": "Where it was declared: the server's `catalog.roots` setting or this request's `root` parameter",
21865
+ "type": "string",
21866
+ "enum": [
21867
+ "configuration",
21868
+ "request"
21869
+ ]
21824
21870
  }
21825
21871
  },
21826
21872
  "required": [
@@ -21986,6 +22032,10 @@
21986
22032
  },
21987
22033
  "kind": {
21988
22034
  "type": "string"
22035
+ },
22036
+ "fingerprint": {
22037
+ "description": "The fingerprint of the spec text now (the promise). Compare it with a result's `specRevision`: different means the promise changed after the result was recorded. Absent: unknown, never \"the same\". Absent from an older server",
22038
+ "type": "string"
21989
22039
  }
21990
22040
  },
21991
22041
  "required": [
@@ -22108,6 +22158,24 @@
22108
22158
  "description": "The fingerprint of its spec text that the layer recorded, when it recorded one. Two results with different revisions were recorded against different promises. Absent: not recorded, never assumed to match",
22109
22159
  "type": "string"
22110
22160
  },
22161
+ "freshness": {
22162
+ "description": "`modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Absent for a clean checkout, on results other than a test's current ones, and from an older server. Advisory; never changes a status",
22163
+ "type": "string",
22164
+ "enum": [
22165
+ "modified",
22166
+ "unknown"
22167
+ ]
22168
+ },
22169
+ "provenance": {
22170
+ "description": "The labels of its layer and check that say what it ran against and who vouches for it: `build`, `platform`, `applies-to`, `applies-to-by`, `published-at`, `enriched-at`. Present on a test's current results when it has any, and absent from an older server. Never decides a result",
22171
+ "type": "object",
22172
+ "propertyNames": {
22173
+ "type": "string"
22174
+ },
22175
+ "additionalProperties": {
22176
+ "type": "string"
22177
+ }
22178
+ },
22111
22179
  "evidence": {
22112
22180
  "type": "object",
22113
22181
  "properties": {
@@ -22761,6 +22829,24 @@
22761
22829
  "description": "The fingerprint of its spec text that the layer recorded, when it recorded one. Two results with different revisions were recorded against different promises. Absent: not recorded, never assumed to match",
22762
22830
  "type": "string"
22763
22831
  },
22832
+ "freshness": {
22833
+ "description": "`modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Absent for a clean checkout, on results other than a test's current ones, and from an older server. Advisory; never changes a status",
22834
+ "type": "string",
22835
+ "enum": [
22836
+ "modified",
22837
+ "unknown"
22838
+ ]
22839
+ },
22840
+ "provenance": {
22841
+ "description": "The labels of its layer and check that say what it ran against and who vouches for it: `build`, `platform`, `applies-to`, `applies-to-by`, `published-at`, `enriched-at`. Present on a test's current results when it has any, and absent from an older server. Never decides a result",
22842
+ "type": "object",
22843
+ "propertyNames": {
22844
+ "type": "string"
22845
+ },
22846
+ "additionalProperties": {
22847
+ "type": "string"
22848
+ }
22849
+ },
22764
22850
  "evidence": {
22765
22851
  "type": "object",
22766
22852
  "properties": {