@crvouga/atlas 1.1.0 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. package/CHANGELOG.md +6 -0
  2. package/dist/reports/src/artifact-query.d.ts +123 -0
  3. package/dist/reports/src/artifact-query.js +234 -0
  4. package/dist/reports/src/evidence-detail.js +2 -1
  5. package/dist/reports/src/evidence-types.d.ts +2 -0
  6. package/dist/reports/src/test-catalog.d.ts +101 -5
  7. package/dist/reports/src/test-catalog.js +140 -5
  8. package/dist/schema/src/api.d.ts +225 -0
  9. package/dist/schema/src/api.js +41 -2
  10. package/dist/server/src/app.js +16 -5
  11. package/dist/server/src/openapi.js +41 -3
  12. package/docs/reference/http-api.md +84 -2
  13. package/docs/reference/reports.md +371 -7
  14. package/docs/reference/schema.md +32 -0
  15. package/docs/rich-evidence.md +31 -0
  16. package/docs/server.md +4 -1
  17. package/docs/ui.md +1 -1
  18. package/llms-full.txt +120 -4
  19. package/package.json +1 -1
  20. package/packages/schema/schemas/effective-delta.schema.json +14 -0
  21. package/packages/schema/schemas/effective-evidence.schema.json +14 -0
  22. package/packages/schema/schemas/evidence-page.schema.json +7 -0
  23. package/packages/schema/schemas/layer-rows.schema.json +7 -0
  24. package/packages/schema/schemas/report-run.schema.json +35 -0
  25. package/packages/schema/schemas/report-runs.schema.json +221 -0
  26. package/packages/schema/schemas/test-catalog.schema.json +7 -0
  27. package/packages/schema/schemas/test-history.schema.json +14 -0
  28. package/packages/server/openapi.json +2473 -1035
  29. package/packages/visualizer/src/explorer/ArtifactsView.tsx +162 -0
  30. package/packages/visualizer/src/explorer/Explorer.tsx +28 -3
  31. package/packages/visualizer/src/explorer/RunsView.tsx +43 -19
  32. package/packages/visualizer/src/explorer/catalog.tsx +97 -12
  33. package/packages/visualizer/src/explorer/filters.tsx +279 -0
  34. package/packages/visualizer/src/explorer/search.ts +81 -6
  35. package/packages/visualizer/src/explorer/views.ts +75 -0
@@ -4,6 +4,257 @@ Layered evidence for any kind of spec: publishing layers, stores (folder, S3, an
4
4
 
5
5
  > Generated from the doc comments of `packages/reports/src` by `bun run docs`. Do not edit: change the comment next to the code.
6
6
 
7
+ ### Run and artifact queries
8
+
9
+ Source: `packages/reports/src/artifact-query.ts`. Import: `@crvouga/atlas/reports/artifact-query`
10
+
11
+ One shareable query for the runs of a project and for the recorded results that carry evidence (recordings,
12
+ screenshots, traces): by artifact tag, label, media type, feature (domain, spec), environment or device, producer,
13
+ branch, revision, outcome and time. The same rules answer `GET /api/reports/runs`, `GET /api/reports/artifacts`
14
+ and the viewer's Runs and Artifacts views, so a link means the same thing everywhere.
15
+
16
+ The semantics are fixed, not inferred:
17
+
18
+ - Across dimensions the filters combine with AND: `artifactTag=demo&type=video&environment=ios` is a demo
19
+ recording on ios.
20
+ - Within a dimension the values combine with OR: `environment=ios,android` is either.
21
+ - When both `artifactTag` and `type` are asked for, one artifact must be both (a `demo`-tagged video, not a demo-tagged
22
+ screenshot beside an untagged video) wherever the report says which videos carry which tags; otherwise a result with
23
+ an artifact of that type and an artifact with that tag matches.
24
+ - Facet counts are disjunctive: the count beside a value is how many records the list would hold if that value were
25
+ the only choice in its dimension, with every other dimension's filter applied. With a value already chosen, the
26
+ other values show what choosing them in addition would add, and the chosen one's count is the list's.
27
+ - A filter never changes a status. Evidence that is not recorded is unknown, never a match.
28
+
29
+ Lists in the address are comma-separated; a comma or backslash inside a value is written `\,` or `\\`
30
+ ({@link splitList}, {@link joinList}), so a label such as `note=a,b` survives.
31
+
32
+ See docs/server.md#reading-the-report-as-an-agent
33
+
34
+ #### `QueryFacetValue`
35
+
36
+ ```ts
37
+ export type QueryFacetValue = { readonly value: string; readonly count: number };
38
+ ```
39
+
40
+ A value of a dimension and how many records have it.
41
+
42
+ #### `QUERY_DIMENSIONS`
43
+
44
+ ```ts
45
+ export const QUERY_DIMENSIONS = …
46
+ ```
47
+
48
+ The dimensions a query narrows by, with their address parameters.
49
+
50
+ #### `QueryDimension`
51
+
52
+ ```ts
53
+ export type QueryDimension = (typeof QUERY_DIMENSIONS)[number];
54
+ ```
55
+
56
+ One of {@link QUERY_DIMENSIONS}.
57
+
58
+ #### `ArtifactFilter`
59
+
60
+ ```ts
61
+ export type ArtifactFilter = {
62
+ /** Words (all of them) found in the title, test id, spec, domain, branch, labels, tags, environment or producer; or, for a run, its title. */
63
+ readonly q?: string;
64
+ /** Artifact tags (`demo` and `#demo` are the same). */
65
+ readonly artifactTag?: readonly string[];
66
+ /** Also match the tags an artifact inherits from its check and run. Default false: the exact artifact query. */
67
+ readonly inherited?: boolean;
68
+ /** Media types as roles: `video`, `image`, `audio`, `snapshot`, `text`, `file`, or a producer's own role. `video` means a playable recording. */
69
+ readonly type?: readonly string[];
70
+ /** `key=value`, or `key` for any value. */
71
+ readonly label?: readonly string[];
72
+ /** Environments or devices (`ios`, `desktop-web`). */
73
+ readonly environment?: readonly string[];
74
+ readonly producer?: readonly string[];
75
+ readonly branch?: readonly string[];
76
+ /** Revisions: a full commit id, or a prefix of at least seven characters. */
77
+ readonly sha?: readonly string[];
78
+ /** Statuses (`passed`, `failed`, `running`, `not-run`). A run's outcome is `failed` when any result failed, else `running`, else `passed`, else `not-run`. */
79
+ readonly outcome?: readonly string[];
80
+ /** The feature domain of a result's spec. Not applicable to runs. */
81
+ readonly domain?: readonly string[];
82
+ /** Spec paths. Not applicable to runs. */
83
+ readonly spec?: readonly string[];
84
+ /** Only records at or after this time (epoch ms): a result's recording time, a run's end. */
85
+ readonly since?: number;
86
+ /** Only records at or before this time (epoch ms). */
87
+ readonly until?: number;
88
+ /** Only records with a playable video recording (the same as `type: ['video']`). */
89
+ readonly video?: boolean;
90
+ };
91
+ ```
92
+
93
+ What a query asks for. Every field is optional; nothing asked is everything. See the module comment for the semantics.
94
+
95
+ #### `FilterName`
96
+
97
+ ```ts
98
+ export type FilterName = QueryDimension | 'q' | 'inherited' | 'since' | 'until' | 'video';
99
+ ```
100
+
101
+ The filters a query used, as they are named in the address and in `filters` of a response.
102
+
103
+ #### `FilterFields`
104
+
105
+ ```ts
106
+ export type FilterFields = {
107
+ /** Lower-cased text a search matches. */
108
+ readonly text: string;
109
+ /** The normalized tags of its artifacts. */
110
+ readonly artifactTags: readonly string[];
111
+ /** The tags its artifacts inherit from checks and runs, and its own. */
112
+ readonly tags: readonly string[];
113
+ /** Distinct media types; `video` only when a recording can be played. */
114
+ readonly types: readonly string[];
115
+ /** Artifacts by type and their own tags. Where the report does not say which artifact carries which tag, every artifact of the record is given the record's artifact tags. */
116
+ readonly media: readonly { readonly type: string; readonly tags: readonly string[] }[];
117
+ /** `key=value`, sorted and distinct. */
118
+ readonly labels: readonly string[];
119
+ readonly environments: readonly string[];
120
+ readonly producers: readonly string[];
121
+ readonly branches: readonly string[];
122
+ readonly shas: readonly string[];
123
+ readonly outcomes: readonly string[];
124
+ /** Absent when the dimension does not apply (runs have no domain). */
125
+ readonly domains?: readonly string[];
126
+ readonly specs?: readonly string[];
127
+ readonly at: number;
128
+ };
129
+ ```
130
+
131
+ The values of a record a query looks at: made by {@link fieldsOfRun} or by the artifact search.
132
+
133
+ #### `QueryFacets`
134
+
135
+ ```ts
136
+ export type QueryFacets = {
137
+ readonly artifactTags: readonly QueryFacetValue[];
138
+ readonly types: readonly QueryFacetValue[];
139
+ /** Values are `key=value`. */
140
+ readonly labels: readonly QueryFacetValue[];
141
+ readonly environments: readonly QueryFacetValue[];
142
+ readonly producers: readonly QueryFacetValue[];
143
+ readonly branches: readonly QueryFacetValue[];
144
+ readonly outcomes: readonly QueryFacetValue[];
145
+ readonly domains?: readonly QueryFacetValue[];
146
+ };
147
+ ```
148
+
149
+ The facets of a query: for each dimension, its values with disjunctive counts (see the module comment).
150
+
151
+ #### `splitList`
152
+
153
+ ```ts
154
+ export function splitList(raw: string): string[]
155
+ ```
156
+
157
+ Splits a comma-separated list from an address: `a,b` is two values; `\,` is a comma and `\\` a backslash inside a
158
+ value. Values are trimmed and empty ones dropped.
159
+
160
+ **Example**
161
+
162
+ ```ts
163
+ splitList('demo,release\\,2'); // ['demo', 'release,2']
164
+ ```
165
+
166
+ #### `joinList`
167
+
168
+ ```ts
169
+ export function joinList(list: readonly string[]): string | undefined
170
+ ```
171
+
172
+ Joins values into one address parameter that {@link splitList} reads back; `undefined` when there are none.
173
+
174
+ #### `parseTime`
175
+
176
+ ```ts
177
+ export function parseTime(raw: string | null | undefined): number | undefined
178
+ ```
179
+
180
+ A time from an address: epoch milliseconds, or a date or date-time Date can read (a date alone is UTC). `undefined` when unreadable.
181
+
182
+ #### `parseArtifactFilter`
183
+
184
+ ```ts
185
+ export function parseArtifactFilter(params: URLSearchParams): ArtifactFilter
186
+ ```
187
+
188
+ Reads a query from URL search parameters: `q`, `video`, `inherited`, `since`, `until` and the repeatable or
189
+ comma-separated {@link QUERY_DIMENSIONS}. Unreadable values are left out, never an error.
190
+
191
+ #### `filterParams`
192
+
193
+ ```ts
194
+ export function filterParams(filter: ArtifactFilter): URLSearchParams
195
+ ```
196
+
197
+ A query as URL search parameters, the form {@link parseArtifactFilter} reads back.
198
+
199
+ #### `usedFilters`
200
+
201
+ ```ts
202
+ export function usedFilters(filter: ArtifactFilter): FilterName[]
203
+ ```
204
+
205
+ Which filters a query uses, by the names an address gives them.
206
+
207
+ #### `isFiltered`
208
+
209
+ ```ts
210
+ export function isFiltered(filter: ArtifactFilter): boolean
211
+ ```
212
+
213
+ Whether a query narrows anything.
214
+
215
+ #### `forRuns`
216
+
217
+ ```ts
218
+ export function forRuns(filter: ArtifactFilter): ArtifactFilter
219
+ ```
220
+
221
+ Every filter in `filter` that may be applied to a record with these fields, as the runs query applies them (no domain or spec).
222
+
223
+ #### `matchesFilter`
224
+
225
+ ```ts
226
+ export function matchesFilter(fields: FilterFields, filter: ArtifactFilter, skip?: QueryDimension): boolean
227
+ ```
228
+
229
+ Whether a record satisfies a query. `skip` leaves one dimension out: what {@link facetsOf} uses to count the values of
230
+ that dimension, and nothing else should need.
231
+
232
+ #### `facetsOf`
233
+
234
+ ```ts
235
+ export function facetsOf(records: readonly FilterFields[], filter: ArtifactFilter): QueryFacets
236
+ ```
237
+
238
+ The facets of a query over every record: for each dimension, the values the records have and how many records
239
+ would match if that value were chosen in its dimension (see the module comment). A value chosen in the query is
240
+ listed even when nothing has it, with a count of 0, so it can be removed.
241
+
242
+ #### `outcomeOfCounts`
243
+
244
+ ```ts
245
+ export function outcomeOfCounts(counts: Readonly<Record<string, number>>): string
246
+ ```
247
+
248
+ The outcome of a run from its results by status: `failed`, else `running`, else `passed`, else `not-run`.
249
+
250
+ #### `labelList`
251
+
252
+ ```ts
253
+ export function labelList(labels: Readonly<Record<string, string>> | undefined): string[]
254
+ ```
255
+
256
+ The `key=value` labels of a record, sorted and distinct.
257
+
7
258
  ### Reviewer tags in a store
8
259
 
9
260
  Source: `packages/reports/src/artifact-tag-store.ts`. Import: `@crvouga/atlas/reports/artifact-tag-store`
@@ -1659,6 +1910,8 @@ export type VideoRef = {
1659
1910
  readonly attempt?: number;
1660
1911
  /** The name of the step that holds it, when `owner` is `step`. */
1661
1912
  readonly step?: string;
1913
+ /** The normalized tags of this attachment itself (`demo`), so a query can ask for a tagged video rather than a result that has a video and some tagged file. Absent when it has none (and in documents of older servers). */
1914
+ readonly tags?: readonly string[];
1662
1915
  /** What identifies the bytes (digest, bundle key or address); references with the same one are the same recording. Absent when `available` is false. */
1663
1916
  readonly recording?: string;
1664
1917
  /** False when the reference names no bytes at all (no bundle key, address or inline text), so it can never be played. */
@@ -7811,7 +8064,9 @@ runs that produced them and which tests each one ran.
7811
8064
  always the report's winner (`combineEvidence`), and history only describes how it got there.
7812
8065
  - `testCatalog` turns the report's winners (plus a history, when there is one) into a `TestCatalog`: a summary with
7813
8066
  insights (slowest, flaky, failing), facets to filter by, and the tests that match a `TestQuery`.
7814
- - `testHistory`, `runList` and `runDetail` answer for one test, every run, and one run.
8067
+ - `testHistory`, `runList` and `runDetail` answer for one test, every run, and one run. `runList` and `artifactList`
8068
+ narrow by tag, label, media type, environment, branch, outcome and time (see `artifact-query.ts`); `artifactList` finds
8069
+ the recorded results that carry evidence across the whole history, not only the latest winners.
7815
8070
  - `catalogMarkdown`, `historyMarkdown`, `runsMarkdown` and `runMarkdown` say the same in Markdown, for an agent or a
7816
8071
  terminal.
7817
8072
 
@@ -8174,7 +8429,15 @@ export type RunSummary = {
8174
8429
  * and references that cannot be played. Absent when it has none, and in a report that carries no video information.
8175
8430
  */
8176
8431
  readonly videos?: VideoCounts;
8177
- };
8432
+ /** The normalized tags of the attachments of its results and layers, sorted and distinct. Absent when none is tagged. */
8433
+ readonly artifactTags?: readonly string[];
8434
+ /** The tags of its checks and layers, which its attachments inherit (what `inherited=1` also matches), sorted and distinct. Absent when none. */
8435
+ readonly tags?: readonly string[];
8436
+ /** The media types (roles) of its attachments, sorted and distinct; `video` only for a playable recording. Absent when it has no attachment. */
8437
+ readonly types?: readonly string[];
8438
+ /** The `key=value` labels of its results and layers, sorted and distinct. Absent when none. */
8439
+ readonly labels?: readonly string[];
8440
+ …
8178
8441
  ```
8179
8442
 
8180
8443
  A run: the layers that name the same execution (`run.id`), else share a CI run URL, else a branch and commit. When,
@@ -8183,13 +8446,11 @@ where, and how its tests went.
8183
8446
  #### `RunQuery`
8184
8447
 
8185
8448
  ```ts
8186
- export type RunQuery = {
8187
- /** Only runs that have at least one playable video recording. `total` then counts only those. */
8188
- readonly video?: boolean;
8189
- };
8449
+ export type RunQuery = ArtifactFilter;
8190
8450
  ```
8191
8451
 
8192
- What `runList` can be asked.
8452
+ What `runList` can be asked: an {@link ArtifactFilter}, of which a run answers every dimension but `domain` and
8453
+ `spec`. `video` keeps only runs that have at least one playable video recording; `total` then counts only those.
8193
8454
 
8194
8455
  #### `RunList`
8195
8456
 
@@ -8202,6 +8463,10 @@ export type RunList = {
8202
8463
  readonly runs: readonly RunSummary[];
8203
8464
  /** How many of all runs (whatever was asked, before `limit`) have a playable video recording. */
8204
8465
  readonly withVideo?: number;
8466
+ /** The filters of the query that were applied. A viewer that asked for another knows the server ignored it. */
8467
+ readonly filters?: readonly FilterName[];
8468
+ /** The values each filter dimension offers with counts over every run (see `facetsOf`). */
8469
+ readonly facets?: QueryFacets;
8205
8470
  };
8206
8471
  ```
8207
8472
 
@@ -8242,6 +8507,70 @@ export type RunDetail = {
8242
8507
 
8243
8508
  `GET /api/reports/runs/detail`: one run and every result it recorded.
8244
8509
 
8510
+ #### `ArtifactResult`
8511
+
8512
+ ```ts
8513
+ export type ArtifactResult = {
8514
+ /** The test's key, as in `TestSummary.key`. */
8515
+ readonly key: string;
8516
+ readonly id: string;
8517
+ readonly title: string;
8518
+ readonly specPath: string | null;
8519
+ /** The domain of its spec, or null when it has none. */
8520
+ readonly domain: string | null;
8521
+ /** When it was recorded, epoch ms. */
8522
+ readonly at: number;
8523
+ readonly status: EvidenceStatus;
8524
+ readonly environment: string | null;
8525
+ readonly producer: string;
8526
+ /** The kind of the part that held it. */
8527
+ readonly kind: string;
8528
+ readonly parameters: Readonly<Record<string, string>>;
8529
+ readonly branch: string | null;
8530
+ readonly sha: string | null;
8531
+ /** The key of its run (see `runKey`). */
8532
+ readonly run: string;
8533
+ readonly runUrl: string | null;
8534
+ /** The id of the layer that recorded it: with `key`, the exact result to open. */
8535
+ readonly layer: string;
8536
+ readonly backend: string;
8537
+ /** Attachments on the result, including those on its steps and attempts. */
8538
+ readonly attachments: number;
8539
+ /** The normalized tags of its attachments. */
8540
+ readonly artifactTags: readonly string[];
8541
+ /** The tags of its check and run, which its attachments inherit. */
8542
+ readonly tags: readonly string[];
8543
+ /** The media types (roles) of its attachments; `video` only for a playable recording. */
8544
+ readonly types: readonly string[];
8545
+ readonly labels: Readonly<Record<string, string>>;
8546
+ readonly videos?: readonly VideoRef[];
8547
+ };
8548
+ ```
8549
+
8550
+ One recorded result that carries evidence (an attachment: a recording, a screenshot, a trace), with where and when it
8551
+ was recorded: the unit `artifactList` finds. It is a result in the history, not only the report's current winner, so
8552
+ a recording tagged `demo` is found with its original run, revision and status even when a later attempt is untagged.
8553
+
8554
+ #### `ArtifactList`
8555
+
8556
+ ```ts
8557
+ export type ArtifactList = {
8558
+ readonly schemaVersion: typeof TEST_CATALOG_VERSION;
8559
+ readonly version: string;
8560
+ /** Results that matched, before `offset` and `limit`. */
8561
+ readonly total: number;
8562
+ readonly offset: number;
8563
+ readonly limit: number;
8564
+ readonly results: readonly ArtifactResult[];
8565
+ /** The filters of the query that were applied. */
8566
+ readonly filters: readonly FilterName[];
8567
+ /** The values each dimension offers with disjunctive counts over every result that carries evidence. */
8568
+ readonly facets: QueryFacets;
8569
+ };
8570
+ ```
8571
+
8572
+ `GET /api/reports/artifacts`: the recorded results that carry evidence and match a query, newest first.
8573
+
8245
8574
  #### `runKey`
8246
8575
 
8247
8576
  ```ts
@@ -8279,6 +8608,7 @@ const catalog = testCatalog(catalogModel(layers, { history }));
8279
8608
  - `get runCount(): number { return this.#runs.size; }` — How many runs it knows of.
8280
8609
  - `runs(): RunSummary[]` — Every run, newest first.
8281
8610
  - `run(key: string): { readonly run: RunSummary; readonly results: RunResult[] } | null` — One run and its results (failed first, then by spec and title), or null.
8611
+ - `evidenceResults(): readonly { readonly result: ArtifactResult; readonly fields: FilterFields }[]` — Every recorded result that carries an attachment, newest first, with the values a query looks at. Built once per layer added. The domain is left to the caller (`artifactList`), which may know more specs than the layers recorded.
8282
8612
  - `testsIn(key: string): ReadonlySet<string>` — The keys of the tests one run recorded.
8283
8613
 
8284
8614
  #### `durationStats`
@@ -8392,6 +8722,32 @@ Every run the history knows of, newest first, `limit` at most (default 100). Wit
8392
8722
  have a playable video recording: the filter and `total` apply to every run before `limit`, and nothing is read from
8393
8723
  any recording's bytes. `withVideo` always counts every run that has one.
8394
8724
 
8725
+ #### `fieldsOfRun`
8726
+
8727
+ ```ts
8728
+ export function fieldsOfRun(run: RunSummary): FilterFields
8729
+ ```
8730
+
8731
+ What a query looks at in a run: see {@link ArtifactFilter}. A run's outcome is `failed` when any result failed.
8732
+
8733
+ #### `artifactList`
8734
+
8735
+ ```ts
8736
+ export function artifactList(history: TestHistory, version: string, query: ArtifactFilter = {}, page: { readonly limit?: number; readonly offset?: number; readonly domainOf?: (specPath: string) => string | undefined } = {}): ArtifactList
8737
+ ```
8738
+
8739
+ The recorded results that carry evidence and match a query, newest first, across the whole history (`limit` default
8740
+ 100). `total` and the facet counts apply to every match before `offset` and `limit`. `domainOf` tells the domain of a
8741
+ spec path when the caller knows more than the layers recorded (a server with a spec set); otherwise it is the
8742
+ recorded spec's domain, else the folder the path implies.
8743
+
8744
+ **Example**
8745
+
8746
+ ```ts
8747
+ const list = artifactList(history, 'v1', { artifactTag: ['demo'], type: ['video'], environment: ['ios'] });
8748
+ list.results.map(r => [r.title, r.run, r.sha, r.status]);
8749
+ ```
8750
+
8395
8751
  #### `runDetail`
8396
8752
 
8397
8753
  ```ts
@@ -8440,6 +8796,14 @@ export function runsMarkdown(list: RunList): string
8440
8796
 
8441
8797
  The run list as Markdown.
8442
8798
 
8799
+ #### `artifactsMarkdown`
8800
+
8801
+ ```ts
8802
+ export function artifactsMarkdown(list: ArtifactList): string
8803
+ ```
8804
+
8805
+ The artifact results as Markdown, for an agent or a terminal.
8806
+
8443
8807
  #### `runMarkdown`
8444
8808
 
8445
8809
  ```ts
@@ -308,6 +308,22 @@ export const ReportRunSchema = …
308
308
 
309
309
  A run: the layers that name the same execution (else share a run URL, else a branch and commit), with what they recorded.
310
310
 
311
+ #### `QueryFacetValueSchema`
312
+
313
+ ```ts
314
+ export const QueryFacetValueSchema = …
315
+ ```
316
+
317
+ A value of a filter dimension and how many records have it.
318
+
319
+ #### `QueryFacetsSchema`
320
+
321
+ ```ts
322
+ export const QueryFacetsSchema = …
323
+ ```
324
+
325
+ The facets of a runs or artifacts query: per dimension, its values with disjunctive counts (the list's count if that value were the only one chosen, every other dimension's filter applied).
326
+
311
327
  #### `RunListSchema`
312
328
 
313
329
  ```ts
@@ -316,6 +332,22 @@ export const RunListSchema = …
316
332
 
317
333
  `GET /api/reports/runs`: runs, newest first.
318
334
 
335
+ #### `ArtifactResultSchema`
336
+
337
+ ```ts
338
+ export const ArtifactResultSchema = …
339
+ ```
340
+
341
+ One recorded result that carries evidence: where and when it was recorded and what it carries.
342
+
343
+ #### `ArtifactListSchema`
344
+
345
+ ```ts
346
+ export const ArtifactListSchema = …
347
+ ```
348
+
349
+ `GET /api/reports/artifacts`: recorded results that carry evidence, newest first, with the facets of the query.
350
+
319
351
  #### `RunDetailSchema`
320
352
 
321
353
  ```ts
@@ -62,6 +62,37 @@ attachments with that tag, and says how many match. Tags describe evidence; they
62
62
  - **Older layers.** A layer without attachment tags reads exactly as before. Tags on an attachment that cannot be read
63
63
  (not a list of strings) are dropped, and the attachment, check and layer stay.
64
64
 
65
+ ## Finding runs and artifacts
66
+
67
+ The viewer's **Runs** and **Artifacts** tabs and the API answer one query, so a link means the same thing everywhere.
68
+ "Demo videos for checkout on iPhone, including failed attempts" is:
69
+
70
+ ```sh
71
+ curl "$ATLAS_SERVER_URL/api/reports/artifacts?artifactTag=demo&type=video&domain=checkout&label=device%3DiPhone"
72
+ ```
73
+
74
+ - **Dimensions.** `artifactTag`, `type` (the media type as a role: `video`, `image`, `audio`, `snapshot`, `text`,
75
+ `file`; `video` means a playable recording), `label` (`key=value`, or `key`), `domain` and `spec` (artifacts only),
76
+ `environment` (the device or client), `producer`, `branch`, `sha` (seven or more characters), `outcome` (`passed`,
77
+ `failed`, `running`, `not-run`), `since` and `until` (epoch milliseconds or an ISO date), and `q` (words).
78
+ - **Semantics.** Filters combine with AND across dimensions and OR within one (`environment=ios,android`). With both
79
+ `artifactTag` and `type`, one attachment must be both where the report says which video carries which tag: a tagged
80
+ screenshot beside an untagged video is not a tagged video. `inherited=1` also counts the tags an attachment inherits
81
+ from its check and run. A comma or backslash inside a value is written `\,` or `\\`; the viewer encodes `#` and spaces.
82
+ - **History.** Artifacts are recorded results, not only the report's current winners: a recording tagged `demo` is
83
+ found with its original run, commit and status even when a later attempt of the test is untagged. Tags a reviewer
84
+ added after upload are kept apart from the layer and are not searched here.
85
+ - **Facets.** Each answer carries `facets`: for every dimension the values on offer. The count beside a value is what
86
+ the list would hold if that value alone were chosen in its dimension with every other filter applied, so the
87
+ chosen value's count is the list's and the others show what adding them would give.
88
+ - **Older servers.** A response names the filters it applied in `filters`. A viewer that asked for one that is missing
89
+ filters what a run summary can answer itself and says plainly that the server cannot filter by the rest (tags, types,
90
+ labels); it never presents an unfiltered list as narrowed. A server without `/api/reports/artifacts` is reported as
91
+ such on the Artifacts tab.
92
+ - **Views.** *Views* saves the current filters under a name in this browser, per project and tab. A saved view is a
93
+ **live query**: opening it asks again and shows what matches the newest report. It is not a snapshot; an immutable
94
+ review collection is a separate feature. *Copy link* gives the address, which holds the whole query.
95
+
65
96
  ## Labels describe, parameters split
66
97
 
67
98
  A result's **slot** is its producer, check id, part kind, spec, environment and **parameters**. The newest completed
package/docs/server.md CHANGED
@@ -375,7 +375,8 @@ with `format=markdown` (or `Accept: text/markdown`), and takes the project prefi
375
375
  | Only some: `q` (all words), `status` (or `flaky`), `environment`, `producer`, `domain`, `spec`, `tag`, `run`; `sort`, `offset`, `limit` | `GET /api/reports/tests?status=failed&environment=ios` |
376
376
  | One test: every recorded result (status, duration, environment, branch, commit, CI run), newest first | `GET /api/reports/tests/history?key=<key>` |
377
377
  | Why Specs, domains and counts disagree: ambiguous spec paths, shared domains, gaps, a reconciliation of the totals | `GET /api/reports/tests/health` |
378
- | The runs that recorded the report (by CI run URL, else branch and commit); `?video=true` keeps those with a playable video | `GET /api/reports/runs` |
378
+ | The runs that recorded the report (by CI run URL, else branch and commit); `?video=true` keeps those with a playable video; `artifactTag`, `type`, `label`, `environment`, `branch`, `outcome`, `since`, `until` narrow it ([filtering](rich-evidence.md#finding-runs-and-artifacts)) | `GET /api/reports/runs` |
379
+ | Recorded results that carry evidence, across the whole history, found by tag, media type, label, feature, environment, branch, revision, outcome and time | `GET /api/reports/artifacts?artifactTag=demo&type=video` |
379
380
  | Every test one run recorded | `GET /api/reports/runs/detail?key=<run key>` |
380
381
 
381
382
  ```sh
@@ -458,6 +459,8 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
458
459
  | `POST /api/projects/{project}/reports/artifact-tags` | publish | Tag or untag one attachment after upload. |
459
460
  | `GET /api/reports/runs` | read | The runs that recorded the report. |
460
461
  | `GET /api/projects/{project}/reports/runs` | read | The runs that recorded the report. |
462
+ | `GET /api/reports/artifacts` | read | Recorded results that carry evidence, across the whole history. |
463
+ | `GET /api/projects/{project}/reports/artifacts` | read | Recorded results that carry evidence, across the whole history. |
461
464
  | `GET /api/reports/runs/detail` | read | Every test one run recorded. |
462
465
  | `GET /api/projects/{project}/reports/runs/detail` | read | Every test one run recorded. |
463
466
  | `GET /api/reports/outline` | read | The specs a report is laid over. |
package/docs/ui.md CHANGED
@@ -125,7 +125,7 @@ The narrowly scoped exceptions are:
125
125
  | Area | Screens/interactions | Mantine composition |
126
126
  | --- | --- | --- |
127
127
  | Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
128
- | Project home | Tests/runs tabs, metrics, search, status/environment/domain filters, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, Button, Accordion |
128
+ | Project home | Tests/runs/artifacts tabs, metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion |
129
129
  | Test detail | Current outcomes, statistics, duration graph, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
130
130
  | Specs | All six existing kinds, overview/index, Gherkin rules/scenarios/examples, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Table, CodeHighlightTabs, ResultRow |
131
131
  | Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |