@crvouga/atlas 0.14.0 → 0.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/CHANGELOG.md +12 -0
  2. package/dist/reports/src/artifact-tag-store.d.ts +20 -0
  3. package/dist/reports/src/artifact-tag-store.js +70 -0
  4. package/dist/reports/src/artifact-tags.d.ts +96 -0
  5. package/dist/reports/src/artifact-tags.js +96 -0
  6. package/dist/reports/src/evidence-detail.d.ts +20 -3
  7. package/dist/reports/src/evidence-detail.js +66 -13
  8. package/dist/reports/src/evidence-types.d.ts +33 -0
  9. package/dist/reports/src/layer-merge.js +2 -2
  10. package/dist/reports/src/reports-explorer-route.d.ts +8 -0
  11. package/dist/reports/src/reports-route.d.ts +8 -0
  12. package/dist/reports/src/test-catalog.d.ts +35 -26
  13. package/dist/reports/src/test-catalog.js +79 -15
  14. package/dist/schema/src/api.d.ts +246 -0
  15. package/dist/schema/src/api.js +59 -5
  16. package/dist/schema/src/layer.d.ts +33 -0
  17. package/dist/schema/src/layer.js +11 -1
  18. package/dist/server/src/app.js +35 -3
  19. package/dist/server/src/llms.js +2 -1
  20. package/dist/server/src/openapi.d.ts +1 -1
  21. package/dist/server/src/openapi.js +20 -3
  22. package/docs/extending.md +3 -0
  23. package/docs/reference/http-api.md +52 -2
  24. package/docs/reference/reports.md +327 -5
  25. package/docs/reference/schema.md +48 -0
  26. package/docs/rich-evidence.md +28 -0
  27. package/docs/server.md +25 -1
  28. package/llms-full.txt +108 -3
  29. package/package.json +1 -1
  30. package/packages/schema/schemas/check-detail.schema.json +28 -0
  31. package/packages/schema/schemas/effective-delta.schema.json +96 -0
  32. package/packages/schema/schemas/effective-evidence.schema.json +96 -0
  33. package/packages/schema/schemas/evidence-page.schema.json +48 -0
  34. package/packages/schema/schemas/layer-rows.schema.json +48 -0
  35. package/packages/schema/schemas/layer.schema.json +70 -0
  36. package/packages/schema/schemas/publish-checks.schema.json +35 -0
  37. package/packages/schema/schemas/report-run.schema.json +71 -0
  38. package/packages/schema/schemas/report-runs.schema.json +38 -1
  39. package/packages/schema/schemas/test-catalog.schema.json +83 -0
  40. package/packages/schema/schemas/test-history.schema.json +110 -0
  41. package/packages/server/openapi.json +1259 -33
  42. package/packages/visualizer/src/evidence/artifact-tags.tsx +285 -0
  43. package/packages/visualizer/src/evidence/artifacts.tsx +18 -12
  44. package/packages/visualizer/src/explorer/RunsView.tsx +128 -10
  45. package/packages/visualizer/src/explorer/catalog.tsx +16 -6
  46. package/packages/visualizer/src/explorer/search.ts +4 -1
@@ -4,6 +4,209 @@ Layered evidence for any kind of spec: publishing layers, stores (folder, S3, an
4
4
 
5
5
  > Generated from the doc comments of `packages/reports/src` by `bun run docs`. Do not edit: change the comment next to the code.
6
6
 
7
+ ### Reviewer tags in a store
8
+
9
+ Source: `packages/reports/src/artifact-tag-store.ts`. Import: `@crvouga/atlas/reports/artifact-tag-store`
10
+
11
+ Where a reviewer's tags for attachments are kept: one small immutable object per edit under
12
+ `curation/artifact-tags/`, named by time and a random id so editors never overwrite one another, and never changed
13
+ or deleted afterwards. The layers they are about are not touched. The list of edits is the audit trail: who changed
14
+ which tags of which attachment, and when. `foldArtifactTags` turns it into the tags that stand now.
15
+
16
+ See docs/server.md#tagging-artifacts
17
+
18
+ #### `ARTIFACT_TAGS_PREFIX`
19
+
20
+ ```ts
21
+ export const ARTIFACT_TAGS_PREFIX = …
22
+ ```
23
+
24
+ The prefix of every recorded edit in a store.
25
+
26
+ #### `ArtifactTagError`
27
+
28
+ ```ts
29
+ export class ArtifactTagError extends Error
30
+ ```
31
+
32
+ The edit was not acceptable (`status` is the HTTP status to answer).
33
+
34
+ #### `recordArtifactTags`
35
+
36
+ ```ts
37
+ export async function recordArtifactTags(store: ObjectStore, input: unknown, by: string, atMs: number): Promise<ArtifactTagEntry>
38
+ ```
39
+
40
+ Validates an edit (`ArtifactTagEditSchema`), normalizes its tags and records it as a new object in `store`, by
41
+ `by` at `atMs`. Throws {@link ArtifactTagError} for a body that is not an edit, a tag that is not a tag (empty, or
42
+ over 64 characters) or an edit that changes nothing. Recording the same edit twice records two entries.
43
+
44
+ #### `readArtifactTagEntries`
45
+
46
+ ```ts
47
+ export async function readArtifactTagEntries(store: ObjectStore): Promise<ArtifactTagEntry[]>
48
+ ```
49
+
50
+ Every recorded edit in `store`, oldest first. An object that cannot be read as an edit is skipped (unknown, never
51
+ guessed at). A store with none gives an empty list. Transport errors throw.
52
+
53
+ ### Artifact tags
54
+
55
+ Source: `packages/reports/src/artifact-tags.ts`. Import: `@crvouga/atlas/reports/artifact-tags`
56
+
57
+ The tags of one attachment (a recording, a screenshot, a trace): how a tag is normalized so `demo` and `#demo`
58
+ are the same tag, which tags an attachment shows (its producer's, its reviewers', and those it inherits from its
59
+ check and run), and how a reviewer's edits fold into the tags that stand now. Pure functions, safe in a browser.
60
+
61
+ Tags describe evidence; they never decide a status, a slot or a winner. A producer writes `tags` on the attachment
62
+ inside an immutable layer. A reviewer's tags live apart, as an append-only list of edits that the server records
63
+ with who made each and when (`artifact-tag-store.ts`); the layer is never rewritten.
64
+
65
+ See docs/rich-evidence.md#tagging-artifacts-and-demos
66
+
67
+ #### `MAX_TAG_LENGTH`
68
+
69
+ ```ts
70
+ export const MAX_TAG_LENGTH = …
71
+ ```
72
+
73
+ The longest a normalized tag may be, in characters.
74
+
75
+ #### `normalizeTag`
76
+
77
+ ```ts
78
+ export function normalizeTag(raw: string): string | null
79
+ ```
80
+
81
+ Normalizes a tag so equal tags compare equal: Unicode NFC, any leading `#` removed, surrounding whitespace
82
+ trimmed, letters lower-cased and each run of whitespace inside turned into one `-`. Every other character is kept,
83
+ so `Smoke`, `#smoke` and ` smoke ` are `smoke`, and `team/payments` or `release=2` stay as they are. Returns
84
+ `null` for a tag that is empty after that (`''`, `'#'`, `' '`) or longer than {@link MAX_TAG_LENGTH}.
85
+
86
+ **Example**
87
+
88
+ ```ts
89
+ normalizeTag('#Demo'); // 'demo'
90
+ normalizeTag('release candidate'); // 'release-candidate'
91
+ normalizeTag('#'); // null
92
+ ```
93
+
94
+ #### `normalizeTags`
95
+
96
+ ```ts
97
+ export function normalizeTags(raw: Iterable<string> | undefined): string[]
98
+ ```
99
+
100
+ Normalizes every tag with {@link normalizeTag}, dropping the ones that are not tags and the repeats; the first of each keeps its place.
101
+
102
+ #### `displayTag`
103
+
104
+ ```ts
105
+ export const displayTag = …
106
+ ```
107
+
108
+ The tag as a person sees it: `#demo`.
109
+
110
+ #### `artifactRef`
111
+
112
+ ```ts
113
+ export function artifactRef(attachment: Pick<Attachment, 'id' | 'bundleKey' | 'name'>): string
114
+ ```
115
+
116
+ How an edit names an attachment: `id:<id>` when it has an `id`, else `key:<bundleKey>` (its bytes), else
117
+ `name:<name>`. It is stable for a layer, which never changes. Two attachments in one check with the same bytes and
118
+ no `id` share a reference, and so share reviewer tags; give them an `id` to tag them apart.
119
+
120
+ #### `ArtifactTarget`
121
+
122
+ ```ts
123
+ export type ArtifactTarget = {
124
+ readonly layer: string;
125
+ readonly check: string;
126
+ readonly kind?: string | undefined;
127
+ readonly artifact: string;
128
+ };
129
+ ```
130
+
131
+ Where a reviewer's tags are about: one attachment of one check of one archived layer.
132
+
133
+ #### `ArtifactTagEntry`
134
+
135
+ ```ts
136
+ export type ArtifactTagEntry = {
137
+ readonly id: string;
138
+ readonly target: ArtifactTarget;
139
+ readonly add: readonly string[];
140
+ readonly remove: readonly string[];
141
+ readonly by: string;
142
+ readonly atMs: number;
143
+ };
144
+ ```
145
+
146
+ One recorded edit. Entries are never changed or removed.
147
+
148
+ #### `targetKey`
149
+
150
+ ```ts
151
+ export const targetKey = …
152
+ ```
153
+
154
+ A string that is equal for equal targets (the part `kind` is optional and then does not matter).
155
+
156
+ #### `ReviewerTag`
157
+
158
+ ```ts
159
+ export type ReviewerTag = { readonly tag: string; readonly by: string; readonly atMs: number };
160
+ ```
161
+
162
+ A tag a reviewer added: who added it last, and when.
163
+
164
+ #### `foldArtifactTags`
165
+
166
+ ```ts
167
+ export function foldArtifactTags(entries: readonly ArtifactTagEntry[]): Map<string, ReviewerTag[]>
168
+ ```
169
+
170
+ Folds the recorded edits (in any order) into the reviewer tags that stand now, by target (see {@link targetKey}).
171
+ Edits apply oldest first, ties by `id`; a tag added, removed and added again stands, with the time and author of
172
+ the last add. A target whose tags were all removed is absent. Entries that are not understood are ignored.
173
+
174
+ #### `TagSource`
175
+
176
+ ```ts
177
+ export type TagSource = 'artifact' | 'reviewer' | 'inherited';
178
+ ```
179
+
180
+ Where a shown tag comes from: the attachment's own producer (`artifact`), a reviewer after the upload (`reviewer`), or the check or run around it (`inherited`).
181
+
182
+ #### `ArtifactTag`
183
+
184
+ ```ts
185
+ export type ArtifactTag = { readonly tag: string; readonly source: TagSource; readonly by?: string; readonly atMs?: number };
186
+ ```
187
+
188
+ One tag an attachment shows, with its source and, for a reviewer's tag, who added it and when.
189
+
190
+ #### `artifactTagsOf`
191
+
192
+ ```ts
193
+ export function artifactTagsOf(attachment: Pick<Attachment, 'tags'>, options: { readonly reviewer?: readonly ReviewerTag[]; readonly inherited?: Iterable<string> } = {}): ArtifactTag[]
194
+ ```
195
+
196
+ The tags one attachment shows, normalized and without repeats: its producer's own (`artifact`), then the
197
+ reviewers' (`reviewer`), then those of its check and run that it does not have itself (`inherited`). A tag it has
198
+ more than once is shown once, by the first source above.
199
+
200
+ #### `hasArtifactTag`
201
+
202
+ ```ts
203
+ export function hasArtifactTag(tags: readonly ArtifactTag[], wanted: string, options: { readonly inherited?: boolean } = {}): boolean
204
+ ```
205
+
206
+ Does an attachment match a tag (either spelling)? An exact query (`inherited: false`, the default) matches only the
207
+ tags the attachment has itself, from its producer or a reviewer; an inherited query also matches the tags it
208
+ inherits from its check and run. The two differ for a recording that is untagged under a tagged run.
209
+
7
210
  ### Reading the files attachments name
8
211
 
9
212
  Source: `packages/reports/src/attachment-files.ts`. Import: `@crvouga/atlas/reports/attachment-files`
@@ -1222,8 +1425,37 @@ read when someone asks for it. Undefined when there is none.
1222
1425
 
1223
1426
  Several holders (for example a check and its layer) are summed: `attachments` also counts those on steps and
1224
1427
  attempts, `steps` counts only each holder's own top-level steps, tags are de-duplicated, and for a repeated
1225
- label the later holder wins. `roles` are the attachments' roles (or, without one, what their media type or file
1226
- name implies), sorted and distinct.
1428
+ label the later holder wins. `artifactTags` are the attachments' own normalized tags and are never inherited. `roles` are the attachments' roles (or, without one, what their media type or file
1429
+ name implies), sorted and distinct. `videos` lists every attachment whose effective media type is video, with the
1430
+ check, retry attempt or step that holds it (see {@link isVideoAttachment}).
1431
+
1432
+ #### `summarizeLayerEvidence`
1433
+
1434
+ ```ts
1435
+ export function summarizeLayerEvidence(...holders: readonly EvidenceHolder[]): EvidenceSummary | undefined
1436
+ ```
1437
+
1438
+ Like {@link summarizeEvidence} for a layer and its parts: the same summary, but a video attached to the layer or a
1439
+ part is owned by the `layer`, not by a check.
1440
+
1441
+ #### `isVideoAttachment`
1442
+
1443
+ ```ts
1444
+ export function isVideoAttachment(attachment: Pick<Attachment, 'name' | 'role' | 'contentType'>): boolean
1445
+ ```
1446
+
1447
+ Whether an attachment is a video, by its effective media type: the producer's `contentType`, else the one its file
1448
+ name implies. A role never decides this on its own (`demo` on a PNG is not a video), except that a `video` role
1449
+ settles a file whose type nothing says (`application/octet-stream`).
1450
+
1451
+ #### `videoCounts`
1452
+
1453
+ ```ts
1454
+ export function videoCounts(videos: readonly VideoRef[] | undefined): VideoCounts | undefined
1455
+ ```
1456
+
1457
+ Counts references, distinct playable recordings and unplayable references. Undefined when there is no reference, so
1458
+ "no video" and "nothing known about video" look the same on the wire.
1227
1459
 
1228
1460
  #### `roleOfType`
1229
1461
 
@@ -1406,6 +1638,51 @@ export type StatusCounts = Readonly<Record<EvidenceStatus, number>>;
1406
1638
 
1407
1639
  How many results have each status (`passed`, `failed`, `running`, `not-run`).
1408
1640
 
1641
+ #### `VideoOwner`
1642
+
1643
+ ```ts
1644
+ export type VideoOwner = 'layer' | 'check' | 'attempt' | 'step';
1645
+ ```
1646
+
1647
+ Who holds a recording: the whole layer (or one of its parts), one check, one retry attempt of a check, or one step.
1648
+
1649
+ #### `VideoRef`
1650
+
1651
+ ```ts
1652
+ export type VideoRef = {
1653
+ /** The attachment's name, for a person. */
1654
+ readonly name: string;
1655
+ readonly owner: VideoOwner;
1656
+ /** The try that holds it, when `owner` is `attempt` or the step is inside one. */
1657
+ readonly attempt?: number;
1658
+ /** The name of the step that holds it, when `owner` is `step`. */
1659
+ readonly step?: string;
1660
+ /** What identifies the bytes (digest, bundle key or address); references with the same one are the same recording. Absent when `available` is false. */
1661
+ readonly recording?: string;
1662
+ /** False when the reference names no bytes at all (no bundle key, address or inline text), so it can never be played. */
1663
+ readonly available: boolean;
1664
+ };
1665
+ ```
1666
+
1667
+ One reference to a video, found by its effective media type (the producer's `contentType`, else the one the file
1668
+ name implies; a `video` role alone counts only when nothing else says what the file is). A reference is not a
1669
+ recording: the same bytes referenced from three checks are three references and one recording.
1670
+
1671
+ #### `VideoCounts`
1672
+
1673
+ ```ts
1674
+ export type VideoCounts = {
1675
+ /** Every reference to a video, playable or not. */
1676
+ readonly references: number;
1677
+ /** Distinct playable recordings: identical bytes referenced twice are one. */
1678
+ readonly recordings: number;
1679
+ /** References that name no bytes. A report that predates availability metadata has none of these and none of the others. */
1680
+ readonly unavailable: number;
1681
+ };
1682
+ ```
1683
+
1684
+ Video availability as counts: references, distinct playable recordings (deduplicated), and references that cannot be played.
1685
+
1409
1686
  #### `EvidenceSummary`
1410
1687
 
1411
1688
  ```ts
@@ -1418,9 +1695,13 @@ export type EvidenceSummary = {
1418
1695
  readonly attempts: number;
1419
1696
  readonly links: number;
1420
1697
  readonly tags: readonly string[];
1698
+ /** The normalized tags of the attachments themselves (`#demo` is `demo`), sorted and distinct. Absent when no attachment is tagged. Unlike `tags` they are never inherited from a check or run. */
1699
+ readonly artifactTags?: readonly string[];
1421
1700
  readonly labels: Readonly<Record<string, string>>;
1422
1701
  /** The distinct roles of its attachments (or what their media types say), to show what kind of evidence there is. */
1423
1702
  readonly roles: readonly string[];
1703
+ /** Its references to videos, wherever they are held (layer, check, attempt, step). Only there when it has any. */
1704
+ readonly videos?: readonly VideoRef[];
1424
1705
  };
1425
1706
  ```
1426
1707
 
@@ -7514,6 +7795,10 @@ export type TestSummary = {
7514
7795
  readonly outcomes: readonly TestOutcome[];
7515
7796
  readonly stats: TestStats;
7516
7797
  readonly tags: readonly string[];
7798
+ /** The normalized tags of the attachments of its current results (`demo`, shown as `#demo`). Never inherited from a check or run; the `tags` above are. Absent in documents of older servers. */
7799
+ readonly artifactTags?: readonly string[];
7800
+ /** The normalized tags its attachments inherit from their check and run (those of the results that have attachments), apart from `artifactTags`. What an inherited artifact query also matches. */
7801
+ readonly inheritedArtifactTags?: readonly string[];
7517
7802
  readonly labels: Readonly<Record<string, string>>;
7518
7803
  /** What its current results carry: attachments, steps, retries, and the kinds of attachment. */
7519
7804
  readonly evidence: { readonly attachments: number; readonly steps: number; readonly attempts: number; readonly roles: readonly string[] };
@@ -7604,6 +7889,8 @@ export type CatalogFacets = {
7604
7889
  readonly domains: readonly FacetValue[];
7605
7890
  readonly specs: readonly (CatalogSpec & { readonly tests: number; readonly counts: StatusCounts })[];
7606
7891
  readonly tags: readonly FacetValue[];
7892
+ /** The tags of attachments, normalized. Absent in documents of older servers. */
7893
+ readonly artifactTags?: readonly FacetValue[];
7607
7894
  readonly branches: readonly FacetValue[];
7608
7895
  readonly labels: Readonly<Record<string, readonly FacetValue[]>>;
7609
7896
  };
@@ -7641,6 +7928,10 @@ export type TestQuery = {
7641
7928
  /** Spec paths (`specs/...`). */
7642
7929
  readonly spec?: readonly string[];
7643
7930
  readonly tag?: readonly string[];
7931
+ /** Only tests with an attachment carrying this tag (`demo` and `#demo` are the same), not counting tags inherited from a check or run: the exact artifact query. `tag` is the inherited one. */
7932
+ readonly artifactTag?: readonly string[];
7933
+ /** With `artifactTag`: also match the tags an attachment inherits from its check and run (a recording without a tag of its own under a run tagged `demo`). Default false: the exact query. */
7934
+ readonly inherited?: boolean;
7644
7935
  /** Only tests recorded in this run (a key from `runList`). */
7645
7936
  readonly run?: string;
7646
7937
  readonly sort?: TestSort;
@@ -7715,20 +8006,39 @@ export type RunSummary = {
7715
8006
  readonly totalDurationMs: number | null;
7716
8007
  /** Up to twenty tests that failed in it. */
7717
8008
  readonly failing: readonly { readonly key: string; readonly title: string; readonly environment: string | null; readonly detail: string }[];
8009
+ /**
8010
+ * Its videos, from the run's layers and from the last result of each test: references, distinct playable recordings
8011
+ * and references that cannot be played. Absent when it has none, and in a report that carries no video information.
8012
+ */
8013
+ readonly videos?: VideoCounts;
7718
8014
  };
7719
8015
  ```
7720
8016
 
7721
8017
  A run: the layers that name the same execution (`run.id`), else share a CI run URL, else a branch and commit. When,
7722
8018
  where, and how its tests went.
7723
8019
 
8020
+ #### `RunQuery`
8021
+
8022
+ ```ts
8023
+ export type RunQuery = {
8024
+ /** Only runs that have at least one playable video recording. `total` then counts only those. */
8025
+ readonly video?: boolean;
8026
+ };
8027
+ ```
8028
+
8029
+ What `runList` can be asked.
8030
+
7724
8031
  #### `RunList`
7725
8032
 
7726
8033
  ```ts
7727
8034
  export type RunList = {
7728
8035
  readonly schemaVersion: typeof TEST_CATALOG_VERSION;
7729
8036
  readonly version: string;
8037
+ /** Runs that matched, before `limit`. */
7730
8038
  readonly total: number;
7731
8039
  readonly runs: readonly RunSummary[];
8040
+ /** How many of all runs (whatever was asked, before `limit`) have a playable video recording. */
8041
+ readonly withVideo?: number;
7732
8042
  };
7733
8043
  ```
7734
8044
 
@@ -7749,6 +8059,8 @@ export type RunResult = {
7749
8059
  readonly at: number;
7750
8060
  readonly layer: string;
7751
8061
  readonly detail: string;
8062
+ /** Every video its evidence references, with who holds it (check, retry attempt or step). Absent when there is none. */
8063
+ readonly videos?: readonly VideoRef[];
7752
8064
  };
7753
8065
  ```
7754
8066
 
@@ -7899,13 +8211,23 @@ export function testHistory(model: CatalogModel, key: string): TestHistoryDocume
7899
8211
 
7900
8212
  One test and its recorded results, or null when the report has no test with that key.
7901
8213
 
8214
+ #### `hasVideo`
8215
+
8216
+ ```ts
8217
+ export function hasVideo(run: Pick<RunSummary, 'videos'>): boolean
8218
+ ```
8219
+
8220
+ Whether a run has at least one playable video recording.
8221
+
7902
8222
  #### `runList`
7903
8223
 
7904
8224
  ```ts
7905
- export function runList(history: TestHistory, version: string, limit = 100): RunList
8225
+ export function runList(history: TestHistory, version: string, limit = 100, query: RunQuery = {}): RunList
7906
8226
  ```
7907
8227
 
7908
- Every run the history knows of, newest first, `limit` at most (default 100).
8228
+ Every run the history knows of, newest first, `limit` at most (default 100). With `query.video` only the runs that
8229
+ have a playable video recording: the filter and `total` apply to every run before `limit`, and nothing is read from
8230
+ any recording's bytes. `withVideo` always counts every run that has one.
7909
8231
 
7910
8232
  #### `runDetail`
7911
8233
 
@@ -7921,7 +8243,7 @@ One run with every result it recorded, or null when no run has that key.
7921
8243
  export function parseTestQuery(params: URLSearchParams): TestQuery
7922
8244
  ```
7923
8245
 
7924
- Reads a `TestQuery` from URL search parameters: `q`, `sort`, `offset`, `limit`, `run`, and repeatable or comma-separated `status`, `environment`, `producer`, `domain`, `spec`, `tag`.
8246
+ Reads a `TestQuery` from URL search parameters: `q`, `sort`, `offset`, `limit`, `run`, and repeatable or comma-separated `status`, `environment`, `producer`, `domain`, `spec`, `tag`, `artifactTag`, and `inherited=1`.
7925
8247
 
7926
8248
  #### `formatMs`
7927
8249
 
@@ -69,6 +69,22 @@ export const BackendInfoSchema = …
69
69
 
70
70
  A backend as the report shows it: where results came from, and whether it could be read.
71
71
 
72
+ #### `VideoRefSchema`
73
+
74
+ ```ts
75
+ export const VideoRefSchema = …
76
+ ```
77
+
78
+ One reference to a video held by a layer, check, retry attempt or step, found by the attachment's effective media type.
79
+
80
+ #### `VideoCountsSchema`
81
+
82
+ ```ts
83
+ export const VideoCountsSchema = …
84
+ ```
85
+
86
+ Video references as counts: how many are referenced, how many distinct recordings can be played, how many cannot.
87
+
72
88
  #### `EvidenceSummarySchema`
73
89
 
74
90
  ```ts
@@ -185,6 +201,38 @@ export const ChecksResultSchema = …
185
201
 
186
202
  `POST /api/checks` answered: the layer that was written.
187
203
 
204
+ #### `ArtifactTargetSchema`
205
+
206
+ ```ts
207
+ export const ArtifactTargetSchema = …
208
+ ```
209
+
210
+ Which attachment a reviewer's tags belong to: one attachment of one check of one archived layer.
211
+
212
+ #### `ArtifactTagEditSchema`
213
+
214
+ ```ts
215
+ export const ArtifactTagEditSchema = …
216
+ ```
217
+
218
+ The body of `POST /api/reports/artifact-tags`: tags to add to and to remove from one attachment. At least one tag overall.
219
+
220
+ #### `ArtifactTagEntrySchema`
221
+
222
+ ```ts
223
+ export const ArtifactTagEntrySchema = …
224
+ ```
225
+
226
+ One recorded edit of an attachment's reviewer tags. Entries are never changed or removed: they are the audit trail.
227
+
228
+ #### `ArtifactTagsSchema`
229
+
230
+ ```ts
231
+ export const ArtifactTagsSchema = …
232
+ ```
233
+
234
+ `GET /api/reports/artifact-tags`: every recorded edit, oldest first. Fold them with `foldArtifactTags`.
235
+
188
236
  #### `HelloEventSchema`
189
237
 
190
238
  ```ts
@@ -26,6 +26,7 @@ decided exactly as before ([the report contract](report-contract.md)). Evidence
26
26
  | `steps` (`parentId`, `offsetMs`, `durationMs`, `status`, `kind`) | check, attempt | Draws the check's timeline: nested steps, when each started, how long it took, which one failed. |
27
27
  | `attempts` | check | Every try of a retried check, with its own status, detail, steps and attachments. A check that passed after failing is flaky. |
28
28
  | `tags`, `labels` | check, layer | Filters and groups (`smoke`, `flaky`, an owner, a browser). |
29
+ | `tags` | attachment | Marks one recording or screenshot, for example `demo`: shown as a `#demo` chip, and searchable. See [Tagging artifacts and demos](#tagging-artifacts-and-demos). |
29
30
  | `annotations`, `links` | check, step, attempt, layer | Notes, warnings and issue links next to the result. |
30
31
  | `specPath` | check | The spec a check belongs to (a `.feature`, a test file), relative to the repository. Checks are grouped by it. |
31
32
  | `specs[]` (`path`, `domain`, `kind`, `title`) | layer | Names each spec and its group (`domain`), so a server can group checks by spec with a title even when no [spec set](integrating.md#tell-atlas-which-spec-a-check-belongs-to) has been published. |
@@ -34,6 +35,33 @@ decided exactly as before ([the report contract](report-contract.md)). Evidence
34
35
  Send `sha`, `branch` and `runUrl` on every layer too. They never change a result, but they tie results to a commit
35
36
  and to the CI run that produced them.
36
37
 
38
+ ## Tagging artifacts and demos
39
+
40
+ To mark one recording as a demo, tag the attachment itself. A tag on a check says something about the result; a tag on
41
+ an attachment says something about that file, and no Gherkin tag, label, environment or producer name is needed:
42
+
43
+ ```ts
44
+ { name: 'checkout.webm', role: 'video', tags: ['demo'], labels: { audience: 'business' } }
45
+ ```
46
+
47
+ The viewer shows it as a `#demo` chip on the artifact card. Selecting the chip filters the result's evidence to the
48
+ attachments with that tag, and says how many match. Tags describe evidence; they never change a status or a slot.
49
+
50
+ - **One spelling.** A tag is normalized before it is compared or stored: a leading `#` is dropped, letters are
51
+ lower-cased, each run of whitespace becomes `-`, and surrounding space is trimmed, so `demo`, `#demo` and `#Demo ` are
52
+ `demo`. Empty tags and tags over 64 characters are not tags and are ignored (or refused, when a reviewer enters them).
53
+ Everything else is a valid tag (`team/payments`, `release=2`), and repeats are shown once.
54
+ - **Inherited.** An attachment is also searchable by the tags of its check and its run. The viewer says `inherited` on
55
+ such a chip, and a switch decides whether the filter counts them: a recording without a tag of its own under a run
56
+ tagged `demo` is found by the inherited query and not by the exact one. In the API,
57
+ `GET /api/reports/tests?artifactTag=demo` is exact and adding `&inherited=1` includes inherited tags.
58
+ - **Reviewers.** After upload, a person with a publish token can add tags to an attachment (the viewer's *Add tag*
59
+ button) and remove the ones a reviewer added. Those tags are not written into the layer, which is immutable: the
60
+ server keeps them as an audit trail (who, when, which tags) and the viewer shows them as `added by <name>`, apart
61
+ from the producer's tags. See [Tagging artifacts](server.md#tagging-artifacts). A producer's own tags stay as uploaded.
62
+ - **Older layers.** A layer without attachment tags reads exactly as before. Tags on an attachment that cannot be read
63
+ (not a list of strings) are dropped, and the attachment, check and layer stay.
64
+
37
65
  ## Labels describe, parameters split
38
66
 
39
67
  A result's **slot** is its producer, check id, part kind, spec, environment and **parameters**. The newest completed
package/docs/server.md CHANGED
@@ -216,6 +216,26 @@ against a server unchanged. A client uploads **in dependency order**: blobs, the
216
216
  entry that makes it visible. The server checks each step and says no to a catalog entry whose manifest is not
217
217
  there (`409`), so nothing half-uploaded is ever shown.
218
218
 
219
+ ## Tagging artifacts
220
+
221
+ A reviewer can mark a recording `#demo` after it was uploaded without uploading it again. The layer is immutable, so the
222
+ tag is kept beside it: each edit is one small object under `curation/artifact-tags/` in the write backend (or the live
223
+ store of a relay), named by time, never changed or removed. It records the attachment (layer, check, and the attachment's
224
+ `id`, else its bytes' key, else its name), the normalized tags added and removed, the name of the token that made the
225
+ edit, and the server's time. Nothing about the layer, its results or the report's winners changes.
226
+
227
+ - `GET /api/reports/artifact-tags` (read) returns every edit, oldest first. Fold them to get the tags that stand now: the
228
+ last edit of a tag wins.
229
+ - `POST /api/reports/artifact-tags` (publish, `Authorization: Bearer`) records an edit. A read token, an anonymous reader
230
+ and the browser's sign-in box are refused (`403`/`401`) like any upload, so a read-only visitor can see the tags and
231
+ not change them. The viewer asks for a token when a reviewer adds a tag and keeps it only in that tab.
232
+ - Producer tags (`tags` on the attachment) and reviewer tags are kept apart everywhere: the viewer says who added a
233
+ reviewer tag and when, and a reviewer can remove only reviewer tags. `capabilities.artifactTags` on `/api/server` says
234
+ a server has this.
235
+
236
+ See [Tagging artifacts and demos](rich-evidence.md#tagging-artifacts-and-demos) for the tag rules. Reviewer tags are
237
+ shown in the viewer's evidence; the tests API (`artifactTag`) searches the producers' tags.
238
+
219
239
  ## Streaming
220
240
 
221
241
  `GET /api/events` is a server-sent-events stream:
@@ -256,7 +276,7 @@ with `format=markdown` (or `Accept: text/markdown`), and takes the project prefi
256
276
  | Only some: `q` (all words), `status` (or `flaky`), `environment`, `producer`, `domain`, `spec`, `tag`, `run`; `sort`, `offset`, `limit` | `GET /api/reports/tests?status=failed&environment=ios` |
257
277
  | One test: every recorded result (status, duration, environment, branch, commit, CI run), newest first | `GET /api/reports/tests/history?key=<key>` |
258
278
  | Why Specs, domains and counts disagree: ambiguous spec paths, shared domains, gaps, a reconciliation of the totals | `GET /api/reports/tests/health` |
259
- | The runs that recorded the report (by CI run URL, else branch and commit) | `GET /api/reports/runs` |
279
+ | The runs that recorded the report (by CI run URL, else branch and commit); `?video=true` keeps those with a playable video | `GET /api/reports/runs` |
260
280
  | Every test one run recorded | `GET /api/reports/runs/detail?key=<run key>` |
261
281
 
262
282
  ```sh
@@ -330,6 +350,10 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
330
350
  | `GET /api/projects/{project}/reports/tests/health` | read | Why specs, domains and counts disagree. |
331
351
  | `GET /api/reports/tests/history` | read | One test and every result recorded for it. |
332
352
  | `GET /api/projects/{project}/reports/tests/history` | read | One test and every result recorded for it. |
353
+ | `GET /api/reports/artifact-tags` | read | Reviewer tags of attachments: every recorded edit. |
354
+ | `GET /api/projects/{project}/reports/artifact-tags` | read | Reviewer tags of attachments: every recorded edit. |
355
+ | `POST /api/reports/artifact-tags` | publish | Tag or untag one attachment after upload. |
356
+ | `POST /api/projects/{project}/reports/artifact-tags` | publish | Tag or untag one attachment after upload. |
333
357
  | `GET /api/reports/runs` | read | The runs that recorded the report. |
334
358
  | `GET /api/projects/{project}/reports/runs` | read | The runs that recorded the report. |
335
359
  | `GET /api/reports/runs/detail` | read | Every test one run recorded. |