@crvouga/atlas 1.10.0 → 1.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. package/CHANGELOG.md +15 -1
  2. package/dist/reports/src/review-collection-store.d.ts +47 -0
  3. package/dist/reports/src/review-collection-store.js +214 -0
  4. package/dist/reports/src/review-collection.d.ts +88 -0
  5. package/dist/reports/src/review-collection.js +144 -0
  6. package/dist/reports/src/test-catalog.d.ts +2 -0
  7. package/dist/reports/src/test-catalog.js +1 -1
  8. package/dist/schema/src/api.d.ts +574 -0
  9. package/dist/schema/src/api.js +152 -1
  10. package/dist/server/src/app.js +4 -1
  11. package/dist/server/src/openapi.js +46 -1
  12. package/dist/server/src/review-collections.d.ts +39 -0
  13. package/dist/server/src/review-collections.js +179 -0
  14. package/docs/extending.md +8 -1
  15. package/docs/reference/http-api.md +122 -0
  16. package/docs/reference/reports.md +294 -0
  17. package/docs/reference/schema.md +131 -0
  18. package/docs/reference/server.md +44 -0
  19. package/docs/rich-evidence.md +77 -0
  20. package/docs/server.md +41 -0
  21. package/docs/ui.md +67 -0
  22. package/llms-full.txt +315 -1
  23. package/package.json +1 -1
  24. package/packages/schema/schemas/review-collection.schema.json +398 -0
  25. package/packages/schema/schemas/review-manifest.schema.json +475 -0
  26. package/packages/server/openapi.json +6729 -4084
  27. package/packages/visualizer/src/app/router.tsx +9 -1
  28. package/packages/visualizer/src/evidence/applicability-view.tsx +233 -0
  29. package/packages/visualizer/src/evidence/applicability.test.ts +131 -0
  30. package/packages/visualizer/src/evidence/applicability.ts +241 -0
  31. package/packages/visualizer/src/evidence/artifact-tags.tsx +2 -1
  32. package/packages/visualizer/src/evidence/artifacts.tsx +88 -42
  33. package/packages/visualizer/src/evidence/behavior-panel.tsx +55 -26
  34. package/packages/visualizer/src/evidence/feature-page.tsx +25 -4
  35. package/packages/visualizer/src/evidence/media-timing.test.ts +85 -0
  36. package/packages/visualizer/src/evidence/media-timing.ts +150 -0
  37. package/packages/visualizer/src/evidence/moments.tsx +457 -0
  38. package/packages/visualizer/src/evidence/outcome-summary.tsx +27 -0
  39. package/packages/visualizer/src/evidence/outcome.ts +7 -2
  40. package/packages/visualizer/src/explorer/ArtifactsView.tsx +19 -5
  41. package/packages/visualizer/src/explorer/Explorer.tsx +7 -2
  42. package/packages/visualizer/src/explorer/RecordingTextSearch.tsx +355 -0
  43. package/packages/visualizer/src/explorer/TestDetail.tsx +2 -2
  44. package/packages/visualizer/src/explorer/collection-model.test.ts +31 -0
  45. package/packages/visualizer/src/explorer/collection-model.ts +45 -0
  46. package/packages/visualizer/src/explorer/collections.tsx +513 -0
  47. package/packages/visualizer/src/explorer/playlist-entries.ts +29 -0
  48. package/packages/visualizer/src/explorer/playlist.tsx +164 -37
  49. package/packages/visualizer/src/explorer/recording-text.test.ts +149 -0
  50. package/packages/visualizer/src/explorer/recording-text.ts +490 -0
  51. package/packages/visualizer/src/explorer/search.ts +17 -3
package/llms-full.txt CHANGED
@@ -1014,6 +1014,7 @@ decided exactly as before ([the report contract](report-contract.md)). Evidence
1014
1014
  | `attempts` | check | Every try of a retried check, with its own status, detail, steps and attachments. A check that passed after failing is flaky. |
1015
1015
  | `tags`, `labels` | check, layer | Filters and groups (`smoke`, `flaky`, an owner, a browser). |
1016
1016
  | `tags` | attachment | Marks one recording or screenshot, for example `demo`: shown as a `#demo` chip, and searchable. See [Tagging artifacts and demos](#tagging-artifacts-and-demos). |
1017
+ | `extensions['atlas.media']` | attachment (video) | Where the check's steps are in the recording, so a click on a step seeks the video. See [Place steps in a recording](#place-steps-in-a-recording). |
1017
1018
  | `scope` | attachment, attempt, check, part, layer | What a recording does and does not show: where it ran, whether data and dependencies were real, whether audio was captured. See [Evidence scope](#evidence-scope). |
1018
1019
  | `annotations`, `links` | check, step, attempt, layer | Notes, warnings and issue links next to the result. |
1019
1020
  | `specPath` | check | The spec a check belongs to (a `.feature`, a test file), relative to the repository. Checks are grouped by it. |
@@ -1096,6 +1097,44 @@ Every field is optional, and every value except the booleans is an open string.
1096
1097
  - **It decides nothing.** A scope never changes a status, a slot or a winner. A scope an older Atlas cannot read is
1097
1098
  dropped, and the attachment, check and layer stay.
1098
1099
 
1100
+ ## Place steps in a recording
1101
+
1102
+ A step says when it started on the check's clock (`offsetMs`, after the check started). A video has its own clock, which
1103
+ began when the recorder did, and the two are not assumed to agree: a recorder usually starts before the first step, and a
1104
+ paced demo may have delays put into the video that the check never waited for. To let a reviewer click a step and land on
1105
+ its moment in the video, say how the clocks relate in the `atlas.media` extension of the video attachment:
1106
+
1107
+ ```ts
1108
+ {
1109
+ name: 'checkout.webm', role: 'video', durationMs: 9000,
1110
+ extensions: {
1111
+ 'atlas.media': {
1112
+ checkStartMs: 1200, // the check started 1.2 s into the video
1113
+ pauses: [{ atMs: 900, durationMs: 2000, reason: 'title card' }], // 2 s put into the video before the step at +900 ms
1114
+ marks: [{ step: 'pay', mediaMs: 4100 }] // a step whose moment in the video you know exactly
1115
+ }
1116
+ }
1117
+ }
1118
+ ```
1119
+
1120
+ Every field is optional and none of it decides a status, a slot or a winner. A step is placed at `checkStartMs` + its
1121
+ `offsetMs` + the `durationMs` of every pause whose `atMs` is at or before the step; a mark names the moment outright and
1122
+ wins. Anything else is unavailable and said so, never guessed.
1123
+
1124
+ - **Without timing, nothing is placed.** A video with no `atlas.media`, no `checkStartMs`, or a step with no `offsetMs` and
1125
+ no mark, is never assumed to start with the check, and a step is never placed from the wording of its name. Selecting
1126
+ such a step says why it cannot be shown, and moves nothing.
1127
+ - **Outside the video is not clamped.** A moment before the start, or after `durationMs` (or the length the player reports),
1128
+ is unavailable. A declaration with a field Atlas cannot read (a pause without a duration) is reported and places nothing.
1129
+ - **Pacing is not latency.** A video with `pauses`, or with a `scope` of `pacing: 'paced'`, is labelled as paced: the time
1130
+ between its steps includes deliberate pauses and is not application latency, and no measurement is changed by it.
1131
+ - **Each try has its own clock.** Put the video and the `atlas.media` of a retry on its `attempts[]` entry, with that
1132
+ try's `steps`. A try is aligned only by its own video, never by the check's or an earlier try's.
1133
+ - **Link to a moment.** The viewer copies a link that names the run (`layer`), the exact recording (`clip`, its digest or
1134
+ stored key) and the position (`clipAt`, milliseconds into the video). Opening it selects that recording of that run, even
1135
+ when a newer run exists, and goes to the position; a recording the run does not hold is reported, never replaced.
1136
+ - **Older layers.** A layer without `atlas.media` reads exactly as before; the video plays and its steps list as ever.
1137
+
1099
1138
  ## Product latency
1100
1139
 
1101
1140
  A test's `durationMs` includes fixture setup, deliberate waits, demo pacing, retries and cleanup, so it says how long the
@@ -1170,6 +1209,44 @@ curl "$ATLAS_SERVER_URL/api/reports/artifacts?artifactTag=demo&type=video&domain
1170
1209
  **live query**: opening it asks again and shows what matches the newest report. It is not a snapshot; an immutable
1171
1210
  review collection is a separate feature. *Copy link* gives the address, which holds the whole query.
1172
1211
 
1212
+ ## Words in recordings
1213
+
1214
+ A reviewer who remembers a phrase that was spoken or shown in a video can search for it on the **Artifacts** tab
1215
+ (**Search inside recordings**, the `inside` address parameter). Atlas never transcribes video or reads a screen: it
1216
+ searches text you attach, and nothing is read from a recording until words are typed. Attach it beside the video, at the
1217
+ same level (the check, or the retry attempt that recorded the video):
1218
+
1219
+ ```ts
1220
+ attachments: [
1221
+ { id: 'video', name: 'checkout.webm', contentType: 'video/webm', data: video },
1222
+ // Spoken words: WebVTT, SubRip, JSON segments or plain text.
1223
+ { name: 'checkout.vtt', role: 'transcript', contentType: 'text/vtt', data: transcript,
1224
+ extensions: { 'atlas.text': { recording: 'video', generated: true, generator: 'my-speech-tool' } } },
1225
+ // Text shown on screen, with its own timing.
1226
+ { name: 'screen.json', role: 'ocr', contentType: 'application/json', data: JSON.stringify({ segments: [{ start: 2, end: 4, text: 'Order total', confidence: 0.9 }] }) }
1227
+ ]
1228
+ ```
1229
+
1230
+ - **Which attachment is text.** One with `extensions["atlas.text"]`, a `role` of `transcript`, `ocr`, `screen-text`,
1231
+ `captions` or `subtitles`, or a WebVTT or SubRip type. `extensions["atlas.text"].kind` (`speech`, `screen`, `caption`)
1232
+ says what it is; otherwise the role does. It belongs to the video that lists its `id` in `captions`, or that
1233
+ `extensions["atlas.text"].recording` names by `id`, else to the only video at the same level.
1234
+ - **Segments.** WebVTT and SubRip cues; JSON as `{ "segments": [...] }` (or `cues`, `results`, `lines`, or a bare list) of
1235
+ `{ text, start, end }` in seconds or `{ text, startMs, endMs }`, with an optional `confidence` between 0 and 1. A
1236
+ segment whose time is missing or invalid is still searched and is shown with no time; the viewer offers the recording
1237
+ and says seeking is unavailable, and never guesses a moment.
1238
+ - **Honesty about the text.** `generated: true` (with `generator`) marks text a tool produced; results show it with its
1239
+ confidence and say it may contain errors. A result always shows the indexed words, never a reworded quotation. A
1240
+ match finds a moment; it is not a new assertion, and the result's own outcome and the retry attempt stay beside it.
1241
+ - **States are different things.** Searched with no match; not indexed (no text was supplied); unsupported (the video is
1242
+ silent by its `scope.audioCaptured: false`, or the text is in a form that is not read, or sits at a `uri`); failed
1243
+ (`extensions["atlas.text"].state: "failed"` with a `reason`, a file left out with `omitted`, or text that did not
1244
+ parse). The ones not searched are listed apart and never counted as "no match".
1245
+ - **Access.** The text is read through the same report API, and so the same access rules, that play the recording. A file
1246
+ the viewer may not read is not indexed and appears in no snippet or count. The index exists only in the open page: there
1247
+ is no stored or cross-project index. Sending evidence to a transcription or OCR service is the producer's explicit
1248
+ decision and happens outside Atlas.
1249
+
1173
1250
  ## Labels describe, parameters split
1174
1251
 
1175
1252
  A result's **slot** is its producer, check id, part kind, spec, environment and **parameters**. The newest completed
@@ -1724,6 +1801,37 @@ edit, and the server's time. Nothing about the layer, its results or the report'
1724
1801
  See [Tagging artifacts and demos](rich-evidence.md#tagging-artifacts-and-demos) for the tag rules. Reviewer tags are
1725
1802
  shown in the viewer's evidence; the tests API (`artifactTag`) searches the producers' tags.
1726
1803
 
1804
+ ## Review collections
1805
+
1806
+ A team can publish a named, curated review of a few behaviors: recordings from several explicit executions, notes on
1807
+ what each is for, and the gaps nobody recorded yet. A **review collection** is kept beside the report under
1808
+ `curation/review-collections/` (never as a layer, a check or a run), so it cannot pass or fail anything, decides no
1809
+ slot or winner, and never claims its entries are one execution: every entry keeps the layer, run, commit, environment and
1810
+ status it was recorded with.
1811
+
1812
+ - **Freeze.** `POST /api/reports/review-collections` (publish) takes a title, optional `purpose`, `audience` and `tags`
1813
+ (such as `demo`), the ordered `entries` (`layer` and `check` from `GET /api/reports/artifacts`, optionally `kind`
1814
+ and one `recording`, and a `caption`) and optional `gaps`. The viewer's **Freeze as review collection** button on the
1815
+ Artifacts tab sends the filtered set as it is listed. The server resolves each entry to the result it holds and
1816
+ records what the report says about it; an entry it cannot resolve refuses the whole request (`422`, naming the entry).
1817
+ The order is the curator's: a failed, retried or incomplete result stays beside a later passing replay, each with its
1818
+ own status, time and tries.
1819
+ - **Immutable.** A revision is never changed or deleted by the server. Later runs and changed tags do not alter it. To
1820
+ change a collection, publish a new revision with `basedOn: { id, revision }` naming the newest (`409` otherwise); older
1821
+ revisions stay readable (`?revision=`).
1822
+ - **Notes and marks** (`POST .../review-collections/reviews`, publish) are separate immutable records with the token's name
1823
+ and the time: a `note`, a `gap` note (optionally at a `timecodeMs` of the entry's recording), `reviewed` and `unreviewed`.
1824
+ They are curation: they change no outcome, layer or winner, and a reviewed mark is not an acceptance sign-off.
1825
+ - **When a source goes away** (`GET .../review-collections/detail`) the entry stays where it was with its frozen snapshot
1826
+ and an `availability` of `unavailable` (its layer is gone, for example collected) or `unverified` (held, but the report
1827
+ no longer lists it), with the reason. Nothing is removed or replaced.
1828
+ - **Sharing** adds no access. The share URL is the viewer's `?tab=collections&collection=<id>`; reading needs the same
1829
+ read access as the report, and a visitor can Read, Watch and Inspect only what the attachment routes already allow them.
1830
+ Nothing is public by default.
1831
+ - **Export.** `GET .../review-collections/export` is a portable manifest: the exact sources, each file with its recorded
1832
+ name, a safe name and path, its `sha256` when the report holds one (`null` is unknown, not verified), and every missing
1833
+ source or file. It holds no bytes. `capabilities.reviewCollections` on `/api/server` says a server has all this.
1834
+
1727
1835
  ## Streaming
1728
1836
 
1729
1837
  `GET /api/events` is a server-sent-events stream:
@@ -1864,6 +1972,16 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
1864
1972
  | `GET /api/projects/{project}/reports/artifact-tags` | read | Reviewer tags of attachments: every recorded edit. |
1865
1973
  | `POST /api/reports/artifact-tags` | publish | Tag or untag one attachment after upload. |
1866
1974
  | `POST /api/projects/{project}/reports/artifact-tags` | publish | Tag or untag one attachment after upload. |
1975
+ | `GET /api/reports/review-collections` | read | Review collections: curated sets of recordings, newest first. |
1976
+ | `GET /api/projects/{project}/reports/review-collections` | read | Review collections: curated sets of recordings, newest first. |
1977
+ | `POST /api/reports/review-collections` | publish | Publish an immutable review collection (or a new revision of one). |
1978
+ | `POST /api/projects/{project}/reports/review-collections` | publish | Publish an immutable review collection (or a new revision of one). |
1979
+ | `GET /api/reports/review-collections/detail` | read | One review collection revision with how its sources stand now. |
1980
+ | `GET /api/projects/{project}/reports/review-collections/detail` | read | One review collection revision with how its sources stand now. |
1981
+ | `POST /api/reports/review-collections/reviews` | publish | Add a note, a time-coded gap note or a reviewed mark to a collection entry. |
1982
+ | `POST /api/projects/{project}/reports/review-collections/reviews` | publish | Add a note, a time-coded gap note or a reviewed mark to a collection entry. |
1983
+ | `GET /api/reports/review-collections/export` | read | Portable index (manifest) of one review collection revision. |
1984
+ | `GET /api/projects/{project}/reports/review-collections/export` | read | Portable index (manifest) of one review collection revision. |
1867
1985
  | `GET /api/reports/runs` | read | The runs that recorded the report. |
1868
1986
  | `GET /api/projects/{project}/reports/runs` | read | The runs that recorded the report. |
1869
1987
  | `GET /api/reports/artifacts` | read | Recorded results that carry evidence, across the whole history. |
@@ -2887,7 +3005,7 @@ carry any of these. Every one is optional.
2887
3005
  | `annotations[]` | Notes about the result: a warning, a deprecation, an accessibility finding, a coverage figure. |
2888
3006
  | `attempts[]` | Every try, when the check was retried. |
2889
3007
  | `tags[]` | Free labels to filter by: `smoke`, `payments`. |
2890
- | `labels{}` | Facts about the result that are not its identity: `owner`, `severity`, `browser`, `build`. |
3008
+ | `labels{}` | Facts about the result that are not its identity: `owner`, `severity`, `browser`, `build`. The viewer reads `build`, `platform`, `applies-to` and `applies-to-by` to tell whether evidence applies to a release (see [the viewer conventions](ui.md#behaviors-read-watch-inspect)). |
2891
3009
  | `scope{}` | What the evidence does and does not show: surface, device kind, synthetic data, scripted dependencies, audio, pacing, what was asserted and its limits. Unknown when absent. See [Evidence scope](rich-evidence.md#evidence-scope). |
2892
3010
  | `links[]` | Where to read more: the CI run, the issue, the trace client. |
2893
3011
  | `extensions{}` | Structured data of your own or a tool's, under a name you own. |
@@ -2978,6 +3096,13 @@ when you do not know, and the viewer says the cause is unknown. It is never infe
2978
3096
  failed before the behavior started comes from the failed step's `kind`: `hook`, `setup`, `fixture`, `before`,
2979
3097
  `beforeAll`, `beforeEach` or `given` mean preparation; `when`, `then`, `assertion` or `expect` mean the behavior.
2980
3098
 
3099
+ ### Placing steps in a recording
3100
+
3101
+ A video may declare how its clock relates to the check's under the `atlas.media` extension (`checkStartMs`, `pauses`,
3102
+ `marks`), so the viewer can seek it from a step and mark the step playing. Without it the video plays as ever and no step
3103
+ is placed. The fields, the rules and the link to a moment are in
3104
+ [Place steps in a recording](rich-evidence.md#place-steps-in-a-recording).
3105
+
2981
3106
  ## Attach evidence
2982
3107
 
2983
3108
  ### From the command line
@@ -3992,6 +4117,15 @@ when Watch is on screen.
3992
4117
  - The promise's revision (the spec's fingerprint now) and the evidence's revision (the fingerprint the layer recorded)
3993
4118
  are shown separately. A known difference demotes a recording to "Recording is older" with an alert; a missing
3994
4119
  fingerprint is "not recorded", never a match.
4120
+ - Whether the evidence applies is a separate reading (`evidence/applicability.ts`), shown on Watch and Inspect and as a
4121
+ row badge when it is stale (or unknown against a named release): `applies`, `stale` (an earlier promise or another
4122
+ build), `unknown`, or `none`. It never changes a status or a coverage state. Age is when the checks ran
4123
+ (`recordedAtMs`), never when the layer was uploaded or annotated. A release is named in the address (`release` build,
4124
+ `releaseSha`, `releasePlatform`); a branch name or a passing run never supplies one. Another platform, a missing
4125
+ build or commit, or a checkout with local changes (`freshness`) is `unknown`, never an exact match. Producers say
4126
+ what they know with labels: `build`, `platform`, and `applies-to` with `applies-to-by` (the policy that asserts the
4127
+ evidence still holds for those builds; without it the assertion is refused). The "Stale or unverified" filter
4128
+ (`evidence=stale`) lists every behavior whose evidence does not apply, including those with none.
3995
4129
  - A recording is always labelled illustrative: it shows one run and does not approve a behavior for production.
3996
4130
  `demo`/`representative` tags only add a "Representative demo" label.
3997
4131
  - The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
@@ -4051,6 +4185,64 @@ over the same filtered set the page lists, in the same order (`explorer/playlist
4051
4185
  (paced, scripted, silent, synthetic) and the sentence that the length is the recording's, not production latency. Developer
4052
4186
  details opens the test pinned to that run's record and closes the playlist.
4053
4187
 
4188
+ ## Steps and moments in a recording
4189
+
4190
+ Wherever a result's steps sit beside its recording (a result's Timeline under Inspect, the Watch tab of a behavior, an
4191
+ opened watch playlist), `evidence/moments.tsx` joins them. Placement is from the recording's own declared timing
4192
+ (`evidence/media-timing.ts`, the `atlas.media` extension; see [Place steps in a recording](rich-evidence.md#place-steps-in-a-recording)),
4193
+ never from an assumption that the recording starts with the check.
4194
+
4195
+ - A step has a "Recording m:ss.s" button that seeks the selected recording; the step's test-clock offset stays visible beside it.
4196
+ A step that cannot be placed (no timing, no origin, no offset, outside the recording, unreadable declaration) is "Not aligned"
4197
+ and selecting it states the reason in a status alert. Nothing is guessed and the video is not moved.
4198
+ - While playing, the step under the playhead gets `aria-current="step"` and "Playing now". It never moves focus, scrolls
4199
+ the page or announces itself. Seeking only scrolls the player into view.
4200
+ - A paced recording (pauses declared, or a `pacing: paced` scope) says that gaps between steps include deliberate pauses and are
4201
+ not application latency.
4202
+ - A failed step has "Jump to the failing moment" in the bar above the timeline, and the failure summary has the same action
4203
+ (`requestFailureMoment`). If the recording is not on screen, or the step cannot be placed, it says so.
4204
+ - "Copy link to this moment" names the run (`layer`), the exact recording (`clip`) and the position (`clipAt`, ms), as the
4205
+ explorer address `/?test=…&layer=…&clip=…&clipAt=…` (the playlist links to the result it is playing). Opening it selects that
4206
+ recording of that exact run and seeks once the player knows its length; a position past the end or a recording that the run
4207
+ does not hold is reported and nothing else is played in its place. If the browser refuses to copy, the link is shown to copy.
4208
+ - One provider serves one holder: the check, or one try. A try's steps are placed by that try's recording and timing only.
4209
+
4210
+ ## Review collections
4211
+
4212
+ The **Collections** tab (`explorer/collections.tsx`, `?tab=collections&collection=<id>&rev=<n>`) lists and opens review
4213
+ collections, and the Artifacts tab's **Freeze as review collection** button (a Mantine `Modal`) freezes the filtered set as it is
4214
+ listed. A collection is curation, not a test run (`@crvouga/atlas/reports/review-collection`, `docs/server.md#review-collections`).
4215
+
4216
+ - The page never shows one status for the whole collection. Every entry shows the status it was **recorded** with ("Failed as
4217
+ recorded"), its time, run, commit, layer and tries in order, so entries from several runs are never read as one execution; the
4218
+ note at the top says how many runs they come from. The order is the curator's: a failed or incomplete take stays beside a
4219
+ later passing replay and is never collapsed to the newest pass.
4220
+ - A revision is frozen. Opening an older revision says a newer one exists; a change is a new revision, never an edit.
4221
+ - An entry whose source is gone (`unavailable`) or cannot be checked (`unverified`) stays in its place with the reason and
4222
+ what it was when frozen, and its recording says it is not available. Nothing is removed or replaced.
4223
+ - Notes, time-coded gap notes (`m:ss`, or seconds; a mistake is said, not guessed) and the **Reviewed** switch are separate
4224
+ records with who and when. They change no outcome and the page says a mark is not an approval. Writing needs a token that
4225
+ may publish, held only in this tab; a read-only visitor is refused with the server's reason and can still Read, Watch and
4226
+ Inspect. **Download manifest** is the portable index; the share link adds no access.
4227
+
4228
+ ## Search inside recordings
4229
+
4230
+ The Artifacts tab has a **Search inside recordings** section (`explorer/RecordingTextSearch.tsx`; words in `inside`). It
4231
+ applies the shared matcher (`find/`) to the transcript, caption and screen-text segments producers attached
4232
+ (`explorer/recording-text.ts`; format in [rich evidence](rich-evidence.md#words-in-recordings)), so phrases, close
4233
+ spellings, meaning and typed filters (`status:failed`) behave as in every search box.
4234
+
4235
+ - Nothing is read until words are typed; then each listed result's evidence and text files are read a few at a time
4236
+ through the report API (the same access rules as playing the recording). The video is never read. A file the viewer
4237
+ may not read is not indexed and is reported as not searched, never counted or quoted. There is no stored index.
4238
+ - A result shows the indexed words exactly as indexed, why it matched (exact, close spelling, meaning), the source (spoken,
4239
+ on screen, caption), generated text with its generator, confidence and a warning, the time range, the owning result and
4240
+ retry attempt, and the result's own outcome. A match is never described as the behavior passing.
4241
+ - "No match" is only said about searched recordings. Recordings with no text (`Not indexed`), silent or unsupported ones
4242
+ and unreadable text (`Failed`) are counted separately and listed under **Recordings that were not searched**.
4243
+ - A result opens the playlist on that result, that attempt's video and, when its time is usable and reachable, that time.
4244
+ With no usable time the recording opens at its start and a note says seeking is unavailable.
4245
+
4054
4246
  ## Icons and notation marks
4055
4247
 
4056
4248
  Use named imports from `@tabler/icons-react` for all UI glyphs. The icon package supplies SVG paths;
@@ -5349,6 +5541,128 @@ Scoped to one project. The same project may be selected on the unprefixed API us
5349
5541
 
5350
5542
  Responses: `201` The recorded entry.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `413` Request too large.; `422` Not a tag edit: a bad target, an empty or over-long tag, or nothing to change.
5351
5543
 
5544
+ ### Review collections: curated sets of recordings, newest first
5545
+
5546
+ `GET /api/reports/review-collections` · role: read
5547
+
5548
+ The newest revision of every review collection, in brief: id, revision and how many exist, title, purpose, audience, tags, how many entries and gaps it has, who published it and when. A collection is curation kept beside the report; it is not a layer, a check or a run, and it has no status. Open one with `/api/reports/review-collections/detail?id=`. Its share URL is the viewer's `?tab=collections&collection=<id>` and needs the same read access as the rest of the report.
5549
+
5550
+ - `project` (query): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5551
+
5552
+ Responses: `200` The collections.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.
5553
+
5554
+ ### Review collections: curated sets of recordings, newest first
5555
+
5556
+ `GET /api/projects/{project}/reports/review-collections` · role: read
5557
+
5558
+ The newest revision of every review collection, in brief: id, revision and how many exist, title, purpose, audience, tags, how many entries and gaps it has, who published it and when. A collection is curation kept beside the report; it is not a layer, a check or a run, and it has no status. Open one with `/api/reports/review-collections/detail?id=`. Its share URL is the viewer's `?tab=collections&collection=<id>` and needs the same read access as the rest of the report.
5559
+
5560
+ Scoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.
5561
+
5562
+ - `project` (path, required): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5563
+
5564
+ Responses: `200` The collections.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.
5565
+
5566
+ ### Publish an immutable review collection (or a new revision of one)
5567
+
5568
+ `POST /api/reports/review-collections` · role: publish
5569
+
5570
+ Freezes an ordered set of references to recorded results as one named collection, recorded under the name of the publish (or admin) token that sends it. Take each entry's `layer` and `check` (and `kind`) from `GET /api/reports/artifacts`, and optionally one of its recordings (`videos[].recording`). The server resolves every entry to the result the report holds and records what the report says about it (status, time, run, commit, environment, tries, videos): the request does not claim them. An entry it cannot resolve refuses the whole request (`422`, naming the entry). The order is yours and is never re-sorted; failed, rejected and incomplete results stay in it beside a later passing replay.
5571
+
5572
+ Entries from different runs are never one execution: each keeps its own layer, run and status. A collection has no status of its own, adds no check and cannot change any result or winner. A revision is immutable. To change a collection, publish a new revision with `basedOn: { id, revision }` naming the newest revision (`409` otherwise); the earlier ones stay readable. Notes and reviewed marks are separate (`/reviews`). A reader-only token, a browser sign-in box and an anonymous reader are refused; nothing is shared publicly by default.
5573
+
5574
+ - `project` (query): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5575
+
5576
+ Responses: `201` The recorded revision.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `404` `basedOn` names no collection.; `409` `basedOn.revision` is not the newest.; `413` Request too large.; `422` Not a collection, or an entry the report does not hold.; `503` The server is still reading its backends. Retry after the number of seconds in `Retry-After`.
5577
+
5578
+ ### Publish an immutable review collection (or a new revision of one)
5579
+
5580
+ `POST /api/projects/{project}/reports/review-collections` · role: publish
5581
+
5582
+ Freezes an ordered set of references to recorded results as one named collection, recorded under the name of the publish (or admin) token that sends it. Take each entry's `layer` and `check` (and `kind`) from `GET /api/reports/artifacts`, and optionally one of its recordings (`videos[].recording`). The server resolves every entry to the result the report holds and records what the report says about it (status, time, run, commit, environment, tries, videos): the request does not claim them. An entry it cannot resolve refuses the whole request (`422`, naming the entry). The order is yours and is never re-sorted; failed, rejected and incomplete results stay in it beside a later passing replay.
5583
+
5584
+ Entries from different runs are never one execution: each keeps its own layer, run and status. A collection has no status of its own, adds no check and cannot change any result or winner. A revision is immutable. To change a collection, publish a new revision with `basedOn: { id, revision }` naming the newest revision (`409` otherwise); the earlier ones stay readable. Notes and reviewed marks are separate (`/reviews`). A reader-only token, a browser sign-in box and an anonymous reader are refused; nothing is shared publicly by default.
5585
+
5586
+ Scoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.
5587
+
5588
+ - `project` (path, required): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5589
+
5590
+ Responses: `201` The recorded revision.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `404` `basedOn` names no collection.; `409` `basedOn.revision` is not the newest.; `413` Request too large.; `422` Not a collection, or an entry the report does not hold.; `503` The server is still reading its backends. Retry after the number of seconds in `Retry-After`.
5591
+
5592
+ ### One review collection revision with how its sources stand now
5593
+
5594
+ `GET /api/reports/review-collections/detail` · role: read
5595
+
5596
+ The revision (the newest without `revision`), every revision number, the review notes and reviewed marks of that revision, and for each entry `availability`: `available` (its layer is held and the report lists the result), `unverified` (the layer is held but the report no longer lists the result or recording) or `unavailable` (the layer is gone, for example deleted by garbage collection), with the reason. An entry whose source is not available stays in its place with its frozen snapshot; it is never removed or replaced. Entries keep the status each layer recorded, with their own times, in the order the curator chose. A reviewed mark is curation: it changes no outcome and is not an approval.
5597
+
5598
+ - `id` (query, required): A collection id from the list.
5599
+ - `revision` (query): A revision number. Default: the newest.
5600
+ - `project` (query): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5601
+
5602
+ Responses: `200` The revision.; `400` No `id`, or a bad `revision`.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `404` No such collection or revision.
5603
+
5604
+ ### One review collection revision with how its sources stand now
5605
+
5606
+ `GET /api/projects/{project}/reports/review-collections/detail` · role: read
5607
+
5608
+ The revision (the newest without `revision`), every revision number, the review notes and reviewed marks of that revision, and for each entry `availability`: `available` (its layer is held and the report lists the result), `unverified` (the layer is held but the report no longer lists the result or recording) or `unavailable` (the layer is gone, for example deleted by garbage collection), with the reason. An entry whose source is not available stays in its place with its frozen snapshot; it is never removed or replaced. Entries keep the status each layer recorded, with their own times, in the order the curator chose. A reviewed mark is curation: it changes no outcome and is not an approval.
5609
+
5610
+ Scoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.
5611
+
5612
+ - `project` (path, required): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5613
+ - `id` (query, required): A collection id from the list.
5614
+ - `revision` (query): A revision number. Default: the newest.
5615
+
5616
+ Responses: `200` The revision.; `400` No `id`, or a bad `revision`.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `404` No such collection or revision.
5617
+
5618
+ ### Add a note, a time-coded gap note or a reviewed mark to a collection entry
5619
+
5620
+ `POST /api/reports/review-collections/reviews` · role: publish
5621
+
5622
+ Records one immutable entry, by the name of the publish (or admin) token and the server's time: a `note` or a `gap` (text required; with `entry` and optionally `timecodeMs`, a moment in that entry's recording), or a `reviewed` / `unreviewed` mark for one entry. It changes no check outcome, no layer, no collection revision and no report winner; a mark is not an acceptance sign-off. The notes of a revision are in its `detail`.
5623
+
5624
+ - `project` (query): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5625
+
5626
+ Responses: `201` The recorded note.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `404` No such collection or revision.; `413` Request too large.; `422` Not a note: missing text, an unknown entry, or a mark without an entry.
5627
+
5628
+ ### Add a note, a time-coded gap note or a reviewed mark to a collection entry
5629
+
5630
+ `POST /api/projects/{project}/reports/review-collections/reviews` · role: publish
5631
+
5632
+ Records one immutable entry, by the name of the publish (or admin) token and the server's time: a `note` or a `gap` (text required; with `entry` and optionally `timecodeMs`, a moment in that entry's recording), or a `reviewed` / `unreviewed` mark for one entry. It changes no check outcome, no layer, no collection revision and no report winner; a mark is not an acceptance sign-off. The notes of a revision are in its `detail`.
5633
+
5634
+ Scoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.
5635
+
5636
+ - `project` (path, required): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5637
+
5638
+ Responses: `201` The recorded note.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `404` No such collection or revision.; `413` Request too large.; `422` Not a note: missing text, an unknown entry, or a mark without an entry.
5639
+
5640
+ ### Portable index (manifest) of one review collection revision
5641
+
5642
+ `GET /api/reports/review-collections/export` · role: read
5643
+
5644
+ A manifest an archive can be built from: the collection, each entry with its exact source (layer, check, kind, recording), the status and run it was recorded with, how it stands now, whether it is marked reviewed, and the files it references with their name as recorded, a safe name and path (`files/<position>-<safeName>`), and the recorded `sha256` when the report holds one (`null` is unknown, not verified). A source that cannot be read now is listed under `missing` and its files are marked `missing`; nothing is left out silently. The manifest contains no bytes; recordings are fetched from the attachment routes, which keep their own access control. Sent as an attachment.
5645
+
5646
+ - `id` (query, required): A collection id.
5647
+ - `revision` (query): A revision number. Default: the newest.
5648
+ - `project` (query): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5649
+
5650
+ Responses: `200` The manifest.; `400` No `id`, or a bad `revision`.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `404` No such collection or revision.
5651
+
5652
+ ### Portable index (manifest) of one review collection revision
5653
+
5654
+ `GET /api/projects/{project}/reports/review-collections/export` · role: read
5655
+
5656
+ A manifest an archive can be built from: the collection, each entry with its exact source (layer, check, kind, recording), the status and run it was recorded with, how it stands now, whether it is marked reviewed, and the files it references with their name as recorded, a safe name and path (`files/<position>-<safeName>`), and the recorded `sha256` when the report holds one (`null` is unknown, not verified). A source that cannot be read now is listed under `missing` and its files are marked `missing`; nothing is left out silently. The manifest contains no bytes; recordings are fetched from the attachment routes, which keep their own access control. Sent as an attachment.
5657
+
5658
+ Scoped to one project. The same project may be selected on the unprefixed API using ?project= or X-Atlas-Project. Unknown projects return 404 on reads and are created by an authenticated upload.
5659
+
5660
+ - `project` (path, required): Project slug. Alternatively send X-Atlas-Project or use the project-prefixed route. Omit for the default project. Conflicting selectors are refused.
5661
+ - `id` (query, required): A collection id.
5662
+ - `revision` (query): A revision number. Default: the newest.
5663
+
5664
+ Responses: `200` The manifest.; `400` No `id`, or a bad `revision`.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `404` No such collection or revision.
5665
+
5352
5666
  ### The runs that recorded the report
5353
5667
 
5354
5668
  `GET /api/reports/runs` · role: read
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@crvouga/atlas",
3
- "version": "1.10.0",
3
+ "version": "1.12.0",
4
4
  "description": "Atlas: specs of any kind, reported as layered evidence. Publish checks from any runner (JUnit, CTRF, a script), combine them across nodes, branches, environments and backends, gate CI on the result, and run statecharts against the real app.",
5
5
  "license": "MIT",
6
6
  "type": "module",