@crvouga/atlas 1.5.0 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,6 +1,18 @@
1
1
  # Changelog: @crvouga/atlas
2
2
 
3
- ## 1.5.0 (2026-10-29)
3
+ ## 1.7.0 (2026-10-29)
4
+
5
+ ### Features
6
+
7
+ - explain selected, aggregate and historical outcomes without contradictory summaries ([018d2fe](https://github.com/crvouga/atlas/commit/018d2fe91a9c17cbefa8f9d24ad6f3d70550c5d1))
8
+
9
+ ## 1.6.0 (2026-10-28)
10
+
11
+ ### Features
12
+
13
+ - unify fuzzy semantic search with complete filterable sortable results ([3b0b90e](https://github.com/crvouga/atlas/commit/3b0b90e4960dcbcbcfee924f33aca4ce02beaaec))
14
+
15
+ ## 1.5.0 (2026-10-28)
4
16
 
5
17
  ### Features
6
18
 
package/docs/extending.md CHANGED
@@ -104,6 +104,23 @@ every kind of evidence and values from the future) and `layer-extensibility.test
104
104
  fields survive at every level, malformed evidence is dropped without losing the result, and a reader of the
105
105
  first version of the format still reads it all.
106
106
 
107
+ ### Saying what failed
108
+
109
+ A failed check or step may declare the expectation it broke under the `atlas.failure` extension, so a reviewer sees
110
+ it before the raw log. Every field is optional, and the declaration never changes the result.
111
+
112
+ ```json
113
+ {
114
+ "name": "The list includes tea", "kind": "then", "status": "failed",
115
+ "extensions": { "atlas.failure": { "expected": "tea in the list", "actual": "the list is empty", "attribution": "product" } }
116
+ }
117
+ ```
118
+
119
+ `expected` and `actual` are shown as written. `attribution` is `product`, `test-harness` or `environment`; leave it out
120
+ when you do not know, and the viewer says the cause is unknown. It is never inferred from the message. Whether setup
121
+ failed before the behavior started comes from the failed step's `kind`: `hook`, `setup`, `fixture`, `before`,
122
+ `beforeAll`, `beforeEach` or `given` mean preparation; `when`, `then`, `assertion` or `expect` mean the behavior.
123
+
107
124
  ## Attach evidence
108
125
 
109
126
  ### From the command line
package/docs/ui.md CHANGED
@@ -87,6 +87,20 @@ when Watch is on screen.
87
87
  - The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
88
88
  none are listed under "No domain declared"; the Tests view reports domain problems).
89
89
 
90
+ ## Outcome summary
91
+
92
+ A test or behavior detail states what the **selected** result says apart from what the other current sources say
93
+ (`evidence/outcome.ts`, `outcome-summary.tsx`). The heading is the selected source's outcome first ("Selected result
94
+ passed; another scoped source failed"), never the combined status; the combined status is labelled as the worst current
95
+ result. Each other current source is one click away, and sources that share an environment are told apart by status,
96
+ time, revision and their place among them. A failure leads with whether the behavior was reached (decided only by the
97
+ recorded step `kind`: a `hook`, `setup`, `fixture`, `before*` or `given` step failing is "Behavior not verified: setup
98
+ failed", `when`, `then`, `assertion` or `expect` is "Expectation not met", anything else is unknown), then the expected
99
+ and observed values and the closest step and video, then the original output. The cause (product, test harness,
100
+ environment) is unknown unless the producer declared it under the `atlas.failure` extension; it is never read from error
101
+ wording. Records of one source whose outcome changed are listed oldest first, and a later pass is never called a fix.
102
+ In the explorer the chosen source is the `source` search parameter (a layer id; unlike `layer` it does not pin a run).
103
+
90
104
  ## Icons and notation marks
91
105
 
92
106
  Use named imports from `@tabler/icons-react` for all UI glyphs. The icon package supplies SVG paths;
@@ -101,6 +115,16 @@ visible notation label; marks are decorative. Sources, licenses and the DMN nota
101
115
  and update this mapping when adding a format. Brand artwork preserves upstream colors independently of
102
116
  Atlas's theme. SVG data plots and statechart arrow markers are geometry, not handcrafted UI icons.
103
117
 
118
+ ## Search
119
+
120
+ All search boxes (finder, test list, Runs tab, an opened run) use `src/find/query.ts` for the query language and
121
+ `src/find/rank.ts` for matching; do not add a second matcher or filter on `includes`. Build documents from
122
+ `src/find/behaviors.ts` so a behavior has the same fields and facets everywhere. Match first, then count, filter and sort
123
+ the whole set, then page. Show filters typed in the box as removable chips, show what was not understood
124
+ (`SearchNotes`), show why a result matched and mark exact ones, and say when matching is words only (no embedder, or
125
+ a failed one). A bounded list says it is bounded and offers all results. Selection follows result identity, never a
126
+ position, when results change. The query, filters and sort stay in the address.
127
+
104
128
  ## Accessibility and responsive behavior
105
129
 
106
130
  The shell collapses its index below Mantine's `md` breakpoint; its named Burger toggles mobile navigation.
@@ -151,7 +175,7 @@ The narrowly scoped exceptions are:
151
175
  | Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
152
176
  | Evidence | Steps, retries, annotations, facts, unknown extensions, safe downloads and previews | List, Badge, Accordion, Code, Image, native media |
153
177
  | Recordings | Pictures/video/timeline, actor selection, moments, failure jumps, timing, model-state differences | Tabs, Slider, Button, ScrollArea, Table, Progress |
154
- | Search/help | Ranked and semantic search, shortcut help, recording shortcuts | Spotlight, Modal, Kbd |
178
+ | Search/help | Ranked, filterable and semantic search across behaviors, runs and results; complete results; shortcut help, recording shortcuts | Spotlight, TextInput, NativeSelect, Button chips, Alert, Modal, Kbd |
155
179
  | Administration | Token access, S3 storage notice, garbage collection policy (saved in S3), live timeouts, report rules, limits, raw JSON, conflicts | Same provider/theme, Fieldset, Mantine inputs, Accordion, Alert |
156
180
 
157
181
  The hash routes (`/`, `/chart/$chartId`, `/specs/$kind`), query contracts, project scoping, streaming,
package/llms-full.txt CHANGED
@@ -2900,6 +2900,23 @@ every kind of evidence and values from the future) and `layer-extensibility.test
2900
2900
  fields survive at every level, malformed evidence is dropped without losing the result, and a reader of the
2901
2901
  first version of the format still reads it all.
2902
2902
 
2903
+ ### Saying what failed
2904
+
2905
+ A failed check or step may declare the expectation it broke under the `atlas.failure` extension, so a reviewer sees
2906
+ it before the raw log. Every field is optional, and the declaration never changes the result.
2907
+
2908
+ ```json
2909
+ {
2910
+ "name": "The list includes tea", "kind": "then", "status": "failed",
2911
+ "extensions": { "atlas.failure": { "expected": "tea in the list", "actual": "the list is empty", "attribution": "product" } }
2912
+ }
2913
+ ```
2914
+
2915
+ `expected` and `actual` are shown as written. `attribution` is `product`, `test-harness` or `environment`; leave it out
2916
+ when you do not know, and the viewer says the cause is unknown. It is never inferred from the message. Whether setup
2917
+ failed before the behavior started comes from the failed step's `kind`: `hook`, `setup`, `fixture`, `before`,
2918
+ `beforeAll`, `beforeEach` or `given` mean preparation; `when`, `then`, `assertion` or `expect` mean the behavior.
2919
+
2903
2920
  ## Attach evidence
2904
2921
 
2905
2922
  ### From the command line
@@ -3919,6 +3936,20 @@ when Watch is on screen.
3919
3936
  - The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
3920
3937
  none are listed under "No domain declared"; the Tests view reports domain problems).
3921
3938
 
3939
+ ## Outcome summary
3940
+
3941
+ A test or behavior detail states what the **selected** result says apart from what the other current sources say
3942
+ (`evidence/outcome.ts`, `outcome-summary.tsx`). The heading is the selected source's outcome first ("Selected result
3943
+ passed; another scoped source failed"), never the combined status; the combined status is labelled as the worst current
3944
+ result. Each other current source is one click away, and sources that share an environment are told apart by status,
3945
+ time, revision and their place among them. A failure leads with whether the behavior was reached (decided only by the
3946
+ recorded step `kind`: a `hook`, `setup`, `fixture`, `before*` or `given` step failing is "Behavior not verified: setup
3947
+ failed", `when`, `then`, `assertion` or `expect` is "Expectation not met", anything else is unknown), then the expected
3948
+ and observed values and the closest step and video, then the original output. The cause (product, test harness,
3949
+ environment) is unknown unless the producer declared it under the `atlas.failure` extension; it is never read from error
3950
+ wording. Records of one source whose outcome changed are listed oldest first, and a later pass is never called a fix.
3951
+ In the explorer the chosen source is the `source` search parameter (a layer id; unlike `layer` it does not pin a run).
3952
+
3922
3953
  ## Icons and notation marks
3923
3954
 
3924
3955
  Use named imports from `@tabler/icons-react` for all UI glyphs. The icon package supplies SVG paths;
@@ -3933,6 +3964,16 @@ visible notation label; marks are decorative. Sources, licenses and the DMN nota
3933
3964
  and update this mapping when adding a format. Brand artwork preserves upstream colors independently of
3934
3965
  Atlas's theme. SVG data plots and statechart arrow markers are geometry, not handcrafted UI icons.
3935
3966
 
3967
+ ## Search
3968
+
3969
+ All search boxes (finder, test list, Runs tab, an opened run) use `src/find/query.ts` for the query language and
3970
+ `src/find/rank.ts` for matching; do not add a second matcher or filter on `includes`. Build documents from
3971
+ `src/find/behaviors.ts` so a behavior has the same fields and facets everywhere. Match first, then count, filter and sort
3972
+ the whole set, then page. Show filters typed in the box as removable chips, show what was not understood
3973
+ (`SearchNotes`), show why a result matched and mark exact ones, and say when matching is words only (no embedder, or
3974
+ a failed one). A bounded list says it is bounded and offers all results. Selection follows result identity, never a
3975
+ position, when results change. The query, filters and sort stay in the address.
3976
+
3936
3977
  ## Accessibility and responsive behavior
3937
3978
 
3938
3979
  The shell collapses its index below Mantine's `md` breakpoint; its named Burger toggles mobile navigation.
@@ -3983,7 +4024,7 @@ The narrowly scoped exceptions are:
3983
4024
  | Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
3984
4025
  | Evidence | Steps, retries, annotations, facts, unknown extensions, safe downloads and previews | List, Badge, Accordion, Code, Image, native media |
3985
4026
  | Recordings | Pictures/video/timeline, actor selection, moments, failure jumps, timing, model-state differences | Tabs, Slider, Button, ScrollArea, Table, Progress |
3986
- | Search/help | Ranked and semantic search, shortcut help, recording shortcuts | Spotlight, Modal, Kbd |
4027
+ | Search/help | Ranked, filterable and semantic search across behaviors, runs and results; complete results; shortcut help, recording shortcuts | Spotlight, TextInput, NativeSelect, Button chips, Alert, Modal, Kbd |
3987
4028
  | Administration | Token access, S3 storage notice, garbage collection policy (saved in S3), live timeouts, report rules, limits, raw JSON, conflicts | Same provider/theme, Fieldset, Mantine inputs, Accordion, Alert |
3988
4029
 
3989
4030
  The hash routes (`/`, `/chart/$chartId`, `/specs/$kind`), query contracts, project scoping, streaming,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@crvouga/atlas",
3
- "version": "1.5.0",
3
+ "version": "1.7.0",
4
4
  "description": "Atlas: specs of any kind, reported as layered evidence. Publish checks from any runner (JUnit, CTRF, a script), combine them across nodes, branches, environments and backends, gate CI on the result, and run statecharts against the real app.",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -154,6 +154,34 @@ Components read only the view model (`AtlasView`), never raw files.
154
154
  its own worker. Positions are cached under a hash of the chart's structure only, so new results
155
155
  never move anything.
156
156
 
157
+ ## Searching behaviors, runs and recordings
158
+
159
+ Every search box (the finder opened with `Ctrl+K` or `/`, the test list, the Runs tab and the results of an opened
160
+ run) reads one query language and matches with one engine (`src/find/query.ts`, `src/find/rank.ts`), so the same words
161
+ make the same behaviors eligible everywhere.
162
+
163
+ - **Words** match titles, steps, descriptions, specs, tags, environments and failure messages by stem, everyday synonym
164
+ and close spelling (`savde answer` finds "Keep a saved answer"), and the start of a longer word while it is typed.
165
+ Every distinctive word must be found; a word present in most behaviors is optional. When an embedder is available the
166
+ results are also re-ranked by meaning and a paraphrase with no word in common is added (`Similar meaning`); when none
167
+ is, or it fails, the box says it is matching words and spellings only.
168
+ - **Exact matches.** `"a quoted phrase"` and identifiers with `/`, `_` or `.` (`memory/keep`) must appear as written, are
169
+ marked `Exact` and are listed first.
170
+ - **Filters** narrow by structure instead of matching words: `status:failed` (`passed`, `failed`, `flaky`, `not-run`,
171
+ `running`), `env:ios` (also `platform:`, `environment:`, `device:`), `producer:`, `domain:`, `spec:`, `tag:`, `branch:`
172
+ and `has:video` (a playable recording, not the word). Values of one field are alternatives (`status:failed,flaky`);
173
+ different fields all hold. Each shows as a removable chip, and the status, environment and Has video buttons are the
174
+ same filters without the syntax.
175
+ - **Nothing is dropped silently.** An unknown operator, an unknown status, an unclosed quote, or a filter a list does not
176
+ have (`domain:` on runs) is listed under the box with what to type instead.
177
+ - **Complete results.** Counts, filters and sorting (including Best match, offered with words to match) apply to the whole
178
+ dataset before anything is shown in pages. The finder shows its first 40 results, says so, and opens all of them
179
+ (`Shift+Enter`, or the explorer's list for the same query). The query, filters and sort are in the address, so a link
180
+ and the back button restore them. Results that change under the cursor keep the highlighted result highlighted by
181
+ identity, and say so when it is no longer listed.
182
+
183
+ The standing rules are in [the UI conventions](../../docs/ui.md#search).
184
+
157
185
  ## Fixtures
158
186
 
159
187
  `fixtures/` holds seven scenarios generated by `scripts/make-fixtures.ts` from `example/specs`, a
@@ -6,12 +6,9 @@
6
6
  * record: run, environment, source, attempts). Also the coverage summary of the whole feature. A recording is always
7
7
  * labelled as illustrative evidence with its own status, provenance and revision; it never stands for approval.
8
8
  */
9
- import type { EvidenceHolder } from '@crvouga/atlas/reports/evidence-detail';
10
- import { isVideoAttachment } from '@crvouga/atlas/reports/evidence-detail';
11
9
  import { effectiveScope } from '@crvouga/atlas/reports/evidence-scope';
12
10
  import type { EvidenceResult } from '@crvouga/atlas/reports/layer-merge';
13
11
  import { environmentLabel } from '@crvouga/atlas/reports/layer-axes';
14
- import type { Attachment } from '@crvouga/atlas/schema';
15
12
  import { Alert, Badge, Box, Group, List, Paper, Stack, Tabs, Text, Title } from '@mantine/core';
16
13
  import { useMemo, type ReactNode } from 'react';
17
14
 
@@ -23,6 +20,7 @@ import { ScopeNotice } from './evidence-scope';
23
20
  import { plural, statusWord, useNow } from './format';
24
21
  import { exampleIndexFor, exampleRows, fillExample, type FeatureStep } from './gherkin';
25
22
  import { Origin, StatusMark } from './marks';
23
+ import { outcomeHeadline, videosIn } from './outcome';
26
24
 
27
25
  /** The short, readable form of a spec fingerprint (`<length>:<hash>:<hash>`): its length and the start of its hash. */
28
26
  export function shortRevision(fingerprint: string): string {
@@ -80,19 +78,6 @@ function expectedSteps(steps: readonly FeatureStep[]): FeatureStep[] {
80
78
  return at < 0 ? [] : steps.slice(at);
81
79
  }
82
80
 
83
- function videosIn(holder: EvidenceHolder | undefined): Attachment[] {
84
- if (!holder) return [];
85
- const own = (holder.attachments ?? []).filter(isVideoAttachment);
86
- return [
87
- ...own,
88
- ...(holder.steps ?? []).flatMap((step) => (step.attachments ?? []).filter(isVideoAttachment)),
89
- ...(holder.attempts ?? []).flatMap((attempt) => [
90
- ...(attempt.attachments ?? []).filter(isVideoAttachment),
91
- ...(attempt.steps ?? []).flatMap((step) => (step.attachments ?? []).filter(isVideoAttachment))
92
- ])
93
- ];
94
- }
95
-
96
81
  /** The recording of one result, read only when this is on screen. */
97
82
  function Player({ result }: { result: EvidenceResult }) {
98
83
  const detail = useCheckDetail(result.layer, result.row);
@@ -171,6 +156,13 @@ export function BehaviorPanel({
171
156
  const steps = useMemo(() => (entry ? fillExample(scenario.steps, entry) : scenario.steps), [scenario, entry]);
172
157
  const expected = expectedSteps(steps);
173
158
  const named = example !== null && entry === null;
159
+ // The coverage state is the check's combined status; this says what the recording or result shown here recorded, apart from it.
160
+ const outcome =
161
+ subject && behavior.slot ? (
162
+ <Text component="div" fw={600} data-outcome-headline>
163
+ {outcomeHeadline({ layer: subject.layer, row: subject.row }, behavior.slot).text}
164
+ </Text>
165
+ ) : null;
174
166
  return (
175
167
  <Tabs
176
168
  defaultValue={start ?? (demo ? 'watch' : state === 'running' || behavior.slot?.live ? 'inspect' : 'read')}
@@ -190,6 +182,7 @@ export function BehaviorPanel({
190
182
  {COVERAGE[state].meaning}
191
183
  </Text>
192
184
  </Group>
185
+ {outcome}
193
186
  {example !== null && (
194
187
  <Text component="div" c="dimmed">
195
188
  Example: {example}
@@ -209,6 +202,7 @@ export function BehaviorPanel({
209
202
  <CoverageBadge state={state} />
210
203
  {behavior.representative && <Badge variant="outline">Representative demo</Badge>}
211
204
  </Group>
205
+ {outcome}
212
206
  {expected.length > 0 && (
213
207
  <Box component="section" aria-label="Expected outcome">
214
208
  <Title order={4} mb="xs">
@@ -9,12 +9,14 @@ import { useQuery } from '@tanstack/react-query';
9
9
  import { useMemo, useState, type ReactNode } from 'react';
10
10
  import { TestProgress, useTestProgress } from '../live/test-progress';
11
11
  import { reportJson } from './api';
12
- import { ResultEvidence } from './artifacts';
12
+ import { ResultEvidence, useCheckDetail } from './artifacts';
13
13
  import { useEvidence } from './evidence-context';
14
14
  import { ago, duration, plural, statusWord, useNow } from './format';
15
15
  import type { FeatureExamples, FeatureStep } from './gherkin';
16
16
  import { Bounded, Origin, StatusMark } from './marks';
17
17
  import { ModelTrace } from './model-trace';
18
+ import { OutcomeSummary } from './outcome-summary';
19
+ import { failedReportStep, failedStep, recordOf, sourceLabels, type ResultRecord } from './outcome';
18
20
  import { recordedModelStates } from './quint';
19
21
  /** What the address says is open: a check, and optionally one particular layer's record of it. */
20
22
  export type CheckSelection = {
@@ -49,13 +51,6 @@ function environmentMarks(slot: CheckSlot) {
49
51
  if (seen.size < 2) return [];
50
52
  return [...seen].map(([id, status]) => ({ id, status }));
51
53
  }
52
- function outcomeName(outcome: EvidenceResult) {
53
- const environment = outcome.layer.environment ? environmentLabel(outcome.layer.environment) : '';
54
- const parameters = Object.entries(outcome.parameters)
55
- .map(([name, value]) => `${name}: ${value}`)
56
- .join(', ');
57
- return [environment, parameters].filter(Boolean).join(' · ') || outcome.layer.producer;
58
- }
59
54
  export function ResultRow({
60
55
  label,
61
56
  slot,
@@ -93,7 +88,7 @@ export function ResultRow({
93
88
  <Flex component="span" display="inline-flex" gap="xs" align="center" wrap="wrap">
94
89
  {marks.map((mark) => (
95
90
  <Text key={mark.id || 'unspecified'} data-status={mark.status} span inherit>
96
- {environmentLabel(mark.id || null)}
91
+ {environmentLabel(mark.id || null)} {statusWord(mark.status).toLowerCase()}
97
92
  </Text>
98
93
  ))}
99
94
  </Flex>
@@ -276,6 +271,14 @@ type Props = {
276
271
  pinned?: HistoryEntry;
277
272
  /** Choosing a run in the history always names its layer, even when that result is a current one, so the choice stays pinned. */
278
273
  pin?: boolean;
274
+ /** The layer id of the current source the address selected (without pinning a run). Used with `onSource`; otherwise the choice is local. */
275
+ source?: string;
276
+ /** Called with the layer id when a source is chosen (null for the newest), so the page can keep the choice in its address. */
277
+ onSource?: (layerId: string | null) => void;
278
+ /** Whether the outcome summary states its own heading. False when the page already shows it. */
279
+ headline?: boolean;
280
+ /** Every record of this check the page knows (a node's history of it), when the layers held here are only the winners. */
281
+ records?: readonly ResultRecord[];
279
282
  };
280
283
  /** Everything recorded for one check: the chosen result, how it went, and every other run of it. */
281
284
  export function CheckDetail(props: Props) {
@@ -306,16 +309,28 @@ export function CheckDetail(props: Props) {
306
309
  </>
307
310
  );
308
311
  }
309
- function RecordedCheckDetail({ slot, layerId, select, reads, written, pinned, pin }: Props) {
312
+ function RecordedCheckDetail({ slot, layerId, select, reads, written, pinned, pin, source, onSource, headline = true, records }: Props) {
310
313
  const { evidence, backends, ready, pending } = useEvidence();
311
314
  const now = useNow();
312
315
  const [variant, setVariant] = useState(0);
313
316
  const history = useMemo(() => checkHistory(evidence, slot.specPath, slot.id), [evidence, slot.specPath, slot.id]);
314
- const winner = slot.outcomes[Math.min(variant, slot.outcomes.length - 1)]!;
317
+ const sourceAt = onSource && source ? slot.outcomes.findIndex((outcome) => outcome.layer.id === source) : -1;
318
+ const index = Math.min(onSource ? Math.max(sourceAt, 0) : variant, slot.outcomes.length - 1);
319
+ const winner = slot.outcomes[index]!;
315
320
  const earlier = pinned ?? (layerId ? history.find((entry) => entry.layer.id === layerId) : undefined);
316
321
  const chosen: HistoryEntry = earlier ?? { layer: winner.layer, row: winner.row };
317
322
  const superseded = !slot.outcomes.some((outcome) => outcome.layer.id === chosen.layer.id);
318
323
  const { paths, report } = useRecordedReport(chosen);
324
+ const recorded = useCheckDetail(chosen.layer, chosen.row);
325
+ const point = useMemo(
326
+ () => (chosen.row.status === 'failed' ? (failedStep(recorded.data?.check.steps) ?? failedReportStep(report.data)) : undefined),
327
+ [chosen.row.status, recorded.data, report.data]
328
+ );
329
+ const labels = sourceLabels(slot.outcomes, now);
330
+ const choose = (at: number) => {
331
+ setVariant(at);
332
+ onSource?.(at === 0 ? null : slot.outcomes[at]!.layer.id);
333
+ };
319
334
  const moments = report.data ? flattenFrames(report.data.steps).length : 0;
320
335
  const timings = superseded ? [] : slot.diagnostics.filter((entry) => entry.layer.id === chosen.layer.id || slot.diagnostics.length === 1);
321
336
  const note = superseded ? 'An earlier result. A newer run replaced it in the combined report.' : qualification(winner.warning);
@@ -328,12 +343,32 @@ function RecordedCheckDetail({ slot, layerId, select, reads, written, pinned, pi
328
343
  : '';
329
344
  return (
330
345
  <Stack gap="sm" miw={0}>
346
+ <OutcomeSummary
347
+ slot={slot}
348
+ selected={chosen}
349
+ history={records && records.length > history.length ? records : history.map(recordOf)}
350
+ point={point}
351
+ detail={recorded.data}
352
+ now={now}
353
+ showHeadline={headline}
354
+ choose={(layer) => {
355
+ const at = slot.outcomes.findIndex((outcome) => outcome.layer.id === layer);
356
+ if (at >= 0) choose(at);
357
+ else select({ layer });
358
+ }}
359
+ />
331
360
  {slot.outcomes.length > 1 && !earlier && (
332
361
  <Flex role="group" aria-label="Environments" gap="xs" align="center" wrap="wrap">
333
- {slot.outcomes.map((outcome, index) => (
334
- <Button key={outcome.key} type="button" aria-pressed={index === variant} onClick={() => setVariant(index)} variant="light">
362
+ {slot.outcomes.map((outcome, at) => (
363
+ <Button
364
+ key={outcome.key}
365
+ type="button"
366
+ aria-pressed={at === index}
367
+ onClick={() => choose(at)}
368
+ variant="light"
369
+ >
335
370
  <StatusMark status={outcome.row.status} size={12} />
336
- {outcomeName(outcome)}
371
+ {labels[at]!.text}
337
372
  </Button>
338
373
  ))}
339
374
  </Flex>
@@ -410,7 +445,7 @@ function RecordedCheckDetail({ slot, layerId, select, reads, written, pinned, pi
410
445
  {moments > 0 && (
411
446
  <Button
412
447
  type="button"
413
- onClick={() => select({ view: 'recording', check: slot.id, layer: earlier || variant > 0 ? chosen.layer.id : null })}
448
+ onClick={() => select({ view: 'recording', check: slot.id, layer: earlier || index > 0 ? chosen.layer.id : null })}
414
449
  variant="filled"
415
450
  >
416
451
  Open recording{' '}
@@ -0,0 +1,208 @@
1
+ /**
2
+ * @module Outcome summary
3
+ *
4
+ * What the selected result says, stated apart from the other current sources, and the failure explained before its
5
+ * raw output: the heading names the selected source's outcome and what the other scoped sources recorded (each one
6
+ * click away), whether the behavior was reached, the recorded expected and observed values, the closest video and
7
+ * step, the chronology of the source, and who is to blame, which stays unknown unless the producer declared it. The
8
+ * original diagnostics remain below, untouched.
9
+ */
10
+ import type { CheckSlot, HistoryEntry } from '@crvouga/atlas/reports/evidence-view';
11
+ import type { CheckDetail } from '@crvouga/atlas/reports/evidence-detail';
12
+ import { effectiveScope, scopeBadges } from '@crvouga/atlas/reports/evidence-scope';
13
+ import { Badge, Box, Button, Flex, List, Paper, Stack, Text, Title } from '@mantine/core';
14
+
15
+ import { ago, duration, shortSha, statusWord } from './format';
16
+ import { StatusMark } from './marks';
17
+ import {
18
+ chronology,
19
+ failureNote,
20
+ outcomeHeadline,
21
+ passedAfterFailure,
22
+ recordOf,
23
+ type ResultRecord,
24
+ reachOf,
25
+ sourceLabels,
26
+ videosIn,
27
+ type FailurePoint
28
+ } from './outcome';
29
+
30
+ const BLAME: Record<string, string> = {
31
+ product: 'the product',
32
+ 'test-harness': 'the test harness',
33
+ environment: 'the environment'
34
+ };
35
+
36
+ /**
37
+ * The outcome of the selected result in words, the other current sources beside it, and, for a failure, where and
38
+ * how it failed. `point` is the failed step found in the recording (undefined when none was recorded); `detail` is the
39
+ * check's full record, for its videos and scope. `choose` selects one of the other current sources; `showHeadline`
40
+ * is false when the page already shows the heading.
41
+ */
42
+ export function OutcomeSummary({
43
+ slot,
44
+ selected,
45
+ history,
46
+ point,
47
+ detail,
48
+ now,
49
+ choose,
50
+ showHeadline = true
51
+ }: {
52
+ slot: CheckSlot;
53
+ selected: HistoryEntry;
54
+ history: readonly ResultRecord[];
55
+ point: FailurePoint | undefined;
56
+ detail: CheckDetail | undefined;
57
+ now: number;
58
+ choose: (layerId: string) => void;
59
+ showHeadline?: boolean;
60
+ }) {
61
+ const headline = outcomeHeadline(selected, slot);
62
+ const failed = selected.row.status === 'failed';
63
+ const reach = failed ? reachOf(point) : undefined;
64
+ const note = point?.note ?? failureNote(detail?.check.extensions);
65
+ const every = sourceLabels(slot.outcomes, now);
66
+ const labels = headline.others.map((outcome) => every[slot.outcomes.indexOf(outcome)]!);
67
+ const video = point?.videos[0] ?? videosIn(detail?.check)[0];
68
+ const scope = video ? effectiveScope(video.scope, detail?.check.scope, detail?.inheritedScope) : undefined;
69
+ const timeline = chronology(recordOf(selected), history);
70
+ return (
71
+ <Paper component="section" aria-label="Outcome summary" data-outcome={selected.row.status} withBorder p="md">
72
+ <Stack gap="sm" miw={0}>
73
+ {showHeadline && (
74
+ <Title order={3} data-outcome-headline>
75
+ {headline.text}
76
+ </Title>
77
+ )}
78
+ {headline.others.length > 0 && (
79
+ <Stack gap="xs" component="section" aria-label="Other current sources">
80
+ <Text component="div" c="dimmed">
81
+ Other current sources of this check
82
+ </Text>
83
+ <List listStyleType="none" spacing="xs">
84
+ {headline.others.map((outcome, index) => (
85
+ <List.Item key={outcome.key}>
86
+ <Button type="button" variant="light" onClick={() => choose(outcome.layer.id)} aria-label={`Open ${labels[index]!.text}`}>
87
+ <StatusMark status={outcome.row.status} size={12} />
88
+ <Text span inherit>
89
+ {labels[index]!.text}
90
+ </Text>
91
+ </Button>
92
+ </List.Item>
93
+ ))}
94
+ </List>
95
+ </Stack>
96
+ )}
97
+ {reach && (
98
+ <Flex gap="xs" align="center" wrap="wrap" data-reach={reach.state}>
99
+ <Badge variant={reach.state === 'setup-failed' ? 'filled' : 'light'} color={reach.state === 'unknown' ? 'gray' : 'red'}>
100
+ {reach.state === 'setup-failed'
101
+ ? 'Behavior not verified'
102
+ : reach.state === 'expectation-failed'
103
+ ? 'Expectation not met'
104
+ : 'Failure point unknown'}
105
+ </Badge>
106
+ <Text span inherit>
107
+ {reach.text}
108
+ </Text>
109
+ </Flex>
110
+ )}
111
+ {failed && (note?.expected !== undefined || note?.actual !== undefined) && (
112
+ <Box component="dl" aria-label="Expected and observed" m={0}>
113
+ {note?.expected !== undefined && (
114
+ <>
115
+ <Text component="dt" fw={600}>
116
+ Expected
117
+ </Text>
118
+ <Text component="dd" m={0}>
119
+ {note.expected}
120
+ </Text>
121
+ </>
122
+ )}
123
+ {note?.actual !== undefined && (
124
+ <>
125
+ <Text component="dt" fw={600}>
126
+ Observed
127
+ </Text>
128
+ <Text component="dd" m={0}>
129
+ {note.actual}
130
+ </Text>
131
+ </>
132
+ )}
133
+ </Box>
134
+ )}
135
+ {failed && (point || video) && (
136
+ <Stack gap="xs" component="section" aria-label="Closest evidence">
137
+ {point && (
138
+ <Text component="div">
139
+ Closest step:{' '}
140
+ <Text span inherit fw={600}>
141
+ {point.name}
142
+ </Text>
143
+ {point.offsetMs !== undefined && (
144
+ <Text span inherit c="dimmed">
145
+ {' '}
146
+ at {duration(point.offsetMs) || '0 ms'} into the check
147
+ </Text>
148
+ )}
149
+ </Text>
150
+ )}
151
+ {video && (
152
+ <Text component="div">
153
+ Closest video:{' '}
154
+ <Text span inherit fw={600}>
155
+ {video.description ?? video.name}
156
+ </Text>
157
+ <Text span inherit c="dimmed">
158
+ {' '}
159
+ {point?.videos.length ? 'attached to that step' : 'recorded for the whole check'}
160
+ {' · '}
161
+ {scopeBadges(scope)
162
+ .map((badge) => badge.text)
163
+ .join(', ')}
164
+ </Text>
165
+ </Text>
166
+ )}
167
+ </Stack>
168
+ )}
169
+ {failed && (
170
+ <Text component="div" c="dimmed" data-attribution={note?.attribution ?? 'unknown'}>
171
+ Cause:{' '}
172
+ {note?.attribution
173
+ ? `${BLAME[note.attribution]}, as declared by the producer`
174
+ : 'unknown. Nothing recorded says whether the product, the test harness or the environment is at fault.'}
175
+ </Text>
176
+ )}
177
+ {timeline.length > 0 && (
178
+ <Stack gap="xs" component="section" aria-label="Chronology of this source">
179
+ <Text component="div" c="dimmed">
180
+ Records of this source, oldest first
181
+ </Text>
182
+ <List listStyleType="none" spacing="xs" type="ordered">
183
+ {timeline.map(({ record, selected: current }) => (
184
+ <List.Item key={`${record.layer}|${record.kind}`} aria-current={current || undefined} data-status={record.status}>
185
+ <Flex gap="xs" align="center" wrap="wrap">
186
+ <StatusMark status={record.status} size={12} />
187
+ <Text span inherit fw={current ? 600 : undefined}>
188
+ {statusWord(record.status)}
189
+ </Text>
190
+ <Text span inherit c="dimmed">
191
+ {ago(record.at, now)} · {shortSha(record.sha) ?? 'revision not recorded'}
192
+ {current ? ' · selected' : ''}
193
+ </Text>
194
+ </Flex>
195
+ </List.Item>
196
+ ))}
197
+ </List>
198
+ {passedAfterFailure(timeline) && (
199
+ <Text component="div" role="status">
200
+ The latest run passed after an earlier failure. Nothing recorded shows that the failure was fixed or why it passed.
201
+ </Text>
202
+ )}
203
+ </Stack>
204
+ )}
205
+ </Stack>
206
+ </Paper>
207
+ );
208
+ }