@crvouga/atlas 1.4.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/docs/ui.md +21 -1
- package/llms-full.txt +21 -1
- package/package.json +1 -1
- package/packages/visualizer/src/evidence/artifacts.tsx +17 -8
- package/packages/visualizer/src/evidence/behavior-panel.tsx +291 -0
- package/packages/visualizer/src/evidence/behavior.test.ts +99 -0
- package/packages/visualizer/src/evidence/behavior.ts +268 -0
- package/packages/visualizer/src/evidence/check-detail.tsx +4 -0
- package/packages/visualizer/src/evidence/feature-page.tsx +82 -36
- package/packages/visualizer/src/evidence/gherkin.ts +31 -1
- package/packages/visualizer/src/evidence/spec-index.tsx +48 -15
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# Changelog: @crvouga/atlas
|
|
2
2
|
|
|
3
|
+
## 1.5.0 (2026-10-29)
|
|
4
|
+
|
|
5
|
+
### Features
|
|
6
|
+
|
|
7
|
+
- connect readable behavior with recordings and coverage gaps (#55) ([0a22022](https://github.com/crvouga/atlas/commit/0a22022dac9f9956aa77214af5de7af4fd2b2d88))
|
|
8
|
+
|
|
3
9
|
## 1.4.0 (2026-10-28)
|
|
4
10
|
|
|
5
11
|
### Features
|
package/docs/ui.md
CHANGED
|
@@ -67,6 +67,26 @@ and component defaults still live only in `ui/theme.tsx`. Quint/Alloy grammar so
|
|
|
67
67
|
[`ui/grammars/README.md`](../packages/visualizer/src/ui/grammars/README.md). Unknown formats remain plain
|
|
68
68
|
text, and parse failures must leave the source visible. Syntax highlighting never means a spec passed.
|
|
69
69
|
|
|
70
|
+
## Behaviors: Read, Watch, Inspect
|
|
71
|
+
|
|
72
|
+
A Gherkin page reads as behaviors: each scenario, or each example row of an outline (never the outline template as an
|
|
73
|
+
extra), is one behavior in one coverage state (`evidence/behavior.ts`). Opening a behavior shows Mantine `Tabs`:
|
|
74
|
+
**Read** (the steps with the example's values filled in), **Watch** (what is expected next to the matched recording,
|
|
75
|
+
its status and provenance) and **Inspect** (the existing check detail: runs, environments, attempts, files). A behavior
|
|
76
|
+
with a playable recording opens on Watch, one in a run in progress on Inspect, otherwise on Read. The video is read only
|
|
77
|
+
when Watch is on screen.
|
|
78
|
+
|
|
79
|
+
- States stay distinct and none but `demonstrated` is a pass with proof: recording missing, recording of an older spec
|
|
80
|
+
revision, no recording, API only, failed, running, skipped, parked (tagged `@parked`, `@wip`, `@pending`) and not executed.
|
|
81
|
+
Counts are in the Coverage section above the source; an empty state is left out except `demonstrated`.
|
|
82
|
+
- The promise's revision (the spec's fingerprint now) and the evidence's revision (the fingerprint the layer recorded)
|
|
83
|
+
are shown separately. A known difference demotes a recording to "Recording is older" with an alert; a missing
|
|
84
|
+
fingerprint is "not recorded", never a match.
|
|
85
|
+
- A recording is always labelled illustrative: it shows one run and does not approve a behavior for production.
|
|
86
|
+
`demo`/`representative` tags only add a "Representative demo" label.
|
|
87
|
+
- The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
|
|
88
|
+
none are listed under "No domain declared"; the Tests view reports domain problems).
|
|
89
|
+
|
|
70
90
|
## Icons and notation marks
|
|
71
91
|
|
|
72
92
|
Use named imports from `@tabler/icons-react` for all UI glyphs. The icon package supplies SVG paths;
|
|
@@ -127,7 +147,7 @@ The narrowly scoped exceptions are:
|
|
|
127
147
|
| Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
|
|
128
148
|
| Project home | Tests/runs/artifacts tabs, a run's Artifacts (files of its layers, shared recordings, direct links), metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion |
|
|
129
149
|
| Test detail | Current outcomes, statistics, duration graph, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
|
|
130
|
-
| Specs | All six existing kinds, overview/index, Gherkin rules/scenarios/examples, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Table, CodeHighlightTabs, ResultRow |
|
|
150
|
+
| Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
|
|
131
151
|
| Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
|
|
132
152
|
| Evidence | Steps, retries, annotations, facts, unknown extensions, safe downloads and previews | List, Badge, Accordion, Code, Image, native media |
|
|
133
153
|
| Recordings | Pictures/video/timeline, actor selection, moments, failure jumps, timing, model-state differences | Tabs, Slider, Button, ScrollArea, Table, Progress |
|
package/llms-full.txt
CHANGED
|
@@ -3899,6 +3899,26 @@ and component defaults still live only in `ui/theme.tsx`. Quint/Alloy grammar so
|
|
|
3899
3899
|
[`ui/grammars/README.md`](../packages/visualizer/src/ui/grammars/README.md). Unknown formats remain plain
|
|
3900
3900
|
text, and parse failures must leave the source visible. Syntax highlighting never means a spec passed.
|
|
3901
3901
|
|
|
3902
|
+
## Behaviors: Read, Watch, Inspect
|
|
3903
|
+
|
|
3904
|
+
A Gherkin page reads as behaviors: each scenario, or each example row of an outline (never the outline template as an
|
|
3905
|
+
extra), is one behavior in one coverage state (`evidence/behavior.ts`). Opening a behavior shows Mantine `Tabs`:
|
|
3906
|
+
**Read** (the steps with the example's values filled in), **Watch** (what is expected next to the matched recording,
|
|
3907
|
+
its status and provenance) and **Inspect** (the existing check detail: runs, environments, attempts, files). A behavior
|
|
3908
|
+
with a playable recording opens on Watch, one in a run in progress on Inspect, otherwise on Read. The video is read only
|
|
3909
|
+
when Watch is on screen.
|
|
3910
|
+
|
|
3911
|
+
- States stay distinct and none but `demonstrated` is a pass with proof: recording missing, recording of an older spec
|
|
3912
|
+
revision, no recording, API only, failed, running, skipped, parked (tagged `@parked`, `@wip`, `@pending`) and not executed.
|
|
3913
|
+
Counts are in the Coverage section above the source; an empty state is left out except `demonstrated`.
|
|
3914
|
+
- The promise's revision (the spec's fingerprint now) and the evidence's revision (the fingerprint the layer recorded)
|
|
3915
|
+
are shown separately. A known difference demotes a recording to "Recording is older" with an alert; a missing
|
|
3916
|
+
fingerprint is "not recorded", never a match.
|
|
3917
|
+
- A recording is always labelled illustrative: it shows one run and does not approve a behavior for production.
|
|
3918
|
+
`demo`/`representative` tags only add a "Representative demo" label.
|
|
3919
|
+
- The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
|
|
3920
|
+
none are listed under "No domain declared"; the Tests view reports domain problems).
|
|
3921
|
+
|
|
3902
3922
|
## Icons and notation marks
|
|
3903
3923
|
|
|
3904
3924
|
Use named imports from `@tabler/icons-react` for all UI glyphs. The icon package supplies SVG paths;
|
|
@@ -3959,7 +3979,7 @@ The narrowly scoped exceptions are:
|
|
|
3959
3979
|
| Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
|
|
3960
3980
|
| Project home | Tests/runs/artifacts tabs, a run's Artifacts (files of its layers, shared recordings, direct links), metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion |
|
|
3961
3981
|
| Test detail | Current outcomes, statistics, duration graph, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
|
|
3962
|
-
| Specs | All six existing kinds, overview/index, Gherkin rules/scenarios/examples, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Table, CodeHighlightTabs, ResultRow |
|
|
3982
|
+
| Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
|
|
3963
3983
|
| Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
|
|
3964
3984
|
| Evidence | Steps, retries, annotations, facts, unknown extensions, safe downloads and previews | List, Badge, Accordion, Code, Image, native media |
|
|
3965
3985
|
| Recordings | Pictures/video/timeline, actor selection, moments, failure jumps, timing, model-state differences | Tabs, Slider, Button, ScrollArea, Table, Progress |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@crvouga/atlas",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.5.0",
|
|
4
4
|
"description": "Atlas: specs of any kind, reported as layered evidence. Publish checks from any runner (JUnit, CTRF, a script), combine them across nodes, branches, environments and backends, gate CI on the result, and run statecharts against the real app.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -146,6 +146,13 @@ function sizeOf(bytes: number | undefined) {
|
|
|
146
146
|
if (bytes < 1024) return `${bytes} B`;
|
|
147
147
|
return bytes < 1024 * 1024 ? `${(bytes / 1024).toFixed(0)} KiB` : `${(bytes / 1024 / 1024).toFixed(1)} MiB`;
|
|
148
148
|
}
|
|
149
|
+
/** Where an attachment's bytes are, if a browser can reach them: its stored file in the layer, else its safe `http(s)` address. */
|
|
150
|
+
export function attachmentSource(attachment: Attachment, layer: string): { path: string | null; url: string | null } {
|
|
151
|
+
const path = attachment.bundleKey
|
|
152
|
+
? `/layers/${encodeURIComponent(layer)}/blobs/${attachment.bundleKey.split('/').map(encodeURIComponent).join('/')}`
|
|
153
|
+
: null;
|
|
154
|
+
return { path, url: path ? reportAssetUrl(path) : safeUri(attachment.uri) };
|
|
155
|
+
}
|
|
149
156
|
/**
|
|
150
157
|
* One attachment as a card: it plays or shows itself when it can, and always says what it is (name, role, type, size,
|
|
151
158
|
* duration, tags) and how to get it. A file that has no address, has an address that is not a web address, or that
|
|
@@ -154,10 +161,7 @@ function sizeOf(bytes: number | undefined) {
|
|
|
154
161
|
*/
|
|
155
162
|
export function ArtifactCard({ attachment, layer, link, note, selected }: { attachment: Attachment; layer: string; link?: ReactNode; note?: ReactNode; selected?: boolean }) {
|
|
156
163
|
const type = mediaType(attachment);
|
|
157
|
-
const path = attachment
|
|
158
|
-
? `/layers/${encodeURIComponent(layer)}/blobs/${attachment.bundleKey.split('/').map(encodeURIComponent).join('/')}`
|
|
159
|
-
: null;
|
|
160
|
-
const url = path ? reportAssetUrl(path) : safeUri(attachment.uri);
|
|
164
|
+
const { path, url } = attachmentSource(attachment, layer);
|
|
161
165
|
const renderer = artifactRenderers.find((candidate) => candidate.matches(attachment, type));
|
|
162
166
|
const body = renderer?.render(attachment, { url, path, type });
|
|
163
167
|
const item = useRef<HTMLLIElement>(null);
|
|
@@ -421,20 +425,25 @@ function Evidence({ holder, layer, steps }: { holder: EvidenceHolder; layer: str
|
|
|
421
425
|
</ScopeProvider>
|
|
422
426
|
);
|
|
423
427
|
}
|
|
424
|
-
/**
|
|
425
|
-
export function
|
|
428
|
+
/** The full evidence of one result, read only when `enabled` and the result has some to read (an archived layer). Shared by every view of the result, so it is read once. */
|
|
429
|
+
export function useCheckDetail(layer: LayerHeader, row: LayerRow, enabled = true) {
|
|
426
430
|
const wanted = row.evidence !== undefined || layer.evidence !== undefined;
|
|
427
|
-
|
|
431
|
+
return useQuery({
|
|
428
432
|
queryKey: ['evidence-detail', layer.backend, layer.id, row.kind, row.id],
|
|
429
433
|
queryFn: ({ signal }) =>
|
|
430
434
|
reportJson<CheckDetail>(
|
|
431
435
|
`/evidence/detail/${encodeURIComponent(layer.backend)}/${encodeURIComponent(layer.id)}?check=${encodeURIComponent(row.id)}&kind=${encodeURIComponent(row.kind)}`,
|
|
432
436
|
signal
|
|
433
437
|
),
|
|
434
|
-
enabled: wanted && /^[0-9a-f]{64}$/.test(layer.id),
|
|
438
|
+
enabled: enabled && wanted && /^[0-9a-f]{64}$/.test(layer.id),
|
|
435
439
|
staleTime: Number.POSITIVE_INFINITY,
|
|
436
440
|
retry: false
|
|
437
441
|
});
|
|
442
|
+
}
|
|
443
|
+
/** Everything one result carries beyond its outcome, read when it is opened, and what the run around it carried. */
|
|
444
|
+
export function ResultEvidence({ layer, row }: { layer: LayerHeader; row: LayerRow }) {
|
|
445
|
+
const wanted = row.evidence !== undefined || layer.evidence !== undefined;
|
|
446
|
+
const detail = useCheckDetail(layer, row);
|
|
438
447
|
if (!wanted) return null;
|
|
439
448
|
if (detail.isLoading)
|
|
440
449
|
return (
|
|
@@ -0,0 +1,291 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @module Behavior panel
|
|
3
|
+
*
|
|
4
|
+
* How a product reader meets one behavior of a feature: Read (the promise and what is expected, with an outline's
|
|
5
|
+
* concrete example filled in), Watch (the matched recording beside what it should show) and Inspect (the technical
|
|
6
|
+
* record: run, environment, source, attempts). Also the coverage summary of the whole feature. A recording is always
|
|
7
|
+
* labelled as illustrative evidence with its own status, provenance and revision; it never stands for approval.
|
|
8
|
+
*/
|
|
9
|
+
import type { EvidenceHolder } from '@crvouga/atlas/reports/evidence-detail';
|
|
10
|
+
import { isVideoAttachment } from '@crvouga/atlas/reports/evidence-detail';
|
|
11
|
+
import { effectiveScope } from '@crvouga/atlas/reports/evidence-scope';
|
|
12
|
+
import type { EvidenceResult } from '@crvouga/atlas/reports/layer-merge';
|
|
13
|
+
import { environmentLabel } from '@crvouga/atlas/reports/layer-axes';
|
|
14
|
+
import type { Attachment } from '@crvouga/atlas/schema';
|
|
15
|
+
import { Alert, Badge, Box, Group, List, Paper, Stack, Tabs, Text, Title } from '@mantine/core';
|
|
16
|
+
import { useMemo, type ReactNode } from 'react';
|
|
17
|
+
|
|
18
|
+
import { attachmentSource, useCheckDetail } from './artifacts';
|
|
19
|
+
import { COVERAGE, COVERAGE_ORDER, type Behavior, type CoverageCounts, type CoverageState, type Revision } from './behavior';
|
|
20
|
+
import { PlainSteps } from './check-detail';
|
|
21
|
+
import { useEvidence } from './evidence-context';
|
|
22
|
+
import { ScopeNotice } from './evidence-scope';
|
|
23
|
+
import { plural, statusWord, useNow } from './format';
|
|
24
|
+
import { exampleIndexFor, exampleRows, fillExample, type FeatureStep } from './gherkin';
|
|
25
|
+
import { Origin, StatusMark } from './marks';
|
|
26
|
+
|
|
27
|
+
/** The short, readable form of a spec fingerprint (`<length>:<hash>:<hash>`): its length and the start of its hash. */
|
|
28
|
+
export function shortRevision(fingerprint: string): string {
|
|
29
|
+
const [length, hash] = fingerprint.split(':');
|
|
30
|
+
return hash ? `${hash.slice(0, 8)} (${length} characters)` : fingerprint.slice(0, 12);
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
/** A behavior's coverage state, always in words. */
|
|
34
|
+
export function CoverageBadge({ state }: { state: CoverageState }) {
|
|
35
|
+
return (
|
|
36
|
+
<Badge data-coverage={state} color={COVERAGE[state].color} variant="light" title={COVERAGE[state].meaning}>
|
|
37
|
+
{COVERAGE[state].label}
|
|
38
|
+
</Badge>
|
|
39
|
+
);
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
/**
|
|
43
|
+
* The promise and the evidence side by side, each with its own revision. A mismatch is stated, not hidden; a missing
|
|
44
|
+
* fingerprint is stated as unknown rather than treated as a match.
|
|
45
|
+
*/
|
|
46
|
+
export function RevisionLine({ revision }: { revision: Revision }) {
|
|
47
|
+
return (
|
|
48
|
+
<Stack gap="xs" data-revision={revision.match}>
|
|
49
|
+
<Group gap="md" wrap="wrap" c="dimmed">
|
|
50
|
+
<Text span inherit>
|
|
51
|
+
Promise revision:{' '}
|
|
52
|
+
<Text span inherit fw={600}>
|
|
53
|
+
{revision.promise ? shortRevision(revision.promise) : 'not recorded'}
|
|
54
|
+
</Text>
|
|
55
|
+
</Text>
|
|
56
|
+
<Text span inherit>
|
|
57
|
+
Evidence revision:{' '}
|
|
58
|
+
<Text span inherit fw={600}>
|
|
59
|
+
{revision.evidence ? shortRevision(revision.evidence) : 'not recorded'}
|
|
60
|
+
</Text>
|
|
61
|
+
</Text>
|
|
62
|
+
</Group>
|
|
63
|
+
{revision.match === 'different' && (
|
|
64
|
+
<Alert color="orange" role="status">
|
|
65
|
+
This evidence was recorded against an earlier revision of the spec. It does not show that the current promise holds.
|
|
66
|
+
</Alert>
|
|
67
|
+
)}
|
|
68
|
+
{revision.match === 'unknown' && (
|
|
69
|
+
<Text component="div" c="dimmed">
|
|
70
|
+
The revisions cannot be compared, so this evidence is not tied to the current promise.
|
|
71
|
+
</Text>
|
|
72
|
+
)}
|
|
73
|
+
</Stack>
|
|
74
|
+
);
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/** The `Then` steps (and the `And`/`But` after them): what should be true when the behavior has run. */
|
|
78
|
+
function expectedSteps(steps: readonly FeatureStep[]): FeatureStep[] {
|
|
79
|
+
const at = steps.findIndex((step) => step.keyword === 'Then');
|
|
80
|
+
return at < 0 ? [] : steps.slice(at);
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
function videosIn(holder: EvidenceHolder | undefined): Attachment[] {
|
|
84
|
+
if (!holder) return [];
|
|
85
|
+
const own = (holder.attachments ?? []).filter(isVideoAttachment);
|
|
86
|
+
return [
|
|
87
|
+
...own,
|
|
88
|
+
...(holder.steps ?? []).flatMap((step) => (step.attachments ?? []).filter(isVideoAttachment)),
|
|
89
|
+
...(holder.attempts ?? []).flatMap((attempt) => [
|
|
90
|
+
...(attempt.attachments ?? []).filter(isVideoAttachment),
|
|
91
|
+
...(attempt.steps ?? []).flatMap((step) => (step.attachments ?? []).filter(isVideoAttachment))
|
|
92
|
+
])
|
|
93
|
+
];
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
/** The recording of one result, read only when this is on screen. */
|
|
97
|
+
function Player({ result }: { result: EvidenceResult }) {
|
|
98
|
+
const detail = useCheckDetail(result.layer, result.row);
|
|
99
|
+
const videos = useMemo(() => videosIn(detail.data?.check), [detail.data]);
|
|
100
|
+
const sources = videos.flatMap((attachment) => {
|
|
101
|
+
const { url } = attachmentSource(attachment, result.layer.id);
|
|
102
|
+
return url ? [{ attachment, url }] : [];
|
|
103
|
+
});
|
|
104
|
+
if (detail.isLoading) {
|
|
105
|
+
return (
|
|
106
|
+
<Text role="status" component="div" c="dimmed">
|
|
107
|
+
Loading the recording…
|
|
108
|
+
</Text>
|
|
109
|
+
);
|
|
110
|
+
}
|
|
111
|
+
if (detail.isError) {
|
|
112
|
+
return (
|
|
113
|
+
<Alert role="alert" color="red">
|
|
114
|
+
The recording could not be read.
|
|
115
|
+
</Alert>
|
|
116
|
+
);
|
|
117
|
+
}
|
|
118
|
+
if (!sources.length) {
|
|
119
|
+
return (
|
|
120
|
+
<Text component="div" c="dimmed">
|
|
121
|
+
This result names a recording, but no file can be played from here.
|
|
122
|
+
</Text>
|
|
123
|
+
);
|
|
124
|
+
}
|
|
125
|
+
return (
|
|
126
|
+
<Stack gap="sm">
|
|
127
|
+
{sources.slice(0, 1).map(({ attachment, url }) => (
|
|
128
|
+
<Stack key={url} gap="xs">
|
|
129
|
+
{/* What the recording does and does not show, as its producer declared it; undeclared says unknown. */}
|
|
130
|
+
<ScopeNotice
|
|
131
|
+
scope={effectiveScope(attachment.scope, detail.data?.check.scope, detail.data?.inheritedScope)}
|
|
132
|
+
name={attachment.name}
|
|
133
|
+
/>
|
|
134
|
+
<Box
|
|
135
|
+
component="video"
|
|
136
|
+
src={url}
|
|
137
|
+
controls
|
|
138
|
+
preload="metadata"
|
|
139
|
+
aria-label={`Recording: ${attachment.description ?? attachment.name}`}
|
|
140
|
+
maw="100%"
|
|
141
|
+
w="100%"
|
|
142
|
+
mah="70vh"
|
|
143
|
+
/>
|
|
144
|
+
</Stack>
|
|
145
|
+
))}
|
|
146
|
+
{sources.length > 1 && (
|
|
147
|
+
<Text component="div" c="dimmed">
|
|
148
|
+
{plural(sources.length - 1, 'more recording', 'more recordings')} in Inspect.
|
|
149
|
+
</Text>
|
|
150
|
+
)}
|
|
151
|
+
</Stack>
|
|
152
|
+
);
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
/** Read, Watch and Inspect for one behavior. `inspect` is the technical record, shown only under Inspect. */
|
|
156
|
+
export function BehaviorPanel({
|
|
157
|
+
behavior,
|
|
158
|
+
inspect,
|
|
159
|
+
start
|
|
160
|
+
}: {
|
|
161
|
+
behavior: Behavior;
|
|
162
|
+
inspect: ReactNode;
|
|
163
|
+
/** The tab to open on; by default Watch when there is something to watch, Inspect while a run is in progress (its progress lives there), else Read. */ start?:
|
|
164
|
+
'read' | 'watch' | 'inspect';
|
|
165
|
+
}) {
|
|
166
|
+
const { backends } = useEvidence();
|
|
167
|
+
const now = useNow();
|
|
168
|
+
const { scenario, example, demo, subject, revision, state } = behavior;
|
|
169
|
+
const at = example === null ? -1 : exampleIndexFor(scenario, example);
|
|
170
|
+
const entry = at < 0 ? null : exampleRows(scenario)[at]!;
|
|
171
|
+
const steps = useMemo(() => (entry ? fillExample(scenario.steps, entry) : scenario.steps), [scenario, entry]);
|
|
172
|
+
const expected = expectedSteps(steps);
|
|
173
|
+
const named = example !== null && entry === null;
|
|
174
|
+
return (
|
|
175
|
+
<Tabs
|
|
176
|
+
defaultValue={start ?? (demo ? 'watch' : state === 'running' || behavior.slot?.live ? 'inspect' : 'read')}
|
|
177
|
+
keepMounted={false}
|
|
178
|
+
aria-label={`${scenario.name}${example ? `: ${example}` : ''}`}
|
|
179
|
+
>
|
|
180
|
+
<Tabs.List mb="sm">
|
|
181
|
+
<Tabs.Tab value="read">Read</Tabs.Tab>
|
|
182
|
+
<Tabs.Tab value="watch">Watch</Tabs.Tab>
|
|
183
|
+
<Tabs.Tab value="inspect">Inspect</Tabs.Tab>
|
|
184
|
+
</Tabs.List>
|
|
185
|
+
<Tabs.Panel value="read">
|
|
186
|
+
<Stack gap="sm" miw={0}>
|
|
187
|
+
<Group gap="sm">
|
|
188
|
+
<CoverageBadge state={state} />
|
|
189
|
+
<Text component="span" c="dimmed">
|
|
190
|
+
{COVERAGE[state].meaning}
|
|
191
|
+
</Text>
|
|
192
|
+
</Group>
|
|
193
|
+
{example !== null && (
|
|
194
|
+
<Text component="div" c="dimmed">
|
|
195
|
+
Example: {example}
|
|
196
|
+
</Text>
|
|
197
|
+
)}
|
|
198
|
+
<PlainSteps steps={steps} examples={entry ? [] : scenario.examples} />
|
|
199
|
+
{named && (
|
|
200
|
+
<Text component="div" c="dimmed">
|
|
201
|
+
The recorded example name does not match one row of the examples, so the values are not filled in.
|
|
202
|
+
</Text>
|
|
203
|
+
)}
|
|
204
|
+
</Stack>
|
|
205
|
+
</Tabs.Panel>
|
|
206
|
+
<Tabs.Panel value="watch">
|
|
207
|
+
<Stack gap="sm" miw={0}>
|
|
208
|
+
<Group gap="sm">
|
|
209
|
+
<CoverageBadge state={state} />
|
|
210
|
+
{behavior.representative && <Badge variant="outline">Representative demo</Badge>}
|
|
211
|
+
</Group>
|
|
212
|
+
{expected.length > 0 && (
|
|
213
|
+
<Box component="section" aria-label="Expected outcome">
|
|
214
|
+
<Title order={4} mb="xs">
|
|
215
|
+
Expected
|
|
216
|
+
</Title>
|
|
217
|
+
<PlainSteps steps={expected} />
|
|
218
|
+
</Box>
|
|
219
|
+
)}
|
|
220
|
+
{demo ? (
|
|
221
|
+
<>
|
|
222
|
+
<Player result={demo} />
|
|
223
|
+
<Text component="div" c="dimmed">
|
|
224
|
+
Illustrative recording of one run. It shows what happened in that run and does not approve the behavior for production.
|
|
225
|
+
</Text>
|
|
226
|
+
</>
|
|
227
|
+
) : (
|
|
228
|
+
<Text component="div" c="dimmed">
|
|
229
|
+
Nothing to watch. {COVERAGE[state].meaning}
|
|
230
|
+
</Text>
|
|
231
|
+
)}
|
|
232
|
+
{subject && (
|
|
233
|
+
<Group gap="xs" wrap="wrap" align="center">
|
|
234
|
+
<StatusMark status={subject.row.status} size={14} />
|
|
235
|
+
<Text span inherit fw={600}>
|
|
236
|
+
{statusWord(subject.row.status)}
|
|
237
|
+
</Text>
|
|
238
|
+
<Text span inherit c="dimmed">
|
|
239
|
+
in {environmentLabel(subject.layer.environment)}
|
|
240
|
+
</Text>
|
|
241
|
+
<Origin layer={subject.layer} backends={backends} now={now} />
|
|
242
|
+
</Group>
|
|
243
|
+
)}
|
|
244
|
+
<RevisionLine revision={revision} />
|
|
245
|
+
</Stack>
|
|
246
|
+
</Tabs.Panel>
|
|
247
|
+
<Tabs.Panel value="inspect">
|
|
248
|
+
<Stack gap="sm" miw={0}>
|
|
249
|
+
<RevisionLine revision={revision} />
|
|
250
|
+
{inspect}
|
|
251
|
+
</Stack>
|
|
252
|
+
</Tabs.Panel>
|
|
253
|
+
</Tabs>
|
|
254
|
+
);
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
/**
|
|
258
|
+
* What the feature's behaviors add up to, one count per state. A state with none is left out except
|
|
259
|
+
* `demonstrated`, which is always named so its absence is not mistaken for a pass.
|
|
260
|
+
*/
|
|
261
|
+
export function CoverageSummary({ counts, total }: { counts: CoverageCounts; total: number }) {
|
|
262
|
+
if (total === 0) return null;
|
|
263
|
+
return (
|
|
264
|
+
<Paper component="section" aria-label="Coverage" withBorder p="md" my="md">
|
|
265
|
+
<Stack gap="xs">
|
|
266
|
+
<Title order={2}>Coverage</Title>
|
|
267
|
+
<Text component="div">
|
|
268
|
+
{counts.demonstrated} of {plural(total, 'behavior', 'behaviors')} demonstrated with a recording of the current spec.
|
|
269
|
+
</Text>
|
|
270
|
+
<List listStyleType="none" spacing="xs">
|
|
271
|
+
{COVERAGE_ORDER.filter((state) => state === 'demonstrated' || counts[state] > 0).map((state) => (
|
|
272
|
+
<List.Item key={state} data-coverage={state}>
|
|
273
|
+
<Group gap="sm" wrap="wrap">
|
|
274
|
+
<CoverageBadge state={state} />
|
|
275
|
+
<Text span inherit fw={600}>
|
|
276
|
+
{counts[state]}
|
|
277
|
+
</Text>
|
|
278
|
+
<Text span inherit c="dimmed">
|
|
279
|
+
{COVERAGE[state].meaning}
|
|
280
|
+
</Text>
|
|
281
|
+
</Group>
|
|
282
|
+
</List.Item>
|
|
283
|
+
))}
|
|
284
|
+
</List>
|
|
285
|
+
<Text component="div" c="dimmed">
|
|
286
|
+
A recording shows one run. It is illustrative evidence and does not mean the behavior is live or ready for production.
|
|
287
|
+
</Text>
|
|
288
|
+
</Stack>
|
|
289
|
+
</Paper>
|
|
290
|
+
);
|
|
291
|
+
}
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
import type { LayerHeader, LayerRow } from '@crvouga/atlas/reports/evidence-types';
|
|
2
|
+
import type { CheckSlot } from '@crvouga/atlas/reports/evidence-view';
|
|
3
|
+
import type { EvidenceResult } from '@crvouga/atlas/reports/layer-merge';
|
|
4
|
+
import { expect, test } from 'bun:test';
|
|
5
|
+
|
|
6
|
+
import { behaviorOf, compareRevisions, coverageCounts, isRepresentative, scenarioBehaviors, type RevisionSource } from './behavior';
|
|
7
|
+
import { exampleIndexFor, featureModel, fillExample, exampleRows } from './gherkin';
|
|
8
|
+
|
|
9
|
+
const FEATURE = featureModel(`Feature: Lists
|
|
10
|
+
@demo
|
|
11
|
+
Scenario: Add
|
|
12
|
+
When adding tea
|
|
13
|
+
|
|
14
|
+
@parked
|
|
15
|
+
Scenario: Archive
|
|
16
|
+
When archiving
|
|
17
|
+
|
|
18
|
+
Scenario Outline: Several
|
|
19
|
+
When adding "<item>" for <who>
|
|
20
|
+
Examples:
|
|
21
|
+
| item | who |
|
|
22
|
+
| Tea | Ann |
|
|
23
|
+
| Milk | Bo |
|
|
24
|
+
`)!;
|
|
25
|
+
const [ADD, ARCHIVE, SEVERAL] = FEATURE.rules[0]!.scenarios;
|
|
26
|
+
|
|
27
|
+
const header = (layer: string, environment: string | null = 'desktop-web'): LayerHeader => ({
|
|
28
|
+
id: layer, backend: 'b', producer: 'p', packageDir: null, recordedAtMs: 1, sha: null, branch: null, environment, runUrl: null, parameters: {},
|
|
29
|
+
recording: null, live: false, counts: { passed: 0, failed: 0, running: 0, 'not-run': 0 }, execution: layer
|
|
30
|
+
});
|
|
31
|
+
function result(layer: string, row: Partial<LayerRow>, environment: string | null = 'desktop-web'): EvidenceResult {
|
|
32
|
+
return {
|
|
33
|
+
key: layer, slot: 's', layer: header(layer, environment), parameters: {}, warning: null,
|
|
34
|
+
row: { kind: 'behavior', id: 'x', title: 'Lists: Add', status: 'passed', specPath: 'specs/a.feature', detail: '', measurements: [], testId: null, variant: {}, durationMs: null, ...row }
|
|
35
|
+
};
|
|
36
|
+
}
|
|
37
|
+
const slot = (status: CheckSlot['status'], outcomes: EvidenceResult[], title = 'Lists: Add'): CheckSlot => ({
|
|
38
|
+
key: title, id: title, title, specPath: 'specs/a.feature', status, outcomes, diagnostics: [], latestAtMs: 1, live: false
|
|
39
|
+
});
|
|
40
|
+
const VIDEO = { attachments: 1, steps: 0, annotations: 0, attempts: 0, links: 0, tags: [], labels: {}, roles: ['video'], videos: [{ name: 'a.webm', owner: 'check' as const, recording: 'sha256:1', available: true }] };
|
|
41
|
+
const revisions = (current: string | null, recorded: Record<string, string> = {}): RevisionSource => ({ current, recorded: (id) => recorded[id] });
|
|
42
|
+
const stateOf = (s: CheckSlot | null, rev = revisions('f1', { l1: 'f1', l2: 'f0' }), scenario = ADD!) => behaviorOf(scenario, s, null, rev, 'k').state;
|
|
43
|
+
|
|
44
|
+
test('revisions compare only when both are known', () => {
|
|
45
|
+
expect(compareRevisions('a', 'a').match).toBe('same');
|
|
46
|
+
expect(compareRevisions('a', 'b').match).toBe('different');
|
|
47
|
+
expect(compareRevisions('a', undefined).match).toBe('unknown');
|
|
48
|
+
expect(compareRevisions(null, 'a').match).toBe('unknown');
|
|
49
|
+
});
|
|
50
|
+
|
|
51
|
+
test('only a passing result with a playable recording of the current promise is demonstrated', () => {
|
|
52
|
+
expect(stateOf(slot('passed', [result('l1', { evidence: VIDEO })]))).toBe('demonstrated');
|
|
53
|
+
// The same recording of an older revision is not proof of the new promise.
|
|
54
|
+
expect(stateOf(slot('passed', [result('l2', { evidence: VIDEO })]))).toBe('outdated-recording');
|
|
55
|
+
// An unplayable reference is a gap, and a pass with nothing to watch says so.
|
|
56
|
+
expect(stateOf(slot('passed', [result('l1', { evidence: { ...VIDEO, videos: [{ ...VIDEO.videos[0]!, available: false }] } })]))).toBe('recording-missing');
|
|
57
|
+
expect(stateOf(slot('passed', [result('l1', { recording: { state: 'skipped', reason: 'too-large' } })]))).toBe('recording-missing');
|
|
58
|
+
expect(stateOf(slot('passed', [result('l1', {})]))).toBe('no-recording');
|
|
59
|
+
expect(stateOf(slot('passed', [result('l1', {}, 'api')]))).toBe('api-only');
|
|
60
|
+
// A video on the API does not turn it into API-only evidence: it can be watched.
|
|
61
|
+
expect(stateOf(slot('passed', [result('l1', { evidence: VIDEO }, 'api')]))).toBe('demonstrated');
|
|
62
|
+
});
|
|
63
|
+
|
|
64
|
+
test('failed, running, skipped, parked and unexecuted stay distinct and never read as passes', () => {
|
|
65
|
+
expect(stateOf(slot('failed', [result('l1', { status: 'failed', evidence: VIDEO })]))).toBe('failed');
|
|
66
|
+
expect(stateOf(slot('running', [result('l1', { status: 'running' })]))).toBe('running');
|
|
67
|
+
expect(stateOf(slot('not-run', [result('l1', { status: 'not-run' })]))).toBe('skipped');
|
|
68
|
+
expect(stateOf(null, undefined, ARCHIVE)).toBe('parked');
|
|
69
|
+
expect(stateOf(null)).toBe('unexecuted');
|
|
70
|
+
// One environment passing with a clip does not hide another that failed.
|
|
71
|
+
expect(stateOf(slot('failed', [result('l1', { evidence: VIDEO }), result('l3', { status: 'failed' })]))).toBe('failed');
|
|
72
|
+
});
|
|
73
|
+
|
|
74
|
+
test('a demo tag marks the scenario as the representative one', () => {
|
|
75
|
+
expect(isRepresentative(['#Demo'])).toBe(true);
|
|
76
|
+
expect(isRepresentative(['smoke'])).toBe(false);
|
|
77
|
+
expect(behaviorOf(ADD!, null, null, revisions(null), 'k').representative).toBe(true);
|
|
78
|
+
expect(behaviorOf(ARCHIVE!, null, null, revisions(null), 'k').representative).toBe(false);
|
|
79
|
+
});
|
|
80
|
+
|
|
81
|
+
test('an outline is one behavior per example, never the template as well', () => {
|
|
82
|
+
const tea = slot('passed', [result('l1', {})], 'Lists: Several — Tea');
|
|
83
|
+
const behaviors = scenarioBehaviors(SEVERAL!, [tea], 'Lists', revisions('f1'));
|
|
84
|
+
expect(behaviors.map((behavior) => [behavior.example, behavior.state])).toEqual([
|
|
85
|
+
['Tea', 'no-recording'],
|
|
86
|
+
['Milk, Bo', 'unexecuted']
|
|
87
|
+
]);
|
|
88
|
+
expect(coverageCounts(behaviors).unexecuted).toBe(1);
|
|
89
|
+
// With nothing recorded there are still only the two examples.
|
|
90
|
+
expect(scenarioBehaviors(SEVERAL!, [], 'Lists', revisions(null))).toHaveLength(2);
|
|
91
|
+
});
|
|
92
|
+
|
|
93
|
+
test('a recorded example is matched to its row by cells or column pairs, and its values are filled in', () => {
|
|
94
|
+
expect(exampleIndexFor(SEVERAL!, 'Milk')).toBe(1);
|
|
95
|
+
expect(exampleIndexFor(SEVERAL!, 'item=Tea, who=Ann')).toBe(0);
|
|
96
|
+
expect(exampleIndexFor(SEVERAL!, 'Unrelated wording')).toBe(-1);
|
|
97
|
+
const filled = fillExample(SEVERAL!.steps, exampleRows(SEVERAL!)[1]!);
|
|
98
|
+
expect(filled[0]!.text).toBe('adding "Milk" for Bo');
|
|
99
|
+
});
|
|
@@ -0,0 +1,268 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @module Behavior coverage
|
|
3
|
+
*
|
|
4
|
+
* Reads a feature the way a product reader does: each scenario (or each concrete example of an outline) is one
|
|
5
|
+
* behavior, and each behavior is in exactly one coverage state. The states keep apart what a green count would blur:
|
|
6
|
+
* a passing test with a playable recording of the current promise is `demonstrated`; a passing test with no
|
|
7
|
+
* recording, with a recording of an older revision of the spec, or only in API environments is not. Nothing here
|
|
8
|
+
* decides an outcome: statuses and winners come from the combined report, and a state is a reading of them.
|
|
9
|
+
* A missing fingerprint, recording or result is unknown or missing, never a pass.
|
|
10
|
+
*/
|
|
11
|
+
import type { CheckSlot } from '@crvouga/atlas/reports/evidence-view';
|
|
12
|
+
import type { EvidenceResult } from '@crvouga/atlas/reports/layer-merge';
|
|
13
|
+
|
|
14
|
+
import { exampleIndexFor, exampleRows, scenarioLabel, type FeatureScenario } from './gherkin';
|
|
15
|
+
|
|
16
|
+
/** Where a behavior stands, from the most reassuring to the least. */
|
|
17
|
+
export type CoverageState =
|
|
18
|
+
| 'demonstrated'
|
|
19
|
+
| 'outdated-recording'
|
|
20
|
+
| 'recording-missing'
|
|
21
|
+
| 'no-recording'
|
|
22
|
+
| 'api-only'
|
|
23
|
+
| 'failed'
|
|
24
|
+
| 'running'
|
|
25
|
+
| 'skipped'
|
|
26
|
+
| 'parked'
|
|
27
|
+
| 'unexecuted';
|
|
28
|
+
|
|
29
|
+
/** What each state is called and why, in the order a summary lists them. */
|
|
30
|
+
export const COVERAGE: Readonly<Record<CoverageState, { readonly label: string; readonly meaning: string; readonly color: string }>> = {
|
|
31
|
+
demonstrated: {
|
|
32
|
+
label: 'Demonstrated',
|
|
33
|
+
meaning: 'A passing result with a recording of the current promise.',
|
|
34
|
+
color: 'green'
|
|
35
|
+
},
|
|
36
|
+
'outdated-recording': {
|
|
37
|
+
label: 'Recording is older',
|
|
38
|
+
meaning: 'A passing result, but the recording is of an earlier revision of the spec.',
|
|
39
|
+
color: 'orange'
|
|
40
|
+
},
|
|
41
|
+
'recording-missing': {
|
|
42
|
+
label: 'Recording missing',
|
|
43
|
+
meaning: 'A passing result whose expected recording was not published or cannot be played.',
|
|
44
|
+
color: 'orange'
|
|
45
|
+
},
|
|
46
|
+
'no-recording': {
|
|
47
|
+
label: 'No recording',
|
|
48
|
+
meaning: 'A passing result with nothing to watch.',
|
|
49
|
+
color: 'gray'
|
|
50
|
+
},
|
|
51
|
+
'api-only': {
|
|
52
|
+
label: 'API only',
|
|
53
|
+
meaning: 'A passing result recorded only through the API, so there is no screen to watch.',
|
|
54
|
+
color: 'gray'
|
|
55
|
+
},
|
|
56
|
+
failed: {
|
|
57
|
+
label: 'Failed',
|
|
58
|
+
meaning: 'The latest result failed.',
|
|
59
|
+
color: 'red'
|
|
60
|
+
},
|
|
61
|
+
running: {
|
|
62
|
+
label: 'Running',
|
|
63
|
+
meaning: 'A run is checking it now.',
|
|
64
|
+
color: 'blue'
|
|
65
|
+
},
|
|
66
|
+
skipped: {
|
|
67
|
+
label: 'Skipped',
|
|
68
|
+
meaning: 'It was reported as skipped or not run.',
|
|
69
|
+
color: 'gray'
|
|
70
|
+
},
|
|
71
|
+
parked: {
|
|
72
|
+
label: 'Parked',
|
|
73
|
+
meaning: 'The scenario is tagged as parked or work in progress, and has no result.',
|
|
74
|
+
color: 'gray'
|
|
75
|
+
},
|
|
76
|
+
unexecuted: {
|
|
77
|
+
label: 'Not executed',
|
|
78
|
+
meaning: 'No test reported a result for it.',
|
|
79
|
+
color: 'gray'
|
|
80
|
+
}
|
|
81
|
+
};
|
|
82
|
+
|
|
83
|
+
/** The order a coverage summary lists states in. */
|
|
84
|
+
export const COVERAGE_ORDER = Object.keys(COVERAGE) as CoverageState[];
|
|
85
|
+
|
|
86
|
+
/** How the spec the evidence was recorded against compares with the spec now. */
|
|
87
|
+
export type Revision = {
|
|
88
|
+
/** Fingerprint of the spec now (the promise). Null when the outline has none. */
|
|
89
|
+
readonly promise: string | null;
|
|
90
|
+
/** Fingerprint of the spec when the evidence was recorded. Null when the layer did not record one. */
|
|
91
|
+
readonly evidence: string | null;
|
|
92
|
+
/** `same`, `different`, or `unknown` when either side is missing. */
|
|
93
|
+
readonly match: 'same' | 'different' | 'unknown';
|
|
94
|
+
};
|
|
95
|
+
|
|
96
|
+
/** Compares two fingerprints: unknown unless both are there. */
|
|
97
|
+
export function compareRevisions(promise: string | null | undefined, evidence: string | null | undefined): Revision {
|
|
98
|
+
const now = promise || null;
|
|
99
|
+
const then = evidence || null;
|
|
100
|
+
return {
|
|
101
|
+
promise: now,
|
|
102
|
+
evidence: then,
|
|
103
|
+
match: now === null || then === null ? 'unknown' : now === then ? 'same' : 'different'
|
|
104
|
+
};
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
/** The tags that mark a scenario as deliberately not running, and the state each gives it. */
|
|
108
|
+
const TAG_STATE: Readonly<Record<string, CoverageState>> = {
|
|
109
|
+
parked: 'parked',
|
|
110
|
+
wip: 'parked',
|
|
111
|
+
pending: 'parked',
|
|
112
|
+
skip: 'skipped',
|
|
113
|
+
skipped: 'skipped',
|
|
114
|
+
ignore: 'skipped'
|
|
115
|
+
};
|
|
116
|
+
|
|
117
|
+
/** The tags that say a recording was picked to stand for a behavior. */
|
|
118
|
+
const REPRESENTATIVE = new Set(['demo', 'representative']);
|
|
119
|
+
|
|
120
|
+
/** Whether a recorded result or its scenario is tagged as the representative demo. */
|
|
121
|
+
export function isRepresentative(tags: readonly string[]): boolean {
|
|
122
|
+
return tags.some((tag) => REPRESENTATIVE.has(tag.replace(/^[#@]/, '').toLowerCase()));
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
/** A result's playable video references: those that name bytes. */
|
|
126
|
+
function playable(result: EvidenceResult): number {
|
|
127
|
+
return result.row.evidence?.videos?.filter((video) => video.available).length ?? 0;
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
/** Environments a reader cannot watch: a result recorded only through these has no screen. */
|
|
131
|
+
function isApi(environment: string | null): boolean {
|
|
132
|
+
return environment !== null && /^(api|http|rest|graphql|service|backend)(-|$)/.test(environment);
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
/** What is known about one behavior. */
|
|
136
|
+
export type Behavior = {
|
|
137
|
+
/** Stable within the page: the scenario and its example. */
|
|
138
|
+
readonly key: string;
|
|
139
|
+
readonly scenario: FeatureScenario;
|
|
140
|
+
/** The example label of an outline's row (what the producer named it), or null for a plain scenario. */
|
|
141
|
+
readonly example: string | null;
|
|
142
|
+
/** The recorded check, or null when nothing reported on it. */
|
|
143
|
+
readonly slot: CheckSlot | null;
|
|
144
|
+
readonly state: CoverageState;
|
|
145
|
+
/** The result to watch: the newest of those with a playable recording, preferring one of the current revision. */
|
|
146
|
+
readonly demo: EvidenceResult | null;
|
|
147
|
+
/** The result the revision line and provenance describe: the demo, else the newest. */
|
|
148
|
+
readonly subject: EvidenceResult | null;
|
|
149
|
+
readonly revision: Revision;
|
|
150
|
+
/** Whether the scenario or the recorded result is tagged as the representative demo. */
|
|
151
|
+
readonly representative: boolean;
|
|
152
|
+
};
|
|
153
|
+
|
|
154
|
+
/** What the page knows about the spec as it is now and as each layer recorded it. */
|
|
155
|
+
export type RevisionSource = {
|
|
156
|
+
/** The spec's fingerprint now. */
|
|
157
|
+
readonly current: string | null;
|
|
158
|
+
/** The fingerprint a layer recorded for this spec, if it did. */
|
|
159
|
+
readonly recorded: (layerId: string) => string | undefined;
|
|
160
|
+
};
|
|
161
|
+
|
|
162
|
+
/**
|
|
163
|
+
* The state of one behavior from its recorded check (or none). A failing, running or skipped result is that, whatever
|
|
164
|
+
* recordings exist. A pass is `demonstrated` only with a playable recording that is not known to be of another
|
|
165
|
+
* revision; a recording whose revision cannot be compared is still shown, with the revision stated as unknown.
|
|
166
|
+
*/
|
|
167
|
+
export function behaviorOf(
|
|
168
|
+
scenario: FeatureScenario,
|
|
169
|
+
slot: CheckSlot | null,
|
|
170
|
+
example: string | null,
|
|
171
|
+
revisions: RevisionSource,
|
|
172
|
+
key: string
|
|
173
|
+
): Behavior {
|
|
174
|
+
const outcomes = slot?.outcomes ?? [];
|
|
175
|
+
const revisionOf = (result: EvidenceResult | null) =>
|
|
176
|
+
compareRevisions(revisions.current, result ? revisions.recorded(result.layer.id) : undefined);
|
|
177
|
+
const withVideo = outcomes.filter((result) => result.row.status === 'passed' && playable(result) > 0);
|
|
178
|
+
const demo = withVideo.find((result) => revisionOf(result).match !== 'different') ?? withVideo[0] ?? null;
|
|
179
|
+
const subject = demo ?? outcomes[0] ?? null;
|
|
180
|
+
const revision = revisionOf(subject);
|
|
181
|
+
const representative =
|
|
182
|
+
isRepresentative(scenario.tags) ||
|
|
183
|
+
outcomes.some((result) => isRepresentative([...(result.row.evidence?.tags ?? []), ...(result.row.evidence?.artifactTags ?? [])]));
|
|
184
|
+
const base = {
|
|
185
|
+
key,
|
|
186
|
+
scenario,
|
|
187
|
+
example,
|
|
188
|
+
slot,
|
|
189
|
+
demo,
|
|
190
|
+
subject,
|
|
191
|
+
revision,
|
|
192
|
+
representative
|
|
193
|
+
};
|
|
194
|
+
if (!slot || outcomes.length === 0) {
|
|
195
|
+
const tagged = scenario.tags.map((tag) => TAG_STATE[tag.toLowerCase()]).find((state) => state !== undefined);
|
|
196
|
+
return { ...base, state: tagged ?? 'unexecuted' };
|
|
197
|
+
}
|
|
198
|
+
if (slot.status === 'failed') return { ...base, state: 'failed' };
|
|
199
|
+
if (slot.status === 'running') return { ...base, state: 'running' };
|
|
200
|
+
if (slot.status === 'not-run') return { ...base, state: 'skipped' };
|
|
201
|
+
if (demo)
|
|
202
|
+
return {
|
|
203
|
+
...base,
|
|
204
|
+
state: revision.match === 'different' ? 'outdated-recording' : 'demonstrated'
|
|
205
|
+
};
|
|
206
|
+
const passed = outcomes.filter((result) => result.row.status === 'passed');
|
|
207
|
+
// A recording was promised and is not here, or is referenced without bytes: that is a gap, not an API test.
|
|
208
|
+
if (
|
|
209
|
+
passed.some((result) => result.row.recording && !['uploaded', 'referenced', 'not-applicable'].includes(result.row.recording.state)) ||
|
|
210
|
+
passed.some((result) => (result.row.evidence?.videos?.length ?? 0) > 0)
|
|
211
|
+
) {
|
|
212
|
+
return { ...base, state: 'recording-missing' };
|
|
213
|
+
}
|
|
214
|
+
if (passed.length > 0 && passed.every((result) => isApi(result.layer.environment))) return { ...base, state: 'api-only' };
|
|
215
|
+
return { ...base, state: 'no-recording' };
|
|
216
|
+
}
|
|
217
|
+
|
|
218
|
+
/**
|
|
219
|
+
* Every behavior of a scenario. A plain scenario is one. An outline is one per example row, never the template as an
|
|
220
|
+
* extra: a recorded example is filed under the row it names; a row nothing reported on is `unexecuted` (or parked);
|
|
221
|
+
* a recorded example whose label names no row is kept as its own behavior and the unclaimed rows are counted against
|
|
222
|
+
* the recorded results that remain, so the total never exceeds the rows by inventing a template.
|
|
223
|
+
*/
|
|
224
|
+
export function scenarioBehaviors(
|
|
225
|
+
scenario: FeatureScenario,
|
|
226
|
+
slots: readonly CheckSlot[],
|
|
227
|
+
feature: string,
|
|
228
|
+
revisions: RevisionSource
|
|
229
|
+
): Behavior[] {
|
|
230
|
+
const rows = exampleRows(scenario);
|
|
231
|
+
if (rows.length === 0) {
|
|
232
|
+
// A scenario with no examples fills one slot per environment/parameter set, which the slot already groups.
|
|
233
|
+
const slot = slots[0] ?? null;
|
|
234
|
+
if (slots.length <= 1) return [behaviorOf(scenario, slot, null, revisions, scenario.name)];
|
|
235
|
+
return slots.map((each) => behaviorOf(scenario, each, null, revisions, each.key));
|
|
236
|
+
}
|
|
237
|
+
const claimed = new Map<number, CheckSlot[]>();
|
|
238
|
+
const loose: CheckSlot[] = [];
|
|
239
|
+
for (const slot of slots) {
|
|
240
|
+
const label = scenarioLabel(feature, slot.title).slice(scenario.name.length + 3);
|
|
241
|
+
const at = exampleIndexFor(scenario, label);
|
|
242
|
+
if (at < 0) loose.push(slot);
|
|
243
|
+
else claimed.set(at, [...(claimed.get(at) ?? []), slot]);
|
|
244
|
+
}
|
|
245
|
+
const behaviors: Behavior[] = [];
|
|
246
|
+
const unmatched = rows.map((_, at) => at).filter((at) => !claimed.has(at));
|
|
247
|
+
// Results whose wording matches no row stand in for the rows nothing claimed, in order, before any row is called unexecuted.
|
|
248
|
+
const spare = [...loose];
|
|
249
|
+
rows.forEach((entry, at) => {
|
|
250
|
+
const found = claimed.get(at) ?? (unmatched.includes(at) && spare.length ? [spare.shift()!] : []);
|
|
251
|
+
const label = found[0] ? scenarioLabel(feature, found[0].title).slice(scenario.name.length + 3) : entry.row.join(', ');
|
|
252
|
+
if (found.length === 0) behaviors.push(behaviorOf(scenario, null, label, revisions, `${scenario.name}:${at}`));
|
|
253
|
+
for (const slot of found) behaviors.push(behaviorOf(scenario, slot, label, revisions, slot.key));
|
|
254
|
+
});
|
|
255
|
+
for (const slot of spare)
|
|
256
|
+
behaviors.push(behaviorOf(scenario, slot, scenarioLabel(feature, slot.title).slice(scenario.name.length + 3), revisions, slot.key));
|
|
257
|
+
return behaviors;
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
/** Behaviors counted by state. Every state is present, so an absent one reads as zero, not as unknown. */
|
|
261
|
+
export type CoverageCounts = Readonly<Record<CoverageState, number>>;
|
|
262
|
+
|
|
263
|
+
/** Counts behaviors by coverage state; only `demonstrated` is a demonstrated pass. */
|
|
264
|
+
export function coverageCounts(behaviors: readonly Behavior[]): CoverageCounts {
|
|
265
|
+
const counts = Object.fromEntries(COVERAGE_ORDER.map((state) => [state, 0])) as Record<CoverageState, number>;
|
|
266
|
+
for (const behavior of behaviors) counts[behavior.state] += 1;
|
|
267
|
+
return counts;
|
|
268
|
+
}
|
|
@@ -61,12 +61,15 @@ export function ResultRow({
|
|
|
61
61
|
slot,
|
|
62
62
|
open,
|
|
63
63
|
onToggle,
|
|
64
|
+
badge,
|
|
64
65
|
children
|
|
65
66
|
}: {
|
|
66
67
|
label: string;
|
|
67
68
|
slot: CheckSlot | null;
|
|
68
69
|
open: boolean;
|
|
69
70
|
onToggle: () => void;
|
|
71
|
+
/** A short extra mark after the label, such as a coverage state. */
|
|
72
|
+
badge?: ReactNode;
|
|
70
73
|
children: ReactNode;
|
|
71
74
|
}) {
|
|
72
75
|
const now = useNow();
|
|
@@ -96,6 +99,7 @@ export function ResultRow({
|
|
|
96
99
|
</Flex>
|
|
97
100
|
)}
|
|
98
101
|
</Text>
|
|
102
|
+
{badge}
|
|
99
103
|
{slot?.live && (
|
|
100
104
|
<Text span inherit>
|
|
101
105
|
Running now
|
|
@@ -1,8 +1,11 @@
|
|
|
1
1
|
import type { CheckSlot, SpecEvidence } from '@crvouga/atlas/reports/evidence-view';
|
|
2
2
|
import { Alert, Box, Container, List, Stack, Text, Title } from '@mantine/core';
|
|
3
3
|
import { useMemo } from 'react';
|
|
4
|
-
import type
|
|
4
|
+
import { outlinePath, type SpecKind } from '../spec-kind';
|
|
5
5
|
import { CheckDetail, PlainSteps, ResultRow, type CheckSelection, type SelectCheck } from './check-detail';
|
|
6
|
+
import { behaviorOf, coverageCounts, scenarioBehaviors, type Behavior, type RevisionSource } from './behavior';
|
|
7
|
+
import { BehaviorPanel, CoverageBadge, CoverageSummary } from './behavior-panel';
|
|
8
|
+
import { useEvidence } from './evidence-context';
|
|
6
9
|
import { featureModel, scenarioLabel, slotsByScenario, type FeatureModel, type FeatureScenario } from './gherkin';
|
|
7
10
|
import { Bounded } from './marks';
|
|
8
11
|
import { SpecHeader } from './spec-header';
|
|
@@ -23,7 +26,10 @@ function parse(text: string): {
|
|
|
23
26
|
try {
|
|
24
27
|
return { feature: featureModel(text), error: null };
|
|
25
28
|
} catch (error) {
|
|
26
|
-
return {
|
|
29
|
+
return {
|
|
30
|
+
feature: null,
|
|
31
|
+
error: error instanceof Error ? error.message : String(error)
|
|
32
|
+
};
|
|
27
33
|
}
|
|
28
34
|
}
|
|
29
35
|
/** A feature read as its rules and scenarios, each scenario with what was recorded when it ran. */
|
|
@@ -35,43 +41,81 @@ export function FeaturePage({ file, text, kind, entry, selection, select }: Prop
|
|
|
35
41
|
() => (feature ? slotsByScenario(feature, slots) : { byScenario: new Map<string, CheckSlot[]>(), leftOver: slots }),
|
|
36
42
|
[feature, slots]
|
|
37
43
|
);
|
|
38
|
-
const
|
|
39
|
-
const
|
|
40
|
-
|
|
44
|
+
const { specs, evidence } = useEvidence();
|
|
45
|
+
const specPath = outlinePath(file);
|
|
46
|
+
const revisions = useMemo<RevisionSource>(() => {
|
|
47
|
+
const recorded = new Map(evidence.map((layer) => [layer.layer.id, layer.fingerprints[specPath]]));
|
|
48
|
+
return {
|
|
49
|
+
current: specs.find((spec) => spec.path === specPath)?.fingerprint ?? null,
|
|
50
|
+
recorded: (id) => recorded.get(id)
|
|
51
|
+
};
|
|
52
|
+
}, [specs, evidence, specPath]);
|
|
53
|
+
const behaviors = useMemo(() => {
|
|
54
|
+
const byScenario = new Map<FeatureScenario, Behavior[]>();
|
|
55
|
+
for (const scenario of feature?.rules.flatMap((rule) => rule.scenarios.filter((each) => !each.background)) ?? []) {
|
|
56
|
+
byScenario.set(scenario, scenarioBehaviors(scenario, filed.byScenario.get(scenario.name) ?? [], feature!.name, revisions));
|
|
57
|
+
}
|
|
58
|
+
// A result no scenario claims is still a behavior that was checked, so it is counted.
|
|
59
|
+
const stray: FeatureScenario = {
|
|
60
|
+
keyword: 'Scenario',
|
|
61
|
+
name: '',
|
|
62
|
+
description: '',
|
|
63
|
+
background: false,
|
|
64
|
+
tags: [],
|
|
65
|
+
steps: [],
|
|
66
|
+
examples: []
|
|
67
|
+
};
|
|
68
|
+
const other = filed.leftOver.map((slot) => behaviorOf({ ...stray, name: slot.title }, slot, null, revisions, slot.key));
|
|
69
|
+
return { byScenario, all: [...[...byScenario.values()].flat(), ...other] };
|
|
70
|
+
}, [feature, filed, revisions]);
|
|
71
|
+
const counts = useMemo(() => coverageCounts(behaviors.all), [behaviors]);
|
|
72
|
+
const planned = behaviors.all.length;
|
|
41
73
|
const toggle = (id: string) => select({ check: selection.check === id ? null : id, layer: null });
|
|
42
|
-
const rows = (scenario: FeatureScenario) =>
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
const id = `scenario:${scenario.name}`;
|
|
74
|
+
const rows = (scenario: FeatureScenario) =>
|
|
75
|
+
(behaviors.byScenario.get(scenario) ?? []).map((behavior) => {
|
|
76
|
+
if (runningOnly && behavior.slot?.status !== 'running') return null;
|
|
77
|
+
const { slot } = behavior;
|
|
78
|
+
const written = <PlainSteps steps={scenario.steps} examples={scenario.examples} />;
|
|
79
|
+
const id = slot ? slot.id : `scenario:${scenario.name}${behavior.example ? `:${behavior.example}` : ''}`;
|
|
80
|
+
const label = slot
|
|
81
|
+
? scenarioLabel(feature!.name, slot.title)
|
|
82
|
+
: behavior.example
|
|
83
|
+
? `${scenario.name} — ${behavior.example}`
|
|
84
|
+
: scenario.name;
|
|
48
85
|
return (
|
|
49
|
-
<ResultRow
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
{
|
|
86
|
+
<ResultRow
|
|
87
|
+
key={id}
|
|
88
|
+
label={label}
|
|
89
|
+
slot={slot}
|
|
90
|
+
badge={<CoverageBadge state={behavior.state} />}
|
|
91
|
+
open={selection.check === id}
|
|
92
|
+
onToggle={() => toggle(id)}
|
|
93
|
+
>
|
|
94
|
+
<BehaviorPanel
|
|
95
|
+
behavior={behavior}
|
|
96
|
+
start={selection.check === id && selection.layer ? 'inspect' : undefined}
|
|
97
|
+
inspect={
|
|
98
|
+
slot ? (
|
|
99
|
+
<CheckDetail
|
|
100
|
+
slot={slot}
|
|
101
|
+
layerId={selection.check === id ? selection.layer : undefined}
|
|
102
|
+
select={(patch) => select({ check: id, ...patch })}
|
|
103
|
+
reads="steps"
|
|
104
|
+
written={written}
|
|
105
|
+
/>
|
|
106
|
+
) : (
|
|
107
|
+
<>
|
|
108
|
+
<Text component="div" c="dimmed">
|
|
109
|
+
No completed test report for this scenario yet.
|
|
110
|
+
</Text>
|
|
111
|
+
{written}
|
|
112
|
+
</>
|
|
113
|
+
)
|
|
114
|
+
}
|
|
115
|
+
/>
|
|
54
116
|
</ResultRow>
|
|
55
117
|
);
|
|
56
|
-
}
|
|
57
|
-
return recorded.map((slot) => (
|
|
58
|
-
<ResultRow
|
|
59
|
-
key={slot.key}
|
|
60
|
-
label={scenarioLabel(feature!.name, slot.title)}
|
|
61
|
-
slot={slot}
|
|
62
|
-
open={selection.check === slot.id}
|
|
63
|
-
onToggle={() => toggle(slot.id)}
|
|
64
|
-
>
|
|
65
|
-
<CheckDetail
|
|
66
|
-
slot={slot}
|
|
67
|
-
layerId={selection.check === slot.id ? selection.layer : undefined}
|
|
68
|
-
select={(patch) => select({ check: slot.id, ...patch })}
|
|
69
|
-
reads="steps"
|
|
70
|
-
written={written}
|
|
71
|
-
/>
|
|
72
|
-
</ResultRow>
|
|
73
|
-
));
|
|
74
|
-
};
|
|
118
|
+
});
|
|
75
119
|
return (
|
|
76
120
|
<Container component="article" size="xl" w="100%" py="xl">
|
|
77
121
|
<SpecHeader
|
|
@@ -87,6 +131,7 @@ export function FeaturePage({ file, text, kind, entry, selection, select }: Prop
|
|
|
87
131
|
</Text>
|
|
88
132
|
)}
|
|
89
133
|
</SpecHeader>
|
|
134
|
+
<CoverageSummary counts={counts} total={behaviors.all.length} />
|
|
90
135
|
<SourceText text={text} file={file} withResults />
|
|
91
136
|
<Box component="section" id="spec-results" aria-label="Recorded results" tabIndex={-1}>
|
|
92
137
|
<TestFilter slots={slots} />
|
|
@@ -112,7 +157,8 @@ export function FeaturePage({ file, text, kind, entry, selection, select }: Prop
|
|
|
112
157
|
.map((scenario, at) => (
|
|
113
158
|
<Stack key={at} gap="sm" miw={0}>
|
|
114
159
|
<Text component="div" c="dimmed">
|
|
115
|
-
Before each scenario
|
|
160
|
+
Before each scenario
|
|
161
|
+
{scenario.name ? `: ${scenario.name}` : ''}
|
|
116
162
|
</Text>
|
|
117
163
|
<PlainSteps steps={scenario.steps} />
|
|
118
164
|
</Stack>
|
|
@@ -13,6 +13,8 @@ export type FeatureScenario = {
|
|
|
13
13
|
readonly name: string;
|
|
14
14
|
readonly description: string;
|
|
15
15
|
readonly background: boolean;
|
|
16
|
+
/** The scenario's own tags, without the `@` (a parked or skipped scenario says so here). */
|
|
17
|
+
readonly tags: readonly string[];
|
|
16
18
|
readonly steps: readonly FeatureStep[];
|
|
17
19
|
readonly examples: readonly FeatureExamples[];
|
|
18
20
|
};
|
|
@@ -50,13 +52,14 @@ export function featureModel(text: string): FeatureModel | null {
|
|
|
50
52
|
for (const entry of child.rule ? child.rule.children : [child]) {
|
|
51
53
|
const background = entry.background;
|
|
52
54
|
const scenario = entry.scenario;
|
|
53
|
-
if (background) group.scenarios.push({ keyword: background.keyword, name: background.name, description: prose(background.description), background: true, steps: steps(background.steps), examples: [] });
|
|
55
|
+
if (background) group.scenarios.push({ keyword: background.keyword, name: background.name, description: prose(background.description), background: true, tags: [], steps: steps(background.steps), examples: [] });
|
|
54
56
|
if (scenario) {
|
|
55
57
|
group.scenarios.push({
|
|
56
58
|
keyword: scenario.keyword,
|
|
57
59
|
name: scenario.name,
|
|
58
60
|
description: prose(scenario.description),
|
|
59
61
|
background: false,
|
|
62
|
+
tags: [...new Set([...feature.tags, ...(child.rule?.tags ?? []), ...scenario.tags].map((tag) => tag.name.replace(/^@/, '')))],
|
|
60
63
|
steps: steps(scenario.steps),
|
|
61
64
|
examples: scenario.examples.map((example) => ({ name: example.name, header: cells(example.tableHeader), rows: example.tableBody.map(cells) }))
|
|
62
65
|
});
|
|
@@ -96,3 +99,30 @@ export function slotsByScenario(feature: FeatureModel, slots: readonly CheckSlot
|
|
|
96
99
|
}
|
|
97
100
|
return { byScenario, leftOver };
|
|
98
101
|
}
|
|
102
|
+
|
|
103
|
+
/** The rows of an outline's examples, each with the names of its columns. */
|
|
104
|
+
export function exampleRows(scenario: Pick<FeatureScenario, 'examples'>): { readonly header: readonly string[]; readonly row: readonly string[] }[] {
|
|
105
|
+
return scenario.examples.flatMap((example) => example.rows.map((row) => ({ header: example.header, row })));
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
/** Whether a recorded example label (what follows `Scenario — `) names one row: by its cells, joined, or by `column=value` pairs. */
|
|
109
|
+
function namesRow(label: string, entry: { readonly header: readonly string[]; readonly row: readonly string[] }) {
|
|
110
|
+
const joined = [entry.row.join(', '), entry.row.join(' '), entry.row.join(' | '), entry.header.map((name, at) => `${name}=${entry.row[at] ?? ''}`).join(', ')];
|
|
111
|
+
return joined.includes(label) || entry.row.includes(label);
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
/** The index (in {@link exampleRows}) of the row a recorded label stands for, or -1 when the label names no single row (the producer worded it its own way, or two rows read alike). */
|
|
115
|
+
export function exampleIndexFor(scenario: Pick<FeatureScenario, 'examples'>, label: string): number {
|
|
116
|
+
const rows = exampleRows(scenario);
|
|
117
|
+
const found = rows.flatMap((entry, at) => (namesRow(label, entry) ? [at] : []));
|
|
118
|
+
return found.length === 1 ? found[0]! : -1;
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
/** The steps of a scenario with one example's values put in for its `<column>` placeholders, so the concrete inputs read as written. */
|
|
122
|
+
export function fillExample(steps: readonly FeatureStep[], entry: { readonly header: readonly string[]; readonly row: readonly string[] }): FeatureStep[] {
|
|
123
|
+
const fill = (text: string) => text.replace(/<([^<>]+)>/g, (whole, name: string) => {
|
|
124
|
+
const at = entry.header.indexOf(name);
|
|
125
|
+
return at < 0 ? whole : (entry.row[at] ?? whole);
|
|
126
|
+
});
|
|
127
|
+
return steps.map((step) => ({ ...step, text: fill(step.text), docString: step.docString === null ? null : fill(step.docString), table: step.table.map((row) => row.map(fill)) }));
|
|
128
|
+
}
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { addSummaries, NO_EVIDENCE, type SpecSummary } from '@crvouga/atlas/reports/evidence-view';
|
|
2
|
-
import { Box, Button, Group, NavLink, Stack, Text, Title } from '@mantine/core';
|
|
2
|
+
import { Box, Button, Group, NavLink, SegmentedControl, Stack, Text, TextInput, Title } from '@mantine/core';
|
|
3
3
|
import { Link, useSearch } from '@tanstack/react-router';
|
|
4
|
-
import { useMemo } from 'react';
|
|
4
|
+
import { useMemo, useState } from 'react';
|
|
5
5
|
import { folderOf, outlinePath, specFile, type SpecKind } from '../spec-kind';
|
|
6
6
|
import { useUiStore, type ResultFilter } from '../state/ui-store';
|
|
7
7
|
import { useEvidence } from './evidence-context';
|
|
@@ -17,30 +17,42 @@ const FILTERS: readonly {
|
|
|
17
17
|
{ id: 'failed', label: 'Failing', keeps: (summary) => summary.counts.failed > 0 },
|
|
18
18
|
{ id: 'none', label: 'None', keeps: (summary) => summary.status === 'none' }
|
|
19
19
|
];
|
|
20
|
+
const NO_DOMAIN = 'No domain declared';
|
|
20
21
|
/** Every spec of one kind, grouped by folder, each with what the combined report holds for it. */
|
|
21
22
|
export function SpecIndex({ kind }: { kind: SpecKind }) {
|
|
22
23
|
const evidence = useEvidence();
|
|
23
24
|
const search = useSearch({ strict: false });
|
|
24
25
|
const filter = useUiStore((s) => s.resultFilter);
|
|
25
26
|
const setFilter = useUiStore((s) => s.setResultFilter);
|
|
27
|
+
const [query, setQuery] = useState('');
|
|
28
|
+
const [grouping, setGrouping] = useState<'folder' | 'domain'>('folder');
|
|
26
29
|
const open = typeof search.spec === 'string' ? specFile(search.spec) : '';
|
|
30
|
+
// The domain a spec declares in its metadata is used as given; a spec that declares none is not given one.
|
|
31
|
+
const domains = useMemo(() => new Map(evidence.specs.map((spec) => [spec.path, spec.domain])), [evidence.specs]);
|
|
27
32
|
const rows = useMemo(
|
|
28
33
|
() =>
|
|
29
34
|
evidence.files.filter(kind.accepts).map((file) => ({
|
|
30
35
|
file,
|
|
31
36
|
title: evidence.titleOf(file),
|
|
37
|
+
domain: domains.get(outlinePath(file)) || '',
|
|
32
38
|
summary: evidence.view.specs.get(outlinePath(file))?.summary ?? NO_EVIDENCE
|
|
33
39
|
})),
|
|
34
40
|
// `titleOf` reads the outline the view was built from.
|
|
35
|
-
[evidence.files, evidence.view, kind]
|
|
41
|
+
[evidence.files, evidence.view, kind, domains]
|
|
36
42
|
);
|
|
37
43
|
const keeps = FILTERS.find((entry) => entry.id === filter)!.keeps;
|
|
38
|
-
const
|
|
44
|
+
const needle = query.trim().toLowerCase();
|
|
45
|
+
const visible = rows.filter(
|
|
46
|
+
(row) => keeps(row.summary) && (!needle || `${row.title} ${row.file} ${row.domain}`.toLowerCase().includes(needle))
|
|
47
|
+
);
|
|
39
48
|
const groups = useMemo(() => {
|
|
40
|
-
const
|
|
41
|
-
for (const row of visible)
|
|
42
|
-
|
|
43
|
-
|
|
49
|
+
const byGroup = new Map<string, typeof visible>();
|
|
50
|
+
for (const row of visible) {
|
|
51
|
+
const group = grouping === 'domain' ? row.domain || NO_DOMAIN : folderOf(row.file);
|
|
52
|
+
byGroup.set(group, [...(byGroup.get(group) ?? []), row]);
|
|
53
|
+
}
|
|
54
|
+
return [...byGroup];
|
|
55
|
+
}, [visible, grouping]);
|
|
44
56
|
return (
|
|
45
57
|
<Box aria-label={`${kind.label} ${kind.many}`} component="nav" p="md">
|
|
46
58
|
<Group role="group" aria-label="Show" gap="xs" mb="md">
|
|
@@ -59,19 +71,40 @@ export function SpecIndex({ kind }: { kind: SpecKind }) {
|
|
|
59
71
|
</Button>
|
|
60
72
|
))}
|
|
61
73
|
</Group>
|
|
74
|
+
<Group gap="md" align="end" mb="md">
|
|
75
|
+
<TextInput
|
|
76
|
+
label={`Search ${kind.many}`}
|
|
77
|
+
type="search"
|
|
78
|
+
value={query}
|
|
79
|
+
onChange={(event) => setQuery(event.currentTarget.value)}
|
|
80
|
+
flex={1}
|
|
81
|
+
miw="12rem"
|
|
82
|
+
/>
|
|
83
|
+
<SegmentedControl
|
|
84
|
+
aria-label="Group by"
|
|
85
|
+
value={grouping}
|
|
86
|
+
onChange={(value) => setGrouping(value === 'domain' ? 'domain' : 'folder')}
|
|
87
|
+
data={[
|
|
88
|
+
{ label: 'Folder', value: 'folder' },
|
|
89
|
+
{ label: 'Domain', value: 'domain' }
|
|
90
|
+
]}
|
|
91
|
+
/>
|
|
92
|
+
</Group>
|
|
62
93
|
{evidence.filesPending && <Text component="div">Loading…</Text>}
|
|
63
94
|
{evidence.filesError && <Text component="div">The file list could not be loaded.</Text>}
|
|
64
95
|
{!evidence.filesPending && !evidence.filesError && visible.length === 0 && (
|
|
65
96
|
<Text component="div">
|
|
66
97
|
{rows.length === 0
|
|
67
98
|
? `No ${kind.many} in this spec set.`
|
|
68
|
-
:
|
|
69
|
-
? `No
|
|
70
|
-
: filter === '
|
|
71
|
-
? `No
|
|
72
|
-
: filter === '
|
|
73
|
-
? `No ${kind.many}
|
|
74
|
-
:
|
|
99
|
+
: needle
|
|
100
|
+
? `No ${kind.many} match "${query.trim()}".`
|
|
101
|
+
: filter === 'running'
|
|
102
|
+
? `No running ${kind.many}.`
|
|
103
|
+
: filter === 'failed'
|
|
104
|
+
? `No failing ${kind.many}.`
|
|
105
|
+
: filter === 'results'
|
|
106
|
+
? `No ${kind.many} with results yet.`
|
|
107
|
+
: `Every ${kind.one} has results.`}
|
|
75
108
|
</Text>
|
|
76
109
|
)}
|
|
77
110
|
{groups.map(([folder, entries]) => (
|