@crvouga/atlas 1.6.0 → 1.8.0

package/docs/reference/reports.md

@@ -2608,6 +2608,22 @@ export type EvidenceMeasurement = {
   readonly unit: string;
   /** The allowance for `value`, in the same unit, when there is one. */
   readonly limit?: number;
+  /** What the number is, in an open vocabulary. `product-latency` is a user-visible response time between `from` and `to`; absent is just a number. */
+  readonly kind?: string;
+  /** The event the measured interval starts at. */
+  readonly from?: string;
+  /** The event the measured interval ends at. */
+  readonly to?: string;
+  /** The interval the name claims, when the measured one is only part of it. */
+  readonly intended?: { readonly from?: string; readonly to?: string };
+  /** How it was measured. */
+  readonly source?: string;
+  /** `observed` (default), `timeout`, `failed`; otherwise `value` is how long the observation waited, at least. */
+  readonly outcome?: string;
+  /** The version or method of the measurement. */
+  readonly instrumentation?: string;
+  /** What the observation is comparable with: `device`, `build`. */
+  readonly labels?: Readonly<Record<string, string>>;
 };
 ```
 
@@ -4073,6 +4089,335 @@ export function splitViewerTestId(
 
 The inverse of {@link viewerTestId}: splits at the first `/`; null when either side is not a safe key segment.
 
+### Product latency
+
+Source: `packages/reports/src/latency.ts`. Import: `@crvouga/atlas/reports/latency`
+
+Tells how quickly a product responded apart from how long a test took. A test's `durationMs` includes fixture setup,
+deliberate waits, demo pacing, retries and cleanup, so it says nothing about a user's wait. Product latency is only
+what a producer measured and named as such: a measurement of `kind` `product-latency` between two named events
+(`from`, `to`), optionally with a budget (`limit`), an outcome (`observed`, `timeout`, `failed`), the instrumentation,
+and labels (`device`, `build`) in its `labels`. Pure functions, safe in a browser.
+
+The rules, each of which keeps a number from looking healthier or more certain than the evidence is:
+
+- **Only named measurements are latency.** A check's duration, a video's length, a recorded step and a number with no
+  `kind` are never product latency. No latency recorded is unknown, not fast.
+- **Boundaries are part of the claim.** Without both `from` and `to`, nothing is claimed from the numbers. When
+  `intended` names a longer interval than the one measured (speech end to TTS dispatch, claimed as microphone to
+  audible response) the measurement is partial and says so.
+- **Percentiles need samples.** p50 needs {@link MIN_P50_SAMPLES} observations and p95 {@link MIN_P95_SAMPLES}; below
+  that they are withheld with the reason, never shown as if supported.
+- **A timeout or failed observation is a sample.** Its value is how long it waited, so a lower bound: it counts
+  in the sample, in the percentiles and as a budget breach, and the summary says the figures are lower bounds.
+- **Comparisons are of the same metric.** Different boundaries are not compared; a change of instrumentation,
+  environment, budget or other condition between the two sides is disclosed beside the change.
+
+It decides nothing: a latency, a budget breach and a comparison never change a status, a slot or a winner.
+
+See docs/rich-evidence.md#product-latency
+
+#### `PRODUCT_LATENCY`
+
+```ts
+export const PRODUCT_LATENCY = …
+```
+
+The `kind` of a measurement that is a user-visible product latency.
+
+#### `MIN_P50_SAMPLES`
+
+```ts
+export const MIN_P50_SAMPLES = …
+```
+
+Observations needed before a p50 is shown.
+
+#### `MIN_P95_SAMPLES`
+
+```ts
+export const MIN_P95_SAMPLES = …
+```
+
+Observations needed before a p95 is shown.
+
+#### `isProductLatency`
+
+```ts
+export function isProductLatency(measurement: Pick<EvidenceMeasurement, 'kind'>): boolean
+```
+
+Whether a measurement is a product latency: only the producer's `kind` makes it one. A name, a unit or a value never does.
+
+#### `productLatency`
+
+```ts
+export function productLatency(measurements: readonly EvidenceMeasurement[] | undefined): readonly EvidenceMeasurement[] | undefined
+```
+
+The product-latency measurements of a check, or undefined when it has none (so a result with no latency carries
+nothing). Measurements that are not well formed (no finite value, no name) are left out.
+
+#### `BoundaryState`
+
+```ts
+export type BoundaryState = 'declared' | 'partial' | 'undeclared';
+```
+
+How well the ends of a measurement are known: `declared` (both named), `partial` (it stops short of, or starts after, what it claims) or `undeclared`.
+
+#### `Boundary`
+
+```ts
+export type Boundary = {
+  readonly state: BoundaryState;
+  readonly from?: string;
+  readonly to?: string;
+  /** The interval the measurement's name claims, when it is longer than what was measured. */
+  readonly intended?: string;
+  /** `submit pressed to confirmation visible`, `to confirmation visible (start not declared)`. */
+  readonly label: string;
+};
+```
+
+The ends of a measurement, in words, and whether they support a claim about the user's wait.
+
+#### `boundaryOf`
+
+```ts
+export function boundaryOf(measurement: Pick<EvidenceMeasurement, 'from' | 'to' | 'intended'>): Boundary
+```
+
+What a measurement spans. `undeclared` when either end is missing: the numbers are shown and no product claim is made.
+`partial` when `intended` names a different end than the measured one: the interval is the measured one, never the intended.
+
+#### `ObservationOutcome`
+
+```ts
+export type ObservationOutcome = 'observed' | 'timeout' | 'failed' | 'unclear';
+```
+
+What happened to one observation: `observed`, or it `timeout`ed or `failed`, or the producer said something Atlas cannot read (`unclear`).
+
+#### `LatencyEntry`
+
+```ts
+export type LatencyEntry = {
+  /** When it was recorded, epoch milliseconds. */
+  readonly at: number;
+  readonly environment: string | null;
+  readonly parameters: Readonly<Record<string, string>>;
+  readonly sha: string | null;
+  readonly layer: string;
+  readonly run?: string;
+  readonly latency?: readonly EvidenceMeasurement[];
+  readonly evidence?: { readonly labels: Readonly<Record<string, string>> };
+};
+```
+
+One recorded result, as much of it as this module reads. A `TestResultEntry` is one.
+
+#### `LatencyObservation`
+
+```ts
+export type LatencyObservation = {
+  readonly name: string;
+  readonly unit: string;
+  readonly value: number;
+  readonly limit?: number;
+  readonly outcome: ObservationOutcome;
+  readonly boundary: Boundary;
+  readonly source?: string;
+  readonly instrumentation?: string;
+  readonly cohort: Readonly<Record<string, string>>;
+  readonly at: number;
+  readonly layer: string;
+  readonly environment: string | null;
+  readonly parameters: Readonly<Record<string, string>>;
+  readonly labels: Readonly<Record<string, string>>;
+  readonly sha: string | null;
+};
+```
+
+One observation of a latency. Time units are converted to milliseconds.
+
+#### `latencyObservations`
+
+```ts
+export function latencyObservations(entries: readonly LatencyEntry[]): LatencyObservation[]
+```
+
+The observations in recorded results, in the order given. Entries with no latency contribute none.
+
+#### `LatencyGrouping`
+
+```ts
+export type LatencyGrouping = 'none' | 'device' | 'build' | 'window';
+```
+
+How observations are grouped: not at all, by `device`, by `build`, or by the UTC `window` (day) they were recorded in.
+
+#### `LATENCY_GROUPINGS`
+
+```ts
+export const LATENCY_GROUPINGS: readonly LatencyGrouping[]
+```
+
+The groupings, in the order a control offers them.
+
+#### `groupOf`
+
+```ts
+export function groupOf(observation: LatencyObservation, by: LatencyGrouping): string
+```
+
+The group an observation falls in. `device` is the measurement's cohort, else the result's label, parameter or
+environment; `build` is the cohort or label, else the commit. One that names nothing is `unknown device` or
+`unknown build`: it is never merged into a named group.
+
+#### `LatencyMetric`
+
+```ts
+export type LatencyMetric = {
+  /** Equal for observations of the same name, unit and boundaries. */
+  readonly key: string;
+  readonly name: string;
+  readonly unit: string;
+  readonly boundary: Boundary;
+};
+```
+
+A measured interval: the name, the unit and the ends. Observations of one metric are comparable; of two metrics they are not.
+
+#### `LatencySummary`
+
+```ts
+export type LatencySummary = {
+  readonly metric: LatencyMetric;
+  /** The group (a device, a build, a day, or all). */
+  readonly group: string;
+  /** Every observation, including those that timed out or failed. */
+  readonly samples: number;
+  /** Observations that completed. */
+  readonly observed: number;
+  readonly timeouts: number;
+  readonly failures: number;
+  /** Observations over their budget, plus every timeout or failure when a budget is set. */
+  readonly breaches: number;
+  /** The budget, from the newest observation that has one. */
+  readonly budget?: number;
+  /** The median, or undefined with the reason in `withheld`. */
+  readonly p50?: number;
+  readonly p95?: number;
+  /** Why a figure is not shown, in words. */
+  readonly withheld: readonly string[];
+  /** True when a timeout or failure's waiting time is in the figures: they are lower bounds. */
+  readonly lowerBound: boolean;
+  readonly max: number;
+  /** The newest observation's value. */
+  readonly last: number;
+  readonly lastAt: number;
+  /** Whether the numbers support a statement about the user's wait: false when the boundaries are not declared. */
+  readonly claim: boolean;
+  /** What the reader must not assume, in words: partial interval, missing boundaries, a single observation. */
+  readonly gaps: readonly string[];
+  readonly instrumentation: readonly string[];
+  readonly environments: readonly string[];
+  /** Other facts the observations differ by: cohort entries and parameters, as `key=value`. */
+  readonly conditions: readonly string[];
+  readonly budgets: readonly number[];
+};
+```
+
+What the observations of one metric in one group show, and what they cannot.
+
+#### `LatencyReview`
+
+```ts
+export type LatencyReview = {
+  readonly by: LatencyGrouping;
+  readonly metrics: readonly { readonly metric: LatencyMetric; readonly groups: readonly LatencySummary[] }[];
+  /** Results that recorded no product latency. They are unknown, not fast. */
+  readonly unmeasured: number;
+  /** Results that did. */
+  readonly measured: number;
+};
+```
+
+The metrics of a result set, each with its summary in every group, groups newest first.
+
+#### `latencyReview`
+
+```ts
+export function latencyReview(entries: readonly LatencyEntry[], by: LatencyGrouping = 'none'): LatencyReview
+```
+
+Reads recorded results (any order) into a review: for every metric, a summary in every group. A result with no
+product latency counts only in `unmeasured`. Metrics are listed by name; groups newest first.
+
+- `by`: How observations are grouped. Default `none`.
+
+**Example**
+
+```ts
+const review = latencyReview(history.history, 'build');
+review.metrics[0]?.groups[0]?.p95; // undefined until 20 observations of that build
+```
+
+#### `LatencyDelta`
+
+```ts
+export type LatencyDelta = {
+  readonly baseline: number;
+  readonly candidate: number;
+  /** Candidate minus baseline, in the metric's unit. Positive is slower. */
+  readonly absolute: number;
+  /** The absolute change over the baseline, or null when the baseline is zero. */
+  readonly relative: number | null;
+};
+```
+
+A change of one figure from the baseline to the candidate.
+
+#### `LatencyChange`
+
+```ts
+export type LatencyChange = {
+  /** Whether a change is stated at all. */
+  readonly comparable: boolean;
+  /** `p50` or `p95` when both sides support it; `last` compares the newest observation of each, which is not a percentile. */
+  readonly basis?: 'p50' | 'p95' | 'last';
+  readonly p50?: LatencyDelta;
+  readonly p95?: LatencyDelta;
+  readonly last?: LatencyDelta;
+  /** Why no change is stated. */
+  readonly reasons: readonly string[];
+  /** Differences in how the two sides were measured or run. A change should be read with them. */
+  readonly disclosures: readonly string[];
+};
+```
+
+What changed between two summaries, or why that cannot be said.
+
+#### `compareLatency`
+
+```ts
+export function compareLatency(baseline: LatencySummary, candidate: LatencySummary): LatencyChange
+```
+
+The change from a baseline to a candidate of the same metric. Different metrics (names, units or boundaries) are not
+compared and the reason is given. When both sides support p50 (and p95) the change is of those; otherwise only the
+newest observation of each is compared, labelled as such. Changes of instrumentation, environments, other
+conditions, budget and partial or lower-bound figures are listed in `disclosures`, never hidden.
+
+#### `formatLatency`
+
+```ts
+export function formatLatency(value: number | undefined, unit: string): string
+```
+
+`500 ms`, `1.2 s`: a latency in the largest unit that keeps it readable. Non-time units print as given.
+
 ### Environment axis of a layer
 
 Source: `packages/reports/src/layer-axes.ts`. Import: `@crvouga/atlas/reports/layer-axes`
@@ -8318,6 +8663,8 @@ export type TestResultEntry = {
   readonly detail: string;
   /** What it carried beyond its outcome (attachments, steps, retries, tags, labels), when anything. */
   readonly evidence?: EvidenceSummary;
+  /** Its product-latency measurements (`kind: product-latency`), when it recorded any. Separate from `durationMs`, which is how long the test took (setup, waits and pacing included). */
+  readonly latency?: readonly EvidenceMeasurement[];
 };
 ```
 

package/docs/reference/schema.md

@@ -742,6 +742,22 @@ export type LayerRun = z.output<typeof LayerRunSchema>;
 
 The execution of a layer, parsed from `LayerRunSchema`.
 
+#### `PRODUCT_LATENCY_KIND`
+
+```ts
+export const PRODUCT_LATENCY_KIND = …
+```
+
+The `kind` of a measurement that is a named, user-visible product latency (see docs/rich-evidence.md#product-latency).
+
+#### `MeasurementBoundarySchema`
+
+```ts
+export const MeasurementBoundarySchema = …
+```
+
+The two ends of a measured interval, named in the producer's words: `{ from: 'submit pressed', to: 'confirmation visible' }`.
+
 #### `MeasurementSchema`
 
 ```ts
@@ -751,6 +767,18 @@ export const MeasurementSchema = …
 A number a check measured, such as a latency or a size, with its unit and an optional allowance. A measurement over
 its `limit` is reported and never changes the check's status.
 
+A measurement of `kind` `product-latency` is how quickly the product responded to a user, between two named events.
+It is not the check's duration (which includes setup, waits and pacing) and a number with no `kind` is never read as
+product latency. All the fields after `limit` are optional and describe the measurement; none decides a status.
+
+#### `Measurement`
+
+```ts
+export type Measurement = z.output<typeof MeasurementSchema>;
+```
+
+A measurement as stored in a layer, parsed from `MeasurementSchema`.
+
 #### `ExtensionsSchema`
 
 ```ts

package/docs/rich-evidence.md

@@ -109,6 +109,49 @@ Every field is optional, and every value except the booleans is an open string.
 - **It decides nothing.** A scope never changes a status, a slot or a winner. A scope an older Atlas cannot read is
   dropped, and the attachment, check and layer stay.
 
+## Product latency
+
+A test's `durationMs` includes fixture setup, deliberate waits, demo pacing, retries and cleanup, so it says how long the
+test ran and nothing about how quickly the product responded. Name the numbers that do, as measurements of `kind`
+`product-latency` between two events:
+
+```ts
+measurements: [
+  { name: 'Fixture setup', value: 20_000, unit: 'ms' },              // just a number: never read as latency
+  {
+    name: 'Confirmation', value: 500, unit: 'ms', kind: 'product-latency',
+    from: 'submit pressed', to: 'confirmation visible',             // the boundaries are part of the claim
+    limit: 800,                                                     // the budget
+    source: 'app-instrumentation', instrumentation: 'spans-v2',
+    labels: { device: 'pixel-8', build: '4821' }                    // what it is comparable with
+  }
+]
+```
+
+The test page shows **Product latency** apart from the test statistics, whose durations are labelled as test execution
+time. Every field after `unit` is optional and none of them decides a status, a slot or a winner.
+
+- **Only named measurements are latency.** A check's duration, a video's length, a step and a number with no `kind` are
+  never product latency. A test that recorded none shows that its latency is unknown, not fast.
+- **Boundaries are part of the claim.** Without both `from` and `to` the numbers are shown but no percentile or product
+  claim is made, and the page says so. When the interval you measured is only part of what the name claims, say what it
+  claims in `intended` (`{ "to": "audible response" }` on a measurement that ends at `tts dispatch`) and the viewer labels
+  it a partial interval, never the complete one.
+- **Percentiles need samples.** p50 needs 5 observations and p95 needs 20; fewer shows `Withheld` with the count it has.
+  One observation shows no spread or confidence.
+- **A timeout is a sample.** `outcome: "timeout"` or `"failed"` (anything but `observed`) means `value` is how long the
+  observation waited. It stays in the sample count and the percentiles, counts as over budget whenever a `limit` is set,
+  and marks the figures as lower bounds, so a stuck product never looks healthy.
+- **Groups.** The viewer groups observations by `device`, `build` or the UTC day they were recorded, taking `device` and
+  `build` from the measurement's `labels`, else the result's labels, parameters, environment and commit. Each group has
+  its own sample count, p50/p95, budget and breaches. A group that names nothing is `unknown device` or `unknown build`.
+- **Baselines.** A group is compared with a chosen baseline by absolute and relative change of p50 (and p95), or of the
+  newest observation when percentiles are not supported on both sides. Measurements with different names, units or
+  boundaries are not compared. A change of `instrumentation`, environment, other labels or parameters, or budget between
+  the two sides is listed beside the change.
+- **Reading it from the API.** `GET /api/reports/tests/history` entries carry `latency` (their product-latency
+  measurements) next to `durationMs`. The pure functions are in `@crvouga/atlas/reports/latency`.
+
 ## Finding runs and artifacts
 
 The viewer's **Runs** and **Artifacts** tabs and the API answer one query, so a link means the same thing everywhere.

package/docs/ui.md

@@ -87,6 +87,29 @@ when Watch is on screen.
 - The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
   none are listed under "No domain declared"; the Tests view reports domain problems).
 
+## Product latency
+
+A test's duration and its product latency are different things and the viewer never mixes them (`explorer/ProductLatency.tsx`,
+`@crvouga/atlas/reports/latency`). Statistics and the trend are labelled test execution time. Product latency is read only from
+measurements named `product-latency`; it shows boundaries, group (device, build, day), samples, budget and breaches, and a
+baseline comparison that lists changes of instrumentation or conditions. Percentiles and product claims that the evidence
+cannot support are shown as `Withheld` with the reason, a partial interval is badged, timeouts count as samples and breaches,
+and a test with no latency says it is unknown, not fast.
+
+## Outcome summary
+
+A test or behavior detail states what the **selected** result says apart from what the other current sources say
+(`evidence/outcome.ts`, `outcome-summary.tsx`). The heading is the selected source's outcome first ("Selected result
+passed; another scoped source failed"), never the combined status; the combined status is labelled as the worst current
+result. Each other current source is one click away, and sources that share an environment are told apart by status,
+time, revision and their place among them. A failure leads with whether the behavior was reached (decided only by the
+recorded step `kind`: a `hook`, `setup`, `fixture`, `before*` or `given` step failing is "Behavior not verified: setup
+failed", `when`, `then`, `assertion` or `expect` is "Expectation not met", anything else is unknown), then the expected
+and observed values and the closest step and video, then the original output. The cause (product, test harness,
+environment) is unknown unless the producer declared it under the `atlas.failure` extension; it is never read from error
+wording. Records of one source whose outcome changed are listed oldest first, and a later pass is never called a fix.
+In the explorer the chosen source is the `source` search parameter (a layer id; unlike `layer` it does not pin a run).
+
 ## Icons and notation marks
 
 Use named imports from `@tabler/icons-react` for all UI glyphs. The icon package supplies SVG paths;
@@ -156,7 +179,7 @@ The narrowly scoped exceptions are:
 | --- | --- | --- |
 | Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
 | Project home | Tests/runs/artifacts tabs, a run's Artifacts (files of its layers, shared recordings, direct links), metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion |
-| Test detail | Current outcomes, statistics, duration graph, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
+| Test detail | Current outcomes, statistics, duration graph, product latency with budgets and baselines, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
 | Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
 | Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
 | Evidence | Steps, retries, annotations, facts, unknown extensions, safe downloads and previews | List, Badge, Accordion, Code, Image, native media |

package/llms-full.txt

@@ -1096,6 +1096,49 @@ Every field is optional, and every value except the booleans is an open string.
 - **It decides nothing.** A scope never changes a status, a slot or a winner. A scope an older Atlas cannot read is
   dropped, and the attachment, check and layer stay.
 
+## Product latency
+
+A test's `durationMs` includes fixture setup, deliberate waits, demo pacing, retries and cleanup, so it says how long the
+test ran and nothing about how quickly the product responded. Name the numbers that do, as measurements of `kind`
+`product-latency` between two events:
+
+```ts
+measurements: [
+  { name: 'Fixture setup', value: 20_000, unit: 'ms' },              // just a number: never read as latency
+  {
+    name: 'Confirmation', value: 500, unit: 'ms', kind: 'product-latency',
+    from: 'submit pressed', to: 'confirmation visible',             // the boundaries are part of the claim
+    limit: 800,                                                     // the budget
+    source: 'app-instrumentation', instrumentation: 'spans-v2',
+    labels: { device: 'pixel-8', build: '4821' }                    // what it is comparable with
+  }
+]
+```
+
+The test page shows **Product latency** apart from the test statistics, whose durations are labelled as test execution
+time. Every field after `unit` is optional and none of them decides a status, a slot or a winner.
+
+- **Only named measurements are latency.** A check's duration, a video's length, a step and a number with no `kind` are
+  never product latency. A test that recorded none shows that its latency is unknown, not fast.
+- **Boundaries are part of the claim.** Without both `from` and `to` the numbers are shown but no percentile or product
+  claim is made, and the page says so. When the interval you measured is only part of what the name claims, say what it
+  claims in `intended` (`{ "to": "audible response" }` on a measurement that ends at `tts dispatch`) and the viewer labels
+  it a partial interval, never the complete one.
+- **Percentiles need samples.** p50 needs 5 observations and p95 needs 20; fewer shows `Withheld` with the count it has.
+  One observation shows no spread or confidence.
+- **A timeout is a sample.** `outcome: "timeout"` or `"failed"` (anything but `observed`) means `value` is how long the
+  observation waited. It stays in the sample count and the percentiles, counts as over budget whenever a `limit` is set,
+  and marks the figures as lower bounds, so a stuck product never looks healthy.
+- **Groups.** The viewer groups observations by `device`, `build` or the UTC day they were recorded, taking `device` and
+  `build` from the measurement's `labels`, else the result's labels, parameters, environment and commit. Each group has
+  its own sample count, p50/p95, budget and breaches. A group that names nothing is `unknown device` or `unknown build`.
+- **Baselines.** A group is compared with a chosen baseline by absolute and relative change of p50 (and p95), or of the
+  newest observation when percentiles are not supported on both sides. Measurements with different names, units or
+  boundaries are not compared. A change of `instrumentation`, environment, other labels or parameters, or budget between
+  the two sides is listed beside the change.
+- **Reading it from the API.** `GET /api/reports/tests/history` entries carry `latency` (their product-latency
+  measurements) next to `durationMs`. The pure functions are in `@crvouga/atlas/reports/latency`.
+
 ## Finding runs and artifacts
 
 The viewer's **Runs** and **Artifacts** tabs and the API answer one query, so a link means the same thing everywhere.
@@ -2830,7 +2873,7 @@ carry any of these. Every one is optional.
 | `scope{}` | What the evidence does and does not show: surface, device kind, synthetic data, scripted dependencies, audio, pacing, what was asserted and its limits. Unknown when absent. See [Evidence scope](rich-evidence.md#evidence-scope). |
 | `links[]` | Where to read more: the CI run, the issue, the trace client. |
 | `extensions{}` | Structured data of your own or a tool's, under a name you own. |
-| `measurements[]` | Numbers, with units and an allowance (already part of the contract). |
+| `measurements[]` | Numbers, with units and an allowance (already part of the contract). A measurement of `kind` `product-latency` names a user-visible response time between two events; see [Product latency](rich-evidence.md#product-latency). |
 
 An attachment may carry its own `tags[]` (`demo`): shown as `#demo` and searchable; see
 [Tagging artifacts and demos](rich-evidence.md#tagging-artifacts-and-demos).
@@ -2900,6 +2943,23 @@ every kind of evidence and values from the future) and `layer-extensibility.test
 fields survive at every level, malformed evidence is dropped without losing the result, and a reader of the
 first version of the format still reads it all.
 
+### Saying what failed
+
+A failed check or step may declare the expectation it broke under the `atlas.failure` extension, so a reviewer sees
+it before the raw log. Every field is optional, and the declaration never changes the result.
+
+```json
+{
+  "name": "The list includes tea", "kind": "then", "status": "failed",
+  "extensions": { "atlas.failure": { "expected": "tea in the list", "actual": "the list is empty", "attribution": "product" } }
+}
+```
+
+`expected` and `actual` are shown as written. `attribution` is `product`, `test-harness` or `environment`; leave it out
+when you do not know, and the viewer says the cause is unknown. It is never inferred from the message. Whether setup
+failed before the behavior started comes from the failed step's `kind`: `hook`, `setup`, `fixture`, `before`,
+`beforeAll`, `beforeEach` or `given` mean preparation; `when`, `then`, `assertion` or `expect` mean the behavior.
+
 ## Attach evidence
 
 ### From the command line
@@ -3919,6 +3979,29 @@ when Watch is on screen.
 - The spec index searches titles, paths and declared domains, and can group by the domain from spec metadata (specs with
   none are listed under "No domain declared"; the Tests view reports domain problems).
 
+## Product latency
+
+A test's duration and its product latency are different things and the viewer never mixes them (`explorer/ProductLatency.tsx`,
+`@crvouga/atlas/reports/latency`). Statistics and the trend are labelled test execution time. Product latency is read only from
+measurements named `product-latency`; it shows boundaries, group (device, build, day), samples, budget and breaches, and a
+baseline comparison that lists changes of instrumentation or conditions. Percentiles and product claims that the evidence
+cannot support are shown as `Withheld` with the reason, a partial interval is badged, timeouts count as samples and breaches,
+and a test with no latency says it is unknown, not fast.
+
+## Outcome summary
+
+A test or behavior detail states what the **selected** result says apart from what the other current sources say
+(`evidence/outcome.ts`, `outcome-summary.tsx`). The heading is the selected source's outcome first ("Selected result
+passed; another scoped source failed"), never the combined status; the combined status is labelled as the worst current
+result. Each other current source is one click away, and sources that share an environment are told apart by status,
+time, revision and their place among them. A failure leads with whether the behavior was reached (decided only by the
+recorded step `kind`: a `hook`, `setup`, `fixture`, `before*` or `given` step failing is "Behavior not verified: setup
+failed", `when`, `then`, `assertion` or `expect` is "Expectation not met", anything else is unknown), then the expected
+and observed values and the closest step and video, then the original output. The cause (product, test harness,
+environment) is unknown unless the producer declared it under the `atlas.failure` extension; it is never read from error
+wording. Records of one source whose outcome changed are listed oldest first, and a later pass is never called a fix.
+In the explorer the chosen source is the `source` search parameter (a layer id; unlike `layer` it does not pin a run).
+
 ## Icons and notation marks
 
 Use named imports from `@tabler/icons-react` for all UI glyphs. The icon package supplies SVG paths;
@@ -3988,7 +4071,7 @@ The narrowly scoped exceptions are:
 | --- | --- | --- |
 | Shared shell | Project switching, spec navigation, run selection, connection status, diagnostics, mobile index | AppShell, Burger, NavLink, Menu, Popover, NativeSelect, Drawer |
 | Project home | Tests/runs/artifacts tabs, a run's Artifacts (files of its layers, shared recordings, direct links), metrics, search, status/environment/domain/artifact-tag filters, run and artifact facets with saved views, grouping, sorting, show more | Container, Tabs, SimpleGrid, Paper, TextInput, MultiSelect, Popover, Button, Accordion |
-| Test detail | Current outcomes, statistics, duration graph, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
+| Test detail | Current outcomes, statistics, duration graph, product latency with budgets and baselines, history, artifacts, recording; the `layer` address parameter pins it to one run's result, labelled "This run" against "Latest combined evidence" | Drawer, Table, Badge, Alert, Stage |
 | Specs | All six existing kinds, overview/index with title search and folder or declared-domain grouping, Gherkin rules/scenarios/examples with a coverage summary and Read / Watch / Inspect per behavior, Quint declarations/traces, DMN tables, source text | SpecHeader, Accordion, Tabs, Table, CodeHighlightTabs, ResultRow |
 | Statecharts | Graph, screen/event/journey details, developer metadata, chart test results | Mantine cards/navigation and product compositions; graph exception above |
 | Evidence | Steps, retries, annotations, facts, unknown extensions, safe downloads and previews | List, Badge, Accordion, Code, Image, native media |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@crvouga/atlas",
-  "version": "1.6.0",
+  "version": "1.8.0",
   "description": "Atlas: specs of any kind, reported as layered evidence. Publish checks from any runner (JUnit, CTRF, a script), combine them across nodes, branches, environments and backends, gate CI on the result, and run statecharts against the real app.",
   "license": "MIT",
   "type": "module",