@crvouga/atlas 0.16.0 → 1.0.0

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog: @crvouga/atlas
 
+## 1.0.0 (2026-10-28)
+
+### ⚠️ Breaking changes
+
+- store all reports and user state in S3 only, with a GC policy set in the UI (#51) ([157ea67](https://github.com/crvouga/atlas/commit/157ea6734439f9d890b3de992aad72f4bca2d597))
+
+## 0.17.0 (2026-10-28)
+
+### Features
+
+- report recording completeness separately from test success ([a8dd9d4](https://github.com/crvouga/atlas/commit/a8dd9d4b12d0c9cfbb7cb94c48d571e4aa5050ef))
+
 ## 0.16.0 (2026-10-28)
 
 ### Features

package/README.md

@@ -18,7 +18,7 @@ backends, and tells you in one command whether the product still does what it wa
   versioned layer format with a golden fixture, and an `ObjectStore` interface any bucket fits.
 - **One node, many projects.** `atlas server` runs a Bun API and serves the Atlas client from the same host. Reports, specs, recordings and live progress are partitioned by project slug.
   Every laptop and CI job sends results to it as they finish (`atlas report publish`, or `atlas report stream` while
-  a suite runs), it keeps them on its own disk (or in S3, R2, MinIO), and every client sees the combined report
+  a suite runs), it keeps them in an S3-compatible bucket (AWS S3, R2, MinIO; required, nothing else is used), and every client sees the combined report
   update live, receiving only what changed ([the server](docs/server.md), [deploying](docs/deploying.md)).
 - **Fast and fail-closed.** Publishing is two small writes. A thousand layers of a hundred checks
   combine in about a second. An unreadable layer, a downed backend or a check still running is never
@@ -67,7 +67,7 @@ bun atlas          # starts the project-agnostic Atlas node at http://localhost:
 bun example:run    # runs tests and streams results, current steps and progress as they execute
 ```
 
-The server starts independently with data in `./atlas-data`; it never loads example specs or reports. `bun example:run` publishes the Todo example under `todo-app`. Select a project using the top-right picker, then use the spec dropdown breadcrumbs to navigate. Other projects set `ATLAS_SERVER_URL`, `ATLAS_PROJECT` (for example `checkout-service`) and their publish token. Every uploader talks to the node; storage credentials stay on the server.
+The server starts independently; it never loads example specs or reports. **It needs S3-compatible storage** (see [Required S3 storage](#required-s3-storage)): copy `.env.example` to `.env` and fill in the `ATLAS_S3_*` variables first, or it exits at once and lists every one that is missing. `bun example:run` publishes the Todo example under `todo-app`. Select a project using the top-right picker, then use the spec dropdown breadcrumbs to navigate. Other projects set `ATLAS_SERVER_URL`, `ATLAS_PROJECT` (for example `checkout-service`) and their publish token. Every uploader talks to the node; storage credentials stay on the server.
 
 Every Gherkin scenario runs, including sharing and renaming. Test reports show running spinners,
 live step logs and a Running status filter. Producers send heartbeats every
@@ -88,8 +88,11 @@ or package installation. [`latest` on GHCR](https://github.com/users/crvouga/pac
 the newest checked `main` image, independently of npm releases.
 
 1. Leave the start command unset. Add a long random secret as `ATLAS_TOKEN` (at least 12 characters;
-   use 48 or more) and set `RAILWAY_RUN_UID=0` for [Railway volume permissions](https://docs.railway.com/volumes#permissions).
-2. Attach a persistent volume at `/data` and keep one replica so reports and server settings survive updates.
+   use 48 or more).
+2. Add the five required storage variables (`ATLAS_S3_ENDPOINT`, `ATLAS_S3_BUCKET`, `ATLAS_S3_REGION`,
+   `ATLAS_S3_ACCESS_KEY_ID`, `ATLAS_S3_SECRET_ACCESS_KEY`; see [Required S3 storage](#required-s3-storage)) for a
+   bucket you own, and keep one replica. All reports and settings live in that bucket, so no volume is needed and
+   the service refuses to start, listing what is missing, without them.
 3. Set the healthcheck path to `/readyz` and generate a public domain. Railway supplies `PORT`;
    leave `ATLAS_PORT` unset. Open the domain and sign in with the token as the password (any username).
 4. Enable **Settings → Source → Configure Auto Updates → Anytime** to follow `latest`.
@@ -100,7 +103,7 @@ For the exact image from a run, open **[CI](https://github.com/crvouga/atlas/act
 run → Summary → Atlas CI**. The summary includes a copyable full commit tag, the immutable registry digest,
 whether that run updated `latest`, all job results, npm release details and artifact links. Use the commit
 tag to pin or roll back a deployment. The [deployment guide](docs/deploying.md#run-it) covers optional reader
-and publisher tokens, bucket storage and sending results from your projects.
+and publisher tokens, the S3 bucket and sending results from your projects.
 
 ### Deploy without authentication
 
@@ -231,8 +234,8 @@ flag).
 
 **The server** is `atlas serve`: one process that serves the client, takes in results and streams the report to
 every client. Point `ATLAS_STORE` at it and every developer's laptop and every CI job sends results with the same
-commands that write to a folder; everyone then looks at one report, live. Results are kept in its data folder
-unless you choose a bucket. `atlas dev` runs the same server with the client's dev server in front, so
+commands that write to a folder; everyone then looks at one report, live. Results are kept in an S3-compatible bucket
+and nowhere else ([required storage](#required-s3-storage)). `atlas dev` runs the same server with the client's dev server in front, so
 development is one host too. See [the server](docs/server.md) and
 [running Atlas for a project](docs/deploying.md).
 
@@ -240,7 +243,7 @@ Deploy the server using the [Railway setup above](#deploy-to-railway-without-cod
 [deployment guide](docs/deploying.md#run-it) for other hosts.
 
 ```sh
-atlas serve                                          # the project's server, at :8080 (ATLAS_TOKEN says who may upload)
+atlas serve                                          # the project's server, at :8080 (needs ATLAS_S3_*; ATLAS_TOKEN says who may upload)
 
 export ATLAS_STORE=https://atlas.example.com ATLAS_TOKEN=…   # on every laptop and CI job
 atlas report publish --producer unit --junit out.xml --specs ./specs     # when a suite is done
@@ -248,6 +251,47 @@ my-suite --json | atlas report stream --producer e2e --environment ios  # or as
 atlas report status --branch main --ignore-modified  # the gate, answered by the server
 ```
 
+### Required S3 storage
+
+The Atlas server keeps **all reports and all user state in S3-compatible object storage, and nowhere else**: no local
+disk, no memory, no database, no browser storage. *User state* is one JSON document in the bucket,
+`server/state.json`, that holds the **garbage collection policy** (defined in the Atlas UI at `/admin`) and the
+settings saved there. The server reads the policy from that document each time it collects. Until someone saves a
+policy, garbage collection is off and nothing is ever deleted.
+
+Because of that, the storage settings are **required environment variables with no defaults** (see
+[`.env.example`](https://github.com/crvouga/atlas/blob/main/.env.example)), and the server **refuses to start** (exit status 2) without them:
+
+| Variable | Required | Meaning |
+| --- | --- | --- |
+| `ATLAS_S3_ENDPOINT` | yes | S3 API endpoint, e.g. `https://s3.eu-west-1.amazonaws.com` |
+| `ATLAS_S3_BUCKET` | yes | Bucket for reports and user state; it must already exist |
+| `ATLAS_S3_REGION` | yes | Bucket region (`auto` for Cloudflare R2) |
+| `ATLAS_S3_ACCESS_KEY_ID` | yes | Access key of an account that may list, read, write and delete in the bucket |
+| `ATLAS_S3_SECRET_ACCESS_KEY` | yes | Its secret key |
+| `ATLAS_S3_PREFIX` | no | Folder inside the bucket (default: the bucket root) |
+| `ATLAS_S3_PATH_STYLE` | no | `1` (default) for `endpoint/bucket/key`, `0` for virtual-hosted style |
+| `ATLAS_S3_SESSION_TOKEN` | no | Only for temporary credentials |
+
+If a variable is missing or malformed, or S3 rejects the settings, Atlas lists **every** variable with `[MISSING]`
+or `[INVALID]` next to the ones that are wrong, says why storage is required, and exits non-zero:
+
+```text
+==============================================================================
+  ATLAS CANNOT START: S3 STORAGE IS NOT USABLE
+==============================================================================
+  Why:    the S3 configuration is missing or invalid.
+  Reason: Atlas stores ALL reports and ALL user state (the garbage collection policy and saved
+          settings) in S3-compatible object storage, and nowhere else. …
+  Required environment variables:
+  [MISSING] ATLAS_S3_ENDPOINT           S3 API endpoint of the object storage service
+  [ok]      ATLAS_S3_BUCKET             Bucket that holds reports and user state; it must already exist
+  …
+```
+
+See [the server guide](docs/server.md#required-s3-storage) for the details and
+[user state and garbage collection](docs/server.md#user-state-and-garbage-collection) for how the policy works.
+
 **The client** shows each spec (Gherkin, Quint, DMN, Alloy, OpenSLO, and the rest by file type) with
 its combined result per environment and the layer behind it. See
 [integrating Atlas](docs/integrating.md#6-browse-it).

package/dist/core/src/cli.d.ts

@@ -9,6 +9,6 @@
  * @see docs/reference/cli.md
  */
 /** The text `atlas --help` prints: every command, and the flags of the statechart commands. */
-export declare const USAGE = "atlas \u2014 executable statecharts\n\n  atlas lint   [--config atlas.config.ts | --specs <dir>] [--require-implementations]\n  atlas plan   [--config atlas.config.ts] [--state <name>]\n  atlas run    [--config atlas.config.ts] [--mode fast|showcase] [--workers n] [--state <name>]\n               [--seed <name>]... [--paths id,id] [--shard i/n] [--rerun-failed | --reuse <run-dir>]\n               [--no-retry] [--output <dir>]\n  atlas merge  [--config atlas.config.ts] [--output <dir>] <run-dir> <run-dir>...\n  atlas export --specs <dir> --format scxml|mermaid|markdown|json [--out <file>]\n  atlas server [--data <dir>] [--port 8080] [--store <url>]...   one Atlas node: API, projects and client\n  atlas client build [--no-data] [--out <dir>]                    build the project-agnostic Atlas client\n  atlas dev    [--port 5180]                                     develop the Atlas server and client on one host\n  atlas init   [--server <url>] [--write]                         integrate this project: detect its suites, print the plan (atlas init --help)\n  atlas report publish|stream|status|merge|doctor|policy \u2026       layered evidence for any kind of spec (atlas report --help)\n\n  --workers       path attempts that run at once (default: one per CPU, capped by the driver)\n  --seed          only paths that start from this seed (\"start\" for paths setup starts); repeatable\n  --shard 2/4     run the second quarter of the plan; merge the shards' runs afterwards\n  --rerun-failed  keep the newest run's passed paths and run only the rest\n";
+export declare const USAGE = "atlas \u2014 executable statecharts\n\n  atlas lint   [--config atlas.config.ts | --specs <dir>] [--require-implementations]\n  atlas plan   [--config atlas.config.ts] [--state <name>]\n  atlas run    [--config atlas.config.ts] [--mode fast|showcase] [--workers n] [--state <name>]\n               [--seed <name>]... [--paths id,id] [--shard i/n] [--rerun-failed | --reuse <run-dir>]\n               [--no-retry] [--output <dir>]\n  atlas merge  [--config atlas.config.ts] [--output <dir>] <run-dir> <run-dir>...\n  atlas export --specs <dir> --format scxml|mermaid|markdown|json [--out <file>]\n  atlas server [--port 8080]                                     one Atlas node: API, projects and client; needs ATLAS_S3_* (atlas serve --help)\n  atlas client build [--no-data] [--out <dir>]                    build the project-agnostic Atlas client\n  atlas dev    [--port 5180]                                     develop the Atlas server and client on one host\n  atlas init   [--server <url>] [--write]                         integrate this project: detect its suites, print the plan (atlas init --help)\n  atlas report publish|stream|status|merge|doctor|policy \u2026       layered evidence for any kind of spec (atlas report --help)\n\n  --workers       path attempts that run at once (default: one per CPU, capped by the driver)\n  --seed          only paths that start from this seed (\"start\" for paths setup starts); repeatable\n  --shard 2/4     run the second quarter of the plan; merge the shards' runs afterwards\n  --rerun-failed  keep the newest run's passed paths and run only the rest\n";
 /** Runs `atlas` with `argv` (default: the process arguments) and returns the exit code. Throws for a command that is wrong, which the `atlas` binary reports as a message and exit code 2. */
 export declare function main(argv?: string[]): Promise<number>;

package/dist/core/src/cli.js

@@ -28,7 +28,7 @@ export const USAGE = `atlas — executable statecharts
                [--no-retry] [--output <dir>]
   atlas merge  [--config atlas.config.ts] [--output <dir>] <run-dir> <run-dir>...
   atlas export --specs <dir> --format scxml|mermaid|markdown|json [--out <file>]
-  atlas server [--data <dir>] [--port 8080] [--store <url>]...   one Atlas node: API, projects and client
+  atlas server [--port 8080]                                     one Atlas node: API, projects and client; needs ATLAS_S3_* (atlas serve --help)
   atlas client build [--no-data] [--out <dir>]                    build the project-agnostic Atlas client
   atlas dev    [--port 5180]                                     develop the Atlas server and client on one host
   atlas init   [--server <url>] [--write]                         integrate this project: detect its suites, print the plan (atlas init --help)

package/dist/reports/src/attachment-files.d.ts

@@ -34,7 +34,9 @@ export type ReadOptions = {
  * attachment gave none.
  *
  * A file that is missing, not allowed or too large is left out with a warning: evidence that could not be
- * found never stops the results from being published. Attachments without a `path` are kept untouched.
+ * found never stops the results from being published. The exception in kind is a video of a check that declared
+ * `recordingExpectation: "required"`: it stays as an attachment without bytes whose `omitted` says why (`not-found`,
+ * `not-allowed`, `too-large`, `over-budget`), so the recording receipt can name the gap. Attachments without a `path` are kept untouched.
  *
  * @param holder A check, step, attempt or any object that may have `attachments`, `steps` and `attempts`.
  * @param roots Folders a relative path is resolved against (in order), and the only folders an untrusted path may name.

package/dist/reports/src/attachment-files.js

@@ -11,13 +11,16 @@
 import { readFile, realpath, stat } from 'node:fs/promises';
 import path from 'node:path';
 import { contentTypeOf } from './media-types.js';
+import { isVideoAttachment } from './recording-receipt.js';
 /**
  * Reads the files that attachments name (`path`), at every level (the holder, its `steps` and its `attempts`,
  * recursively), and hands their bytes over as `data` with a `contentType` guessed from the file name when the
  * attachment gave none.
  *
  * A file that is missing, not allowed or too large is left out with a warning: evidence that could not be
- * found never stops the results from being published. Attachments without a `path` are kept untouched.
+ * found never stops the results from being published. The exception in kind is a video of a check that declared
+ * `recordingExpectation: "required"`: it stays as an attachment without bytes whose `omitted` says why (`not-found`,
+ * `not-allowed`, `too-large`, `over-budget`), so the recording receipt can name the gap. Attachments without a `path` are kept untouched.
  *
  * @param holder A check, step, attempt or any object that may have `attachments`, `steps` and `attempts`.
  * @param roots Folders a relative path is resolved against (in order), and the only folders an untrusted path may name.
@@ -27,7 +30,7 @@ export async function readAttachmentFiles(holder, roots, options = {}) {
     const warnings = [];
     const maxBytes = options.maxBytes ?? 512 * 1024 * 1024;
     let attached = 0, bytes = 0;
-    async function walk(value) {
+    async function walk(value, expectsRecording = value.recordingExpectation === 'required') {
         const out = { ...value };
         if (Array.isArray(out.attachments)) {
             const kept = [];
@@ -40,17 +43,24 @@ export async function readAttachmentFiles(holder, roots, options = {}) {
                 const name = String(given.name ?? file);
                 const { path: _path, ...rest } = given;
                 const found = await locate(file, roots, options.trusted === true);
+                // A video that a check was expected to have stays in the layer as metadata with the reason it was left out
+                // (never the path), so the recording receipt can name the gap instead of the file vanishing.
+                const left = (reason) => { if (expectsRecording && isVideoAttachment({ ...rest, contentType: rest.contentType ?? contentTypeOf(name) }))
+                    kept.push({ ...rest, omitted: reason }); };
                 if (found.problem) {
                     warnings.push(`${name}: ${found.problem}`);
+                    left(found.problem.startsWith('file not found') ? 'not-found' : 'not-allowed');
                     continue;
                 }
                 const size = (await stat(found.file)).size;
                 if (size > maxBytes) {
                     warnings.push(`${name}: ${(size / 1024 / 1024).toFixed(0)} MiB is over the limit of ${Math.round(maxBytes / 1024 / 1024)} MiB`);
+                    left('too-large');
                     continue;
                 }
                 if (options.budget && size > options.budget.remaining) {
                     warnings.push(`${name}: the total size of attachments would pass the budget (raise --max-total-attachment-mb, or attach a uri)`);
+                    left('over-budget');
                     continue;
                 }
                 if (options.budget)
@@ -64,13 +74,13 @@ export async function readAttachmentFiles(holder, roots, options = {}) {
         if (Array.isArray(out.steps)) {
             const steps = [];
             for (const step of out.steps)
-                steps.push(await walk(step));
+                steps.push(await walk(step, expectsRecording));
             out.steps = steps;
         }
         if (Array.isArray(out.attempts)) {
             const attempts = [];
             for (const attempt of out.attempts)
-                attempts.push(await walk(attempt));
+                attempts.push(await walk(attempt, expectsRecording));
             out.attempts = attempts;
         }
         return out;

package/dist/reports/src/evidence-types.d.ts

@@ -123,6 +123,23 @@ export type LayerHeader = {
     readonly execution: string;
     /** What the run as a whole carries: its logs, links and labels. */
     readonly evidence?: EvidenceSummary;
+    /** Counts from the layer's recording receipt: how many expected recordings were published. Absent when the layer declared no expectations: its completeness is unknown, not complete. Never changes a status. */
+    readonly recordings?: RecordingCounts;
+};
+/** How many recordings a layer expected and what became of them (see `RecordingReceipt`). */
+export type RecordingCounts = {
+    /** Checks that required a recording: `uploaded + referenced + skipped + unresolved`. */
+    readonly expected: number;
+    /** Stored in the layer, with a checksum. */
+    readonly uploaded: number;
+    /** Given only by `uri`: Atlas cannot vouch for it. */
+    readonly referenced: number;
+    /** Captured but left out, with a reason (too large, not allowed…). */
+    readonly skipped: number;
+    /** No video was attached at all. */
+    readonly unresolved: number;
+    /** Intentionally unrecorded checks: neither expected nor missing. */
+    readonly notApplicable: number;
 };
 /** One check of one report part. */
 export type LayerRow = {
@@ -145,6 +162,11 @@ export type LayerRow = {
     readonly durationMs: number | null;
     /** What this result carries beyond its outcome. Absent when nothing. */
     readonly evidence?: EvidenceSummary;
+    /** What became of this check's expected recording: `uploaded`, `referenced`, `skipped`, `unresolved` or `not-applicable`, with a reason when it is not uploaded. Absent when the check declared no expectation. Independent of `status`. */
+    readonly recording?: {
+        readonly state: string;
+        readonly reason?: string;
+    };
 };
 /** The checks of one layer, read separately from its header. */
 export type LayerRows = {

package/dist/reports/src/garbage.d.ts

@@ -0,0 +1,39 @@
+import type { ObjectStore } from './object-store.js';
+import type { GcPolicy } from '../../schema/src/user-state.js';
+/** One layer the policy would delete. */
+export type GcCandidate = {
+    readonly id: string;
+    readonly catalogKey: string;
+    readonly recordedAtMs: number;
+};
+/** What a policy would delete from a store right now. */
+export type GcPlan = {
+    /** Layers to delete, oldest first. */
+    readonly candidates: readonly GcCandidate[];
+    /** Layers in the catalog that stay. */
+    readonly kept: number;
+};
+/** The outcome of applying a plan. */
+export type GcOutcome = {
+    readonly deletedLayers: number;
+    readonly deletedBlobs: number;
+};
+/**
+ * Decides which layers a policy deletes, without deleting anything. `eligible` is the set of layer ids the caller has
+ * read and knows no longer decide any result (`ReportIndex.superseded()`); a layer not in it is never a candidate.
+ * Lists the catalog only. With `policy.enabled === false` the plan is empty.
+ *
+ * @example
+ * const plan = await planGarbage(store, { policy, nowMs: Date.now(), eligible: index.superseded() });
+ */
+export declare function planGarbage(store: ObjectStore, input: {
+    readonly policy: GcPolicy;
+    readonly nowMs: number;
+    readonly eligible: ReadonlySet<string>;
+}): Promise<GcPlan>;
+/**
+ * Deletes the layers of a plan, then the blobs only they used. Store errors throw after whatever was already deleted;
+ * running the same plan again finishes the job, because every delete is idempotent and the catalog entry (what
+ * makes a layer visible) always goes before the content.
+ */
+export declare function collectGarbage(store: ObjectStore, plan: GcPlan): Promise<GcOutcome>;

package/dist/reports/src/garbage.js

@@ -0,0 +1,82 @@
+/**
+ * @module Layer garbage collection
+ *
+ * Applies a garbage collection policy (`GcPolicy`, chosen by a person in the Atlas UI and kept in the user state) to the
+ * layers in a store. Atlas is otherwise deletion-free, so this is deliberately conservative: a layer is deleted only when
+ * (1) the caller says the report has read it and every result it carries has been replaced by a newer layer (`eligible`),
+ * (2) it was recorded longer ago than the policy allows, and (3) it is not among the newest `keepLatest` layers. Deleting
+ * it therefore cannot change the combined report. Its catalog entry goes first, so nothing visible ever points at a
+ * missing manifest; its manifest next; then the captured-file blobs that no remaining layer refers to. If any remaining
+ * layer cannot be read, no blob is deleted: what cannot be read is unknown, never unused. Specs, runs and commit
+ * inventories are not touched.
+ *
+ * @see docs/server.md#user-state-and-garbage-collection
+ */
+import { eachLimited } from './completion-order.js';
+import { LAYER_INDEX, layerKey, readLayer } from './layer-store.js';
+const DAY_MS = 86_400_000;
+const CATALOG = /^layers\/index\/(\d{16})-([0-9a-f]{64})\.json$/;
+/**
+ * Decides which layers a policy deletes, without deleting anything. `eligible` is the set of layer ids the caller has
+ * read and knows no longer decide any result (`ReportIndex.superseded()`); a layer not in it is never a candidate.
+ * Lists the catalog only. With `policy.enabled === false` the plan is empty.
+ *
+ * @example
+ * const plan = await planGarbage(store, { policy, nowMs: Date.now(), eligible: index.superseded() });
+ */
+export async function planGarbage(store, input) {
+    const entries = (await store.listKeys(LAYER_INDEX)).flatMap(key => {
+        const match = CATALOG.exec(key);
+        return match ? [{ id: match[2], catalogKey: key, recordedAtMs: Number.MAX_SAFE_INTEGER - Number(match[1]) }] : [];
+    });
+    // Catalog keys sort newest first (the time is inverted), so the layers to always keep are the first `keepLatest`.
+    entries.sort((a, b) => b.recordedAtMs - a.recordedAtMs || (a.id < b.id ? -1 : 1));
+    if (!input.policy.enabled)
+        return { candidates: [], kept: entries.length };
+    const cutoff = input.nowMs - input.policy.maxAgeDays * DAY_MS;
+    const candidates = entries.slice(input.policy.keepLatest).filter(entry => entry.recordedAtMs < cutoff && input.eligible.has(entry.id));
+    return { candidates: candidates.reverse(), kept: entries.length - candidates.length };
+}
+/**
+ * Deletes the layers of a plan, then the blobs only they used. Store errors throw after whatever was already deleted;
+ * running the same plan again finishes the job, because every delete is idempotent and the catalog entry (what
+ * makes a layer visible) always goes before the content.
+ */
+export async function collectGarbage(store, plan) {
+    if (!plan.candidates.length)
+        return { deletedLayers: 0, deletedBlobs: 0 };
+    const doomed = new Set(plan.candidates.map(candidate => candidate.id));
+    // Which blobs the doomed layers used, read before anything is deleted. A layer that cannot be read stays: unknown is not garbage.
+    const used = new Map();
+    const readable = [];
+    await eachLimited(plan.candidates, 16, async (candidate) => {
+        const layer = await readLayer(store, candidate.id).catch(() => null);
+        if (!layer)
+            return;
+        used.set(candidate.id, new Set(Object.values(layer.bundle).flatMap(file => file.parts.map(part => part.blob))));
+        readable.push(candidate);
+    });
+    if (!readable.length)
+        return { deletedLayers: 0, deletedBlobs: 0 };
+    await store.deleteMany(readable.map(candidate => candidate.catalogKey));
+    await store.deleteMany(readable.map(candidate => layerKey(candidate.id)));
+    const removed = new Set(readable.map(candidate => candidate.id));
+    const orphanable = new Set([...removed].flatMap(id => [...used.get(id)]));
+    // Blobs that any remaining layer still points at must stay. If a remaining layer cannot be read, keep every blob.
+    const remaining = (await store.listKeys(LAYER_INDEX)).flatMap(key => CATALOG.exec(key)?.[2] ?? []).filter(id => !removed.has(id));
+    let unreadable = false;
+    await eachLimited(remaining, 16, async (id) => {
+        const layer = await readLayer(store, id).catch(() => null);
+        if (!layer) {
+            unreadable = true;
+            return;
+        }
+        for (const file of Object.values(layer.bundle))
+            for (const part of file.parts)
+                orphanable.delete(part.blob);
+    });
+    if (unreadable)
+        return { deletedLayers: removed.size, deletedBlobs: 0 };
+    await store.deleteMany([...orphanable]);
+    return { deletedLayers: removed.size, deletedBlobs: orphanable.size };
+}

package/dist/reports/src/layer-merge.js

@@ -74,10 +74,13 @@ export function layerEvidence(summary, backend) {
         return found;
     };
     const rows = [];
+    const receipt = summary.recordings;
+    const expectations = receipt ? new Map(receipt.items.map(item => [item.check, item])) : null;
     for (const part of summary.parts) {
         for (const check of part.checks) {
             const test = tests.get(check.id);
             const evidence = summarizeEvidence(check);
+            const item = expectations?.get(check.id);
             rows.push({
                 kind: part.kind,
                 id: check.id,
@@ -90,6 +93,7 @@ export function layerEvidence(summary, backend) {
                 variant: test ? { system: test.system, runtime: test.runtime, seed: test.seed } : {},
                 durationMs: test?.durationMs ?? check.durationMs ?? null,
                 ...(evidence ? { evidence } : {}),
+                ...(item ? { recording: { state: item.state, ...(item.reason ? { reason: item.reason } : {}) } } : {}),
             });
         }
     }
@@ -113,6 +117,7 @@ export function layerEvidence(summary, backend) {
         counts: outcomeCounts(rows),
         execution: textFingerprint(layerExecutionKey(summary)),
         ...(layerSummaryEvidence ? { evidence: layerSummaryEvidence } : {}),
+        ...(receipt ? { recordings: { expected: receipt.expected, uploaded: receipt.uploaded, referenced: receipt.referenced, skipped: receipt.skipped, unresolved: receipt.unresolved, notApplicable: receipt.notApplicable } } : {}),
     };
     const fingerprints = {};
     for (const spec of summary.specs)

package/dist/reports/src/layer-types.d.ts

@@ -15,7 +15,7 @@ import type { SourceRecord } from './protocol.js';
 /** String choices (such as `{ device: 'iphone-15' }`) that make a result its own slot. Compared as sorted key/value pairs. */
 export type LayerParameters = Readonly<Record<string, string>>;
 /** What a layer may carry as a whole. All optional and open: see docs/extending.md. */
-export type LayerExtras = Partial<Pick<import('../../schema/src/layer.js').LayerDocument, 'tags' | 'labels' | 'links' | 'attachments' | 'annotations' | 'extensions'>>;
+export type LayerExtras = Partial<Pick<import('../../schema/src/layer.js').LayerDocument, 'tags' | 'labels' | 'links' | 'attachments' | 'annotations' | 'extensions' | 'recordings'>>;
 /** A layer without its captured files: what composition and the viewer's lists read. */
 export type LayerSummary = LayerExtras & {
     /** `modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Absent for a clean checkout. Advisory; never changes a status. */

package/dist/reports/src/publish-checks.js

@@ -15,6 +15,7 @@ import { normalizeEnvironment } from './layer-axes.js';
 import { publishLayerBundle } from './layer-store.js';
 import { contentTypeOf } from './media-types.js';
 import { parseWith } from './parse-with.js';
+import { isVideoAttachment, recordingReceipt } from './recording-receipt.js';
 import { sha256Hex } from './sigv4.js';
 /** A small text value stays in the layer's own document; anything longer is stored as a file. */
 const INLINE_TEXT_BYTES = 8 * 1024;
@@ -28,7 +29,7 @@ function decodeBase64(text) {
  * screenshot attached ten times is stored once), referred to by `bundleKey`. What a producer wrote as `data`,
  * `base64` or a long `text` becomes a reference; a short `text` stays inline; a `uri` is left alone.
  */
-async function materialize(holder, entries, limit) {
+async function materialize(holder, entries, limit, expectsRecording = holder.recordingExpectation === 'required') {
     const out = { ...holder };
     if (Array.isArray(out.attachments)) {
         out.attachments = await Promise.all(out.attachments.map(async (given) => {
@@ -41,6 +42,11 @@ async function materialize(holder, entries, limit) {
                 bytes = encoder.encode(text);
             if (!bytes)
                 return { ...kept, ...(text !== undefined ? { bytes: encoder.encode(text).length } : {}) };
+            if (bytes.length > limit && expectsRecording && isVideoAttachment({ ...kept, ...(kept.contentType ? {} : { contentType: contentTypeOf(String(kept.name)) }) })) {
+                // A recording the check was expected to have is reported as left out (and why) instead of failing the whole publish.
+                const { text: _inline, ...rest } = kept;
+                return { ...rest, contentType: kept.contentType ?? contentTypeOf(String(kept.name)), bytes: bytes.length, sha256: await sha256Hex(bytes), omitted: 'too-large' };
+            }
             if (bytes.length > limit)
                 throw new Error(`publishChecks: attachment "${String(kept.name)}" is ${bytes.length} bytes, over the limit of ${limit}. Store it elsewhere and attach its uri`);
             const sha256 = await sha256Hex(bytes);
@@ -51,9 +57,9 @@ async function materialize(holder, entries, limit) {
         }));
     }
     if (Array.isArray(out.steps))
-        out.steps = await Promise.all(out.steps.map(step => materialize(step, entries, limit)));
+        out.steps = await Promise.all(out.steps.map(step => materialize(step, entries, limit, expectsRecording)));
     if (Array.isArray(out.attempts))
-        out.attempts = await Promise.all(out.attempts.map(attempt => materialize(attempt, entries, limit)));
+        out.attempts = await Promise.all(out.attempts.map(attempt => materialize(attempt, entries, limit, expectsRecording)));
     return out;
 }
 /**
@@ -81,6 +87,7 @@ export async function publishChecks(store, input, options = {}) {
     const entries = new Map();
     const checks = await Promise.all(args.checks.map(check => materialize(check, entries, limit)));
     const layerEvidence = await materialize({ attachments: args.attachments }, entries, limit);
+    const recordings = recordingReceipt(checks);
     const total = [...entries.values()].reduce((sum, entry) => sum + entry.bytes.length, 0);
     const totalLimit = options.maxTotalAttachmentBytes ?? 2 * 1024 * 1024 * 1024;
     if (total > totalLimit)
@@ -110,6 +117,7 @@ export async function publishChecks(store, input, options = {}) {
         parts: [part],
         source: null,
         recording: null,
+        ...(recordings ? { recordings } : {}),
         ...(args.tags?.length ? { tags: args.tags } : {}),
         ...(args.labels && Object.keys(args.labels).length ? { labels: Object.fromEntries(Object.entries(args.labels).sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0))) } : {}),
         ...(args.links?.length ? { links: args.links } : {}),

package/dist/reports/src/recording-receipt.d.ts

@@ -0,0 +1,48 @@
+/**
+ * @module Recording receipts
+ *
+ * Answers "did the recordings we expected get published?" apart from "did the tests pass?". `recordingReceipt` reads the
+ * checks a producer is about to publish (after their files were stored) and describes, per check that declared a
+ * `recordingExpectation`, whether its video was uploaded, only referenced, left out (with a reason) or never attached.
+ * `recordingCompleteness` reduces the winning results of a report to the figures `atlas report status` prints. Neither
+ * reads a clock or the environment, and neither changes a status, a slot or a winner: a receipt describes publication, it
+ * does not decide a result. A layer with no receipt is Unknown, never Complete.
+ *
+ * @see docs/rich-evidence.md#recording-completeness
+ */
+import type { RecordingCompleteness, RecordingReceipt } from '../../schema/src/layer.js';
+type Loose = Record<string, unknown>;
+/** Is this attachment a screen recording: a role of `video`, or a `video/*` media type? */
+export declare function isVideoAttachment(attachment: Loose): boolean;
+/**
+ * The receipt for the checks of one layer, or `undefined` when no check declares a `recordingExpectation` (the layer then
+ * claims nothing). Call it with checks whose attachment bytes are already stored (`bundleKey`, `bytes` and `sha256` set).
+ *
+ * A `required` check is `uploaded` when any video of it has stored bytes, `referenced` when one has only a `uri`,
+ * `skipped` when its videos were left out (the attachment's `omitted` is the reason) and `unresolved` when it has no
+ * video at all. A `not-applicable` check is listed but is neither expected nor missing. Items are sorted by check id so the
+ * same input is always the same receipt.
+ */
+export declare function recordingReceipt(checks: readonly Loose[]): RecordingReceipt | undefined;
+/** One result as {@link recordingCompleteness} reads it. */
+export type RecordingResult = {
+    readonly spec: string | null;
+    readonly check: string;
+    readonly environment: string | null;
+    readonly layer: string;
+    /** Whether the layer the result came from carried a receipt. */
+    readonly hasReceipt: boolean;
+    /** The result's item in that receipt, when its check declared an expectation. */
+    readonly recording?: {
+        readonly state: string;
+        readonly reason?: string;
+    };
+};
+/**
+ * Reduces the winning results of a report to its recording completeness. `incomplete` when any expected recording is
+ * skipped or unresolved; otherwise `unknown` when a contributing layer has no receipt (nothing is claimed for it, and a
+ * report with no results at all is unknown too); otherwise `complete` when something was expected and `none-expected` when
+ * nothing was. Gaps are sorted so the output never depends on read order.
+ */
+export declare function recordingCompleteness(results: readonly RecordingResult[]): RecordingCompleteness;
+export {};