@crvouga/atlas 0.17.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (70) hide show
  1. package/CHANGELOG.md +12 -0
  2. package/README.md +52 -8
  3. package/dist/core/src/cli.d.ts +1 -1
  4. package/dist/core/src/cli.js +1 -1
  5. package/dist/reports/src/garbage.d.ts +39 -0
  6. package/dist/reports/src/garbage.js +82 -0
  7. package/dist/reports/src/report-index.d.ts +6 -0
  8. package/dist/reports/src/report-index.js +14 -0
  9. package/dist/schema/src/index.d.ts +1 -0
  10. package/dist/schema/src/index.js +1 -0
  11. package/dist/schema/src/server-config.d.ts +7 -49
  12. package/dist/schema/src/server-config.js +8 -21
  13. package/dist/schema/src/user-state.d.ts +115 -0
  14. package/dist/schema/src/user-state.js +54 -0
  15. package/dist/server/src/app.d.ts +16 -2
  16. package/dist/server/src/app.js +62 -0
  17. package/dist/server/src/config.d.ts +16 -21
  18. package/dist/server/src/config.js +48 -57
  19. package/dist/server/src/dev.d.ts +1 -1
  20. package/dist/server/src/dev.js +20 -7
  21. package/dist/server/src/index.d.ts +5 -1
  22. package/dist/server/src/index.js +5 -1
  23. package/dist/server/src/main.d.ts +23 -7
  24. package/dist/server/src/main.js +72 -26
  25. package/dist/server/src/openapi.js +24 -5
  26. package/dist/server/src/projects.js +89 -14
  27. package/dist/server/src/settings.d.ts +23 -7
  28. package/dist/server/src/settings.js +50 -60
  29. package/dist/server/src/storage.d.ts +68 -0
  30. package/dist/server/src/storage.js +125 -0
  31. package/dist/server/src/user-state.d.ts +31 -0
  32. package/dist/server/src/user-state.js +74 -0
  33. package/docs/agents.md +1 -1
  34. package/docs/deploying.md +38 -39
  35. package/docs/reference/cli.md +12 -7
  36. package/docs/reference/http-api.md +30 -2
  37. package/docs/reference/reports.md +71 -0
  38. package/docs/reference/schema.md +102 -16
  39. package/docs/reference/server.md +263 -41
  40. package/docs/report-backends.md +0 -144
  41. package/docs/report-contract.md +1 -1
  42. package/docs/runners.md +1 -1
  43. package/docs/server.md +141 -36
  44. package/docs/statecharts.md +1 -1
  45. package/docs/ui.md +8 -1
  46. package/docs/workspaces.md +5 -2
  47. package/llms-full.txt +238 -234
  48. package/llms.txt +1 -1
  49. package/package.json +2 -2
  50. package/packages/reports/STORAGE.md +1 -1
  51. package/packages/schema/schemas/gc-policy.schema.json +36 -0
  52. package/packages/schema/schemas/gc-run-result.schema.json +106 -0
  53. package/packages/schema/schemas/gc-snapshot.schema.json +122 -0
  54. package/packages/schema/schemas/server-config.schema.json +0 -41
  55. package/packages/schema/schemas/server-editable-settings.schema.json +0 -41
  56. package/packages/schema/schemas/server-settings-snapshot.schema.json +0 -82
  57. package/packages/schema/schemas/user-state.schema.json +140 -0
  58. package/packages/server/README.md +11 -6
  59. package/packages/server/openapi.json +510 -126
  60. package/packages/visualizer/src/admin/SettingsPage.tsx +183 -88
  61. package/packages/visualizer/src/evidence/check-detail.tsx +6 -1
  62. package/packages/visualizer/src/evidence/evidence-context.tsx +9 -2
  63. package/packages/visualizer/src/evidence/load-state.tsx +65 -0
  64. package/packages/visualizer/src/evidence/overview.tsx +4 -6
  65. package/packages/visualizer/src/evidence/recording-page.tsx +17 -7
  66. package/packages/visualizer/src/evidence/spec-header.tsx +8 -6
  67. package/packages/visualizer/src/evidence/spec-page.tsx +98 -15
  68. package/packages/visualizer/src/find/Finder.tsx +45 -7
  69. package/packages/visualizer/src/spec-kind.test.ts +14 -1
  70. package/packages/visualizer/src/spec-kind.ts +28 -2
package/llms-full.txt CHANGED
@@ -153,7 +153,7 @@ workspace selection and test scripts. An existing `atlas.yml` not generated by A
153
153
  workflows may still need manual integration. Non-JavaScript suites need their language/tool setup in CI.
154
154
 
155
155
  Without a server, start with [local verification](quickstart.md). To run a server later, `atlas serve`
156
- starts one (`http://127.0.0.1:8080`, data in `./atlas-data`); [the server guide](server.md) covers configuration
156
+ starts one (`http://127.0.0.1:8080`; it needs S3-compatible storage configured with the `ATLAS_S3_*` variables, see `.env.example`); [the server guide](server.md) covers configuration
157
157
  and authentication. Keep synthetic failure checks in a scratch store rather than a shared production report.
158
158
 
159
159
  ## What an agent can rely on
@@ -979,7 +979,7 @@ Playwright, Vitest, Bun, WebDriver/Appium and Detox already have Atlas statechar
979
979
 
980
980
  ## Operate the singleton
981
981
 
982
- Run `atlas server` once for the projects the node hosts; point local and CI publishers at that URL and set each project's `ATLAS_PROJECT`. Configure backends using environment variables or open **Settings** in the client (`/admin`). Authentication and listening addresses remain deployment configuration. Settings changes are validated and saved atomically for the next server restart. Environment variables and flags override saved values. A backend change does not migrate historical data: retain the old backend as a read source when switching the upload destination.
982
+ Run `atlas server` once for the projects the node hosts; point local and CI publishers at that URL and set each project's `ATLAS_PROJECT`. The node keeps everything in the S3 bucket named by the required `ATLAS_S3_*` variables ([required S3 storage](server.md#required-s3-storage)); open **Settings** in the client (`/admin`) to define the garbage collection policy and report settings, which are saved as JSON in that bucket. Authentication, listening addresses and storage remain deployment configuration. Settings changes are validated and saved for the next server restart. Environment variables and flags override saved values.
983
983
 
984
984
  Native reporters send a heartbeat every 5 seconds. The server interrupts invocations after 30 seconds of silence, or after the configurable progress-stall timeout. It preserves finished results and never marks missing tests as passed. See [live test runs](server.md#live-test-runs-and-heartbeats).
985
985
  </doc>
@@ -1254,22 +1254,24 @@ development and in production.
1254
1254
  CI job ──────┼── PUT /api/store/… POST /api/checks ─▶│
1255
1255
  device lab ──┘ atlas report publish | stream │ atlas report status (the gate: asks /api/status)
1256
1256
  │
1257
- backends: ./atlas-data (default), S3, other folders, another Atlas server
1257
+ one S3-compatible bucket: ALL reports and ALL user state (required)
1258
1258
  ```
1259
1259
 
1260
1260
  - **Clients upload directly to it.** `atlas report publish`, `atlas report stream`, `createReporter` and a plain
1261
1261
  `curl` all end up as layers on the server. Nothing else needs deploying: no bucket access for CI, no artifact
1262
1262
  merging, no static site to rebuild.
1263
- - **Its default backend is its own filesystem** (`./atlas-data`). That folder is the whole state: back it up,
1264
- mount it on a volume, or put the state in a bucket instead ([backends](#backends)). Several backends are one
1265
- report, as ever.
1263
+ - **Its only storage is an S3-compatible bucket.** All reports and all user state (the garbage collection policy,
1264
+ saved settings) live there and nowhere else: not on its disk, in memory, in a database or in the browser. The
1265
+ storage settings are required and the server refuses to start without them
1266
+ ([required S3 storage](#required-s3-storage)).
1266
1267
  - **The report is live.** Every client holds one server-sent-events connection; when results arrive the server
1267
1268
  says so and each client reads only what changed ([streaming](#streaming)).
1268
1269
  - **Layered, never partitioned**, as everywhere in Atlas: the server only stores immutable, content-addressed
1269
1270
  layers and works out the report from them. Replace it, restore it, or run a second one on a copy of the data:
1270
1271
  the same layers give the same report ([the contract](report-contract.md)).
1271
1272
 
1272
- - [Run it](#run-it) · [Configuration](#configuration) · [Backends](#backends) · [Tokens and roles](#tokens-and-roles)
1273
+ - [Run it](#run-it) · [Required S3 storage](#required-s3-storage) · [User state and garbage collection](#user-state-and-garbage-collection)
1274
+ - [Configuration](#configuration) · [Storage](#storage) · [Tokens and roles](#tokens-and-roles)
1273
1275
  - [Sending results](#sending-results) · [Streaming](#streaming) · [API reference](#api-reference)
1274
1276
  - [What the server refuses](#what-the-server-refuses) · [Performance](#performance) · [Operating it](#operating-it)
1275
1277
 
@@ -1278,7 +1280,8 @@ development and in production.
1278
1280
  ```sh
1279
1281
  bun add -d @crvouga/atlas
1280
1282
  export ATLAS_TOKEN=$(openssl rand -hex 24) # admin token; see "Tokens and roles"
1281
- bunx atlas serve # http://127.0.0.1:8080, data in ./atlas-data
1283
+ cp .env.example .env # then fill in the ATLAS_S3_* variables: storage is required
1284
+ bunx atlas serve # http://127.0.0.1:8080, everything kept in your S3 bucket
1282
1285
  ```
1283
1286
 
1284
1287
  <!-- generated:runtime-requirements -->
@@ -1294,21 +1297,118 @@ server in front of it. Vite proxies `/api` (server-sent events included) to the
1294
1297
  uploaders and the client all use **one host**, exactly as in production, and a change to the client shows at once.
1295
1298
 
1296
1299
  ```sh
1297
- bunx atlas dev --data .atlas-data # http://127.0.0.1:5180
1300
+ bunx atlas dev # http://127.0.0.1:5180 (same ATLAS_S3_* variables)
1298
1301
  ATLAS_STORE=http://127.0.0.1:5180 bunx atlas report publish --producer unit --junit out.xml
1299
1302
  ```
1300
1303
 
1301
1304
  For Railway, deploy `ghcr.io/crvouga/atlas:latest` directly: the image contains the server and
1302
1305
  viewer, and every green `main` push updates it. See [running Atlas for a project](deploying.md#run-it) for
1303
- Railway's volume, tokens, healthcheck and image auto updates, or for Docker, Kubernetes and CI.
1306
+ Railway's variables, tokens, healthcheck and image auto updates, or for Docker, Kubernetes and CI.
1304
1307
  For anonymous reading, uploads and administration, use `ghcr.io/crvouga/atlas-insecure:latest` with no tokens
1305
1308
  configured; see [deploying without authentication](deploying.md#deploy-without-authentication).
1306
1309
 
1310
+ ## Required S3 storage
1311
+
1312
+ The server keeps **all reports and all user state in S3-compatible object storage, and nowhere else**. There is no
1313
+ local-disk, in-memory or database fallback and no flag to choose another place, so the storage settings are
1314
+ **required environment variables with no defaults** (they hold secrets, so they are never read from a file or a flag):
1315
+
1316
+ | Variable | Required | Meaning |
1317
+ | --- | --- | --- |
1318
+ | `ATLAS_S3_ENDPOINT` | yes | S3 API endpoint, e.g. `https://s3.eu-west-1.amazonaws.com`, `https://<account>.r2.cloudflarestorage.com`, `http://minio:9000` |
1319
+ | `ATLAS_S3_BUCKET` | yes | Bucket for reports and user state. It must already exist |
1320
+ | `ATLAS_S3_REGION` | yes | Bucket region (`auto` for Cloudflare R2, `us-east-1` for most self-hosted servers) |
1321
+ | `ATLAS_S3_ACCESS_KEY_ID` | yes | Access key of an account that may list, read, write and delete in the bucket and prefix |
1322
+ | `ATLAS_S3_SECRET_ACCESS_KEY` | yes | Its secret key |
1323
+ | `ATLAS_S3_PREFIX` | no | A folder inside the bucket for everything Atlas writes. Default: the bucket root |
1324
+ | `ATLAS_S3_PATH_STYLE` | no | `1` (default) for `endpoint/bucket/key`, `0` for virtual-hosted `bucket.endpoint/key` |
1325
+ | `ATLAS_S3_SESSION_TOKEN` | no | Only for temporary credentials |
1326
+
1327
+ [`.env.example`](https://github.com/crvouga/atlas/blob/main/.env.example) lists them with explanations. The server does not look for
1328
+ ambient AWS credentials (`AWS_*`, profiles, instance roles) and the old `ATLAS_STORE`, `ATLAS_WRITE`, `ATLAS_DATA`,
1329
+ `--store`, `--write`, `--data` and the `data`, `backends` and `write` keys of `atlas.server.json` are gone; a
1330
+ configuration file that still has one of the keys is refused with a message that says so.
1331
+
1332
+ **It fails fast and loud.** Before anything listens, the server checks every variable, then proves the bucket is
1333
+ usable (lists it, writes and deletes a scratch object) and reads the user state. If any step fails it prints a banner
1334
+ that lists **every** variable with `[MISSING]`, `[INVALID]`, `[ok]` or `[not set]`, says why storage is required (and
1335
+ what S3 answered, when S3 was reached), never prints a secret, and exits with status **2**:
1336
+
1337
+ ```text
1338
+ ==============================================================================
1339
+ ATLAS CANNOT START: S3 STORAGE IS NOT USABLE
1340
+ ==============================================================================
1341
+ Why: the S3 configuration is missing or invalid.
1342
+ Reason: Atlas stores ALL reports and ALL user state (the garbage collection policy and saved
1343
+ settings) in S3-compatible object storage, and nowhere else. There is no local-disk,
1344
+ in-memory or database fallback, so the server refuses to start without working S3.
1345
+
1346
+ Required environment variables:
1347
+ [MISSING] ATLAS_S3_ENDPOINT S3 API endpoint of the object storage service
1348
+ [ok] ATLAS_S3_BUCKET Bucket that holds reports and user state; it must already exist
1349
+ [INVALID] ATLAS_S3_REGION must be lowercase letters, digits and hyphens, like eu-west-1
1350
+ …
1351
+ Fix these 2: ATLAS_S3_ENDPOINT, ATLAS_S3_REGION
1352
+ ```
1353
+
1354
+ Credentials that S3 rejects, a bucket that does not exist and a user state object that cannot be read stop the
1355
+ server in the same way, with the answer from S3 in the banner. `atlas dev` behaves the same.
1356
+
1357
+ ## User state and garbage collection
1358
+
1359
+ Everything a person using Atlas has chosen is **user state**: one JSON document, `server/state.json` under
1360
+ `ATLAS_S3_PREFIX`, in the same bucket as the reports. It is never kept on the server's disk or in the browser.
1361
+
1362
+ ```json
1363
+ {
1364
+ "schemaVersion": 1,
1365
+ "gc": { "enabled": true, "maxAgeDays": 90, "keepLatest": 20, "intervalHours": 24 },
1366
+ "settings": { "refreshSeconds": 30 }
1367
+ }
1368
+ ```
1369
+
1370
+ (published as [`user-state.schema.json`](https://github.com/crvouga/atlas/blob/main/packages/schema/schemas/user-state.schema.json))
1371
+
1372
+ - **`gc` is the garbage collection policy**, defined in the Atlas UI (**Settings**, `/admin`, *Garbage collection*) and
1373
+ read by the server **from S3 every time it collects**, so a saved policy applies at the next pass without a restart.
1374
+ - **`settings`** are the report and live-run settings saved from the same page. They apply at the next start, below
1375
+ `atlas.server.json`, the environment and the flags in precedence.
1376
+ - A document that does not exist is an empty state; one that exists but cannot be parsed stops the server and is
1377
+ never overwritten.
1378
+
1379
+ **No policy saved yet.** Atlas is deletion-free until someone says otherwise: with no `gc` in the document,
1380
+ garbage collection is **off and nothing is ever deleted**. The server says so at start, the API answers
1381
+ `saved: false, policy: null`, the page shows a "No policy has been saved yet" notice with suggested values that take
1382
+ effect only after you turn collection on and save, and `POST /api/settings/gc/run` answers 409. There is no
1383
+ fallback to a file, an environment variable or built-in numbers.
1384
+
1385
+ **What a policy deletes.** With `enabled: true`, every `intervalHours` (and shortly after start) the server applies the
1386
+ policy to every project. It deletes a report layer only when all of these hold:
1387
+
1388
+ 1. the server has read it and every check it carries has been replaced by a newer layer, so it no longer decides any
1389
+ result (deleting it cannot change the report);
1390
+ 2. it was recorded more than `maxAgeDays` days ago;
1391
+ 3. it is not one of the newest `keepLatest` layers of its project.
1392
+
1393
+ The layer's catalog entry goes first (so nothing visible points at a missing manifest), then its manifest, then the
1394
+ captured files (screenshots, logs, …) that no remaining layer refers to; if any remaining layer cannot be read, no
1395
+ file is deleted. Specs, runs and commit inventories are never touched, and a layer that failed to load, that
1396
+ `include` hides, or that still decides a result is never a candidate. Because a deleted layer is also removed from the
1397
+ test history, history older than the policy allows is gone after the next start. One narrow race remains: a publisher
1398
+ that reuses a stored file at the very moment the last layer that used it is deleted can end up with a layer that
1399
+ points at a missing file; keep `maxAgeDays` well above the time uploads take.
1400
+
1401
+ **Editing it.** `GET /api/settings/gc` and `PUT /api/settings/gc` (admin, with `If-Match` the revision that was read,
1402
+ same-origin in browsers) read and save the policy; `POST /api/settings/gc/run` applies it now, and
1403
+ `POST /api/settings/gc/run?dryRun=1` only counts what would go. A save that races another is refused with 409. The
1404
+ revision is the SHA-256 of the stored document, so saving the policy and saving settings invalidate each other's
1405
+ revision: reload, then save. An object store has no conditional writes, so run one server per bucket.
1406
+
1307
1407
  ## Configuration
1308
1408
 
1309
- Everything has a default, so no configuration is a working server for this machine. Settings come from
1409
+ Everything but the [S3 storage](#required-s3-storage) has a default. Settings come from the settings saved in S3 (the lowest layer), then
1310
1410
  `atlas.server.json` (or `--config <file>` / `ATLAS_CONFIG`), then environment variables, then flags; a later one
1311
- wins. A mistake (an unknown key, a short token, two write targets, listening beyond this machine without tokens)
1411
+ wins. A mistake (an unknown key, a short token, listening beyond this machine without tokens)
1312
1412
  is said in words and the server does not start. The file has a published
1313
1413
  [JSON Schema](https://github.com/crvouga/atlas/blob/main/packages/schema/schemas/server-config.schema.json)
1314
1414
  (`@crvouga/atlas/schemas/server-config.schema.json`).
@@ -1318,9 +1418,6 @@ is said in words and the server does not start. The file has a published
1318
1418
  "$schema": "./node_modules/@crvouga/atlas/packages/schema/schemas/server-config.schema.json",
1319
1419
  "host": "0.0.0.0",
1320
1420
  "port": 8080,
1321
- "data": "./atlas-data",
1322
- "backends": [{ "id": "archive", "url": "s3://my-bucket/atlas/my-project?region=us-east-1", "label": "Archive" }],
1323
- "write": "data",
1324
1421
  "auth": {
1325
1422
  "tokens": [
1326
1423
  { "name": "ci", "role": "publish", "tokenEnv": "ATLAS_CI_TOKEN" },
@@ -1336,11 +1433,16 @@ is said in words and the server does not start. The file has a published
1336
1433
  <!-- generated:server-settings -->
1337
1434
  | Setting | File key | Flag | Environment | Default |
1338
1435
  | --- | --- | --- | --- | --- |
1436
+ | S3 endpoint URL (**required**) | `storage.s3.endpoint` | | `ATLAS_S3_ENDPOINT` | **none: the server refuses to start** |
1437
+ | Bucket for all reports and user state (**required**) | `storage.s3.bucket` | | `ATLAS_S3_BUCKET` | **none: the server refuses to start** |
1438
+ | Bucket region (**required**) | `storage.s3.region` | | `ATLAS_S3_REGION` | **none: the server refuses to start** |
1439
+ | Access key id (**required**) | `storage.s3.accessKeyId` | | `ATLAS_S3_ACCESS_KEY_ID` | **none: the server refuses to start** |
1440
+ | Secret access key (**required**) | `storage.s3.secretAccessKey` | | `ATLAS_S3_SECRET_ACCESS_KEY` | **none: the server refuses to start** |
1441
+ | Folder in the bucket for everything Atlas writes (optional) | `storage.s3.prefix` | | `ATLAS_S3_PREFIX` | the bucket root |
1442
+ | Path-style bucket addressing (`0` for virtual-hosted) (optional) | `storage.s3.pathStyle` | | `ATLAS_S3_PATH_STYLE` | yes |
1443
+ | Session token for temporary credentials (optional) | `storage.s3.sessionToken` | | `ATLAS_S3_SESSION_TOKEN` | none |
1339
1444
  | Address to listen on | `host` | `--host <address>` | `ATLAS_HOST`, `HOST` | `127.0.0.1` |
1340
1445
  | Port (`0` picks a free one) | `port` | `--port <n>` | `ATLAS_PORT`, `PORT` | `8080` |
1341
- | The server's own folder: the default backend and write target | `data` | `--data <dir>` | `ATLAS_DATA` | `./atlas-data` |
1342
- | More backends, read together with the data folder | `backends` | `--store <[id=]url>` (repeatable) | `ATLAS_STORE` | none |
1343
- | The backend uploads go to | `write` | `--write <id>` | `ATLAS_WRITE` | `data` |
1344
1446
  | The built client | `site` | `--site <dir>` | `ATLAS_SITE` | `./atlas-site`, else built once |
1345
1447
  | Whose specs the report is laid over | `outlineBranch` | `--outline-branch <branch>` | `ATLAS_OUTLINE_BRANCH` | `main`, then `master`, then the newest |
1346
1448
  | Only layers recorded on these branches take part | `include.branches` | `--branch <name>` (repeatable) | `ATLAS_ONLY_BRANCH` | all |
@@ -1362,22 +1464,19 @@ is said in words and the server does not start. The file has a published
1362
1464
  `ATLAS_HOST` and `ATLAS_PORT` win over `HOST` and `PORT`, which other tools on a machine often set for their own
1363
1465
  purposes.
1364
1466
 
1365
- ## Backends
1366
-
1367
- A backend is anything that implements `ObjectStore`: a folder, an S3-compatible bucket (`s3://`, `r2://`,
1368
- `gs://`), or **another Atlas server** (`https://…`, with `ATLAS_TOKEN` for it). The server reads every backend
1369
- and shows one report; one that is down is named in the client and skipped, and the server recovers by itself.
1467
+ ## Storage
1370
1468
 
1371
- - **Uploads go to one backend**: the `data` folder, or the one named by `write` / `ATLAS_WRITE`. Name a bucket
1372
- to make it the system of record, so the server holds nothing a replacement would lose:
1373
-
1374
- ```sh
1375
- ATLAS_STORE=project=s3://my-bucket/atlas/my-project ATLAS_WRITE=project atlas serve
1376
- ```
1469
+ The server has one backend, the S3 bucket named by the [required variables](#required-s3-storage); it reads every
1470
+ layer in it and shows one report. If the bucket becomes unreachable the server keeps serving what it has, names the
1471
+ backend in the client and `/readyz`, and recovers by itself.
1377
1472
 
1378
- - **Layers written to a backend by something else** (CI with bucket access, `atlas report merge`) are found at
1379
- the next refresh. Layers uploaded through the server are in the report at once.
1380
- - **A server can be a backend of another**: a local server reading the project's shared one shows both.
1473
+ - **Uploads go to the bucket** and are in the report at once. Layers written to the bucket by something else (CI
1474
+ with bucket access, `atlas report merge`) are found at the next refresh.
1475
+ - **Projects** are folders in the bucket (`projects/<slug>/`), as [below](#multiple-projects-on-one-node).
1476
+ - **Clients are different.** `atlas report publish --store …` on a laptop or CI job may name any store (a folder,
1477
+ a bucket, this server's URL); this page is about the server.
1478
+ - **Library use.** `createAtlasServer({ backends, write })` accepts any `ObjectStore` so an application embedding the
1479
+ server, or a test, can bring its own; `atlas serve` and `atlas dev` always use S3.
1381
1480
 
1382
1481
  ## Tokens and roles
1383
1482
 
@@ -1566,6 +1665,9 @@ curl "$ATLAS_SERVER_URL/api/projects/$ATLAS_PROJECT/reports/tests?format=markdow
1566
1665
  | `GET /admin` | read | Server settings page. |
1567
1666
  | `GET /api/settings` | admin | Read server settings. |
1568
1667
  | `PUT /api/settings` | admin | Save settings for the next restart. |
1668
+ | `GET /api/settings/gc` | admin | Read the garbage collection policy. |
1669
+ | `PUT /api/settings/gc` | admin | Save the garbage collection policy. |
1670
+ | `POST /api/settings/gc/run` | admin | Collect garbage now, or preview it. |
1569
1671
  | `GET /healthz` | none | Is the process up. |
1570
1672
  | `GET /readyz` | none | Has the server read its backends. |
1571
1673
  | `GET /api/openapi.json` | none | This document. |
@@ -1694,11 +1796,11 @@ What makes it so, none of it at the cost of the contract:
1694
1796
  - **One node for multiple projects.** Local environments and CI select their project slug and talk to the same node, which tells
1695
1797
  every client at once. Two servers on one backend both work and show the same report, but a client hears about
1696
1798
  an upload only from the server that took it and about the other's at the next refresh, so give a team one.
1697
- - **State is the write backend.** Back up `./atlas-data` (or the bucket). Restoring it, or starting a new server
1698
- on it, shows the same report.
1799
+ - **State is the bucket.** Reports and user state are all in it. Back it up, replicate or version it as any data;
1800
+ starting a new server on a restored copy shows the same report and applies the same garbage collection policy.
1699
1801
  - **Stopping.** `SIGTERM` stops accepting, lets requests in flight finish, ends the event streams and exits 0.
1700
1802
  - **Logs.** One JSON line per request (method, path, status, ms) to stdout, none for the probes. The first lines
1701
- say what it reads and which backend takes uploads, and warn when there are no tokens.
1803
+ say which bucket it uses, what the garbage collection policy in S3 is (or that none is saved), and warn when there are no tokens.
1702
1804
  - **A backend goes down.** The server keeps serving what it has, names the backend in the client and in
1703
1805
  `/readyz`, and recovers by itself. The gate fails closed: unknown is not a pass.
1704
1806
  - **Upgrades.** The layer format is a stable, versioned contract; upgrade the server before the clients that
@@ -1708,7 +1810,10 @@ What makes it so, none of it at the cost of the contract:
1708
1810
 
1709
1811
  Open **Settings** (`/admin`) and enter an admin token. The token stays in page memory. Read and publish tokens cannot edit settings; the browser sign-in box only grants reading. The API uses `Authorization: Bearer` and refuses cross-origin browser writes.
1710
1812
 
1711
- Use the Storage, Live updates, and Report rules forms to edit saved settings. Advanced includes request limits, logging, a JSON editor, and a read-only view of the active configuration. Save changes and restart Atlas to apply them. Atlas validates the configuration and atomically replaces `atlas.server.json`, preserving authentication and deployment settings. Changes apply at the next restart; environment variables and flags retain precedence. Concurrent edits require reloading the current revision. Data is never moved or removed by changing settings. Keep an old backend as a read source when moving future uploads to a new one. Embedded servers can provide their own settings persistence or manage configuration outside this API.
1813
+ The page has a Storage notice (the bucket is set by the `ATLAS_S3_*` variables and cannot be changed here), the **Garbage collection** form, and the Live updates and Report rules forms. Advanced includes request limits, logging, a JSON editor, and a read-only view of the active configuration.
1814
+
1815
+ - **Garbage collection** edits the policy: enabled, delete layers older than N days, always keep the newest N layers, run every N hours. *Save garbage collection policy* writes it to S3 at once (it applies at the next pass, no restart). *Preview deletions* counts what a pass would delete, per project, and deletes nothing. See [user state and garbage collection](#user-state-and-garbage-collection).
1816
+ - **Settings** forms save to the same JSON document in S3 and apply after a restart. Atlas validates them against the whole deployment first, and the configuration file, environment variables and flags retain precedence. Concurrent edits require reloading the current revision. Data is never moved or removed by changing settings. Embedded servers can provide their own persistence or manage settings outside this API.
1712
1817
 
1713
1818
  Native test reporters for CodeceptJS, Mocha, Jest and Node upload directly to this server; see [runner setup](runners.md). Every publisher uses the same server address and only a publish token. Storage credentials belong in the server environment.
1714
1819
 
@@ -1720,13 +1825,13 @@ Start with `POST /api/live/<uuid>`, send ordered check deltas with `PATCH /api/l
1720
1825
 
1721
1826
  The server uses its own receipt clock. By default it interrupts a run after 30 seconds without a heartbeat (`ATLAS_RUN_TIMEOUT_SECONDS`). Heartbeats do not reset progress: after 15 minutes with no changed tests/steps, a separate stall timeout interrupts it (`ATLAS_RUN_STALL_SECONDS`, 0 disables). Both settings are editable in the admin UI. Healthy long tests continue renewing heartbeats; increase or disable the progress timeout for tests that legitimately make no observable progress for longer.
1722
1827
 
1723
- Interrupted runs retain completed results and show why the remaining work stopped. Missing outcomes never become passes. The server persists bounded summaries under `server/live/` in its write backend (an HTTP relay keeps operational leases in its own data folder; the remote server only accepts immutable report objects) and restores outstanding leases as interrupted after a server restart. S3 server credentials need GetObject/PutObject for that prefix; generate the updated `atlas report policy --role both` policy. Producers still receive no backend credentials. Live `run` events carry bounded snapshots directly into test rows, including step logs and interruption reasons. The legacy `atlas:change` announcement remains available for older clients; its optional `liveRunId` identifies lease updates. Reconnecting viewers re-fetch progress, reports, and specifications. With SSE connected, live progress uses a 60-second safety refresh; without SSE it polls every two seconds. Returning to the tab or coming back online also revalidates reports and progress.
1828
+ Interrupted runs retain completed results and show why the remaining work stopped. Missing outcomes never become passes. The server persists bounded summaries under `server/live/` in its bucket and restores outstanding leases as interrupted after a server restart. S3 server credentials need GetObject/PutObject for that prefix; generate the updated `atlas report policy --role both` policy. Producers still receive no backend credentials. Live `run` events carry bounded snapshots directly into test rows, including step logs and interruption reasons. The legacy `atlas:change` announcement remains available for older clients; its optional `liveRunId` identifies lease updates. Reconnecting viewers re-fetch progress, reports, and specifications. With SSE connected, live progress uses a 60-second safety refresh; without SSE it polls every two seconds. Returning to the tab or coming back online also revalidates reports and progress.
1724
1829
 
1725
1830
  In this repository, use `bun atlas` and `bun example:run` in two terminals. Every example Gherkin scenario executes, including sharing and renaming; `@unrecorded` no longer excludes tests.
1726
1831
 
1727
1832
  ## Multiple projects on one node
1728
1833
 
1729
- Start the node with `atlas server` (or `bun atlas` in this repository). The server builds or finds the Atlas client and serves it with the API from one process. It never imports a consumer or an example. In this repository the node uses port 5180 and `./atlas-data`; `bun example:run` publishes the Todo app to project `todo-app`.
1834
+ Start the node with `atlas server` (or `bun atlas` in this repository). The server builds or finds the Atlas client and serves it with the API from one process. It never imports a consumer or an example. In this repository the node uses port 5180 and the bucket named by your `.env`; `bun example:run` publishes the Todo app to project `todo-app`.
1730
1835
 
1731
1836
  Set `ATLAS_PROJECT=todo-app` alongside `ATLAS_SERVER_URL` for native reporters and statechart runs. Report CLI commands accept `--project todo-app` (or `ATLAS_PROJECT`) with an HTTP store. Programmatic HTTP stores, live reporters and native reporter options accept `project: 'todo-app'`. Keep the same slug for the invocation's live progress, final results, spec sets and recordings. Slugs use 1–64 lowercase letters, digits or hyphens, starting and ending with a letter or digit.
1732
1837
 
@@ -1744,14 +1849,14 @@ Project indexes are opened lazily; at most 64 are active in memory. An index wit
1744
1849
 
1745
1850
  This is the production setup. A project runs **one Atlas server**. Developers' machines and every CI job send
1746
1851
  their results to it, as they finish or when they are done, and it shows everyone, developers and the business
1747
- alike, the whole project at once, live. It keeps results on its own disk by default, or in an object-store bucket
1748
- if you prefer; either way nothing is partitioned by run, branch or person: results are layered. The server's API,
1852
+ alike, the whole project at once, live. It keeps results (and all its user state) in an S3-compatible bucket and
1853
+ nowhere else; nothing is partitioned by run, branch or person: results are layered. The server's API,
1749
1854
  configuration and protocol are in **[the Atlas server](server.md)**; this page is the path to production.
1750
1855
 
1751
1856
  ```
1752
1857
  a developer's worktree ─┐ ┌─▶ developers (browser, live)
1753
- a CI job, any branch ───┼─ atlas report publish|stream ─▶ SERVER ◀┤ atlas serve: keeps layers in ./atlas-data
1754
- a device lab, nightly ──┘ (immutable layers, over HTTP) or S3 ├─▶ business or in a bucket (--write)
1858
+ a CI job, any branch ───┼─ atlas report publish|stream ─▶ SERVER ◀┤ atlas serve: keeps layers in S3
1859
+ a device lab, nightly ──┘ (immutable layers, over HTTP) ──▶ S3 ├─▶ business (required, nothing else)
1755
1860
  └─▶ CI gate atlas report status
1756
1861
  ```
1757
1862
 
@@ -1760,7 +1865,7 @@ Why this holds together:
1760
1865
  - **Publishing never conflicts.** A layer is one immutable file named by its content. Two machines that
1761
1866
  publish at once write different files; a retry writes the same one. There is no lock, no index to
1762
1867
  corrupt, nothing to merge.
1763
- - **The server is replaceable.** What it knows is its write backend (a folder, or a bucket). Replace it, restore it
1868
+ - **The server is replaceable.** What it knows is in its bucket. Replace it, restore it
1764
1869
  from a backup, or run a copy on a laptop: they show the same report, because a report is a function of the
1765
1870
  layers ([the contract](report-contract.md)).
1766
1871
  - **One view.** The newest completed result of every check, in every environment, from anyone. A
@@ -1778,13 +1883,15 @@ Why this holds together:
1778
1883
 
1779
1884
  ## 1. Where results are kept
1780
1885
 
1781
- **By default, on the server's own disk** (`./atlas-data`, or the volume you mount at `ATLAS_DATA`). Nothing else
1782
- needs to exist: skip to [section 3](#3-send-results-from-ci-and-from-laptops). Back that folder up like any data
1783
- directory; starting a new server on a restored copy shows the same report.
1886
+ **The server keeps everything in one S3-compatible bucket** (reports and user state, such as the garbage collection
1887
+ policy), set by the required `ATLAS_S3_ENDPOINT`, `ATLAS_S3_BUCKET`, `ATLAS_S3_REGION`, `ATLAS_S3_ACCESS_KEY_ID` and
1888
+ `ATLAS_S3_SECRET_ACCESS_KEY` variables ([details](server.md#required-s3-storage), template in
1889
+ [`.env.example`](https://github.com/crvouga/atlas/blob/main/.env.example)). Without them it refuses to start, listing
1890
+ every variable and what is wrong. There is no disk or database to fall back to, so the server is replaceable:
1891
+ a new one started on the same bucket shows the same report.
1784
1892
 
1785
- **To keep results in a bucket instead** (so a replaced server loses nothing, or CI already has bucket access),
1786
- name it and make it the place uploads go: `ATLAS_STORE=project=s3://my-bucket/atlas/my-project ATLAS_WRITE=project`.
1787
- The rest of this section is that setup. A store is named by a URL. Keys never go in it.
1893
+ **Clients and CI** name stores by URL, and a client may publish straight to a bucket or folder as well as to the
1894
+ server. A store is named by a URL. Keys never go in it.
1788
1895
 
1789
1896
  | Store | URL |
1790
1897
  | --- | --- |
@@ -1932,18 +2039,16 @@ A run is uploaded only when its privacy scan passed: the same rule the static cl
1932
2039
  ## 4. The server
1933
2040
 
1934
2041
  ```sh
1935
- atlas serve # ATLAS_STORE says where; listens on :8080
2042
+ atlas serve # ATLAS_S3_* says where; listens on :8080
1936
2043
  ```
1937
2044
 
1938
- It serves the client, the report and the project's specs and runs from all configured backends. It also
1939
- accepts uploads and announces new results on its event stream. Uploads go to `--data` by default;
1940
- use a named `--store project=s3://… --write project` to put uploads in a shared bucket, with publisher permissions.
2045
+ It serves the client, the report and the project's specs and runs from its bucket. It also
2046
+ accepts uploads and announces new results on its event stream. The bucket's credentials need list, read, write and delete
2047
+ permissions (`atlas report policy` prints a least-privilege policy; delete is for the garbage collection you define in the UI).
1941
2048
 
1942
2049
  | Flag | Environment | Meaning |
1943
2050
  | --- | --- | --- |
1944
- | `--store <url>` (repeatable) | `ATLAS_STORE` | Where layers are. Several stores are one report: repeat the flag, or separate them in the variable with commas or new lines. |
1945
- | `--data <dir>` | `ATLAS_DATA` | Local upload backend. Default `./atlas-data`. |
1946
- | `--write <id>` | `ATLAS_WRITE` | Backend that accepts uploads. Default `data`. |
2051
+ | | `ATLAS_S3_ENDPOINT`, `ATLAS_S3_BUCKET`, `ATLAS_S3_REGION`, `ATLAS_S3_ACCESS_KEY_ID`, `ATLAS_S3_SECRET_ACCESS_KEY` | **Required.** The S3-compatible bucket that holds all reports and user state. Optional: `ATLAS_S3_PREFIX`, `ATLAS_S3_PATH_STYLE`, `ATLAS_S3_SESSION_TOKEN`. |
1947
2052
  | `--port <n>` | `ATLAS_PORT` or `PORT` | Default 8080. `0` picks a free one. |
1948
2053
  | `--host <h>` | `ATLAS_HOST` or `HOST` | Default `127.0.0.1`. Listening beyond this machine requires tokens or `--insecure`. |
1949
2054
  | `--site <dir>` | `ATLAS_SITE` | The built client. Default: `./atlas-site`, else built once into `~/.cache/atlas/site`. |
@@ -1970,9 +2075,10 @@ updated `latest`. Pull request builds and failed publications are labelled accor
1970
2075
  Leave the start command unset: the image includes the server and the prebuilt viewer.
1971
2076
  2. Add `ATLAS_TOKEN` with a long random value (generate one with `openssl rand -hex 24`). It grants admin
1972
2077
  access; optionally add separate `ATLAS_PUBLISH_TOKEN` and `ATLAS_READ_TOKEN` for publishers and readers.
1973
- 3. Attach a volume at `/data` and set `RAILWAY_RUN_UID=0`: Railway mounts volumes as root, while the image
1974
- normally runs as `bun` ([Railway volume permissions](https://docs.railway.com/volumes#permissions)).
1975
- Keep one replica. The volume retains reports and server settings when the image is replaced.
2078
+ 3. Add the five required storage variables (`ATLAS_S3_ENDPOINT`, `ATLAS_S3_BUCKET`, `ATLAS_S3_REGION`,
2079
+ `ATLAS_S3_ACCESS_KEY_ID`, `ATLAS_S3_SECRET_ACCESS_KEY`) for a bucket you own. The service keeps nothing on its own
2080
+ disk, so no volume is needed and a replaced image loses nothing; without the variables it exits and lists each one.
2081
+ Keep one replica.
1976
2082
  4. Set the healthcheck path to `/readyz` and generate a public domain. The image listens on `0.0.0.0` and
1977
2083
  honors Railway's `PORT`; leave `ATLAS_PORT` unset. Open the domain and sign in with your token as the
1978
2084
  password (any username), or use the viewer's token sign-in.
@@ -1986,8 +2092,7 @@ Projects publish to that domain with `ATLAS_STORE=https://YOUR-DOMAIN` and their
1986
2092
  [sending results](#3-send-results-from-ci-and-from-laptops). Hosting Atlas needs no project checkout; publishing
1987
2093
  your project's test results still needs the integration described there.
1988
2094
 
1989
- For bucket storage instead, set `ATLAS_STORE=project=s3://my-bucket/atlas/my-project`, `ATLAS_WRITE=project`,
1990
- and the bucket credentials. Keep the `/data` volume for server settings and local state.
2095
+ Then open `/admin` to define the garbage collection policy; until you save one, nothing is deleted.
1991
2096
 
1992
2097
  ### Deploy without authentication
1993
2098
 
@@ -2003,12 +2108,11 @@ In Railway, create a **Docker image** service with:
2003
2108
  ghcr.io/crvouga/atlas-insecure:latest
2004
2109
  ```
2005
2110
 
2006
- Leave the start command unset. Attach a persistent volume at `/data`, set `RAILWAY_RUN_UID=0`, keep one
2111
+ Leave the start command unset. Add the required `ATLAS_S3_*` storage variables, keep one
2007
2112
  replica, set the healthcheck to `/readyz` and generate a public domain. Leave `ATLAS_PORT` unset so Railway's
2008
2113
  `PORT` is used. Enable **Settings → Source → Configure Auto Updates → Anytime** to follow `latest`.
2009
2114
  Do not set `ATLAS_TOKEN`, `ATLAS_PUBLISH_TOKEN` or `ATLAS_READ_TOKEN`, or add `auth.tokens` to
2010
- `/data/atlas.server.json`: configured tokens still enable authentication. When reusing a volume from an
2011
- authenticated deployment, remove its saved auth tokens first. Open the domain directly, including `/admin`
2115
+ a configuration file: configured tokens still enable authentication. Open the domain directly, including `/admin`
2012
2116
  for settings (leave the token field blank and choose **Load settings**).
2013
2117
 
2014
2118
  Verify anonymous reads and uploads without an Authorization header:
@@ -2029,7 +2133,7 @@ leave its `ATLAS_TOKEN` unset. Use the same publishing commands described in
2029
2133
  For local Docker:
2030
2134
 
2031
2135
  ```sh
2032
- docker run --pull always -p 8080:8080 -v atlas-data:/data ghcr.io/crvouga/atlas-insecure:latest
2136
+ docker run --pull always -p 8080:8080 --env-file .env ghcr.io/crvouga/atlas-insecure:latest
2033
2137
  ```
2034
2138
 
2035
2139
  Both images publish full commit tags for pinning or rollback. A new GHCR package must be made Public
@@ -2039,10 +2143,11 @@ once before Railway can pull without registry credentials; see
2039
2143
  ### Docker
2040
2144
 
2041
2145
  The same published image works anywhere Docker runs. It already contains the client (the same
2042
- for every project: the server tells the page where its data is), and keeps results on the volume at `/data`:
2146
+ for every project: the server tells the page where its data is), and keeps nothing on its own disk: it needs the
2147
+ `ATLAS_S3_*` variables (copy `.env.example` to `.env` and fill it in):
2043
2148
 
2044
2149
  ```sh
2045
- docker run --pull always -p 8080:8080 -v atlas-data:/data \
2150
+ docker run --pull always -p 8080:8080 --env-file .env \
2046
2151
  -e ATLAS_TOKEN=$(openssl rand -hex 24) ghcr.io/crvouga/atlas:latest
2047
2152
  ```
2048
2153
 
@@ -2051,17 +2156,15 @@ back, replace `latest` with the full commit SHA. To build from this repository,
2051
2156
  [`examples/deploy/Dockerfile`](https://github.com/crvouga/atlas/blob/main/examples/deploy/Dockerfile) is the
2052
2157
  alternative for building from an npm version or a local package tarball.
2053
2158
 
2054
- For a bucket as the system of record add `-e ATLAS_STORE=project=s3://my-bucket/atlas/my-project -e ATLAS_WRITE=project -e AWS_REGION=us-east-1`.
2055
- On ECS, EKS or EC2, attach the combined **both** policy (including `server/live/*` for producer leases) to the task, service account or
2056
- instance and set no keys.
2159
+ The bucket's account needs the combined **both** policy (including `server/live/*` for producer leases and `server/state.json`
2160
+ for user state) plus `s3:DeleteObject` for garbage collection. The server does not use ambient AWS roles: pass the keys.
2057
2161
 
2058
2162
  **Try it on one machine.** [`examples/deploy/compose.yaml`](https://github.com/crvouga/atlas/blob/main/examples/deploy/compose.yaml)
2059
- starts the published image on a volume; `compose.s3.yaml` starts it with an S3 server and a bucket as its storage.
2163
+ starts the published image with an S3 server (SeaweedFS) and a bucket as its storage; `compose.s3.yaml` is the same stack without tokens.
2060
2164
  Publish to `http://localhost:8080` from your project and open it.
2061
2165
 
2062
2166
  **Kubernetes.** [`examples/deploy/kubernetes.yaml`](https://github.com/crvouga/atlas/blob/main/examples/deploy/kubernetes.yaml):
2063
- one copy (the server is a singleton for the project), results in the bucket, readiness on `/readyz`, an IRSA
2064
- service account.
2167
+ one copy (the server is a singleton for the project), results in the bucket (keys from a Secret), readiness on `/readyz`.
2065
2168
 
2066
2169
  **Without a container.** `bun add -d @crvouga/atlas`, then `bunx atlas client build --no-data --out
2067
2170
  atlas-site` and `bunx atlas serve`. Everything it can be told is in
@@ -2134,12 +2237,13 @@ Narrow it by choice, per consumer, never by moving data:
2134
2237
  listing (one request per thousand layers) runs at the start and every 30 refreshes (five minutes by
2135
2238
  default), which is also how a layer recorded with an old timestamp is found. One small `GET` per new
2136
2239
  layer, once.
2137
- - **Retention.** Atlas deletes nothing. `atlas report doctor` shows what the bucket holds.
2138
- - **Disaster recovery.** The write backend (the data folder, or the bucket) is the only state. Back it up, replicate or version it as you
2240
+ - **Retention.** Atlas deletes nothing until you define a garbage collection policy in the UI (`/admin`); it is kept in the bucket as JSON and
2241
+ then removes only superseded layers older than you allow ([details](server.md#user-state-and-garbage-collection)). `atlas report doctor` shows what the bucket holds.
2242
+ - **Disaster recovery.** The bucket is the only state (reports and user state). Back it up, replicate or version it as you
2139
2243
  do any data; a server is rebuilt from nothing in the time it takes to read the layers (below). It listens at once
2140
2244
  and answers `/readyz` 503 until it has, so a liveness probe never kills one that is still reading.
2141
- - **A second bucket.** Another team's layers, a device lab, an archive: pass several `--store`s. They are one
2142
- report, attributed to the store they came from.
2245
+ - **A second bucket.** The server reads one bucket. To combine another team's layers or an archive into the
2246
+ report, copy them in with `atlas report merge --into <store>`; clients and `atlas report status` may read several stores.
2143
2247
 
2144
2248
  ## 7. Performance
2145
2249
 
@@ -2392,7 +2496,7 @@ are part of the contract:
2392
2496
  | **Partition independence** | Splitting the same layers across any number of backends, machines or branches, with any overlap, gives the same report. This is what makes partial runs on separate nodes safe. |
2393
2497
  | **Idempotence** | A layer present twice, in one backend or several, counts once. Publishing the same checks again writes the same layer. |
2394
2498
  | **Monotonic** | A layer changes only the slots it fills. Adding a layer never changes a result it did not record. |
2395
- | **Deletion-free** | Nothing in the contract removes or rewrites a layer. Retention is the operator's choice. |
2499
+ | **Deletion-free** | Nothing in the contract removes or rewrites a layer. Retention is the operator's choice: Atlas deletes only under a garbage collection policy an operator saved in the UI, and then only layers that no longer decide any result ([the server](server.md#user-state-and-garbage-collection)). |
2396
2500
  | **Unknown is not a pass** | A layer that cannot be read, a backend that is down, and a check that is `running` or `not-run` never count as passing. The gate fails closed. |
2397
2501
  | **Aggregation by copying** | Layers are immutable and never share a key with different bytes, so combining stores is copying files (`atlas report merge`, `cp -R`, `rsync`, a CI artifact download). Copying is idempotent and commutative. |
2398
2502
 
@@ -2834,150 +2938,7 @@ unaffected.
2834
2938
  </doc>
2835
2939
 
2836
2940
  <doc path="docs/report-backends.md">
2837
- # Report backends and live snapshots
2838
-
2839
- > [!NOTE]
2840
- > Atlas has two kinds of backend, and this page is about the first.
2841
- >
2842
- > - **Run sources** (this page): where the client finds *statechart runs* (`manifest.json` per run)
2843
- > and live snapshots of a run in progress: a directory, HTTP files, or an HTTP API.
2844
- > - **Layer stores**: where *layers of evidence for any kind of spec* are kept, an `ObjectStore`
2845
- > such as a directory or a bucket. Any number of them are one report. See
2846
- > [the report contract](report-contract.md#backends) and [integrating Atlas](integrating.md).
2847
- >
2848
- > Publish a run as a layer (`atlas report publish --manifest`) to have it combine with other suites,
2849
- > environments and branches.
2850
-
2851
- Atlas maps the product's areas, screens, events and journeys from a spec, then overlays evidence
2852
- of what the product did. Specs and reports are independent. Local development, CI, device labs
2853
- and hosted runners can supply reports to the same product atlas simultaneously. The client works
2854
- before the first report exists.
2855
-
2856
- ## Connect sources
2857
-
2858
- The default source is the directory passed to `--runs`. Add sources with a JSON file:
2859
-
2860
- ```json
2861
- {
2862
- "schemaVersion": 1,
2863
- "sources": [
2864
- { "id": "mobile", "label": "Mobile runs", "type": "directory", "runs": "./mobile-runs" },
2865
- { "id": "ci", "label": "CI reports", "type": "http", "baseUrl": "https://reports.example.com/atlas-data/" },
2866
- { "id": "runner", "label": "Hosted runner", "type": "api", "baseUrl": "https://runner.example.com/atlas/", "eventsUrl": "events", "pollMs": 10000 }
2867
- ]
2868
- }
2869
- ```
2870
-
2871
- ```sh
2872
- atlas client dev --specs ./specs --runs ./atlas-runs --sources ./atlas.sources.json
2873
- atlas client build --specs ./specs --runs ./atlas-runs --sources ./atlas.sources.json --out ./atlas-site
2874
- atlas client publish-data --specs ./specs --runs ./atlas-runs --sources ./atlas.sources.json --out ./atlas-data
2875
- ```
2876
-
2877
- Directory paths resolve against the configuration file. Source ids are unique, stable names
2878
- containing letters, numbers, underscores or hyphens. `workspace` is reserved for `--runs`.
2879
- `ATLAS_SOURCES_FILE` is the environment equivalent. Restart dev after changing the file.
2880
- Credentials must not appear in this file or URLs: descriptors are sent to the browser and
2881
- included in static output.
2882
-
2883
- Remote services must allow requests
2884
- from the client's origin. For private services, use an authenticated same-origin gateway. Atlas
2885
- does not download private GitHub artifacts or sign object-storage requests itself: expose their
2886
- published data over HTTP or implement a gateway using the contract below.
2887
-
2888
- ## Transport contracts
2889
-
2890
- | Backend | Index | Snapshot | Media and other reports | Updates |
2891
- | --- | --- | --- | --- | --- |
2892
- | Directory | Generated from run directories | `<run>/manifest.json` | Files beside the manifest | File watcher and polling fallback |
2893
- | HTTP files | `runs/index.json` | `runs/<id>/manifest.json` | `runs/<id>/<path>` | Polling; optional server events |
2894
- | HTTP API | `runs` | `runs/<id>/manifest` | `runs/<id>/files/<path>` | Polling; optional server events |
2895
2941
 
2896
- Locations are relative to `baseUrl`. The index follows the Atlas runs-index contract:
2897
-
2898
- ```json
2899
- {
2900
- "runs": [{
2901
- "id": "build-123",
2902
- "startedAt": "2026-10-09T11:59:00Z",
2903
- "progress": "running",
2904
- "hasManifest": true,
2905
- "execution": {
2906
- "totalPaths": 8,
2907
- "completedPaths": 2,
2908
- "activePaths": ["journey-3"],
2909
- "updatedAt": "2026-10-09T12:00:00Z"
2910
- }
2911
- }]
2912
- }
2913
- ```
2914
-
2915
- Each manifest is a full snapshot of everything received so far, rather than a patch or unfinished
2916
- JSON document. Records can be absent; malformed records are reported individually. Set
2917
- `run.finishedAt` to `null` while running, then a timestamp when complete. Its absence preserves
2918
- the meaning of older completed reports. Optional `run.progress` has the same shape as `execution`
2919
- above. Paths can declare `progress: queued | running | complete`.
2920
-
2921
- Unvisited `not-reached` records appear pending during an active run. Passed and failed records
2922
- appear immediately. A PNG can arrive before its optional WebP thumbnail. Promised media can
2923
- declare `screenshotState: processing` or `clip.state: processing` until ready.
2924
-
2925
- The runner publishes an initial snapshot, each reached screen and completed step, each completed
2926
- path and run completion. File replacements are atomic. Active outcomes stay out of
2927
- `outcomes.json`, so merging and rerunning reuse only completed paths. Results retain deterministic
2928
- plan order across workers.
2929
-
2930
- ## Server events
2931
-
2932
- An optional `eventsUrl`, relative to `baseUrl`, serves `text/event-stream`. Send ordinary messages
2933
- or named `atlas:change` events:
2934
-
2935
- ```text
2936
- event: atlas:change
2937
- id: 124
2938
- data: {"scope":"runs","runIds":["build-123"]}
2939
-
2940
- ```
2941
-
2942
- These invalidate the source's index and named snapshots. Empty `runIds` refreshes cached snapshots
2943
- for this source. Reconnection does the same to recover missed updates. Malformed notices are
2944
- ignored. Polling remains available if a stream disconnects or EventSource is unavailable.
2945
-
2946
- Configured source indexes poll every ten seconds by default (`pollMs` overrides this); selected
2947
- running snapshots poll every two seconds. `VITE_ATLAS_POLL_MS` controls spec polling. Pause live
2948
- updates to keep a snapshot steady; manual Refresh still works. Resuming refreshes immediately.
2949
-
2950
- ## Identity, failures and publishing
2951
-
2952
- Client identities combine source ids and native run ids. Cache keys include the backend location
2953
- and transport. Matching run names cannot mix results or media. Shared links retain the qualified
2954
- identity; old links containing only a run id still resolve to workspace reports.
2955
-
2956
- One unavailable source leaves other sources and the spec usable. Failed refreshes retain the
2957
- last valid index and snapshot, with the failure recorded in data issues. A missing first snapshot
2958
- is a waiting state for a running report. Latest follows the newest report, including active runs;
2959
- choosing a particular report keeps it selected as newer reports arrive.
2960
-
2961
- Publishing includes privacy-checked running snapshots and finished runs. Extra directory sources
2962
- are copied under `data/backends/<id>/`; `sources.json` uses relative URLs. Remote descriptors stay
2963
- remote. Missing or failed privacy checks exclude the run, and temporary files and work directories
2964
- are excluded. `--keep` applies per directory source. Publish updated data to the same host while
2965
- a run is active for static viewers to receive its snapshots through polling.
2966
-
2967
- `VITE_ATLAS_BASE_URL` sets the primary spec/data location; `VITE_ATLAS_SOURCES_URL` can point to a
2968
- descriptor file hosted elsewhere. Published descriptors translate directory sources into HTTP
2969
- URLs and never expose the directory paths.
2970
-
2971
- ## Another transport
2972
-
2973
- `src/data/adapters/report-backend.ts` defines `ReportBackend`: stable identity and cache key,
2974
- report listing, snapshot loading, media resolution and optional change subscription. The query
2975
- layer consumes this separately from the spec adapter. An embedded client can supply a backend
2976
- through `ReportSourcesContext`; a new built-in transport can add a factory alongside
2977
- `createReportBackend`. Screens, maps and journeys require no transport-specific code.
2978
-
2979
- The generated JSON Schemas include `report-sources`, `atlas-change`, `run-info` and `run-summary`
2980
- alongside the existing manifest and record schemas.
2981
2942
  </doc>
2982
2943
 
2983
2944
  <doc path="docs/statecharts.md">
@@ -3063,7 +3024,7 @@ In another terminal run `bun run example:run`: statecharts, Gherkin scenarios an
3063
3024
  to project `todo-app` on that server while testing. Producers renew a heartbeat every 5 seconds; 30 seconds of silence
3064
3025
  marks an invocation interrupted. Final specs and recordings upload when their privacy scan completes.
3065
3026
  Tests show inline running indicators and logs without waiting for the whole suite to finish. Report history
3066
- is retained in `atlas-data/projects/todo-app`; local run artifacts remain in `atlas-runs` and
3027
+ is retained in the server's bucket under `projects/todo-app/`; local run artifacts remain in `atlas-runs` and
3067
3028
  `atlas-reports`. To upload a completed run again without rerunning tests,
3068
3029
  use `bun run example:publish`. Set `ATLAS_SERVER_URL` for a different upload destination (and
3069
3030
  `ATLAS_PUBLISH_TOKEN` when it requires one). Restart `bun atlas` when switching from the older
@@ -3689,8 +3650,11 @@ bun run workspace:status
3689
3650
  ```
3690
3651
 
3691
3652
  Everything binds to `127.0.0.1`. For remote workspaces, use the IDE's port forwarding. Reports and server
3692
- settings stay in this checkout's `.atlas-data/`; live runtime leases stay in `.atlas-workspace/`.
3693
- Keep both directories private to each worktree. Do not share `node_modules`, `.atlas-data/`,
3653
+ settings are kept in S3 only, never in the checkout: `workspace:run` needs the required `ATLAS_S3_*` variables
3654
+ (copy `.env.example` to `.env`, which Bun loads, or export them) and exits at once, listing every missing one,
3655
+ without them. Give each checkout its own bucket or `ATLAS_S3_PREFIX` if their reports must not mix
3656
+ (`workspace:setup` and `workspace:preview` need no credentials). Live runtime leases stay in `.atlas-workspace/`.
3657
+ Keep that directory private to each worktree. Do not share `node_modules`,
3694
3658
  `.atlas-workspace/` or caches across checkouts with different dependencies. Frozen installs use Bun's
3695
3659
  download cache without linking one worktree's dependency tree to another.
3696
3660
 
@@ -3804,6 +3768,13 @@ they preserve evidence semantics, provenance, URL selection, or playback behavio
3804
3768
  `Bounded`/show-more controls. Do not fetch unopened results' recordings eagerly.
3805
3769
  - Loading: named `Loader`/`Skeleton` or explicit text with `role="status"`. Error: `Alert color="red"` and
3806
3770
  a retry when possible. Empty: descriptive `Text` and a relevant action. Use built-in loading/disabled states.
3771
+ - Waiting is bounded. A focused page (a spec, a recording) loads what its link names on its own and never waits for
3772
+ the whole catalog. `loadingPolicy.stallMs` in `evidence/load-state.tsx` is how long a read may take before the page
3773
+ names what is affected, keeps the rest usable and offers `Try again` and a way back that keeps the selected scope.
3774
+ A missing spec or layer is an explicit unavailable state with its path, never an endless `Loading…`. Layers that
3775
+ fail to read are counted (`EvidenceIncomplete`) and the rest stays usable. Search disclosed as incomplete (still
3776
+ reading, or specs it could not read) never reports "no match" as final. `tests/resilient-navigation.test.tsx`
3777
+ holds the request budgets for a large catalog.
3807
3778
  - Spec overview links use a centered `Group`: status, flexible title/folder, then outcome and time. Keep gaps
3808
3779
  explicit with theme spacing; stack metadata on narrow screens instead of crowding the title.
3809
3780
 
@@ -3888,7 +3859,7 @@ The narrowly scoped exceptions are:
3888
3859
  | Evidence | Steps, retries, annotations, facts, unknown extensions, safe downloads and previews | List, Badge, Accordion, Code, Image, native media |
3889
3860
  | Recordings | Pictures/video/timeline, actor selection, moments, failure jumps, timing, model-state differences | Tabs, Slider, Button, ScrollArea, Table, Progress |
3890
3861
  | Search/help | Ranked and semantic search, shortcut help, recording shortcuts | Spotlight, Modal, Kbd |
3891
- | Administration | Token access, storage backends/write destination, live timeouts, report rules, limits, raw JSON, conflicts | Same provider/theme, Fieldset, Mantine inputs, Accordion, Alert |
3862
+ | Administration | Token access, S3 storage notice, garbage collection policy (saved in S3), live timeouts, report rules, limits, raw JSON, conflicts | Same provider/theme, Fieldset, Mantine inputs, Accordion, Alert |
3892
3863
 
3893
3864
  The hash routes (`/`, `/chart/$chartId`, `/specs/$kind`), query contracts, project scoping, streaming,
3894
3865
  report APIs and data adapters remain the migration boundary. `/admin` still serves an independent document:
@@ -4269,7 +4240,7 @@ atlas — executable statecharts
4269
4240
  [--no-retry] [--output <dir>]
4270
4241
  atlas merge [--config atlas.config.ts] [--output <dir>] <run-dir> <run-dir>...
4271
4242
  atlas export --specs <dir> --format scxml|mermaid|markdown|json [--out <file>]
4272
- atlas server [--data <dir>] [--port 8080] [--store <url>]... one Atlas node: API, projects and client
4243
+ atlas server [--port 8080] one Atlas node: API, projects and client; needs ATLAS_S3_* (atlas serve --help)
4273
4244
  atlas client build [--no-data] [--out <dir>] build the project-agnostic Atlas client
4274
4245
  atlas dev [--port 5180] develop the Atlas server and client on one host
4275
4246
  atlas init [--server <url>] [--write] integrate this project: detect its suites, print the plan (atlas init --help)
@@ -4306,14 +4277,18 @@ atlas init — integrate a project with an Atlas server
4306
4277
  ```text
4307
4278
  atlas serve: the Atlas server
4308
4279
 
4309
- atlas serve [--config atlas.server.json] [--data <dir>] [--port 8080] [--host 127.0.0.1]
4310
- [--store <url|id=url>]... [--write <id>] [--site <dir>]
4280
+ atlas serve [--config atlas.server.json] [--port 8080] [--host 127.0.0.1] [--site <dir>]
4311
4281
  [--outline-branch <b>] [--branch <name>]... [--ignore-modified] [--refresh <seconds>]
4312
4282
  [--anonymous-read] [--insecure] [--no-log]
4313
4283
 
4314
4284
  One process serves the client and the API. Test results are uploaded to it from anywhere (laptops, CI) and it
4315
- shows everyone the combined report, live. It keeps its data in --data (default ./atlas-data) and can read more
4316
- backends (--store): S3, other servers, folders. Uploads go to the data folder unless --write names another.
4285
+ shows everyone the combined report, live.
4286
+
4287
+ Storage is required. ALL reports and ALL user state (the garbage collection policy, saved settings) are kept in
4288
+ S3-compatible object storage and nowhere else; there is no local-disk, in-memory or database fallback. Without
4289
+ these environment variables the server prints every one of them with what is wrong, and exits with status 2:
4290
+ ATLAS_S3_ENDPOINT ATLAS_S3_BUCKET ATLAS_S3_REGION ATLAS_S3_ACCESS_KEY_ID ATLAS_S3_SECRET_ACCESS_KEY
4291
+ Optional: ATLAS_S3_PREFIX, ATLAS_S3_PATH_STYLE, ATLAS_S3_SESSION_TOKEN. See .env.example.
4317
4292
 
4318
4293
  Tokens (ATLAS_TOKEN admin, ATLAS_PUBLISH_TOKEN, ATLAS_READ_TOKEN, or "auth.tokens" in the config) are required
4319
4294
  to listen beyond this machine unless --insecure. Every option has an environment variable (docs/deploying.md).
@@ -4324,11 +4299,12 @@ atlas serve: the Atlas server
4324
4299
  ```text
4325
4300
  atlas dev: the Atlas server and the client, for working on either
4326
4301
 
4327
- atlas dev [--port 5180] [--host 127.0.0.1] [--data <dir>] [--store <url|id=url>]... [--config atlas.server.json]
4302
+ atlas dev [--port 5180] [--host 127.0.0.1] [--config atlas.server.json]
4328
4303
 
4329
4304
  Starts the server (API only, on a port of its own) and the client's dev server, which proxies /api to it, so the
4330
4305
  browser talks to one host and a change to the client shows at once. In production `atlas serve` does both from
4331
- one process.
4306
+ one process. Storage is the same as for `atlas serve`: the required ATLAS_S3_* environment variables name the S3
4307
+ bucket that holds all reports and user state; without them it exits with status 2 and lists every variable.
4332
4308
  ```
4333
4309
 
4334
4310
  ## atlas report
@@ -4532,7 +4508,7 @@ Responses: `200` Settings editor.; `401` No token, or an unknown one. Send `Auth
4532
4508
 
4533
4509
  `GET /api/settings` · role: admin
4534
4510
 
4535
- Returns editable persisted storage/report settings, active settings, a revision, and configuration precedence. Authentication settings are excluded. Requires an admin Bearer token. Embedded applications may manage settings outside this API.
4511
+ Returns the editable report and live-run settings saved in the user state in S3, active settings, a revision, and configuration precedence. Storage and authentication settings are excluded. Requires an admin Bearer token. Embedded applications may manage settings outside this API.
4536
4512
 
4537
4513
  Responses: `200` Settings and revision.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `503` Configuration is managed by the embedding application.
4538
4514
 
@@ -4540,12 +4516,40 @@ Responses: `200` Settings and revision.; `401` No token, or an unknown one. Send
4540
4516
 
4541
4517
  `PUT /api/settings` · role: admin
4542
4518
 
4543
- Replaces editable storage/report settings, preserving authentication and other file fields. Atomically persists a validated configuration; no report data is moved. Environment variables and flags override these values after restart. Requires an admin Bearer token, same-origin requests in browsers, and a current revision.
4519
+ Replaces the editable report and live-run settings in the user state in S3 (`server/state.json`), preserving the garbage collection policy and other fields. Persists a validated configuration; no report data is moved. Environment variables and flags override these values after restart. Requires an admin Bearer token, same-origin requests in browsers, and a current revision.
4544
4520
 
4545
4521
  - `If-Match` (header, required): The quoted revision returned by GET /api/settings.
4546
4522
 
4547
4523
  Responses: `200` Saved settings and new revision. Restart required.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `409` Settings changed; reload before saving.; `413` Settings exceed the body limit.; `422` Invalid configuration.; `428` A quoted settings revision is required.; `503` Configuration is managed by the embedding application.
4548
4524
 
4525
+ ### Read the garbage collection policy
4526
+
4527
+ `GET /api/settings/gc` · role: admin
4528
+
4529
+ Returns the policy kept in the user state in S3 (`server/state.json`), or `policy: null, saved: false` when none has been saved yet, in which case garbage collection is off and nothing is deleted. Also returns the revision to send back when saving and the last pass over each project. Requires an admin Bearer token. If the stored user state is unreadable the answer is 503 and the stored object is left untouched.
4530
+
4531
+ Responses: `200` The policy, its revision and the last passes.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `503` The user state in S3 cannot be read, or settings are managed by the embedding application.
4532
+
4533
+ ### Save the garbage collection policy
4534
+
4535
+ `PUT /api/settings/gc` · role: admin
4536
+
4537
+ Replaces the policy in the user state in S3 and leaves the rest of that document alone. The server reads the policy from S3 each time it collects, so the new policy applies at the next pass, without a restart. Requires an admin Bearer token, same-origin requests in browsers, and the current revision.
4538
+
4539
+ - `If-Match` (header, required): The quoted revision returned by GET /api/settings/gc.
4540
+
4541
+ Responses: `200` The saved policy and the new revision.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `409` The state changed; reload before saving.; `413` The body exceeds the limit.; `422` Invalid policy.; `428` A quoted revision is required.; `503` The user state in S3 cannot be read, or settings are managed by the embedding application.
4542
+
4543
+ ### Collect garbage now, or preview it
4544
+
4545
+ `POST /api/settings/gc/run` · role: admin
4546
+
4547
+ Applies the policy saved in S3 to every project now, instead of waiting for the schedule. Only report layers that were read and no longer decide any result, older than the policy allows and beyond its `keepLatest`, are deleted, with the captured files only they used. With `dryRun=1` nothing is deleted and each run reports what it would delete (a disabled policy is previewed as if enabled). Answers 409 when no policy has been saved, and, without `dryRun`, when the saved policy is disabled. Requires an admin Bearer token and a same-origin request in browsers.
4548
+
4549
+ - `dryRun` (query): `1` or `true` to only count what would be deleted.
4550
+
4551
+ Responses: `200` One run per project.; `401` No token, or an unknown one. Send `Authorization: Bearer <token>`.; `403` The token is known but its role may not do this.; `409` No policy has been saved, or it is disabled.
4552
+
4549
4553
  ### Is the process up
4550
4554
 
4551
4555
  `GET /healthz` · role: none