@crvouga/atlas 0.4.0 → 0.5.0

package/docs/report-contract.md

@@ -0,0 +1,400 @@
+# The report contract
+
+This is the normative description of what Atlas reads and writes when it reports on specs of any
+kind: Gherkin features, Quint models, DMN tables, Alloy structures, OpenSLO objectives, statecharts,
+unit tests, linters, anything that ends in "this check passed or failed". If this page and the code
+disagree, that is a bug: [file it](reporting-issues.md).
+
+The contract is small on purpose. It is one document (the **layer**), one rule (how layers compose),
+and one interface (the **backend** that stores them). Everything else, the viewer, the gate, the CLI,
+is built on those three and can be replaced.
+
+- [Concepts](#concepts)
+- [The layer document](#the-layer-document)
+- [How layers compose](#how-layers-compose)
+- [Guarantees](#guarantees)
+- [Statuses and the gate](#statuses-and-the-gate)
+- [Backends](#backends)
+- [Where bytes go](#where-bytes-go)
+- [Versioning and stability](#versioning-and-stability)
+- [Failure behaviour](#failure-behaviour)
+- [Speed](#speed)
+- [Other documents of the contract](#other-documents-of-the-contract)
+
+## Concepts
+
+| Term | Meaning |
+| --- | --- |
+| **Check** | One thing a producer verified, with a stable `id` and a status: `passed`, `failed`, `running` or `not-run`. |
+| **Layer** | One focused recording: the checks one **producer** ran, on one **environment**, from one version and branch, at one time. Immutable, addressed by its content. |
+| **Producer** | A stable name for the suite or tool that makes layers: `gherkin`, `web-e2e`, `quint`, `openslo`. |
+| **Environment** | The client that was checked: `desktop-web`, `mobile-web`, `ios`, `android`, or any other lowercase name. |
+| **Slot** | The cell a check fills: producer, kind, spec, check id, environment and parameters. |
+| **Backend** | A store that holds layers. Any [`ObjectStore`](#backends): a directory, a bucket, a device lab. |
+| **Report** | The result of laying every layer of every backend over the current specs. Computed, never stored. |
+
+Two ideas follow, and the rest of this page is their consequences.
+
+1. **A report is a function of the set of layers.** There is no "the run". Nothing partitions a
+   report by run, commit, branch, machine or backend; those are provenance written on each layer.
+2. **Partial runs are the normal way to build a report.** A node that runs one shard, a branch that
+   re-runs one scenario, a device lab that checks one environment: each publishes the layer of what it
+   ran. The report fills in as layers arrive, and no node needs to know about any other.
+
+## The layer document
+
+`layers/manifests/<id>.json`, schema version **1**. The JSON Schema (draft 2020-12) is shipped as
+[`@crvouga/atlas/schemas/layer.schema.json`](../packages/schema/schemas/layer.schema.json), and the
+Zod schema as `LayerDocumentSchema` from `@crvouga/atlas/schema`.
+
+```json
+{
+  "schemaVersion": 1,
+  "id": "9f2c…64 hex digits",
+  "producer": "gherkin",
+  "recordedAtMs": 1760000000000,
+  "sha": "0123456789abcdef0123456789abcdef01234567",
+  "branch": "main",
+  "runUrl": "https://ci.example.com/runs/42",
+  "environment": "ios",
+  "parameters": { "device": "iphone-15" },
+  "specs": [{ "path": "specs/join.feature", "domain": "list", "kind": "gherkin", "title": "Join a list", "fingerprint": "13:1a2b3c:4d5e6f" }],
+  "parts": [{
+    "id": "gherkin", "kind": "behavior", "title": "Gherkin", "recordedAtMs": 1760000000000, "sha": "0123…",
+    "checks": [
+      { "id": "specs/join.feature > Joins", "title": "Joins", "status": "passed", "detail": "", "specPath": "specs/join.feature", "measurements": [], "durationMs": 120 },
+      { "id": "specs/join.feature > Leaves", "title": "Leaves", "status": "failed", "detail": "expected 2 members, saw 3", "specPath": "specs/join.feature", "measurements": [{ "name": "latency", "value": 1400, "unit": "ms", "limit": 2500 }] }
+    ]
+  }],
+  "source": null,
+  "recording": null,
+  "bundle": {}
+}
+```
+
+| Field | Rule |
+| --- | --- |
+| `schemaVersion` | `1`. A layer without it, written before it existed, is version 1. |
+| `id` | 64 lowercase hex digits: the SHA-256 of this document without `id`. Opaque to readers. The same checks recorded the same way give the same id, so publishing twice is harmless. |
+| `producer` | Required, non-empty. Part of the [slot](#how-layers-compose). |
+| `recordedAtMs` | When the checks ran, epoch milliseconds, `>= 0`. The newest completed result wins its slot. Take it from the test run, never from the moment of publishing, or a retried publish would claim to be newer than a later run. |
+| `sha`, `branch`, `runUrl` | Provenance, or `null`. **Never part of the slot.** |
+| `environment` | The client checked, or absent. Normalised to a lowercase slug: `iOS` → `ios`, `Desktop Web` → `desktop-web`, `iphone` → `ios`, `mobile browser` → `mobile-web`. Any other name is kept. Part of the slot. |
+| `parameters` | Other string choices that make a result its own slot, such as `{ "device": "iphone-15" }`. A parameter named `environment` is read as the environment. Part of the slot. |
+| `freshness` | `modified`: recorded from a checkout with uncommitted changes. `unknown`: origin not captured. Advisory only; see [Statuses](#statuses-and-the-gate). |
+| `specs` | The specs the layer was recorded against, with a `fingerprint` of their text, so a reader can warn when a spec changed since. Optional. |
+| `parts[].kind` | An open category (`behavior`, `performance`, `accessibility`, …). A new kind needs no change to any reader. Part of the slot. |
+| `parts[].checks[]` | `id` (required, stable), `title`, `status`, `detail`, `specPath`, `measurements`, `durationMs`. |
+| `tags`, `labels`, `links`, `attachments`, `annotations`, `extensions` | Evidence, on the layer, a part or a check: what a result carries beyond its outcome. All optional and open. See [evidence](#evidence) and [the guide](extending.md). |
+| `checks[].steps`, `checks[].attempts` | A check's timeline, and every try when it was retried. Optional. |
+| `checks[].specPath` | The spec the check belongs to, as `specs/<path inside the specs directory>`. Anything before the first `specs/` is dropped, so `packages/app/specs/join.feature` and `specs/join.feature` are the same spec. |
+| `measurements[]` | `{ name, value, unit, limit? }`, finite numbers. A measurement over its `limit` is shown, and never fails the check by itself. |
+| `source`, `recording`, `bundle` | Optional timelines and captured files (screenshots, frames) a suite recorded. `null` and `{}` for a layer of plain checks. |
+
+**Unknown fields are allowed and preserved.** Every object in the schema is loose. A later minor
+version may add fields; an older reader ignores them and a copy carries them along.
+
+### Evidence
+
+A check, a part and a layer may carry evidence beyond their outcome. It is the part of the format meant to grow:
+every field is optional, every object is open, every vocabulary is an open string, and none of it ever decides
+a result. [Evidence: attachments, timelines, snapshots and everything else](extending.md) is the guide; the rules
+are these.
+
+| Field | Contents |
+| --- | --- |
+| `attachments[]` | `{ name, role?, contentType?, bundleKey? \| uri? \| text?, bytes?, sha256?, width?, height?, durationMs?, description?, at?, poster?, captions?, id?, extensions? }`. A screenshot, a video, a trace, a log, a DOM snapshot, a HAR, anything. `bundleKey` is a key in the layer's `bundle` (`attachments/<sha256>`); `uri` is elsewhere; `text` is a short inline value. |
+| `steps[]` | `{ id?, parentId?, name, kind?, status?, offsetMs?, durationMs?, detail?, attachments?, annotations?, extensions? }`. A flat, ordered timeline, nested by `parentId`. |
+| `annotations[]` | `{ type, message, level?, at?, source?, links?, extensions? }`. |
+| `attempts[]` | `{ index, status, durationMs?, startedAtMs?, detail?, steps?, attachments?, annotations?, extensions? }`. |
+| `tags[]`, `labels{}`, `links[]` | Facts to filter by. `labels` are facts about the result; unlike `parameters` they never make it a different slot. |
+| `extensions{}` | Structured data under a name the producer owns (`com.example.coverage`). |
+
+The rules, which hold for every version of the format:
+
+1. A new field is **optional**. No field is ever made required, renamed, removed or given a new meaning.
+2. **Every object is open.** Unknown fields are kept at every level, and a copy carries them.
+3. **Vocabularies are open.** `role`, `kind`, `type`, `level` and a step's `status` are strings with conventional
+   values; an unknown one is shown generically, never rejected.
+4. **Only `status`, `freshness` and `schemaVersion` are closed.** They are the whole of what decides a result.
+5. **Evidence never decides a result.** A reader that cannot read a piece of evidence drops that piece and keeps
+   the check, its status and its place in the report.
+6. A file is stored once, in the layer, by content: `attachments/<sha256>` in `layers/blobs/`. Identical files
+   attached anywhere in a layer share it, and the layer's id still depends only on what was recorded.
+7. A path inside a report is data from a test run. A publisher reads the files it names only from the report's own
+   folder and from folders the person chose, never a hidden file or a link out.
+8. Attachment bytes are untrusted. They are served with the type the producer gave (when it is a well-formed media
+   type), never sniffed, under a sandboxing content-security policy, with `cross-origin-resource-policy:
+   same-origin`, and a `uri` is followed only if it is `http` or `https`.
+
+### Choosing a check id
+
+The check `id` is the one thing only the producer can get right, and everything depends on it: the
+newer result replaces the older one only if both used the same id.
+
+- Make it **stable** across runs: derive it from the check's identity (file, suite, name), never from
+  an index, a timestamp, a random seed or a path's position in a plan.
+- Make it **unique** within the layer. Publishing refuses a layer with a repeated id. When importing
+  CTRF or JUnit, a repeated id gets `#2`, `#3`, … so no check is silently counted as another.
+- Keep it **human**. It is what the gate prints.
+- Changing an id retires the old check: its last result stays in the report until the layers that
+  hold it are left out. Rename deliberately.
+
+Atlas's own importers build ids this way:
+
+| Source | Check id | `specPath` |
+| --- | --- | --- |
+| CTRF | `filePath > suite… > name` (parts that exist) | `filePath`, when present |
+| JUnit XML | `file or classname > name` | the `file` attribute, when present |
+| Atlas run (`manifest.json`) | the path's `key`, which the same start and events give in every run | the root chart's file |
+| `publishChecks` / `--checks` | whatever you give | whatever you give |
+
+## How layers compose
+
+A **slot** is the tuple
+
+```
+(producer, kind, specPath, check id, environment, parameters)
+```
+
+with `parameters` compared as sorted key/value pairs. Branch, commit, backend, layer id and run are
+**not** in the slot. Each slot shows exactly one result: the winner among every layer that filled it.
+
+The winner is decided by these rules, in order:
+
+1. A **completed** result (`passed` or `failed`) beats an **unfinished** one (`running` or `not-run`),
+   whatever their times. An unfinished layer never erases a completed outcome.
+2. Between results of the same completeness, the **newer** `recordedAtMs` wins.
+3. At the same time, **`failed` beats `passed`**, and `running` beats `not-run`. A tie fails.
+4. At the same status, the greater `layerId|backendId` wins, so no input order can change the answer.
+
+Consequences you can rely on:
+
+- Desktop web, mobile web, iOS and Android results of the same check sit side by side.
+- A layer that ran one scenario leaves every other scenario as the newest layer that ran it.
+- Re-running a failing check on any branch and passing replaces the failure. Re-running and failing
+  replaces the pass.
+- A spec in the current specs with no layer at all stays visible as having no results. A spec that
+  was removed keeps its evidence, outside the outline.
+- An immutable layer held by several backends is **one** layer, reported by the backend whose id sorts
+  first.
+
+Leaving a layer, backend, environment, branch or version out of the input is how earlier outcomes are
+inspected.
+
+### What takes part
+
+By default **every layer takes part**, wherever it was recorded: that is what a unified report is. The
+publisher records where it came from, and the reader chooses what to include:
+
+- `freshness` is set by the publisher from git: `modified` when the worktree has uncommitted edits to
+  tracked files, `unknown` outside git, and absent for a CI checkout or a clean worktree at its commit.
+  Untracked files do not count. A modified layer takes part with a warning and is never mistaken for the
+  commit's own result.
+- `atlas report status` and `atlas serve` (and `GET /api/status?branch=&ignoreModified=1` on a server) take `--branch <name>` (repeatable) and `--ignore-modified`
+  (`ATLAS_ONLY_BRANCH`, `ATLAS_IGNORE_MODIFIED`). They only choose which layers are read; the rules above are
+  unchanged. A pull request's gate is `--branch "$PR_BRANCH" --branch main --ignore-modified`: the branch
+  laid over `main`, so what it did not re-run is inherited.
+
+## Guarantees
+
+These are tested (`packages/reports/tests/composition-laws.test.ts`, over random partial runs) and
+are part of the contract:
+
+| Guarantee | Meaning |
+| --- | --- |
+| **Order independence** | The report does not depend on the order backends or layers are read in. |
+| **Partition independence** | Splitting the same layers across any number of backends, machines or branches, with any overlap, gives the same report. This is what makes partial runs on separate nodes safe. |
+| **Idempotence** | A layer present twice, in one backend or several, counts once. Publishing the same checks again writes the same layer. |
+| **Monotonic** | A layer changes only the slots it fills. Adding a layer never changes a result it did not record. |
+| **Deletion-free** | Nothing in the contract removes or rewrites a layer. Retention is the operator's choice. |
+| **Unknown is not a pass** | A layer that cannot be read, a backend that is down, and a check that is `running` or `not-run` never count as passing. The gate fails closed. |
+| **Aggregation by copying** | Layers are immutable and never share a key with different bytes, so combining stores is copying files (`atlas report merge`, `cp -R`, `rsync`, a CI artifact download). Copying is idempotent and commutative. |
+
+## Statuses and the gate
+
+| Status | Meaning | Counts as |
+| --- | --- | --- |
+| `passed` | The check ran and its expectation held. | Pass |
+| `failed` | The check ran and its expectation did not hold. | Fail |
+| `running` | The check has started and not finished. | Not done: fails the gate |
+| `not-run` | The check was skipped, pending, blocked, or never reached. | Nothing: never a pass, never a failure |
+
+Tools map to these four. CTRF `skipped`, `pending` and `other`, and JUnit `<skipped>`, are `not-run`.
+An Atlas path that passed on retry (`flaky`) is `passed`, with the detail "Passed on retry (flaky)".
+
+Timings that only qualify a check (parts of kind `performance`) are not counted as checks when
+anything else is recorded. A measurement over its allowance is reported and never fails a check.
+
+**Freshness never changes a status.** A layer whose spec fingerprint no longer matches, whose
+checkout was modified, or whose related code changed still shows its recorded outcome, with a
+warning ("Potentially outdated: spec changed"). Atlas favours quick, possibly stale evidence over
+none, and never invents a result.
+
+### `atlas report status`
+
+The gate passes (exit 0) only when **all** of these hold:
+
+- at least one check has completed (`passed` or `failed`): a report where everything was skipped has
+  no evidence;
+- no slot's winning result is `failed`;
+- no slot's winning result is `running`;
+- every backend and every layer could be read (a notice is a failure);
+- every environment named by `--require-environments` has at least one check;
+- with `--require-all-specs`, every spec in `--specs <dir>` has at least one result.
+
+Exit codes: `0` the report passes, `1` it does not, `2` the command itself is wrong (a store
+directory that does not exist, a missing flag). A CI job can tell a broken invocation from a red report.
+
+`--format json` prints a [`ReportStatus`](../packages/schema/schemas/report-status.schema.json)
+(`schemaVersion: 1`): `ok`, `reasons`, `checks`, `byEnvironment`, `layers`, `backends`, `notices`,
+`specsWithoutEvidence`, `specs` (each spec's counts, per environment) and `failing`. The text and markdown formats are for people and are not a
+contract; parse the JSON.
+
+## Backends
+
+A backend is any implementation of `ObjectStore` (`@crvouga/atlas/reports/object-store`), seven
+async methods over keys and bytes:
+
+| Method | Rule |
+| --- | --- |
+| `get(key)` | The object with its `contentType` and `size`, or `null` when missing. Transport errors throw. |
+| `put(key, bytes, contentType)` | Idempotent: writing a key twice overwrites without error. The content type round-trips. |
+| `head(key)` | `true`/`false`. Never throws for a missing key. |
+| `delete(key)`, `deleteMany(keys)` | A missing key is a no-op. |
+| `getUri(key)` | A URL the host can load directly, or `null`. |
+| `listKeys(prefix?)` | Keys in code-unit order, optionally filtered by prefix. |
+| `withPrefix(prefix)` | A store scoped to a prefix. Keys relative to it. Prefixes compose. |
+
+It needs nothing else: no transactions, no rename, no locks, no conditional writes. That is what
+lets a plain bucket, a directory and a device lab compose.
+
+Atlas ships two backends, both named by one URL (`openStore`, `--store`, `ATLAS_STORE`):
+
+- `directoryStore(dir)` (`@crvouga/atlas/reports/directory-store`): atomic writes, for a CI workspace, a
+  mount or a laptop.
+- `s3Store({ bucket, prefix, region, endpoint })` (`@crvouga/atlas/reports/s3-store`): AWS S3 and every store
+  that speaks its protocol, such as R2 and MinIO, written without an SDK. `s3://bucket/folder?region=…&endpoint=…`.
+  Requests are signed with Signature Version 4, retried with backoff on throttling and server errors,
+  capped in flight, and follow a bucket's region. Credentials are never in a URL: they come from the
+  environment the way an AWS SDK finds them. It is tested against AWS's published signing vectors and a
+  live S3 server in CI. On S3 a missing object answers 403 instead of 404 for an account that cannot list
+  the bucket, so publishers need `s3:ListBucket` too (`atlas report policy` prints the permissions).
+
+For anything else, write an adapter and prove it:
+
+```ts
+import { verifyObjectStore } from '@crvouga/atlas/reports/object-store-conformance';
+
+const { ok, failures } = await verifyObjectStore(myS3Store); // writes under a scratch prefix, cleans up
+```
+
+Several backends are one report: pass them all to `combineBackends`, to the viewer
+(`reportsHost`), or to the CLI (`--store` for directories, `--backends <module>` for the rest). A
+backend that cannot be reached is named in the notices and leaves the others usable; asking again
+retries it.
+
+## Where bytes go
+
+All keys are relative to the backend. Layers are written **bottom up**, so a reader never meets an
+entry whose content has not arrived:
+
+A layer's checks are the contract. Next to them the store holds the two other things a whole project
+needs, written the same way (content first, catalog entry last, nothing overwritten):
+
+- **Spec sets** (`layers/specs/…`): the specs of one commit, so a viewer needs no checkout. A spec set is
+  `{ schemaVersion, id, branch, sha, recordedAtMs, freshness?, files: [{ path, digest, fingerprint, kind,
+  title, domain, bytes }] }` ([`spec-set.schema.json`](../packages/schema/schemas/spec-set.schema.json)). Its
+  id is the digest of the branch, commit and file digests, so one commit's specs are one set however many
+  nodes publish them. Bodies are `layers/specs/blobs/<sha256>.txt`, shared by every set that holds them.
+  The outline a report is laid over is the **newest set from `main`** (then `master`, then the newest);
+  a set from a modified checkout is used only if no committed set exists.
+- **Run bundles** (`layers/runs/<run>/…` and `layers/runs/index/…`): a finished statechart run with its
+  screenshots and clips, for the viewer. Uploaded only when the run's privacy scan passed; the manifest is
+  written last, then the catalog entry.
+
+The keys of a layer, bottom up:
+
+1. `layers/blobs/<sha256>.gz`: recorded files (screenshots, frames), content-addressed and shared
+   between layers. Skipped entirely for a layer of plain checks.
+2. `layers/manifests/<id>.json`: the layer document, written once.
+3. `layers/index/<inverted-time>-<id>.json`: `{"id": "<id>"}`, the catalog entry, written last. The
+   time is inverted so a listing is newest first.
+
+Two producers never write different bytes to one key: every key is derived from content. A crash
+between steps leaves unreferenced blobs, never a catalog entry without its layer.
+
+Everything under `layers/` is immutable and addressed by content, so `copyStore` and `atlas report merge`
+carry spec sets and runs along with layers.
+
+Stores written before layers existed (`commits/<sha>/…`, `versions/…`, `index/…`) are read in place
+and composed with layers; Atlas never rewrites them. `copyStore` and `atlas report merge` copy
+`layers/` only, because commit-keyed objects are mutable.
+
+## Versioning and stability
+
+- `schemaVersion` is the **major** version of the layer document. This is version 1.
+- Within a version, changes are **additive**: a new optional field, a new open `kind`. Readers ignore
+  unknown fields, so an older Atlas reads a newer minor version.
+- The closed vocabularies, `status` and `freshness`, change only with a new major version. A reader
+  never has to guess what an unknown status means.
+- A reader reads every version **at or below its own**. A layer of a newer major version is refused by
+  name ("Written with layer schema version 2; … Upgrade @crvouga/atlas") and shows up as a notice. It
+  is never half understood and never counted as a pass.
+- A breaking change to the document, to the [slot](#how-layers-compose) or to the
+  [winner rules](#how-layers-compose) is a major version of the contract **and** a major release of
+  `@crvouga/atlas` (a `!` commit). It also ships a migration note.
+- Adding evidence (a new optional field, a new open vocabulary value, a new extension namespace) is a minor
+  change and needs no new `schemaVersion`. `packages/reports/tests/fixtures/layer-v1-extended.json` carries every
+  kind of evidence, and `layer-extensibility.test.ts` proves unknown fields survive at every level and that a
+  reader of the first version of the format still reads it.
+- The contract is guarded by `packages/reports/tests/fixtures/layer-v1.json`, a layer that must keep
+  parsing, and by the emitted JSON Schemas, which a test keeps equal to the Zod schemas. **Never edit
+  the fixture to make a test pass.**
+
+## Failure behaviour
+
+| What happens | What Atlas does |
+| --- | --- |
+| A backend is unreachable | Named in `notices` and in the backend list as unavailable; every other backend is used. The gate fails. |
+| A layer is not JSON, fails the schema, or is a newer major version | Skipped with a notice naming the layer and why. The gate fails. |
+| A catalog entry names a layer that is missing | A notice; the gate fails. |
+| Two layers disagree | The [winner rules](#how-layers-compose) decide, the same way everywhere. |
+| A publish is interrupted | Unreferenced blobs may remain. No partial layer is ever visible. Publish again. |
+| A publish is retried | The same layer; nothing changes. |
+| Two nodes publish at once | They write different keys, or identical bytes to the same key. |
+| A required field is missing at publish | `publishChecks` throws, naming each problem by its path. Nothing is written. |
+| A check id repeats within a layer | `publishChecks` throws. Importers number repeats (`#2`). |
+
+## Speed
+
+- Publishing a layer of plain checks is one `head` and two small writes.
+- A report is one linear pass over the checks of every layer. Reading dominates it: each layer is
+  one catalog read and one manifest read, a few at a time (16 by default, 64 for a node on S3).
+- A long-lived reader does not redo that pass. `ReportIndex` reads each immutable layer once, folds it into
+  the winning result of every slot it fills, and drops it unless it still holds a winner, so memory follows
+  the number of checks (plus a few bytes for each layer ever read), not the age of the project, and a refresh
+  with nothing new is one request for the newest page of the catalog. Folding is exact in any order because the winner
+  rule is a total order; a test proves the incremental report equals combining every layer from scratch.
+  The node sends the viewer only that: one request, only winning rows, with an ETag.
+- Measured on a laptop with a directory backend: 300 layers of 50 checks in about 0.2 s; 1,000 layers
+  of 100 checks, each recorded against 1,000 specs (100,000 results), in about 0.9 s. Composing two
+  thousand layers of 100 checks (200,000 results) already in memory takes about 0.3 s.
+  `performance.test.ts` holds budgets ten times looser, to catch an accidental quadratic step on a
+  slow machine.
+- A directory backend lists only the directories a prefix can reach. The viewer polls a small index,
+  reads an immutable layer once and caches it forever, and loads recordings only when opened.
+
+## Other documents of the contract
+
+| Document | Schema | Purpose |
+| --- | --- | --- |
+| Layer | `layer.schema.json` | What a producer publishes. This page. |
+| Publish checks | `publish-checks.schema.json` | The input of `publishChecks` and `atlas report publish --checks`: a layer without the parts the publisher fills in. |
+| Report status | `report-status.schema.json` | The output of `atlas report status --format json`. |
+| Spec set | `spec-set.schema.json` | The specs of one commit, as published beside the layers. |
+| Run manifest | `manifest.schema.json` | What `atlas run` writes for a statechart run. Publish it as a layer with `--manifest`. |
+| Runs index, sources, change notices | `runs-index`, `report-sources`, `atlas-change` | How the viewer finds runs. See [report backends](report-backends.md). |
+| CloudEvents | `cloud-event.schema.json` | Business events a run observed. |
+
+All of them are JSON Schema 2020-12 and ship in the package at `@crvouga/atlas/schemas/<name>.schema.json`.

package/docs/reporting-issues.md

@@ -0,0 +1,160 @@
+# Reporting issues
+
+How people and coding agents in a project that uses Atlas report a bug or a missing feature. Issues
+are the only way a project changes Atlas: maintainers and agents in this repository pick them up,
+fix them here, and the fix reaches you as a new version ([dogfooding.md](dogfooding.md)).
+
+This guide is written for coding agents. Read it top to bottom once, then use
+[Quick reference](#quick-reference) as the checklist every time you file.
+
+## Quick reference
+
+| You found | Kind | Title | Template |
+| --- | --- | --- | --- |
+| Atlas crashes, gives a wrong result, contradicts its README or docs, or does not install, build or type-check | `bug` | `[<part>] bug: <what breaks>` | [bug.md][t-bug] |
+| Atlas lacks an API, option, driver capability, CLI flag, schema field or visualizer view you need | `feature` | `[<part>] feature: <what is missing>` | [feature.md][t-feature] |
+
+`<part>` is the part of `@crvouga/atlas` the issue is about: `core` (the main import and the
+`atlas` command), or the import path after the package name: `playwright`, `bun`, `webdriver`,
+`detox`, `vitest`, `schema`, `reports`; `visualizer` for the `atlas-visualizer` command. Use `core`
+when you cannot tell.
+
+1. [Search for an existing issue](#1-search-first). Comment on it instead of opening a duplicate.
+2. [Leave out](#never-include) secrets, personal data and anything that identifies your product.
+3. Copy the kind's template, fill in every section, and save it to a file.
+4. File it:
+
+   ```sh
+   gh issue create --repo crvouga/atlas \
+     --title "[core] bug: plan skips a transition guarded by a parallel region" \
+     --label agent-reported,bug \
+     --body-file issue.md
+   ```
+
+5. Give your human the issue URL, and note the workaround you are using next to the code it is in.
+
+File one issue per bug or missing feature, so each can be fixed, tested and closed on its own. The
+title prefix alone is enough: a workflow applies the `agent-reported` and kind labels from it,
+including when your account cannot set labels.
+
+## When to file, and when not to
+
+File when:
+
+- **bug**: Atlas throws or hangs; lint, plan or a run gives a result the spec does not support;
+  a driver taps, types, waits or records differently from what its README says; the manifest or a
+  report does not match its JSON Schema; the visualizer shows something other than what the run
+  recorded; the package fails to install, import, build or type-check.
+- **feature**: you need something Atlas does not have: a spec construct, a config option, a driver
+  capability, a CLI flag, a field in the manifest, a view in the visualizer, an export format. Check
+  that part's README and `docs/` first.
+
+Do not file when:
+
+- The cause is your own spec, implementation or configuration: a recogniser that matches the wrong
+  screen, a missing `implementation`, state left over from an earlier run. `atlas lint` findings
+  about your charts are Atlas working.
+- You are on an old version. Upgrade (`bun update @crvouga/atlas --latest`) and try again first.
+- You only suspect a problem. Reproduce it, and say what you ran and what happened.
+
+Never fix Atlas from your project instead of filing: no edits to `node_modules`, no patch, no fork.
+
+## Never include
+
+Issues are public. Before you file, remove:
+
+- API keys, tokens, passwords, cookies and signed URLs. Replace each with `<redacted>`.
+- Real personal data. Use obviously fake values (`jane@example.com`).
+- Your product: its name, real screen and event names, screenshots, clips and run manifests, and
+  your project's source. Rename states and events (`Screen A`, `Submits the form`) and reduce the
+  spec until it shows only the problem.
+
+When you cannot show the problem without something on this list, stop and ask your human.
+
+## 1. Search first
+
+```sh
+gh issue list --repo crvouga/atlas --state all --search "<part> <what breaks or is missing> in:title,body"
+```
+
+If an open issue covers the same thing, add a comment with your evidence (versions, reproduction,
+the behavior you need) instead of opening a duplicate. A 👍 on a `feature` issue also counts as
+demand. If a closed issue covers it, check that you are on the version its release comment names,
+or a later one, before you reopen it.
+
+## 2. Bugs: reduce to a reproduction
+
+A bug report is ready when someone can reproduce it from a clean install without your project.
+
+- **Versions.** The exact installed version (`bun pm ls`, or your package manager's equivalent), Bun
+  or Node, the driver and its browser or device.
+- **The smallest spec.** One chart with only the states and transitions that show the problem.
+  `atlas lint --specs <dir>` needs no config and no implementation; say so when that alone
+  reproduces it.
+- **The smallest config and command.** `atlas.config.ts` with a `functionDriver` or `httpDriver`
+  when the bug is not about a browser; the exact command line.
+- **Expected and actual.** Quote the README or doc sentence Atlas contradicts. Paste the error and
+  stack trace, the lint finding, or the wrong part of the manifest, not the whole file.
+- **Workaround.** What your project does in the meantime, or that you are blocked.
+
+## 3. Features: describe the behavior
+
+Describe what Atlas should do, not how your project is built.
+
+- **What is missing.** What you are trying to specify, run or see, and where Atlas stops.
+- **Proposed surface.** The option, API, spec metadata, CLI flag or view as you would want to use
+  it. A short code block is better than a paragraph.
+- **Behaviors.** Numbered Given / When / Then statements, each one checkable by a test in this
+  repository, covering the failures as well as the happy path:
+
+  ```text
+  B1. Given a chart with a `<parallel>` state, when I run `atlas plan --state Sync`, then only
+      paths that enter the Sync region are planned.
+  B2. Given `--state` names a state that does not exist, then the command exits 1 and names the
+      states it could have been.
+  ```
+
+- **Out of scope.** What you do not need, so the feature stays small.
+
+A feature with a clear surface and concrete behaviors is ready to build. One that only names a goal
+waits, labelled `needs-info`, until someone supplies them.
+
+## 4. File the issue
+
+Copy the template for the kind, fill in every section, delete the HTML comments, and save it to a
+file. Then run:
+
+```sh
+gh issue create --repo crvouga/atlas \
+  --title "<title in the format from the quick reference>" \
+  --label agent-reported,<kind> \
+  --body-file issue.md
+```
+
+| Kind | Template (raw, for copying) |
+| --- | --- |
+| `bug` | <https://raw.githubusercontent.com/crvouga/atlas/main/.github/ISSUE_TEMPLATE/bug.md> |
+| `feature` | <https://raw.githubusercontent.com/crvouga/atlas/main/.github/ISSUE_TEMPLATE/feature.md> |
+
+Each template begins with a `---` front-matter block; leave it out of the body file.
+
+If `gh` is not installed or not authenticated, do not install it or log in on your own. Give your
+human the finished title and body and the link to the matching form, for example
+<https://github.com/crvouga/atlas/issues/new?template=bug.md>.
+
+## What happens next
+
+Maintainers and agents in this repository work the `agent-reported` queue.
+
+- **bug**: they reproduce it, add a failing regression test, fix it, and merge a change that closes
+  the issue. If Atlas behaves as documented, they comment with what they observed and close it.
+- **feature**: they settle the surface with you in the comments if it is not clear, then ship it
+  with a test for each behavior you listed.
+
+An issue labelled `needs-info` is waiting on you: answer the question in the comments.
+
+Merging the fix publishes it. The release then comments on the issue with the exact version that
+carries it. Upgrade (`bun update @crvouga/atlas --latest`) and remove your workaround.
+
+[t-bug]: https://github.com/crvouga/atlas/blob/main/.github/ISSUE_TEMPLATE/bug.md
+[t-feature]: https://github.com/crvouga/atlas/blob/main/.github/ISSUE_TEMPLATE/feature.md