@workerstack/testing 0.0.0-stage → 0.31.0

package/docs/api/migrating-from-oat.md

@@ -0,0 +1,70 @@
+# testing api — Migrating from oat
+
+[← back to the overview](../../README.md)
+
+`@lovrozagar/oat` 0.31 became `@workerstack/testing` 0.32, and oat itself became its `/api` subpath:
+the API tester. The checks, the config shape and the reports are the same; every name changed. This page
+lists each one, old to new. Nothing reads the old names, so change all of them at once.
+
+## Install and run
+
+| oat 0.31                                  | @workerstack/testing 0.32                        |
+| ----------------------------------------- | ------------------------------------------------ |
+| `npm i -D @lovrozagar/oat`                | `npm i -D @workerstack/testing`                  |
+| `oat <command>` (`oat run`, `oat doctor`) | `testing api <command>` (`testing api run`, ...) |
+| `oat help`                                | `testing api help`; `testing` lists the subpaths |
+| `node dist/cli.js <command>`              | `node dist/api/cli.js <command>`                 |
+| `oat: <message>`                          | `testing api: <message>`                         |
+
+## Imports and types
+
+| oat 0.31                                         | @workerstack/testing 0.32                                 |
+| ------------------------------------------------ | --------------------------------------------------------- |
+| `import { defineConfig } from "@lovrozagar/oat"` | `import { defineConfig } from "@workerstack/testing/api"` |
+| `import { steps } from "@lovrozagar/oat/steps"`  | `import { steps } from "@workerstack/testing/api/steps"`  |
+| `OatConfig`                                      | `ApiConfig`                                               |
+
+The package has no root export: `import "@workerstack/testing"` fails with `ERR_PACKAGE_PATH_NOT_EXPORTED`.
+
+## Files and folders
+
+| oat 0.31                           | @workerstack/testing 0.32                 |
+| ---------------------------------- | ----------------------------------------- |
+| `oat.config.ts` (`.mjs`, `.json`)  | `testing.api.config.ts` (`.mjs`, `.json`) |
+| `.oat/runs/<datetime>/`            | `.testing/api/runs/<datetime>/`           |
+| `oat-report.md`, `oat-report.json` | `report.md`, `report.json`                |
+| `.oat/credentials.json`            | `.testing/api/credentials.json`           |
+| `oat.baseline.json`                | `testing.api.baseline.json`               |
+| `oat-out/`                         | `testing-out/`                            |
+
+`--config` is always explicit, so an `oat.config.ts` still runs when you pass it by that name; `testing api
+init` writes `testing.api.config.ts`. Runs under `.oat/runs` are not read: `testing api baseline --from` or
+`testing api diff` pointed at one says so. Make a new baseline from a new run.
+
+## Environment
+
+| oat 0.31                               | @workerstack/testing 0.32                      |
+| -------------------------------------- | ---------------------------------------------- |
+| `OAT_<SCHEME>_TOKEN` written by `init` | `TESTING_API_<SCHEME>_TOKEN` written by `init` |
+| `OAT_DEFER_ENV`                        | `TESTING_API_DEFER_ENV`                        |
+| `OAT_HEAP_SNAPSHOTS`                   | `TESTING_API_HEAP_SNAPSHOTS`                   |
+
+A config reads the env names it names with `env("...")`, so a config `oat init` wrote keeps reading its
+`OAT_*` variables until you rename them. Rename them with the config, so the two stay in step.
+
+## Reports, specs and data
+
+| oat 0.31                                              | @workerstack/testing 0.32                    |
+| ----------------------------------------------------- | -------------------------------------------- |
+| matrix `"kind": "oat.matrix"`                         | `"kind": "testing.api.matrix"`               |
+| JUnit suite `oat.run`                                 | `testing.api.run`                            |
+| loader keys `x-oat-defs`, `x-oat-external`, ...       | `x-testing-defs`, `x-testing-external`, ...  |
+| reference backend bypass `x-oat-fresh: 1`             | `x-testing-fresh: 1`                         |
+| generated values `oat-absent-...`, `oat-baseline-...` | `testing-absent-...`, `testing-baseline-...` |
+| generated names `oat_unknown_parameter`               | `testing_unknown_parameter`                  |
+
+A cleanup script or a log filter that matches `oat-` on records the tester created should match `testing-`.
+
+## Lint plugin (contributors)
+
+The repo's own oxlint plugin is `testing`: rule ids are `testing/<rule>` instead of `oat/<rule>`.

package/docs/api/model.md

@@ -0,0 +1,230 @@
+# testing api — How the API tester reads a document
+
+[← back to the overview](../../README.md)
+
+## How a run works
+
+1. **Load** the OpenAPI document (URL or path). Internal `$ref`s are inlined, and a file's relative `$ref`s to files in its own folder are read in. URL `$ref`s are reported, never fetched.
+2. **Model** entities by inverting `x-invalidate` (or path heuristics) into a read surface per entity.
+3. **Authenticate** every configured principal with the recipe for each scheme the document declares. Credentials refresh on a countdown from `exp` (default 30s buffer) before every dispatch and each async poll.
+4. **Seed** a cohort of records per entity, in parent-before-child order, using each entity's create operation.
+5. **Test** the matrix entity by entity: foundations first, then composition, writes, isolation, declared effects. Entities that can see each other's writes (a parent and its children, a route one invalidates, an effect one lands on another) run in series; entities that share nothing run side by side. Checks inside an entity stay ordered.
+6. **Destructive** operations (`x-destructive`) run as a second user signed up for the purpose, never the suite caller.
+7. **Scenarios** run the flows the config declares for operations the matrix cannot reach on its own (a payment, a consent, an invite accepted by a second user).
+8. **Floor**: every operation nothing above graded is called once with a request built from its schema. A delete that only answered 2xx is read back once.
+9. **Teardown** everything the run created, unless `--keep-fixtures` / `keepFixtures: true`.
+
+Every pass names its tier. `verified`: a later exchange shows a value from the call, such as the read holding the created id, the list without the deleted one, or the session refused after the logout. `contract`: the call got a documented success and its body matched the schema, and nothing later showed more. An operation the API tester could not call at all stays `untested` with the reason (`unresolved param team_id`, `contract call returned 409 …`); declare a scenario for it.
+
+The writer is the principal `writer` names, or the first one. Isolation needs a second principal with a different `tenant`. A rank lattice needs two or more principals that share a `tenant` and differ in `rank`. Invite checks need `x-invite` plus a peer with `inviteAs`.
+
+The API tester never needs ground truth about your data. A filter and its negation must partition the set; a page walk must cover the collection; a record read four ways must read the same.
+
+Each operation is sent the credentials its OpenAPI `security` requires, placed where its scheme says, from the config's recipe for that scheme (see [Security recipes](configuration/principals-and-security.md#security-recipes)). The primary origin is `baseUrl`; extra hosts go in `origins[]`, each with its own spec. A request step whose `path` is an absolute `http(s)` URL is a recorded hop on that URL (a mailed consume page), not an `origins[]` entry. Request bodies follow the document: JSON, `multipart/form-data` (scalars plus dummy, pool, `each` or `resolveUpload` files), or `application/x-www-form-urlencoded`. `hooks.resolveInput` can replace a generated JSON field (a Stripe test `pm_…`); `hooks.resolveHeaders` can attach a one-shot header (Turnstile) per request.
+
+## How the model is derived
+
+`testing api plan` is this model. Checks never see raw paths; they see entities, actions, and query **roles**.
+
+### Entity name and action
+
+**Explicit:** `x-entity: { name, action, identity? }` on the operation.
+
+**Heuristic** (when the tag is absent):
+
+1. Split the path on `/`. Ignore `{param}` segments.
+2. The last non-parameter segment is the noun. `v1` / `v2` / `vN` is never a noun.
+3. Singularize that noun (`stores` → `store`, `batches` → `batch`). Irregulars include `people→person`, `categories→category`, `campuses→campus`, `statuses→status`, `children→child`, `companies→company`, `addresses→address`, `indices→index`, `queries→query`, `properties→property`, `entities→entity`, `inboxes→inbox`. Endings `us|ss|is|os|as|ics|ews|ess|ous|sis` are left alone (`status` stays `status`).
+4. If a non-parameter segment follows the noun (`/rows/aggregate`, `/tables/{id}/restore`), action is `action`.
+5. Otherwise: `GET` collection → `list`, `GET` item → `read`, `POST` collection → `create`, `POST` item → `action`, `PUT`/`PATCH` → `update`, `DELETE` → `delete`.
+
+If no noun can be found, the operation is untracked and `doctor` records an `x-entity` gap.
+
+`--only` and report entity names are these singular names.
+
+### Identity
+
+`x-entity.identity` wins. Else the first of `id`, `uuid`, `slug`, `key`, `name` that is **required** on the item schema, else the first of those that exists, else the trailing path-param suffix (`{table_id}` → `id`). Without an identity the entity is not trackable.
+
+### Read surface
+
+The set of `GET` routes through which an instance is visible.
+
+- Declared: every `"METHOD /path"` in any `x-invalidate` that refers to this entity.
+- Inferred: sibling collection and item routes on the same path prefix as a mutator.
+
+`invalidation.declared-route-changes` only runs when a mutator's `x-invalidate` names **another** entity's route.
+
+### Generated / immutable / soft-delete / tenant
+
+See [OpenAPI meta tags](tags.md#openapi-meta-tags). Fallbacks:
+
+- `readOnly: true` counts as generated (omitted from create bodies).
+- No immutability testing without `x-immutable`.
+- Tenant param: `x-tenant` or a path param matching `org|organization|tenant|workspace|account|project|app` + optional `_id`/`_slug`. Inferred tenants make a cross-tenant read `AMBIGUITY`, not `SECURITY`. With neither a tag nor an inferred name, the check does not apply.
+
+### Idempotency
+
+No meta tag. If create declares a header matching `Idempotency-Key` / `Idempotence-Key` / `X-Idempotency-Key` (spaces ignored, case-insensitive), `idempotency.replay-does-not-duplicate` runs.
+
+### Methods
+
+The API tester reads every operation field of an OAS 3.2 Path Item: `get`, `put`, `post`, `delete`, `options`, `head`, `patch`, `trace` and `query`. `GET`, `HEAD`, `OPTIONS`, `TRACE` and `QUERY` are safe: a read-only check may send them, they never count as writes, and a contract call may send one to any record the run saw. `PUT` and `DELETE` are idempotent too, so a network failure resends them, as it does a safe method. A `QUERY` carries its question in a JSON body like any other request body. A `TRACE` operation is listed but never sent, because fetch refuses the method: the contract call records it as untested with that reason, and the model flags it (`unsendable`).
+
+### Parameters
+
+A required parameter the caller leaves unset is sent with the value the document declares for it (`example`, `examples`, `const`, the first `enum` value, or `default`), in the query, as a header, or as a cookie. An object or array value is kept whole. A header parameter named `Accept`, `Content-Type` or a credential header, and the idempotency key, are never sent from the document. A declared header or cookie under a name the operation's credential is placed in is never sent either; a caller's header there still collides with the credential.
+
+Each value is written in the parameter's OAS `style` and `explode`: `form`, `spaceDelimited`, `pipeDelimited` and `deepObject` in the query and cookies, and `simple`, `label` and `matrix` in the path and headers. The OAS defaults hold when the document says nothing: `form` exploded in the query and cookies, `simple` unexploded in the path and headers. Values are percent-encoded per RFC 3986 unless the parameter sets `allowReserved`; a header is never percent-encoded. A urlencoded body's `encoding` uses the same serializer.
+
+### Discriminated unions
+
+A `oneOf` or `anyOf` of `$ref`s beside a `discriminator` pins each branch to its tag: the `mapping` values that reach the branch's schema, or else its component name. Each branch then requires `propertyName` with one of its own values, so a response whose tag names one branch and whose fields fit another fails its schema, and a generated body carries the tag of the branch it was made from. A union's branch turns with the record's index, as an enum value does, so a cohort makes every branch. A union with a branch that is not a `$ref`, or one the API tester cannot name, stays as JSON Schema reads it. Not read: the `allOf` inheritance pattern, where a parent schema holds the `discriminator` and each child extends the parent with `allOf`; such a parent's tag is not pinned. A `mapping` value that names a schema in another file falls back to that branch's component name, and the union stays unpinned when no name is left. A tag marked `readOnly` (in a request) or `writeOnly` (in a response) leaves the union unpinned, read from the union and each branch's own `properties`, not from inside an `allOf`.
+
+## Seeding
+
+Per entity, the API tester POSTs the create body built from the request schema (JSON, multipart, or urlencoded).
+
+An entity with no create of its own — a table's exports, a webhook's deliveries — is read through its list. When that list sits under a parent (`/tables/{table_id}/exports`), the API tester makes the parent the way it makes any ancestor, sharing the one it already made for this principal, so the entity's checks have a list to read. A path parameter names whatever the collection before it creates, so `/organizations/{organization_id}/webhooks/{id}` finds the webhook even when the model calls it `organization-webhook`.
+
+Default cohort is **7** records, one of each variant, sliced by `cohortSize`:
+
+| variant         | what it is for                                     |
+| --------------- | -------------------------------------------------- |
+| `baseline`      | `"Quarterly Report N"`                             |
+| `lexical-first` | sorts first (`"aaa first alphabetically"`)         |
+| `lexical-last`  | sorts last (`"zzz last alphabetically"`)           |
+| `null-heavy`    | `null` on every nullable field                     |
+| `unicode`       | `"äöüß čćžšđ 日本語 中文 한글 привет مرحبا 🙂"`    |
+| `metacharacter` | `"100% _off_ *everything*"` — LIKE / escape probes |
+| `boundary`      | empty / maxLength / numeric `maximum`              |
+
+Numbers use the ladder `1, 2, 5, 10, 20, 50, 100` so **lexical order ≠ numeric order** (otherwise a TEXT compare looks correct). Enums walk `index % enum.length`. `readOnly` / `x-generated` fields are omitted. Required fields that cannot be generated get a type fallback (`0`, `false`, `[]`, `{}`, `"value"`). Arrays honor `minItems` (never send `[]` when `minItems ≥ 1`). Nested objects stop at depth 4.
+
+Empty schemas (`{}`, `true`, `additionalProperties: {}`) and cyclic `$ref`s after inlining stop the walk — they become a scalar or `{}`, never another object descent. A `RangeError` during generation is a `COVERAGE_GAP` on that entity naming the operationId and JSON pointer (`fixture generation overflow on table.create (/)`), not `blocked by unknown`.
+
+String fields honor `format`, `pattern`, and `minLength` together: `email` → `testing-{variant}-{index}@example.test`, `uri` / `url` → `https://example.test/...`, `uuid` → a fixed-shape UUID, `pattern` → a string that matches (or the field is omitted / the entity is a gap). A generated string is padded to `minLength` (repeat the last character) without breaking `pattern`. `"Quarterly Report N"` is only used when the document does not constrain the string. `minLength` greater than `maxLength` omits an optional field and records `missingRequired` on a required one.
+
+An operation with `x-invite` is not `entity.create`. The API tester does not POST a generated invitee. The invite check sends `granteeField` = the peer's `inviteAs`. Missing `inviteAs` is a coverage gap naming the tag.
+
+A create whose `operationId` appears in any principal `auth.steps`, or that declares `x-fresh-principal`, is not seeded. Those rows were provisioned by the auth flow.
+
+Parent path parameters are created first (depth-first through the owning entity's create). Each principal's `scope` (or a login's `scope.<parameter>` claim, or `hooks.resolvePathValue`) fills parameters the API tester cannot create.
+
+A create that returns `>= 300` on the first variant fails the entity (downstream checks `BLOCKED`). Later variants that fail just shorten the cohort — a partial cohort is still used.
+
+HTTP 429 is retried first — `Retry-After` or exponential backoff, up to five times — on seed, checks, and teardown, whether or not the operation declared `x-rate-limit`. The first 429 is never a seed failure. A leftover 429 after those retries is a gap, not a backend defect. A `Retry-After` longer than `network.maxRetryAfterMs` (default 60s) is not waited for: that 429 stands at once, unretried, and the gap names the wait it asked for.
+
+A 503 with `Retry-After` is waited out and asked again, up to three times, as a well-behaved client would. A 503 that names no wait, or a wait longer than `network.maxRetryAfterMs`, stands, and a read meeting it is not asked again. A read (`GET`, `HEAD`) that meets a 502, 503 or 504 — what a gateway answers on its own, without the request reaching the API — is asked once more after a second; a write never is. Every server error left standing fails its operation (`response.no-server-error`); a lasting 503 is reported as the operation being unavailable, any other 5xx as a crash.
+
+A request that times out is a network outcome only when the host has stopped answering. When the origin still answers a probe, the timeout fails that one operation and the run goes on.
+
+A 402 / plan-limit (`payment_required`, `*_plan_limit`) on create is not a backend defect when the same-tenant list already has a row — typically an earlier `x-effects` create that filled a free-plan quota. The API tester reuses that id so children (a `row` after extract created a `table`) can still seed. It does not invent records. Write-path checks on the adopted entity stand down.
+
+A documented feature-gate 403 is the exception: if create declares `x-feature-gate` and the body is `vars.type: feature_gate` (and `vars.feature` matches the tag when present), the API tester records a `COVERAGE_GAP` naming the tag rather than a seed defect. See [`x-feature-gate`](tags.md#x-feature-gate).
+
+When create is tagged `x-unique`, a first-variant seed **409** with a nonempty same-tenant list **adopts** that row (`world.seed` `COVERAGE_GAP` naming `x-unique` — the create could not insert). Unique-conflict checks still run against that row. Write-path oracles that need a body the API tester submitted stay skipped. 409 with an empty list stays `BLOCKED` (`could not seed`). A 409 without the tag is still today's seed failure. The seed 409 itself is not `create.unique-conflict-rejected` passing — that check is the explicit second POST. See [`x-unique`](tags.md#x-unique). Later variants that 409 only shorten the cohort. Generated values for unique body columns differ across variants (a suffix) without weakening `maxLength` / `pattern`; if the document cannot express two distinct values, the API tester records a gap and skips extra variants.
+
+`--seed` / `seed` makes the bodies identical across runs. It does not make server-assigned ids identical.
+
+A record belongs to the configured principal whose credential its create carried, whatever sent it: a seed, a check, a probe the backend should have refused, a scenario step, a create declared as `PUT`. The API tester reads every answer as it arrives, and each 2xx from an entity's create is ledgered under that principal, scoped where the create's own path put it — a path value wins over the principal's root of the same name, so a create sent under another tenant's root is removed under that root. A principal configured with no credential owns the requests that carry none. A create sent with a credential no configured principal holds — a scenario's own account — is outside the ledger: the scenario that made it cleans it up. Accounts are never ledgered here: the caller's own account (`x-entity.identity: self`) and the record a sign-up creates belong to the account ledger, which removes each as itself, by an admin, or by `hooks.teardownPrincipal` (see [principals](configuration/reference.md)).
+
+Teardown DELETEs created rows (or the `x-cleanup` route) newest-first, and only by an id the backend gave back: a create's response, a listing, an async receipt, or — for a create that answers only an acknowledgment — the entity's read under the id the API tester sent, showing that id and every field the API tester submitted. An id the API tester merely sent is never recorded, so a server that ignores a client-chosen id cannot make the API tester delete somebody else's row. A record whose delete is refused counts as removed when a delete on the same path succeeded earlier in the run: a tenant its creator has left can refuse every later request. A write the backend accepted to a key it should have refused — a tenant or parent id — may have moved a record out of every scope the API tester can reach, so its creator's 404 no longer counts as removal: the record is a leftover, named with the write that moved it. Each record is deleted with its creator's credential first. Failures and missing delete routes are printed as leftovers, each naming the principal that created the record, not as check findings. After each level of deletes, and before the parents above it go, each list is read again with the creator's credential, page by page as the list declares its paging (largest page, at most 50 pages), until it ends or holds every record being looked for: a record still listed after its delete answered 2xx is a leftover too, with that reason. A read that is fenced or cannot be sent verifies nothing and claims nothing; `teardown: { verify: false }` skips it. Every read and listing by a record's creator during the run counts as having seen it; a record its creator never saw is read once with the creator's credential before its delete, and one still unseen after a 2xx delete is a leftover, since nothing HTTP said could have shown it gone — unless the record, or an ancestor of it, was deleted earlier in the run (compared by path segments). An entity with no read makes no such claim. `hooks.verifyRemoved` can settle each record from outside HTTP; see [Proving a record gone](configuration/hooks.md#proving-a-record-gone). `keepFixtures: true` skips teardown altogether.
+
+## Query roles and grammars
+
+Checks do not look for a parameter _named_ `filter`. They resolve **roles** from aliases, then write values in the grammar the document demonstrates.
+
+| role              | aliases (normalized: case, `_` / `-`, `perPage` → `per_page`)                       |
+| ----------------- | ----------------------------------------------------------------------------------- |
+| filter            | `filter`, `where`, `query`, `conditions`                                            |
+| order             | `order`, `order_by`, `sort`, `sort_by`, `ordering`                                  |
+| select            | `select`, `fields`, `field`, `include_fields`, `projection`, `only`                 |
+| search            | `q`, `search`, `query_text`, `term`, `keyword`, `text`                              |
+| search mode       | `search_mode`, `searchmode`, `search_type`, `mode` (only if a search role exists)   |
+| limit (page size) | `limit`, `per_page`, `page_size`, `pagesize`, `count`, `max_results`, `top`, `size` |
+| page              | `page`, `page_number`, `pagenum`, `p`                                               |
+| offset            | `offset`, `skip`, `start`, `from`                                                   |
+| cursor            | `cursor`, `after`, `starting_after`, `next`, `page_token`, `continuation`           |
+
+A bracketed suffix is a _value_ in the name (`fields[articles]`, `filter[status]`), not part of the role. `count` is a page size only when it looks like one (has `maximum` or a default); otherwise it is treated as a total.
+
+A search alias whose declaration says it takes structured input is not free text, and the API tester passes over it for the next alias: a scalar schema that is not a string, a `pattern` containing `:`, or examples or a description that spell `key:value` (`key1:value2` too). Keycloak's `q`, described as `'key1:value2 key2:value2'`, is an attribute query, so its users list searches by `search`. The rule is kept that narrow: a colon in an example (`role:admin`) or the word "attribute" alone can belong to free text.
+
+A bounded integer with a default that matches no alias is still taken as page size. A 1-based integer with no maximum is taken as page number.
+
+**Filter grammars** — how the API tester **writes** a term:
+
+| name        | example                                                                                              | `and` / `or`                       |
+| ----------- | ---------------------------------------------------------------------------------------------------- | ---------------------------------- |
+| `postgrest` | `status.eq.active`, `name.neq.x`, `id.in.(a,b)`, `name.ilike.FOO`, `note.is.null`, `tags.contains.x` | `and(a.eq.1,b.eq.2)`, `or(...)`    |
+| `colon`     | `filter=status=eq:active` (comma-joined terms; no grouping)                                          | not expressible; those checks skip |
+| `equality`  | `?status=active` (one query param per field). Only `eq` is expressible                               | not expressible                    |
+
+Operators the **postgrest** writer can emit: `eq`, `ne`, `neq`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`, `like`, `ilike`, `is`, `contains`. Colon stays on `eq` / `neq` / `gt` / `gte` / `lt` / `lte` / `like`. Equality is `eq` only. Anything a grammar cannot write becomes **did not apply**, not a failed request.
+
+A check that needs `in`, `ilike`, `is`, `contains`, a search mode, or `nullsfirst` / `nullslast` also needs that capability **declared**. The API tester does not infer new operators, modes, or nulls tokens onto an API that never listed them.
+
+**Sort grammars:** `name.asc` (dotted), `-name` (prefixed / JSON:API; ascending is the bare name), `name:asc` (colon), `name asc` (spaced). Dotted (PostgREST-shaped) can also emit `name.asc.nullsfirst` / `name.desc.nullslast` when the capability map allows that token. Colon / prefixed / spaced stay as they are.
+
+**Select grammars:** `id,name` (csv) or `fields[table]=id,name` (bracketed). If the parameter is already named `fields[articles]`, that name is used verbatim.
+
+### How a grammar is inferred
+
+`x-query.grammar` wins when it is `postgrest` | `colon` | `equality`.
+
+Otherwise the API tester concatenates the filter/order/select parameter's `example`, `examples`, and `description`:
+
+- Filter: `/postgrest/i` or `field.op.value` / `status.eq.active` → `postgrest`; `status=eq:` → `colon`; else `equality`. A free-text `filter` string that still looks like equality produces an `x-query` gap telling you to declare the grammar.
+- Sort: `name.asc` → dotted; `name:desc` → colon; `name desc` → spaced; leading `-field` → prefixed; else dotted.
+- Select: parameter name or description contains `fields[…]` → bracketed; else csv.
+
+Without `x-query`, if a filter/order/select/search role resolves, the API tester assumes **every scalar** is filterable/sortable/selectable (`string` / `number` / `integer` / `boolean`, including nullable unions). Sortable-without-tag leaves out the list's own path parameters (`project_id` under `/projects/{project_id}/tables`): every listed record holds the same value, so ordering by it shows nothing. Searchable-without-tag is further narrowed to names matching `name|title|slug|label|description|email`. `doctor` warns. Pagination-only lists stay uncovered.
+
+A query probe goes only to a parameter the list operation declares. Without a declared order, search or select parameter, that axis is empty, whatever `x-query` or config lists for it. Under one parameter per field (`equality`, or a template with `param: "{field}"`), the guessed filterable fields are cut to those the list declares as query parameters; a field `x-query`, config, a harvest or the hook names is kept. A check that needed what was cut does not apply, and its reason names the missing parameter (``the `id` query parameter, which the list operation does not declare``). `doctor` shows the cut map.
+
+That heuristic runs **only** for an axis that was not tagged and not set in config. An explicit empty claim — `filterable: []`, `searchable: null`, `selectable: []`, `sortable: []` — is a claim of none. The API tester will not infer scalars over it.
+
+Searchable / filterable / sortable / selectable from the tag are used as given. `maxLimit` is also taken from a page-size parameter's `schema.maximum` when the tag omits it.
+
+### Declared or skip
+
+Every list check consults one **effective capability map** per entity. Precedence, unchanged in spirit:
+
+| order | source                          | what it contributes                                                                                  |
+| ----- | ------------------------------- | ---------------------------------------------------------------------------------------------------- |
+| 1     | `x-query` on the list operation | fields, structured rows, operators, modes, caps, harvest                                             |
+| 2     | `config.entities[name].query`   | union of fields; overlay of ops / modes / caps                                                       |
+| 3     | `config.query`                  | global defaults for the same keys                                                                    |
+| 4     | heuristic                       | today's scalars / name-regex searchable — **only if that axis was not tagged and not set in config** |
+| 5     | skip                            | the check does not apply                                                                             |
+
+`hooks.resolveQueryCapabilities` runs after 1–3 (and after any `*From` harvest) and may add or replace any axis. Return `null` to keep the merge. Called once per entity after seed, with the seeded parent scope so a follow-up GET can run.
+
+Missing capability → the check does not apply / unresolved. Never `BACKEND_BUG` for an undeclared op, mode, or nulls option.
+
+## Pagination and envelopes
+
+Three page models, all first-class:
+
+- **Page number** (`page` + `limit` roles).
+- **Offset** (`offset` + `limit`). Checks that say "page 3" translate to `offset = (page - 1) * size` (size defaults to 20 only for that translation).
+- **Cursor** (`cursor` role + envelope `nextCursor` or a `Link: rel=next` header).
+
+`hasMore` is taken from the body (`hasMore`, `has_more`, `hasNextPage`, `more`) **or** from a documented `Link` response header. Under Link pagination, **absence** of `rel="next"` means no more pages.
+
+Collection shape is derived from the success JSON schema, not from hardcoded wrapper names:
+
+- Response `type: array` → the body is the list (`key: null`).
+- Otherwise the array property whose items are objects, skipping sidecar names `error(s)`, `warning(s)`, `message(s)`, `meta`, `links`. Resource-named envelopes (`{ tables: [...] }`) work.
+- Sibling keys become envelope fields:
+
+| role       | accepted property names                                     |
+| ---------- | ----------------------------------------------------------- |
+| total      | `count`, `total`, `totalCount`, `total_count`, `totalItems` |
+| hasMore    | `hasMore`, `has_more`, `hasNextPage`, `more`                |
+| nextCursor | `nextCursor`, `next_cursor`, `cursor`, `next`, `endCursor`  |
+| page       | `page`, `pageNumber`, `page_number`, `offset`               |
+| limit      | `limit`, `perPage`, `per_page`, `pageSize`, `page_size`     |
+
+Success schema is the first JSON media type on responses `200`, `201`, `202`, `2XX`, or `default`. Request schema is taken from `requestBody` preferring `multipart/form-data`, then `application/x-www-form-urlencoded`, then JSON. A success response that lists `text/event-stream` is a stream: the API tester consumes it to the end and does not treat the raw body as a JSON schema defect. Media type is the stream tag — there is no `x-stream`.

package/docs/api/reference-defects.md

@@ -0,0 +1,181 @@
+# testing api — Reference defects
+
+[← back to the overview](../../README.md)
+
+## Reference defects (`testing api serve --defects`)
+
+Comma-separated. Each is one named lie the demo API can tell. Primary check is what conformance asserts; extras in parentheses are accepted additional symptoms of the same lie.
+
+| defect                                             | primary check                                   | the lie                                                                                                                           |
+| -------------------------------------------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
+| `STALE_LIST`                                       | `list.read-after-write`                         | create succeeds, list does not show the row                                                                                       |
+| `CREATE_DROPS_FIELD`                               | `create.persists-submitted-fields`              | a submitted field is dropped                                                                                                      |
+| `UPDATE_DROPS_FIELD`                               | `update.persists-submitted-fields`              | an update answers 200 and leaves the table's submitted `description` unchanged                                                    |
+| `READ_DROPS_FIELD`                                 | `consistency.projections-agree`                 | the item read leaves out a field the record and its listing still hold                                                            |
+| `STRING_PAYLOAD_MANGLED`                           | `payload.string-survives`                       | non-ASCII / surrounding whitespace stripped on write                                                                              |
+| `CREATE_500_ON_NON_ASCII`                          | `create.does-not-error`                         | a create whose string holds non-ASCII text answers 500 (also string-survives, status-is-documented, no-server-error)              |
+| `RESPONSE_STATUS_UNDECLARED`                       | `response.status-is-documented`                 | PATCH returns 201 when the document names 200                                                                                     |
+| `CREATED_201_AS_200`                               | `create.status-matches-document`                | create returns 200 when the spec says 201                                                                                         |
+| `RESPONSE_SCHEMA_DRIFT`                            | `schema.success-response-matches-document`      | success body does not match the schema (also read-response-matches-document)                                                      |
+| `READ_BODY_DIVERGES`                               | `schema.read-response-matches-document`         | the item read answers a member its documented schema does not declare; the create and the listing answer as documented            |
+| `WRITE_ONLY_LEAKED`                                | `response.write-only-not-returned`              | behind write-only-field, the item read answers the writeOnly `secret` (also read-response-matches-document: the schema is closed) |
+| `KEY_DELETE_CROSS_TENANT`                          | `tenant.item-not-writable-cross-tenant`         | behind key-lifecycle, another tenant's revoke of an API key succeeds; proved on a key made for the probe, never the run's own     |
+| `STALE_PAST_BOUND`                                 | `consistency.staleness-within-bound`            | behind cached-reads, the read cache keeps a copy 2500 ms where the reads declare a 1 s staleness                                  |
+| `BYPASS_IGNORED`                                   | `consistency.bypass-is-fresh`                   | behind cached-reads, the declared `x-testing-fresh: 1` bypass is accepted and the cache answers anyway                            |
+| `ERROR_SCHEMA_DRIFT`                               | `schema.error-response-matches-document`        | error body does not match the schema                                                                                              |
+| `PROBLEM_JSON_MALFORMED`                           | `schema.problem-json-shape`                     | the 412 problem detail says `status: 400`; its body still fits the declared schema                                                |
+| `DISCRIMINATOR_BRANCH_WRONG`                       | `response.contract-call`                        | the search answers a card's fields under `kind: bank`, or a bank account's under `kind: card`                                     |
+| `PUBLIC_GET_NOT_SUCCESS`                           | `response.public-get-returns-success`           | the public `GET /health` answers 500                                                                                              |
+| `TOKEN_EXPIRY_AS_STRING`                           | `response.contract-call`                        | the token exchange answers `expires_in` as a string where the document declares an integer                                        |
+| `LIMIT_IGNORED`                                    | `pagination.limit-bounds-page-size`             | `limit` is accepted and ignored                                                                                                   |
+| `LIMIT_EXCEEDS_MAX`                                | `pagination.limit-respects-documented-max`      | documented max is not capped                                                                                                      |
+| `LIMIT_OUT_OF_RANGE_SERVED`                        | `pagination.limit-bounds-enforced`              | a page size of 0 or past the maximum is served, though the document declares it refused                                           |
+| `HASMORE_ALWAYS_FALSE`                             | `pagination.has-more-is-accurate`               | `hasMore` is always false                                                                                                         |
+| `OFF_BY_ONE_PAGE`                                  | `pagination.page-walk-covers-set`               | page walk skips or repeats                                                                                                        |
+| `PAGE_PAST_END_REPEATS`                            | `pagination.bounds-handled`                     | a page past the end repeats the last page                                                                                         |
+| `UNSTABLE_SORT`                                    | `pagination.page-walk-covers-set`               | default order is not a total order                                                                                                |
+| `CURSOR_DRIFT`                                     | `pagination.cursor-agrees-with-page`            | cursor and page disagree                                                                                                          |
+| `FILTER_IGNORED`                                   | `filter.unknown-field-rejected`                 | unknown filter field is ignored                                                                                                   |
+| `UNKNOWN_PARAM_EMPTIES_LIST`                       | `query.unknown-parameter-consistent`            | an unknown query parameter empties the listing                                                                                    |
+| `FILTER_EQ_NOT_APPLIED`                            | `filter.equality-selects-exactly-one`           | equality filter is ignored                                                                                                        |
+| `EMPTY_RESULT_RETURNS_ALL`                         | `filter.zero-match-returns-none`                | empty match returns the whole set                                                                                                 |
+| `NEQ_DROPS_NULLS`                                  | `filter.negation-partitions-the-set`            | `neq` drops nulls so the partition leaks                                                                                          |
+| `FILTER_GROUP_COMBINATOR_SWAPPED`                  | `filter.and-composes-as-intersection`           | `and`/`or` are swapped                                                                                                            |
+| `LIKE_UNESCAPED`                                   | `filter.like-metacharacters-escaped`            | `%`/`_` are wildcards in values                                                                                                   |
+| `NUMERIC_COMPARED_AS_TEXT`                         | `filter.numeric-comparison-is-numeric`          | numbers compared as strings                                                                                                       |
+| `ERROR_500_ON_BAD_FILTER`                          | `error.malformed-filter-not-5xx`                | bad filter is 500                                                                                                                 |
+| `FILTER_AFTER_PAGINATION`                          | `query.filter-selects-from-whole-set`           | filter applied after the page is cut                                                                                              |
+| `ORDER_IGNORED`                                    | `sort.order-is-applied`                         | sort param is ignored                                                                                                             |
+| `SORT_DESC_DROPS_NULLS`                            | `sort.reverse-symmetry`                         | desc drops nulls                                                                                                                  |
+| `SORT_NULLS_MODIFIER_IGNORED`                      | `sort.nulls-first-last`                         | `nullsfirst` / `nullslast` are parsed and ignored                                                                                 |
+| `SORT_UNKNOWN_FIELD_IGNORED`                       | `sort.unknown-field-rejected`                   | a sort on an unknown field is dropped instead of refused                                                                          |
+| `SEARCH_IGNORED`                                   | `search.q-narrows-result`                       | search param is ignored                                                                                                           |
+| `SEARCH_ONLY_FIRST_TOKEN`                          | `search.tokens-and`                             | only the first search token is applied                                                                                            |
+| `SEARCH_CASE_SENSITIVE`                            | `search.case-insensitive`                       | search is case-sensitive though the document declares `searchCase: insensitive`                                                   |
+| `SEARCH_EMPTY_MATCHES_NONE`                        | `search.empty-q`                                | `q=` matches nothing though the document declares `searchEmpty: match-all`                                                        |
+| `SEARCH_MODE_REJECTED`                             | `search.mode-accepted`                          | the declared `prefix` search mode is refused                                                                                      |
+| `SELECT_IGNORED`                                   | `select.projection-honored`                     | select param is ignored                                                                                                           |
+| `SELECT_UNKNOWN_IGNORED`                           | `select.unknown-field-rejected`                 | an unknown select field is dropped though the document declares `select.unknown: reject`                                          |
+| `NESTED_SELECT_IGNORED`                            | `select.nested-honored`                         | a nested select returns every field of the relation                                                                               |
+| `COUNT_ALWAYS_ZERO`                                | `count.consistent-with-returned-page`           | total is always 0                                                                                                                 |
+| `COUNT_IGNORES_FILTER`                             | `count.matches-filtered-set`                    | total ignores the filter                                                                                                          |
+| `FILTER_DROPPED_WHEN_SORTED`                       | `query.axes-compose`                            | sort drops the filter                                                                                                             |
+| `FILTER_DROPPED_WHEN_SELECTED`                     | `query.filter-and-select-compose`               | select drops the filter                                                                                                           |
+| `FILTER_DROPPED_WHEN_SEARCHED`                     | `query.search-and-filter-compose`               | search drops the filter                                                                                                           |
+| `FILTER_DROPPED_WHEN_SORTED_AND_SELECTED`          | `query.filter-sort-select-compose`              | the triple drops the filter                                                                                                       |
+| `FILTER_DROPPED_WHEN_SORTED_AND_SEARCHED`          | `query.filter-search-sort-compose`              | the triple drops the filter                                                                                                       |
+| `FILTER_DROPPED_WHEN_SEARCHED_AND_SELECTED`        | `query.filter-search-select-compose`            | the triple drops the filter                                                                                                       |
+| `FILTER_DROPPED_WHEN_SORTED_SEARCHED_AND_SELECTED` | `query.filter-search-sort-select-compose`       | all four axes together drop the filter; every pair and triple composes                                                            |
+| `SEARCH_DROPPED_WHEN_SELECTED`                     | `query.search-and-select-compose`               | select drops the search (also the triples and the four with search and select)                                                    |
+| `SEARCH_DROPPED_WHEN_SORTED`                       | `query.search-and-sort-compose`                 | sort drops the search (also the triples and the four with search and sort)                                                        |
+| `ORDER_DROPPED_WHEN_SELECTED`                      | `query.sort-and-select-compose`                 | select drops the sort (also the triples and the four with sort and select)                                                        |
+| `SPEC_OVERCLAIMS_FILTERABLE`                       | `spec.declared-filterable-is-filterable`        | `x-query` lists a field that 400s                                                                                                 |
+| `SPEC_OVERCLAIMS_SORTABLE`                         | `spec.declared-sortable-is-sortable`            | same for sort                                                                                                                     |
+| `SPEC_OVERCLAIMS_SELECTABLE`                       | `spec.declared-selectable-is-selectable`        | same for select                                                                                                                   |
+| `SPEC_OVERCLAIMS_FILTER_OP`                        | `spec.declared-filterable-ops-accepted`         | `x-query` lists `like` on `slug`, and the backend refuses it                                                                      |
+| `FILTER_IN_FIRST_ONLY`                             | `filter.in-is-union-of-eq`                      | `in.(a,b)` matches only the first value                                                                                           |
+| `FILTER_NIN_FIRST_ONLY`                            | `filter.nin-complements-in`                     | `nin.(a,b)` excludes only the first value                                                                                         |
+| `FILTER_IN_CAP_IGNORED`                            | `filter.in-over-limit-rejected`                 | an `in` list over `maxInValues` is evaluated instead of refused                                                                   |
+| `FILTER_GTE_IS_GT`                                 | `filter.gte-is-gt-or-eq`                        | `gte` is compiled as `gt`                                                                                                         |
+| `FILTER_LTE_IS_LT`                                 | `filter.lte-is-lt-or-eq`                        | `lte` is compiled as `lt`                                                                                                         |
+| `FILTER_LT_IS_LTE`                                 | `filter.ordered-triple-partitions`              | `lt` is compiled as `lte`, so a boundary value is both below and equal to itself                                                  |
+| `FILTER_ILIKE_IS_LIKE`                             | `filter.ilike-is-case-insensitive`              | `ilike` is compiled as `like`                                                                                                     |
+| `FILTER_IS_NULL_MATCHES_ALL`                       | `filter.is-null-selects-nulls`                  | `is.null` matches every row                                                                                                       |
+| `CONTAINS_MATCHES_SUBSTRING`                       | `filter.contains-membership`                    | `contains` matches the array as text, so an element containing the value matches too                                              |
+| `FILTER_ALIAS_MISMAPPED`                           | `filter.alias-matches-canonical`                | the `ne` alias is read as `eq` (also illegal-op)                                                                                  |
+| `FILTER_CONDITION_CAP_IGNORED`                     | `filter.condition-cap-rejected`                 | a filter over `maxFilterConditions` is evaluated instead of refused                                                               |
+| `FILTER_NESTED_COMBINATOR_LEAKS`                   | `filter.nested-and-or-distributes`              | a nested group's combinator replaces the enclosing group's for its remaining terms                                                |
+| `FILTER_ILLEGAL_OP_IGNORED`                        | `filter.illegal-op-rejected`                    | an operator outside the allowlist, or one the grammar does not have, is ignored                                                   |
+| `IGNORES_MALFORMED_FILTER`                         | `filter.malformed-term-rejected`                | a filter term the grammar cannot parse is dropped (200 with the whole set)                                                        |
+| `IGNORES_BAD_SORT_DIRECTION`                       | `sort.invalid-direction-rejected`               | an unknown sort direction is read as ascending (`template` dialect, the one declaring its sort)                                   |
+| `SORT_NUMERIC_AS_TEXT`                             | `sort.numeric-order-is-numeric`                 | a numeric field is ordered lexicographically                                                                                      |
+| `SORT_MULTI_KEY_IGNORED`                           | `sort.multi-key-tiebreak`                       | the second sort key is ignored                                                                                                    |
+| `TIEBREAK_NOT_APPLIED`                             | `sort.stable-tiebreak`                          | ties are broken by another field than the declared one                                                                            |
+| `DEFAULT_ORDER_IGNORED`                            | `sort.default-order-applied`                    | with no sort requested, the listing ignores the documented `defaultOrder`                                                         |
+| `RATE_LIMIT_STRICTER_THAN_DECLARED`                | `spec.declared-rate-limit-is-honored`           | 429 well inside the declared `x-rate-limit` rate                                                                                  |
+| `ACTION_CRASHES`                                   | `response.no-server-error`                      | starting a job answers 500 instead of starting it                                                                                 |
+| `SELECT_FIELD_MISSING`                             | `select.requested-fields-present`               | a requested select field is dropped                                                                                               |
+| `CROSS_TENANT_READ`                                | `tenant.item-not-readable-cross-tenant`         | item GET is global by id                                                                                                          |
+| `CLIENT_ID_IGNORED`                                | `create.keeps-submitted-identity`               | a create stores the record under its own id, not the one the client chose (`client-ids` shape)                                    |
+| `CLIENT_ID_OVERWRITES`                             | `create.unique-conflict-rejected`               | a create with an id the tenant already holds overwrites that record (`client-ids` shape)                                          |
+| `CROSS_TENANT_WRITE`                               | `tenant.item-not-writable-cross-tenant`         | update and delete reach another tenant's record                                                                                   |
+| `EXISTENCE_LEAK_VIA_STATUS`                        | `tenant.denial-does-not-reveal-existence`       | 403 vs 404 reveals the other tenant's row                                                                                         |
+| `TENANT_LEAK_VIA_FILTER`                           | `tenant.filter-does-not-bypass-scope`           | filter drops the tenant predicate                                                                                                 |
+| `FOREIGN_PARENT_ACCEPTED`                          | `tenant.parent-not-reachable-from-another-root` | another tenant's parent is accepted under one's own root                                                                          |
+| `ROLE_MONOTONICITY_BROKEN`                         | `auth.rank-is-monotonic`                        | a lower rank can do more                                                                                                          |
+| `ROLE_WRITE_INVERTED`                              | `auth.rank-is-monotonic-on-writes`              | a lower rank can update where a higher rank cannot                                                                                |
+| `INVITE_NEVER_GRANTS`                              | `auth.invite-grants-then-revokes`               | accept does not grant                                                                                                             |
+| `REVOKE_IGNORED`                                   | `auth.invite-grants-then-revokes`               | revoke leaves the grant                                                                                                           |
+| `CALLER_UPDATE_DROPPED`                            | `auth.self-is-the-caller`                       | a PATCH of the signed-up caller (`/v1/me`) answers 200 and changes nothing                                                        |
+| `ROOT_READ_OPEN`                                   | `read.matches-caller-root`                      | the caller's project overview answers with no token (also missing-credential, invalid-credential, wrong-scheme)                   |
+| `DISPOSABLE_KILLS_CALLER`                          | `auth.destructive-spares-the-caller`            | logout ends every account's sessions, the suite caller's included (also `auth.blocked`)                                           |
+| `IMMUTABLE_WRITABLE`                               | `patch.immutable-field-rejected`                | immutable fields accept writes                                                                                                    |
+| `SOFT_DELETE_LEAK`                                 | `softdelete.absent-from-default-list`           | tombstone stays on the default list                                                                                               |
+| `PARENT_PROJECTION_STALE`                          | `invalidation.declared-route-changes`           | child write does not bump the parent                                                                                              |
+| `EFFECT_NOT_APPLIED`                               | `effects.declared-effect-occurs`                | declared cardinality delta does not happen                                                                                        |
+| `SIDE_EFFECT_NEVER_ARRIVES`                        | `effects.side-effect-arrives`                   | the side effect a write declares with `x-wait` never arrives                                                                      |
+| `ASYNC_NEVER_COMPLETES`                            | `async.reaches-terminal-state`                  | job stays pending                                                                                                                 |
+| `ASYNC_RECEIPT_MISSING_ID`                         | `async.receipt-identifies-the-job`              | receipt has no id                                                                                                                 |
+| `PATCH_REPLACES`                                   | `patch.minimality`                              | PATCH is implemented as replace (also read-response-matches-document behind rich-schema)                                          |
+| `IDEMPOTENCY_IGNORED`                              | `idempotency.replay-does-not-duplicate`         | Idempotency-Key is ignored                                                                                                        |
+| `IDEMPOTENT_REPLAY_INSERTS`                        | `idempotency.replay-does-not-duplicate`         | a replay returns the original but inserts another row                                                                             |
+| `HIDDEN_FROM_CREATOR`                              | `payload.string-survives`                       | a row holding a given value is never served, even to its creator, and outlives its delete                                         |
+| `UNIQUE_NOT_ENFORCED`                              | `create.unique-conflict-rejected`               | a duplicate unique-set write is 2xx instead of 409                                                                                |
+| `DELETE_MISSING_OK`                                | `delete.absent-record-returns-404`              | DELETE missing returns 200                                                                                                        |
+| `DELETE_KEEPS_ROW`                                 | `delete.absent-record-returns-404`              | a delete answers success and keeps the row; teardown also lists each such record as a leftover                                    |
+| `CONCURRENT_WRITE_LOST`                            | `concurrency.no-lost-update`                    | full-row write clobbers a parallel PATCH                                                                                          |
+| `ETAG_IGNORED`                                     | `concurrency.if-match-enforced`                 | an update under a stale `If-Match` is applied instead of refused with 412                                                         |
+| `ENUM_NOT_VALIDATED`                               | `validation.enum-enforced`                      | enum is not enforced (also read-response-matches-document: the listing serves it)                                                 |
+| `MAXLENGTH_NOT_VALIDATED`                          | `validation.max-length-enforced`                | maxLength is not enforced (also read-response-matches-document: the listing serves it)                                            |
+| `REQUIRED_NOT_VALIDATED`                           | `validation.required-enforced`                  | required is not enforced on a create, nor on a replacing PUT update (`request.invalid-body-refused` under `put-update`)           |
+| `CONTENT_TYPE_NOT_ENFORCED`                        | `validation.content-type-enforced`              | wrong Content-Type is accepted                                                                                                    |
+| `WRITE_BODY_NOT_VALIDATED`                         | `request.invalid-body-refused`                  | a table or row update applies what parses of an invalid body and answers 200                                                      |
+| `LIST_DETAIL_DISAGREE`                             | `consistency.projections-agree`                 | list and item show different values (also read-response-matches-document)                                                         |
+| `LISTED_GET_DIFFERS`                               | `list.item-matches-listed-row`                  | the get of a listed notice answers another `title` than the list                                                                  |
+| `COLUMN_NAME_MISMATCH`                             | `create.persists-submitted-fields`              | SQL identifier does not match the field (SQL backends)                                                                            |
+| `COLLATION_INCONSISTENT`                           | `pagination.cursor-agrees-with-page`            | cursor order ≠ page order (SQL)                                                                                                   |
+| `AUTH_IGNORED_ON_ROUTE`                            | `auth.missing-credential-refused`               | one secured route (the notices list) answers with no credential (also invalid-credential, wrong-scheme)                           |
+| `TOKEN_NOT_VERIFIED`                               | `auth.invalid-credential-refused`               | a JWT whose signature was altered still authenticates                                                                             |
+| `ANY_SCHEME_ACCEPTED`                              | `auth.wrong-scheme-refused`                     | a route the document secures with jwt alone takes an API key too                                                                  |
+| `SCOPE_NOT_CHECKED`                                | `auth.scope-enforced`                           | an API key authenticates an operation whose scope it was never granted                                                            |
+| `APIKEY_ALTERNATIVE_BROKEN`                        | `auth.alternative-accepted`                     | a record route documented as jwt or a scoped API key refuses every API key                                                        |
+| `PUBLIC_ROUTE_GUARDED`                             | `auth.anonymous-operation-open`                 | the login the document declares `security: []` demands a credential                                                               |
+| `REVOKED_TOKEN_ACCEPTED`                           | `auth.revoked-credential-refused`               | a logged-out or revoked token, and a revoked API key, keep working (also destructive-spares)                                      |
+| `REFRESH_REUSE_ACCEPTED`                           | `auth.refresh-reuse-refused`                    | a rotated refresh token still refreshes, and its family is never revoked                                                          |
+| `ROLE_NOT_CHECKED`                                 | `auth.role-enforced`                            | a viewer can create, update and delete                                                                                            |
+| `CROSS_TENANT_KEY_ACCEPTED`                        | `auth.cross-tenant-key-refused`                 | an API key is honored under another tenant's project                                                                              |
+| `PRIVILEGED_FIELD_WRITABLE`                        | `auth.privileged-field-ignored`                 | a member's or a viewer's write sets a table's `x-privileged` `verified`                                                           |
+| `PRIVILEGED_FIELD_ON_SIGNUP`                       | `auth.privileged-field-ignored`                 | a sign-up that sets the register's `x-privileged` `staff` gets it, as `signup.list` shows                                         |
+| `LOGIN_NOT_THROTTLED`                              | `auth.login-throttled`                          | with `throttleLogins` on, failed logins never answer 429                                                                          |
+
+```bash
+testing api serve --defects STALE_LIST,PATCH_REPLACES
+testing api run --config labs/api/local.config.ts --base-url <url>
+```
+
+`COLUMN_NAME_MISMATCH` is SQL-only in conformance (the in-memory store has no physical column names). `CONCURRENT_WRITE_LOST` runs on the in-memory engine only: the race it times does not depend on the store.
+
+The `offset-total` shape pages by rows skipped and taken (`skip` / `take`, bounded by each entity's maximum) and answers `{ data, totalCount }` with no more-pages flag; its baseline must be clean and its recall pass runs every common defect.
+
+The `wrapped-rows` shape answers each listed row as `{ document, highlights }`, the record under `document` as a search engine answers it, and its document declares `x-entity.recordPath: $.<rows>[*].document` on every list; its baseline must be clean and its recall pass runs every common defect.
+
+The `template` dialect (`testing api serve --dialect template`) speaks a grammar the API tester knows only from its declaration: `where=and_status_eq_active`, repeated once per term, `ne` for not-equal, `and_x_is_null` / `and_x_isn_null`, comma-joined `in` lists, `_` in a value written `~`, `like` as a bare substring, and sort keys `name_ascending` / `name_descending`. Its document declares both grammars as templates on `x-query`; the reference parses them with its own expressions, never the API tester's renderer.
+
+`STALE_PAST_BOUND` and `BYPASS_IGNORED` exist only behind the `cached-reads` shape, where every item and list read goes through a per-caller cache that keeps an answer 400 ms, declares `x-consistency: { staleness: "1s", bypass: "x-testing-fresh: 1" }`, and answers `Cache-Status: testing-reference; hit` with `Age`, or `fwd=uri-miss` / `fwd=bypass`. A write never clears it: the API tester's verification reads pass because they carry the bypass, and the staleness check measures the window by reading without it.
+
+`CLIENT_ID_IGNORED` and `CLIENT_ID_OVERWRITES` exist only behind the `client-ids` shape, where the client picks a table's integer id (1–999), the create answers `{ "success": true }`, and tables are already present before the API tester arrives. The shape is a correct backend; its recall pass runs these two on top of the common defects.
+
+The security defects lie about the reference's auth. Its document declares two schemes: `jwt`, an HTTP bearer JWT that `auth.token` and `auth.login` issue and `auth.refresh` rotates, and `apiKey`, sent as `Authorization: ApiKey ak_…`, minted by `apiKey.create` with a set of scopes and revoked by `apiKey.revoke`. Record routes take jwt or a key holding `<entity>:read` or `<entity>:write`; key, member and invite management take jwt alone; `auth.token`, `auth.login`, `auth.refresh`, `auth.register`, `gateway.health`, `styles.read` (a deepObject `range` and a pipe-delimited `ids`, refused with 400 in any other style) and `search.query` (an OAS 3.2 `QUERY` whose JSON body and answer are a card or a bank account, a `oneOf` told apart by its `kind` discriminator; a body whose fields fit the other kind is refused with 400) are public. A table's `verified` is marked `x-privileged`, so only admin may set it. Conformance signs each principal in at `auth.token`, rotates the session at `auth.refresh`, and mints scoped keys with `apiKey.create`.
+
+`LOGIN_NOT_THROTTLED` shows only behind its opt-ins: the reference runs with `throttleLogins` (429 with `Retry-After` once one identity fails 5 logins within a minute), and the API tester runs with `checks: { "auth.login-throttled": true }`. Conformance switches both on for that defect alone.
+
+### What every conformance case also asserts
+
+Besides its defect, every case — clean baseline or injected defect — fails when the API tester:
+
+- leaves a record behind after teardown (baselines),
+- leaves a record behind without reporting it (any case: `left N record(s) behind without reporting them`) — teardown must name it as failed, or a create the API tester could not identify must account for it,
+- deletes or changes a record that existed before the run (`touched N record(s) it did not create`), or
+- ledgers for teardown an id the server never issued or stored (`ledgered N id(s) the server never created`) — an id the API tester only sent, say. An id a defective server handed back without storing the record counts as issued.
+
+A defect that by design rewrites other rows would be listed in `DAMAGES_BYSTANDERS` (`src/api/reference/defects/bystanders.ts`) with its reason; none is today.

package/docs/api/releasing.md

@@ -0,0 +1,52 @@
+# testing api — Releasing
+
+[← back to the overview](../../README.md)
+
+A release is a git tag. Pushing `vX.Y.Z` runs [`.github/workflows/release.yml`](../../.github/workflows/release.yml),
+which waits for CI's verdict and the changelog check on the tagged commit, publishes `@workerstack/testing` to npm with provenance and to GitHub
+Packages, and then creates the GitHub Release. npm is the registry people install from; GitHub
+Releases follow npm versions.
+
+## Cutting a release
+
+1. On `main`, with CI green, set `version` in `package.json` to `X.Y.Z` and add its `## X.Y.Z`
+   heading to `CHANGELOG.md`. Commit and push through the hooks, as any change.
+2. Tag that commit `vX.Y.Z` and push the tag: `git tag vX.Y.Z && git push origin vX.Y.Z`.
+3. Watch the run. It publishes only when every job before it passed.
+4. Check the result: `npm view @workerstack/testing@X.Y.Z` answers the version, its page on npmjs.com
+   shows the provenance badge, and the GitHub Release `vX.Y.Z` exists with generated notes.
+
+A tag the workflow could not publish is run again from the Actions tab (`workflow_dispatch`) with
+the tag as input. A version npm or GitHub Packages already has is skipped, not republished.
+
+## What the workflow runs
+
+A tag matches `v*`. Its jobs, in order:
+
+| job               | what it does                                                                                                                                                                      |
+| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `gates`           | `bun scripts/release-gates.ts vX.Y.Z`: passes once `ci.yml` passed on the tagged commit; fails on a red run; runs `ci.yml` on the tag when a later push canceled the commit's run |
+| `changelog`       | `bun scripts/changelog-has.ts vX.Y.Z`: fails unless `CHANGELOG.md` has a `## X.Y.Z` heading                                                                                       |
+| `npmjs.com`       | after both, checks the tag equals `package.json`'s `version`, then `npm publish --access public --provenance`                                                                     |
+| `GitHub Packages` | after both, publishes the same build to `npm.pkg.github.com`                                                                                                                      |
+| `GitHub Release`  | `gh release create vX.Y.Z --generate-notes`                                                                                                                                       |
+
+A release therefore passes exactly the gates `main` does, without running them a second time, and
+nothing is published until they and the changelog check pass.
+
+The npm job also fails on a tag that does not equal `package.json`'s `version`, so a mistyped tag
+publishes nothing.
+
+## Trusted publishing
+
+The npm job holds no token. It exchanges the job's GitHub OIDC token for a short-lived npm publish
+token, which npm grants only to the publisher configured on the package, and `--provenance` links
+the published version to the workflow run that built it. Configure that publisher once, on
+npmjs.com → `@workerstack/testing` → Settings → Trusted Publisher, exactly as:
+
+```
+GitHub Actions / workerstack / testing / release.yml / environment empty / allow npm publish
+```
+
+A `404` or a non-`201` from the token exchange in the job's log means the saved publisher does not
+match this workflow; the job prints the same line to compare against.