@workerstack/testing 0.0.0-stage → 0.31.0

package/docs/api/checks.md

@@ -0,0 +1,265 @@
+# testing api — Checks and verdicts
+
+[← back to the overview](../../README.md)
+
+## Checks
+
+128 checks: 115 judge an entity, and 13 judge an operation: the [12 security checks](#security-checks) and [`request.invalid-body-refused`](#validation-of-other-writes). A check that cannot run says so (`did not apply` + `needs`). A check that depends on a broken primitive is `BLOCKED`. A check that ran and stopped is inconclusive, not a pass.
+
+Order is fixed (foundations first) so cascade suppression has a cause to point at. Mutating checks run alone; read-only checks may share in-flight requests under `maxInFlight`.
+
+`depends` is the `dependsOn` list: if any of those already failed **for this entity**, this check is `BLOCKED` rather than reported as a second defect. Suppression is transitive.
+
+| id                                              | asserts                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | needs                                                                          | depends                                 |
+| ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------- |
+| `list.read-after-write`                         | a just-created record appears on the list                                                                                                                                                                                                                                                                                                                                                                                                                                             | create + a seeded record                                                       | —                                       |
+| `list.item-matches-listed-row`                  | on an entity with a list and a read but no create, the read of the first listed row answers 2xx with a body holding every value that row shows: a nested object is compared by the keys the row shows, so the read may carry more at any depth; an array must match in length and element by element                                                                                                                                                                                  | a list with no create and one listed row                                       | —                                       |
+| `create.persists-submitted-fields`              | every writable field sent on create is held by the record read back by id, or by the echo                                                                                                                                                                                                                                                                                                                                                                                             | create that echoes the record, or an item read                                 | `list.read-after-write`                 |
+| `update.persists-submitted-fields`              | an update stores every field it was sent: one update, on a record the check made, with a new value for each field the update offers that a read returns (never a lifecycle enum such as status or plan, a privileged or a fixed field); a field neither its answer nor a read past the cache holds is dropped, and every later check picks around it. A combined update that is refused is retried one field at a time (at most 8), and a field refused alone is refused, not dropped | create + update + item read                                                    | `create.persists-submitted-fields`      |
+| `create.keeps-submitted-identity`               | a create that takes the record's identity from the client stores it under that id: an acknowledgment-only create's record is read back under the sent id, and an answered identity equals the sent one                                                                                                                                                                                                                                                                                | create whose request body carries a writable identity, and an item read        | —                                       |
+| `payload.string-survives`                       | a documented-valid string is stored exactly; 4xx after an ASCII control is a fail                                                                                                                                                                                                                                                                                                                                                                                                     | update or create+delete, item GET, unconstrained string                        | `list.read-after-write`                 |
+| `create.status-matches-document`                | create status is one the document declared                                                                                                                                                                                                                                                                                                                                                                                                                                            | create                                                                         | —                                       |
+| `response.status-is-documented`                 | every non-create exchange returns a status that operation names (`default` ≠ 201)                                                                                                                                                                                                                                                                                                                                                                                                     | a modeled non-create operation the API tester invoked                          | —                                       |
+| `response.no-server-error`                      | every 5xx the API tester received fails the operation that sent it, wherever it came from (check, scenario, contract call, teardown), even when the document lists it; a request that timed out on a host that still answers is failed the same way                                                                                                                                                                                                                                   | any operation the API tester called                                            | —                                       |
+| `response.public-get-returns-success`           | a GET or HEAD outside any entity, with no parameters, no request body and no security, sent with no credential, answers a success status the document names                                                                                                                                                                                                                                                                                                                           | a public GET or HEAD with no id, no query, and a documented success status     | —                                       |
+| `response.contract-call`                        | each operation no check or scenario graded (an `x-destructive` one aside) is called once with a request built from its schema, or judged on the 2xx exchange that already reached it: a 5xx is `BACKEND_BUG`, a documented success whose body breaks its schema is `SPEC_BUG`, and a 4xx or an undocumented status leaves it untested with the reason                                                                                                                                 | an operation nothing else graded, with every path parameter resolvable         | —                                       |
+| `schema.success-response-matches-document`      | create body validates against the success schema                                                                                                                                                                                                                                                                                                                                                                                                                                      | success schema on create                                                       | `create.status-matches-document`        |
+| `schema.read-response-matches-document`         | every 2xx item-read and listing body validates against its documented success schema, one past 256 KiB included; a sparse-fieldset answer is left alone                                                                                                                                                                                                                                                                                                                               | an item read or a listing                                                      | —                                       |
+| `response.write-only-not-returned`              | no 2xx create, item-read or listing answer holds a field the document marks writeOnly, whatever additionalProperties says; SECURITY when it is credential-like (password, secret, token, api key, `format: password`), SPEC_BUG otherwise                                                                                                                                                                                                                                             | a writeOnly property on the record or a write body                             | —                                       |
+| `schema.error-response-matches-document`        | an error body validates against the documented error schema                                                                                                                                                                                                                                                                                                                                                                                                                           | error schema on the item route                                                 | —                                       |
+| `schema.problem-json-shape`                     | every `application/problem+json` answer is an RFC 9457 object: `type` a URI reference, `title`, `detail` and `instance` strings, `status` the response's own status                                                                                                                                                                                                                                                                                                                   | an answer served as `application/problem+json`                                 | —                                       |
+| `pagination.limit-bounds-page-size`             | page size ≤ the requested limit                                                                                                                                                                                                                                                                                                                                                                                                                                                       | page-size _role_ (aliases include `limit`, `per_page`, …)                      | `list.read-after-write`                 |
+| `pagination.limit-respects-documented-max`      | requesting more than `maxLimit` does not return more                                                                                                                                                                                                                                                                                                                                                                                                                                  | declared `maxLimit` and a larger cohort                                        | `pagination.limit-bounds-page-size`     |
+| `pagination.limit-bounds-enforced`              | a page size of 0 and of `maxLimit + 1` are 4xx, not served                                                                                                                                                                                                                                                                                                                                                                                                                            | `x-query.limitOutOfRange: reject` and a page size with a maximum               | limit-bounds-page-size                  |
+| `pagination.has-more-is-accurate`               | `hasMore` / `Link rel=next` matches whether another page exists                                                                                                                                                                                                                                                                                                                                                                                                                       | page-forward + `hasMore` or `Link rel=next`                                    | `pagination.limit-bounds-page-size`     |
+| `pagination.page-walk-covers-set`               | walking pages covers the collection with no gaps or dupes                                                                                                                                                                                                                                                                                                                                                                                                                             | page or offset + ≥3 records                                                    | `pagination.limit-bounds-page-size`     |
+| `pagination.bounds-handled`                     | a zero, negative, fractional or non-numeric page size, and a page past the end, are refused or served sanely — never a 5xx or a repeat                                                                                                                                                                                                                                                                                                                                                | page-size or page-number parameter + ≥1 record                                 | `pagination.page-walk-covers-set`       |
+| `pagination.cursor-agrees-with-page`            | cursor walk and page walk yield the same set                                                                                                                                                                                                                                                                                                                                                                                                                                          | both cursor and page                                                           | `pagination.limit-bounds-page-size`     |
+| `filter.unknown-field-rejected`                 | a filter on a field that does not exist is not silently ignored                                                                                                                                                                                                                                                                                                                                                                                                                       | a filter expression                                                            | —                                       |
+| `filter.equality-selects-exactly-one`           | `id.eq.<one>` returns that one record                                                                                                                                                                                                                                                                                                                                                                                                                                                 | equality on the identity                                                       | `list.read-after-write`                 |
+| `filter.zero-match-returns-none`                | a filter that matches nothing returns an empty page, not the whole set                                                                                                                                                                                                                                                                                                                                                                                                                | same                                                                           | `list.read-after-write`                 |
+| `filter.negation-partitions-the-set`            | `eq` ∪ `neq` = whole set, intersection empty                                                                                                                                                                                                                                                                                                                                                                                                                                          | eq and neq                                                                     | `list.read-after-write`, equality       |
+| `filter.and-composes-as-intersection`           | `and(A,B)` = A ∩ B                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | two filterable fields + AND                                                    | equality / list                         |
+| `filter.or-composes-as-union`                   | `or(A,B)` = A ∪ B                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | a grammar with `or` (postgrest, or a template's `or`)                          | equality / list                         |
+| `filter.like-metacharacters-escaped`            | `%` `_` `*` in a value are literals, not wildcards                                                                                                                                                                                                                                                                                                                                                                                                                                    | like operator                                                                  | `list.read-after-write`                 |
+| `filter.numeric-comparison-is-numeric`          | `gt`/`lt` on a number uses numeric order, not TEXT (`1,10,2`)                                                                                                                                                                                                                                                                                                                                                                                                                         | numeric field + a filter param                                                 | `list.read-after-write`, unknown-field  |
+| `filter.in-is-union-of-eq`                      | `in.(a,b)` = `eq.a` ∪ `eq.b`                                                                                                                                                                                                                                                                                                                                                                                                                                                          | field allows `in`; ≥2 distinct values                                          | equality                                |
+| `filter.nin-complements-in`                     | `in` ∩ `nin` empty; union = set minus nulls                                                                                                                                                                                                                                                                                                                                                                                                                                           | field allows `in` and `nin`                                                    | `in`                                    |
+| `filter.gte-is-gt-or-eq`                        | `gte.x` = `gt.x` ∪ `eq.x`                                                                                                                                                                                                                                                                                                                                                                                                                                                             | field allows `gte` and `gt`; ordered type                                      | numeric comparison                      |
+| `filter.lte-is-lt-or-eq`                        | `lte.x` = `lt.x` ∪ `eq.x`                                                                                                                                                                                                                                                                                                                                                                                                                                                             | field allows `lte` and `lt`                                                    | numeric comparison                      |
+| `filter.ordered-triple-partitions`              | `lt` ∪ `eq` ∪ `gt` = set minus nulls; pairwise disjoint                                                                                                                                                                                                                                                                                                                                                                                                                               | field allows all three                                                         | numeric comparison                      |
+| `filter.ilike-is-case-insensitive`              | case-flipped value: `ilike` matches; `like` does not (unresolved if both match)                                                                                                                                                                                                                                                                                                                                                                                                       | field allows both                                                              | like                                    |
+| `filter.is-null-selects-nulls`                  | `is.null` = nulls; `is.notnull` = complement                                                                                                                                                                                                                                                                                                                                                                                                                                          | field allows `is`; cohort has a null                                           | foundations                             |
+| `filter.contains-membership`                    | `contains.<el>` = records whose array value includes `el`                                                                                                                                                                                                                                                                                                                                                                                                                             | array field or ops include `contains`                                          | foundations                             |
+| `filter.nested-and-or-distributes`              | `and(A,or(B,C))` = (A∩B) ∪ (A∩C)                                                                                                                                                                                                                                                                                                                                                                                                                                                      | a grammar that nests `and`/`or` (postgrest only); ≥2 filterable fields         | `and` / `or`                            |
+| `filter.alias-matches-canonical`                | each declared alias token selects the same id-set as its target                                                                                                                                                                                                                                                                                                                                                                                                                       | `aliases` non-empty                                                            | equality                                |
+| `filter.malformed-term-rejected`                | a declared term with its conjunction (or, without one, its operator) left out is 4xx, not 200 with the whole set; unresolved when it selects exactly what the well-formed term does (the server spells that equality both ways)                                                                                                                                                                                                                                                       | declared filter grammar whose terms carry a conjunction or operator            | unknown-field                           |
+| `filter.illegal-op-rejected`                    | one op **not** in that field's `ops` is 4xx; on a declared grammar, an operator token the grammar lacks too                                                                                                                                                                                                                                                                                                                                                                           | closed `ops` list, or a declared grammar                                       | unknown-field                           |
+| `filter.empty-in`                               | `in.()` is 4xx (`reject`) or zero rows (`match-none`)                                                                                                                                                                                                                                                                                                                                                                                                                                 | `emptyIn` set; field allows `in`                                               | `in`                                    |
+| `filter.in-over-limit-rejected`                 | `in` list of `maxInValues+1` is 4xx                                                                                                                                                                                                                                                                                                                                                                                                                                                   | `maxInValues` set                                                              | foundations                             |
+| `filter.condition-cap-rejected`                 | `maxFilterConditions+1` `eq` terms is 4xx                                                                                                                                                                                                                                                                                                                                                                                                                                             | `maxFilterConditions` set                                                      | foundations                             |
+| `spec.declared-filterable-ops-accepted`         | every declared field × every op in that field's `ops` returns <400                                                                                                                                                                                                                                                                                                                                                                                                                    | closed `ops`                                                                   | declared-filterable                     |
+| `error.malformed-filter-not-5xx`                | garbage filter text is 4xx, never 5xx                                                                                                                                                                                                                                                                                                                                                                                                                                                 | a filter expression                                                            | —                                       |
+| `query.filter-selects-from-whole-set`           | a filter is applied to the collection, not to the current page                                                                                                                                                                                                                                                                                                                                                                                                                        | filterable + ≥3 records                                                        | list / walk                             |
+| `sort.order-is-applied`                         | requesting a sort actually rearranges the page                                                                                                                                                                                                                                                                                                                                                                                                                                        | order + a sortable field                                                       | `pagination.limit-bounds-page-size`     |
+| `sort.reverse-symmetry`                         | desc is the reverse of asc (nulls included)                                                                                                                                                                                                                                                                                                                                                                                                                                           | order + asc/desc                                                               | order-is-applied                        |
+| `sort.invalid-direction-rejected`               | a direction token the declared sort grammar lacks is 4xx, not read as ascending                                                                                                                                                                                                                                                                                                                                                                                                       | declared sort grammar with a token for each direction                          | order-is-applied                        |
+| `sort.unknown-field-rejected`                   | `order` on an undeclared field is 4xx, not silent ignore                                                                                                                                                                                                                                                                                                                                                                                                                              | order param                                                                    | —                                       |
+| `sort.numeric-order-is-numeric`                 | `1,10,2` sorts as numbers, not text                                                                                                                                                                                                                                                                                                                                                                                                                                                   | numeric sortable field whose lexical order disagrees                           | order-is-applied                        |
+| `sort.nulls-first-last`                         | `nullsfirst` / `nullslast` put nulls at the start / end of asc                                                                                                                                                                                                                                                                                                                                                                                                                        | declared nulls token; a sort grammar with nulls tokens; cohort has a null      | order + reverse                         |
+| `sort.multi-key-tiebreak`                       | ties on the first key are ordered by the second                                                                                                                                                                                                                                                                                                                                                                                                                                       | ≥2 sortable fields; `maxKeys` absent or ≥2; a sort grammar that joins keys     | order + reverse                         |
+| `sort.default-order-applied`                    | omitting `order` matches `defaultOrder`                                                                                                                                                                                                                                                                                                                                                                                                                                               | `defaultOrder` set; walk complete                                              | order + walk                            |
+| `sort.stable-tiebreak`                          | the same `order` twice yields the same sequence                                                                                                                                                                                                                                                                                                                                                                                                                                       | `stableTiebreak` set                                                           | order-is-applied                        |
+| `spec.declared-sortable-nulls-accepted`         | each declared nulls token returns <400                                                                                                                                                                                                                                                                                                                                                                                                                                                | some field/global declares `nulls`                                             | order-is-applied                        |
+| `search.q-narrows-result`                       | a search term that matches one record does not return the whole set                                                                                                                                                                                                                                                                                                                                                                                                                   | search param + searchable fields                                               | `list.read-after-write`                 |
+| `search.tokens-and`                             | `q=a b` = intersection (AND). Sameness of AND and OR passes                                                                                                                                                                                                                                                                                                                                                                                                                           | searchable fields; two tokens that split the cohort                            | `q-narrows`                             |
+| `search.case-insensitive`                       | case-flipped token matches the same set (unresolved if the backend is case-sensitive)                                                                                                                                                                                                                                                                                                                                                                                                 | searchable field with a letter                                                 | `q-narrows`                             |
+| `search.empty-q`                                | `q=` is ignore / match-all / reject as declared                                                                                                                                                                                                                                                                                                                                                                                                                                       | `searchEmpty` set                                                              | `q-narrows`                             |
+| `search.mode-accepted`                          | each declared `searchModes` value is <400 on the mode param                                                                                                                                                                                                                                                                                                                                                                                                                           | `searchModes` set and a mode role                                              | —                                       |
+| `select.projection-honored`                     | `select=id,name` does not return undeclared fields                                                                                                                                                                                                                                                                                                                                                                                                                                    | select param                                                                   | —                                       |
+| `select.requested-fields-present`               | every name in `select=` appears on each returned item                                                                                                                                                                                                                                                                                                                                                                                                                                 | select param; ≥1 selectable field                                              | projection                              |
+| `select.unknown-field-rejected`                 | unknown select name is 4xx (`reject`) or dropped (`ignore`)                                                                                                                                                                                                                                                                                                                                                                                                                           | `select.unknown` set                                                           | projection                              |
+| `select.nested-honored`                         | `rel(col)` returns `rel` as an object/array carrying only `col`                                                                                                                                                                                                                                                                                                                                                                                                                       | `select.nested` and a named relation                                           | projection                              |
+| `count.consistent-with-returned-page`           | envelope total ≥ rows on this page, and is not zero when the page is not                                                                                                                                                                                                                                                                                                                                                                                                              | envelope total                                                                 | `list.read-after-write`                 |
+| `count.matches-filtered-set`                    | filtered total equals the size of the filtered walk                                                                                                                                                                                                                                                                                                                                                                                                                                   | total + a filter                                                               | list, equality                          |
+| `query.axes-compose`                            | filter + sort together: filter still holds on the sorted page                                                                                                                                                                                                                                                                                                                                                                                                                         | filterable + sortable                                                          | filter + sort foundations               |
+| `query.filter-and-select-compose`               | filter + select together                                                                                                                                                                                                                                                                                                                                                                                                                                                              | filterable + select                                                            | same                                    |
+| `query.search-and-filter-compose`               | search + filter together                                                                                                                                                                                                                                                                                                                                                                                                                                                              | filterable + search                                                            | same                                    |
+| `query.filter-sort-select-compose`              | filter + sort + select                                                                                                                                                                                                                                                                                                                                                                                                                                                                | filter + sort + select                                                         | same                                    |
+| `query.filter-search-sort-compose`              | filter + search + sort                                                                                                                                                                                                                                                                                                                                                                                                                                                                | filter + search + sort                                                         | same                                    |
+| `query.filter-search-select-compose`            | filter + search + select                                                                                                                                                                                                                                                                                                                                                                                                                                                              | filter + search + select                                                       | same                                    |
+| `query.sort-and-select-compose`                 | sort + select: order holds; extras dropped; requested fields present                                                                                                                                                                                                                                                                                                                                                                                                                  | sortable + selectable                                                          | sort + select                           |
+| `query.search-and-select-compose`               | search + select: search membership holds; extras dropped                                                                                                                                                                                                                                                                                                                                                                                                                              | searchable + selectable                                                        | search + select                         |
+| `query.search-and-sort-compose`                 | search + sort: search membership holds; remaining rows ordered                                                                                                                                                                                                                                                                                                                                                                                                                        | searchable + sortable                                                          | search + sort                           |
+| `query.filter-search-sort-select-compose`       | all four: filter ∩ search; order holds; extras dropped                                                                                                                                                                                                                                                                                                                                                                                                                                | all four axes declared                                                         | the triples                             |
+| `query.unknown-parameter-consistent`            | an undocumented query parameter is either refused or ignored — it never silently changes the set                                                                                                                                                                                                                                                                                                                                                                                      | a listing + ≥1 record                                                          | `pagination.page-walk-covers-set`       |
+| `spec.declared-filterable-is-filterable`        | every `x-query.filterable` field actually accepts a filter                                                                                                                                                                                                                                                                                                                                                                                                                            | `x-query` naming filterable fields                                             | filter foundations                      |
+| `spec.declared-sortable-is-sortable`            | every `x-query.sortable` field actually accepts a sort                                                                                                                                                                                                                                                                                                                                                                                                                                | `x-query` naming sortable fields                                               | sort foundations                        |
+| `spec.declared-selectable-is-selectable`        | every `x-query.selectable` field actually accepts a select                                                                                                                                                                                                                                                                                                                                                                                                                            | `x-query` naming selectable fields                                             | select                                  |
+| `tenant.item-not-readable-cross-tenant`         | principal B cannot GET principal A's item                                                                                                                                                                                                                                                                                                                                                                                                                                             | second principal, different `tenant`, and a tagged or inferred tenant          | —                                       |
+| `tenant.item-not-writable-cross-tenant`         | principal B cannot update or delete principal A's item                                                                                                                                                                                                                                                                                                                                                                                                                                | second principal in another tenant, create, and update or delete               | `tenant.item-not-readable-cross-tenant` |
+| `tenant.denial-does-not-reveal-existence`       | 404 vs 403 (or equivalent) does not distinguish "exists other tenant" from "missing"                                                                                                                                                                                                                                                                                                                                                                                                  | second principal, and a tagged or inferred tenant                              | `tenant.item-not-readable-cross-tenant` |
+| `tenant.filter-does-not-bypass-scope`           | `filter=id.eq.<other tenant>` does not return that row                                                                                                                                                                                                                                                                                                                                                                                                                                | second principal, a filter, and a tagged or inferred tenant                    | `query.filter-selects-from-whole-set`   |
+| `tenant.parent-not-reachable-from-another-root` | a nested collection does not accept another tenant's parent under the caller's own tenant; inconclusive when the other tenant holds its own parent of that identity (a slug from a shared name)                                                                                                                                                                                                                                                                                       | second tenant, a collection nested below the tenant                            | `tenant.item-not-readable-cross-tenant` |
+| `auth.rank-is-monotonic`                        | a lower rank cannot do what a higher rank is denied                                                                                                                                                                                                                                                                                                                                                                                                                                   | two same-tenant principals at different `rank`                                 | `list.read-after-write`                 |
+| `auth.rank-is-monotonic-on-writes`              | a lower rank cannot create, update or delete where a higher rank is denied                                                                                                                                                                                                                                                                                                                                                                                                            | two same-tenant principals at different `rank`, and a create                   | `auth.rank-is-monotonic`                |
+| `auth.invite-grants-then-revokes`               | invite → accept grants; revoke takes it back                                                                                                                                                                                                                                                                                                                                                                                                                                          | `x-invite` + peer with `inviteAs`                                              | list, cross-tenant                      |
+| `auth.self-is-the-caller`                       | the caller's own route answers 401 with no token (`SECURITY` otherwise); with the caller's credential, its create (when there is one) shows on the next read, every read answers 200, each GET action with no path id answers about the value the create sent, and an update is still there on the next read. With no read, the update alone must succeed                                                                                                                             | `x-entity.identity: self`, and a read or an update                             | —                                       |
+| `read.matches-caller-root`                      | each read answers 401 with no token (`SECURITY` otherwise) and 200 with the caller's credential at the caller's own root                                                                                                                                                                                                                                                                                                                                                              | `x-entity.identity: root` and a read                                           | —                                       |
+| `auth.destructive-spares-the-caller`            | an `x-destructive` operation, run as a second signed-up user, leaves the suite caller's own read answering 200; a DELETE's id leaves its parent list; when the operation's `x-invalidate` names the caller read, the second user's credential is then refused and a declared refresh no longer renews it. A 5xx fails; another refusal leaves it untested                                                                                                                             | `x-destructive`, a way to sign up a second user, and a caller read             | —                                       |
+| `create.unique-conflict-rejected`               | a second create colliding a documented unique set is 409, not 2xx; the list does not grow                                                                                                                                                                                                                                                                                                                                                                                             | `x-unique` with a probeable create body column and a known row                 | —                                       |
+| `update.unique-conflict-rejected`               | PATCHing a **different** row onto another row's unique-set values is 409, not 2xx                                                                                                                                                                                                                                                                                                                                                                                                     | `x-unique`, update, two known rows; skip all-`x-immutable` sets                | —                                       |
+| `patch.immutable-field-rejected`                | PATCHing an `x-immutable` field is rejected or ignored                                                                                                                                                                                                                                                                                                                                                                                                                                | `x-immutable`                                                                  | —                                       |
+| `softdelete.absent-from-default-list`           | a soft-deleted row is gone from the default list                                                                                                                                                                                                                                                                                                                                                                                                                                      | `x-soft-delete`                                                                | `list.read-after-write`                 |
+| `invalidation.declared-route-changes`           | after a write, the other entity's listed route actually changes                                                                                                                                                                                                                                                                                                                                                                                                                       | `x-invalidate` naming another entity                                           | list, persist                           |
+| `effects.declared-effect-occurs`                | `x-effects` cardinality delta is observed on the named list (`count` exact, `min` at-least; nested child lists bind the created parent id)                                                                                                                                                                                                                                                                                                                                            | `x-effects`                                                                    | `list.read-after-write`                 |
+| `effects.side-effect-arrives`                   | after the write, the named GET’s JSON path is occupied before `timeoutMs`                                                                                                                                                                                                                                                                                                                                                                                                             | `x-wait`                                                                       | `list.read-after-write`                 |
+| `async.reaches-terminal-state`                  | `x-async` reaches `until` (poll, or a terminal SSE frame) before `timeoutMs`                                                                                                                                                                                                                                                                                                                                                                                                          | `x-async`                                                                      | —                                       |
+| `async.receipt-identifies-the-job`              | `idFrom` on the receipt (JSON object or SSE event JSON) resolves to a pollable job                                                                                                                                                                                                                                                                                                                                                                                                    | `x-async` + `idFrom`                                                           | —                                       |
+| `patch.minimality`                              | PATCH `{ name }` does not clear other writable fields                                                                                                                                                                                                                                                                                                                                                                                                                                 | update + item route                                                            | —                                       |
+| `idempotency.replay-does-not-duplicate`         | same Idempotency-Key + same body does not create a second row                                                                                                                                                                                                                                                                                                                                                                                                                         | create + documented Idempotency-Key header                                     | list, persist                           |
+| `delete.absent-record-returns-404`              | DELETE of a missing id is 404, not 200                                                                                                                                                                                                                                                                                                                                                                                                                                                | delete                                                                         | —                                       |
+| `concurrency.no-lost-update`                    | two PATCHes to different fields do not clobber each other                                                                                                                                                                                                                                                                                                                                                                                                                             | update + two writable strings                                                  | persist + patch                         |
+| `concurrency.if-match-enforced`                 | an update naming, in `If-Match`, an ETag the record has moved past is refused with 412 and changes nothing; both updates send the record's own writable state, and a weak ETag leaves the check untested                                                                                                                                                                                                                                                                              | create + a read declaring `ETag` + an update declaring `If-Match`              | —                                       |
+| `validation.enum-enforced`                      | a value outside the enum is rejected                                                                                                                                                                                                                                                                                                                                                                                                                                                  | enum in the request schema                                                     | —                                       |
+| `validation.max-length-enforced`                | a string over `maxLength` is rejected                                                                                                                                                                                                                                                                                                                                                                                                                                                 | maxLength                                                                      | —                                       |
+| `validation.required-enforced`                  | omitting a required field is rejected                                                                                                                                                                                                                                                                                                                                                                                                                                                 | required field                                                                 | —                                       |
+| `validation.content-type-enforced`              | a wrong Content-Type is 415 when 415 is documented                                                                                                                                                                                                                                                                                                                                                                                                                                    | documented 415                                                                 | —                                       |
+| `spec.declared-rate-limit-is-honored`           | no 429 arrives while the API tester paces under the rate `x-rate-limit` declares                                                                                                                                                                                                                                                                                                                                                                                                      | `x-rate-limit` with a rate                                                     | —                                       |
+| `consistency.projections-agree`                 | list, item, and filtered views of the same field agree                                                                                                                                                                                                                                                                                                                                                                                                                                | item route + a comparable field                                                | list + persist + filter                 |
+| `consistency.staleness-within-bound`            | after an update or a delete, a read sent past no bypass shows it within the `x-consistency.staleness` it declares; with none declared the window is reported, never failed. Reads at once, then backs off (at most 12 reads)                                                                                                                                                                                                                                                          | create + item read + update or delete                                          | —                                       |
+| `consistency.bypass-is-fresh`                   | right after a write, one read sent its declared bypass shows the write, and no `Cache-Status` says a cache answered                                                                                                                                                                                                                                                                                                                                                                   | `x-consistency.bypass` on the item read; off with config `cache.bypass: false` | —                                       |
+
+### Security checks
+
+These judge an operation rather than an entity: the credential its `security` requires, the schemes it lists, and the sessions behind them. They run in a stage of their own, and their findings name the operation.
+
+- Probes on a read (GET, HEAD, QUERY) run concurrently, before the entity lanes.
+- Probes that write, and probes that change a credential (revoke, refresh, failed logins), run after the lanes and before teardown, alone and in order. A write probe targets only a record the run created, with a body that would be harmless if applied: the record's current state cut to what the request body accepts. A DELETE probe never targets a record another step uses: each DELETE operation gets a throwaway record of its own, made by the entity's create as that record's creator, with no children, ledgered and removed by teardown like any create. It is reused while the probes are refused; a probe whose DELETE goes through consumes it, and the next probe makes another. A DELETE on an entity with no create is untested, with the reason. A write operation with no run-created record to target is untested, with the reason.
+- An operation whose document declares no security, at the root or on the operation, makes each check stand down as untested: add the `security` by [overlay](configuration/reference.md).
+- The probes that need a credential of their own (revoked, refresh-reuse, privileged-field) acquire a dedicated one, a second login or a fresh mint, never the principal's own session. Teardown revokes it.
+
+| id                                | asserts                                                                                                                                                                      | needs                                                                                                             | when                     |
+| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------ |
+| `auth.missing-credential-refused` | a secured operation answers a request with no credential with 401 or 403                                                                                                     | an operation whose security requires a credential                                                                 | read before, write after |
+| `auth.invalid-credential-refused` | a garbage value, and a real JWT with its signature altered, are refused with 401 for each scheme                                                                             | an operation whose security requires a credential                                                                 | read before, write after |
+| `auth.wrong-scheme-refused`       | a valid credential of a scheme the operation does not list is refused                                                                                                        | an operation whose security requires a credential, and a scheme it does not list                                  | read before, write after |
+| `auth.scope-enforced`             | an API key minted without the operation's scope is refused with 403, while one minted with it is let in; refused both ways, the check is unresolved                          | an operation that requires a scope, of a scheme a mint recipe issues                                              | read before, write after |
+| `auth.alternative-accepted`       | each alternative of the operation's security the principal can satisfy, besides the first, works on its own                                                                  | a read whose security documents a second alternative of one scheme                                                | before the lanes         |
+| `auth.anonymous-operation-open`   | an operation declared `security: []` answers without a credential; a public write is sent an unreadable body, and passes on any answer but 401 or 403                        | an operation whose security lets anonymous callers in                                                             | read before, write after |
+| `auth.revoked-credential-refused` | a dedicated credential, proven on one read and revoked through the recipe's declared revoke, is then refused with 401 or 403                                                 | a recipe that declares a revoke, and a GET the credential authenticates                                           | after the lanes          |
+| `auth.refresh-reuse-refused`      | a dedicated session, rotated through the recipe's refresh, cannot be refreshed again with the refresh token the rotation spent                                               | a recipe that declares a refresh, and a GET the credential authenticates                                          | after the lanes          |
+| `auth.role-enforced`              | a principal marked `readOnly` cannot write: its write to a run-created record in its tenant is refused with 401 or 403                                                       | a principal marked `readOnly`, and a run-created record in its tenant to write to                                 | after the lanes          |
+| `auth.cross-tenant-key-refused`   | the peer tenant's API key is refused under the actor's tenant, and accepted under its own                                                                                    | a peer tenant, and a tenant-scoped GET that takes an API key                                                      | before the lanes         |
+| `auth.privileged-field-ignored`   | a principal below the highest rank cannot set a property marked `x-privileged`: the record, re-read by its owner, keeps its value; a sign-up that sets one does not grant it | a property marked `x-privileged`, and a sign-up or a run-created record with a principal ranked below the highest | after the lanes          |
+| `auth.login-throttled`            | repeated failed logins eventually answer 429                                                                                                                                 | a login recipe whose login step the document declares, with a request body schema; and the opt-in                 | after the lanes, opt-in  |
+
+What each needs, and where each route is judged:
+
+- **Per-route guards.** `auth.missing-credential-refused` and `auth.invalid-credential-refused` probe every secured operation. Per-route guard coverage comes from these two: a route that forgets its guard shows there.
+- **Credential lifecycle.** `auth.revoked-credential-refused` and `auth.refresh-reuse-refused` sample one route per scheme. A session's revoke and refresh belong to the credential, not to a route, so each anchors once per scheme on the operation the recipe names for that step (a mint's `revoke`, a login's `revoke`, the recipe's `refresh`). It proves the credential on one read the API tester can send whole: a GET or HEAD that takes the scheme alone, whose path the actor's scope or a declared enum fills, and whose every required query, header and cookie parameter has a declared value. It prefers a read with no required parameters, then the caller's own read or a list, then a GET over a HEAD (some stacks answer HEAD outside their auth layer), then document order. With no such read, the check stands down: "no read this credential protects that the API tester can send". With no revoke or refresh declared, the check is untested and says why. It never guesses an operation by its name.
+- **Roles.** `auth.role-enforced` needs a principal marked [`readOnly: true`](configuration/reference.md). Rank alone never makes a principal read-only. It differs from `auth.rank-is-monotonic-on-writes`: where every rank may write, the ranks are still monotonic, and only this check fires.
+- **Login throttling.** `auth.login-throttled` runs only with the opt-in `checks: { "auth.login-throttled": true }`. It sends failed logins for an identity that does not exist, never a configured principal's, and stops at the first 429 or after 8 attempts. A throttle that counts by address can slow every login from the machine running the API tester for a while, which is why it is off by default.
+- **Targets.** A probe of one record fills each path parameter only from a record the run created of that parameter's own entity: `{token}` in `/invites/{token}/accept` takes an invite the run created, never the table the operation is filed under. With none, the operation is untested with the reason. Each probe kind sends to one target per operation.
+- **Open writes.** A 2xx to `auth.anonymous-operation-open`'s unreadable body passes: the route is open, as declared. Whether it validates what it accepts is [`request.invalid-body-refused`](#validation-of-other-writes)'s to judge. The write is taken as applied: what it made is ledgered and removed when its answer names a new id that reads back, and otherwise reported as a record that may remain.
+- **Privileged fields.** `auth.privileged-field-ignored` needs a request body property marked [`x-privileged: true`](tags.md#x-privileged). With no marker anywhere, it is untested with "no property is marked x-privileged". On the document's sign-up (the operation marked [`x-fresh-principal`](tags.md#x-cost--x-destructive--x-idempotent--x-fresh-principal), or one it leaves unmarked), it signs one account up, anonymously as anyone may, with each marked property set to the value that grants it: its `const`, its enum's one member, or `true` for a boolean; any other property is untested with "no value the API tester can tell is privileged". The account is ledgered before the sign-up is sent and removed at teardown like any other, or named a leftover. Whether the privilege took is read back from the account's own read (`x-entity` identity `self`), the listing of the entity the sign-up makes, or the property in the sign-up's answer, the first that declares it; with none, nothing is sent and the check is untested. A privilege that took is a `SECURITY` finding. A document that lacks the marker takes it by [overlay](configuration/reference.md): `overlay: { schemas: { Table: { properties: { verified: { "x-privileged": true } } } } }`.
+
+### Validation of other writes
+
+The `validation.*-enforced` checks grade creates. `request.invalid-body-refused` grades every other write that takes a JSON request body. Like the security checks, it judges an operation and runs in their stage, after them, alone and in order; its findings name the operation.
+
+| id                             | asserts                                                                                                                                                                                     | needs                                                      |
+| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
+| `request.invalid-body-refused` | one invalid body is refused with a 4xx: the first required property left out, else the first typed property, in document order, sent a value of another type. A 2xx or a 5xx is BACKEND_BUG | a write other than a create that takes a JSON request body |
+
+- **Targets.** A write to one record goes only to a record the run created, as its creator, with the record's current state as the rest of the body. With none, the operation is untested with the reason.
+- **Collections.** A write outside a record is not sent: "an action outside a run-created record could apply a side effect the run cannot undo". The one exception is a POST open to all (`security: []`) that the document declares a signup with [`x-fresh-principal`](tags.md), sent anonymously with the rest of its body drawn under the run's seed. Its 2xx is taken as applied: what it made is ledgered and removed when its answer names a new id that reads back, and otherwise reported as a record that may remain. Any other open POST could be a sign-in, where even a malformed body can count toward a lockout, so it is never sent.
+- **Open sign-ins.** `auth.anonymous-operation-open` still sends its one unreadable body to every open POST, `auth.login` included: proving the route open is its purpose (`PUBLIC_ROUTE_GUARDED` lives on `auth.login`), and a malformed body answered 400 is the least lockout-prone request there is. This check adds nothing there, so the two never send twice.
+- **Credential steps.** An operation a credential recipe names (a login, token, refresh, revoke or mint step, by `operationId` or by method and path) is never sent.
+- **Addresses.** An action on a record (a POST that is not its entity's update, an invite say) carries no deliverable address: every email-format value is `testing-nonexistent-<nonce>@example.invalid`, and every URL-format value has an `.invalid` host.
+
+On a typical untagged CRUD document (create, list, item, `page`/`limit`, maybe `sort`):
+
+- Foundations, PATCH/delete, and schema checks usually run.
+- The query matrix runs when filter/order/select/search roles resolve.
+- Isolation runs only if you configured two principals.
+- Spec-as-adversary and tagged behavior never run — `doctor` says so.
+
+Worked examples of what a finding looks like:
+
+```text
+BACKEND_BUG   table   created row missing from GET /tables
+              list.read-after-write
+
+BACKEND_BUG   product  PATCH { name } cleared description
+              patch.minimality
+
+SECURITY      table    GET /tables/{id} readable with the other tenant's key
+              tenant.item-not-readable-cross-tenant
+
+SPEC_BUG      table    x-query.filterable lists "ghost"; filter=ghost.eq.x is 400
+              spec.declared-filterable-is-filterable
+
+BACKEND_BUG   table    in() is not the union of the equalities it lists
+              filter.in-is-union-of-eq
+
+BACKEND_BUG   table    an illegal filter operator is accepted
+              filter.illegal-op-rejected
+
+COVERAGE_GAP  batch    no x-async; receipt treated as the result
+              async.reaches-terminal-state
+```
+
+## Run-level findings
+
+The run raises these itself, outside any check. They appear in reports like a check's findings.
+
+| id                      | raised when                                                                                                                                                                      | verdict                     |
+| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
+| `world.seed`            | an entity's cohort could not be seeded: the create was refused, fenced, or ran out of values                                                                                     | `COVERAGE_GAP` or `BLOCKED` |
+| `create.does-not-error` | a create answered 5xx for a body generated from the documented schema while seeding                                                                                              | `BACKEND_BUG`               |
+| `world.teardown`        | records the run created could not be removed, or teardown stopped on an error                                                                                                    | `COVERAGE_GAP` or `BLOCKED` |
+| `teardown.verify-hook`  | `hooks.verifyRemoved` threw                                                                                                                                                      | `COVERAGE_GAP`              |
+| `auth.blocked`          | a principal or credential is BLOCKED; the detail names the step (`login`, `signup`, `refresh`) for `<principal>/<scheme>` and the error                                          | `BLOCKED`                   |
+| `profile.skip`          | the active profile excluded an operation, so the checks that depend on it stand down; with the create excluded and no existing records to fall back on, the entity is not tested | `COVERAGE_GAP` or `BLOCKED` |
+| `net.unreachable`       | the network went down and the remaining entities were skipped                                                                                                                    | `COVERAGE_GAP` or `BLOCKED` |
+| `run.interrupted`       | the run was stopped by a signal; what was tested is reported and what the API tester created is removed                                                                          | `COVERAGE_GAP`              |
+| `run.error`             | testing an entity, or a secondary origin, stopped on an unexpected error                                                                                                         | `BLOCKED`                   |
+
+## Verdicts, skips, and exit codes
+
+| verdict        | meaning                                                              | fails `testing api run`? |
+| -------------- | -------------------------------------------------------------------- | ------------------------ |
+| `BACKEND_BUG`  | the handler is wrong                                                 | yes                      |
+| `SPEC_BUG`     | the document is wrong (or disagrees with the handler)                | yes                      |
+| `SECURITY`     | isolation/authz failure and `x-tenant` (or equivalent) was declared  | yes                      |
+| `AMBIGUITY`    | same evidence, but the tenant boundary was only inferred             | yes                      |
+| `COVERAGE_GAP` | the check could not run; the report names the missing tag or surface | no                       |
+| `BLOCKED`      | a check this one depends on already failed                           | no                       |
+
+Separate from findings:
+
+- **did not apply** — entity never had what `needs` lists (printed in the report, not a pass). Example: no `select` parameter → `select.projection-honored` did not apply.
+- **inconclusive** — the check ran and stopped (empty listing, probe 4xx, no shared filterable field, …). Not a pass. The report prints the reason.
+
+Coverage is split **never** (zero entities could run it) vs **partial** (ran on some entities, skipped on others). A clean run still prints both. "Nothing found" and "nothing was looked for" are different.
+
+Cascade suppression is transitive: one root cause is one finding, not a page of consequences. A blocked check has **not** been verified; re-run after the cause is fixed.
+
+`testing api run` exit `1` if any finding has a failing verdict. Gaps and blocked entries do not fail a full run. A targeted run (`--ops`) also exits `1` when any target did not end `passed`.
+
+Console (stdout) after a run:
+
+```
+  50 checks · 5 entities · 842 requests · 41.2s · p95 90ms · 4 checks did not apply
+
+  BACKEND DEFECTS (1)
+    product          PATCH { name } also cleared description
+                     patch.minimality
+
+  DID NOT APPLY — no entity had what these need
+    async.reaches-terminal-state             needs an operation declaring x-async
+
+  leftover teardown printed next
+  report / matrix / graph / progress paths
+```