@workerstack/testing 0.0.0-stage → 0.31.0

package/docs/api/reports.md

@@ -0,0 +1,287 @@
+# testing api — Reports and progress logs
+
+[← back to the overview](../../README.md)
+
+## Reports
+
+Written under `--out` (default `./.testing/api/runs`). Each invocation creates a UTC timestamp folder and points `latest` at it:
+
+```
+.testing/api/runs/2026-08-18T12-00-00Z/
+.testing/api/runs/latest -> 2026-08-18T12-00-00Z
+```
+
+`--out` / `outDir` replace the root, not the leaf — `--out .testing/api/runs/prod` writes `.testing/api/runs/prod/<datetime>/`.
+
+| file                         |                                                                                                                                                                           |
+| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `report.html`                | **start here**: every operation in one group (Failed, Blocked, Effects unproven, Verified, Complete by contract), why, and links to the requests behind it                |
+| `report.md`                  | human report: scope, confidence counts, per-operation status, summary, findings with request/response excerpts, coverage, latency p50/p95/max                             |
+| `report.json`                | same data for CI. Summary only — full exchanges are not inlined here                                                                                                      |
+| `junit.xml`                  | only under `--report junit`: a suite per entity and a case per operation and check, for CI test-result views. See [JUnit and the job summary](#junit-and-the-job-summary) |
+| `matrix.html`                | visual matrix of entities × checks                                                                                                                                        |
+| `matrix.json`                | the same graph (AI-friendly), including a mermaid string                                                                                                                  |
+| `issue-repro/*.sh`           | one executable `curl` script per finding that has evidence. Directory is omitted when the run is clean                                                                    |
+| `progress.log`               | logfmt, one event per line, never truncated                                                                                                                               |
+| `progress.jsonl`             | same events as JSON                                                                                                                                                       |
+| `progress.tsv`               | same columns, tab-separated. `req_id` is the join key to the exchange journal                                                                                             |
+| `progress.json`              | latest snapshot only (overwritten ~1s)                                                                                                                                    |
+| `accounts.json`              | every account the run asked a backend to make, written before each sign-up was sent: its names (never its password), and whether teardown removed it                      |
+| `exchanges.jsonl`            | one line per request (`seq`, `requestId`, method, url, status, bytes, …). Greppable                                                                                       |
+| `exchanges/<requestId>.json` | full exchange: status, headers, described bodies. Missing id → `seq-<n>.json`. Duplicate id → `-<seq>`                                                                    |
+| `blobs/<sha256>`             | content-addressed file parts and oversized / binary bodies                                                                                                                |
+
+The journal is the API tester `Exchange` JSON, not HAR (HAR export is out of scope). Default **on** unless `--profile cheap`. `--no-save-exchanges` skips `exchanges/` entirely. `--quiet` does not. **`--no-save-exchanges` does not reduce transcript RAM** — it only skips the disk journal. The in-memory transcript always drops live `FormData` / `Blob` / `ArrayBuffer` / multi-MiB strings after each hop, keeping citeable metadata and content-addressed body descriptors. Reconstruct bytes from `blobs/<sha256>` when a check genuinely needs them.
+
+Bodies are described, never base64 multipart. JSON / text inline up to 256 KiB, then `blobs/<sha256>`. A response body past 256 KiB is validated against its documented schema when it arrives, while it is whole, so the schema checks judge it like any other. `FormData` file parts and binary downloads (`application/pdf`, spreadsheet, image, `octet-stream`) are always blobs; the same fixture bytes POSTed N times share one file. SSE is parsed incrementally into `{ event, data }[]` frames — the raw `text/event-stream` concatenation is not retained on the transcript.
+
+Redaction is on by default (not opt-in), applied on write:
+
+- Headers: `authorization`, `cookie`, `set-cookie`, `proxy-authorization`, and any `x-*-key` / `x-*-secret` / `x-ia-tester-key` (case-insensitive).
+- JSON bodies, any depth: `access_token`, `refresh_token`, `id_token`, `password`, `token`, `secret`, `api_key` → `"<redacted>"`.
+
+Every report names its scope. The console prints `scope: full · 141 of 212 operations graded` or `scope: targeted · 2 operations (--ops …)` followed by one line per target. `report.md` has a **Scope** line and an **Operations** table: on a full run, every operation in the document with its status and, for a pass, its **tier** (so operations no check ever grades are visible); on a targeted run, the targets only, followed by **Support operations** with call and non-2xx counts. In `matrix.html` / `matrix.json` a check a targeted run did not ask for is `out-of-scope`, distinct from `skipped` (not applicable).
+
+Join `progress.tsv` `req_id` to `exchanges.jsonl` `requestId` (response `x-request-id` / `request-id` / `x-correlation-id` / `correlation-id` wins; else what the API tester sent). `report.md` includes one line such as `1841 exchanges → exchanges/`.
+
+A pass carries `tier` and `evidence`, naming exchanges by their run `seq` (each saved exchange records it). `verified`: `evidence.observation` is a later exchange that holds `value` at `path`; for a delete, a later read answering `status` (404/410), or a later list where `value` is `absent`. `contract`: only `evidence.call` — a documented success whose body matched the schema.
+
+### How far a pass goes
+
+Calling a route and getting a documented answer is not the same as proving it works, and not every route has more to prove. Each pass carries `sufficiency` and a one-line `why`:
+
+| group                    | meaning                                                                                                                                                                      |
+| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Verified**             | a later request shows what the call did (a read returns the written value, a deleted record answers 404)                                                                     |
+| **Complete by contract** | a safe method (`GET` / `HEAD` / `OPTIONS` / `QUERY`) with no declared `x-effects`, or any operation that declares `x-effects: []`: meeting its contract is all it promises   |
+| **Effects unproven**     | met its contract, but whatever it changes was never observed. Either something should read the effect back, or the operation has none and should say so with `x-effects: []` |
+
+Operations that did not pass are **Failed** (a check found a defect) or **Blocked** (no verdict: a dependency failed, or no check could reach it), each with its reason. `report.html` shows the five groups worst first, with counts, the reason per row, and links to the call and the observing request in `exchanges/`. `report.md` opens its Operations table with the same counts under **Confidence**.
+
+### Leftovers
+
+A create that answered 2xx but named no id the API tester could read or confirm, in its body or its `Location` header, left a record teardown cannot remove. `report.md` lists each under **Leftovers** (method, URL, operation, status and exchange number), the console prints `leftovers: N create(s) answered 2xx but no id could be read`, `report.json` holds them as `"teardown": { "unidentified": [...] }`, and the run exits `4`. Declare `x-entity.createdIdFrom` where the answer keeps the id.
+
+A record the API tester identified but could not show gone — its delete was refused, or it is still listed after a 2xx delete — is listed under **Leftovers** too, and the run exits `4`. It reads ``- table `t9` · as beta · DELETE … returned beta: 403``. Every record names the principal that created it, since that principal's tenant is where it remains: the console prints `table t9 (as beta): …`, and `report.json` carries `principal` on each `teardown.failed` and `teardown.records` entry.
+
+Two reasons are teardown's own verdict rather than an answer it got. `its creator could never read it back, so teardown cannot tell whether it is gone` names a record no read or listing by its creator served, not even teardown's read before the delete, so a 2xx delete and an empty list prove nothing. Only the latest read counts: a creator's read that answers 404 or 410 takes back an earlier sighting, unless a delete the backend accepted had already removed the record. `hooks.verifyRemoved says it is still present` names a record the hook found after teardown counted it removed. A hook that threw is one `teardown.verify-hook` gap, and those records keep teardown's verdict.
+
+### Unverified seeds
+
+A create that answers without the record, `{ "id": ..., "message": "created" }` or an empty 201 with `Location`, is read back by the id it gave through the entity's item read. When there is no item read, the read answers something other than the record, or the creator cannot send it (its credential cannot meet the read's security, or the fence refuses it), the entity is seeded with the bodies the API tester sent, and the checks that work out stored values from the records stand down. That is a note, not a finding, one per entity, naming the cause: the console prints `note: table — seeded with the bodies the API tester sent, unverified (read back answered 404): …`, `report.md` lists it under **Observed behavior**, and `report.json` holds it in `observed` as `{ "check": "world.seed", "entity": "table", "note": "…" }`. The cause is `no item read`, `read back answered <status>`, or why the read could not be sent. A backend that misses a record right after creating it (eventual consistency) shows here, rather than quietly turning off its filter checks.
+
+### Known, fixed and the target version
+
+With a `baseline`, `report.md` lists the findings it names under **Known** (each with its reason) and the entries nothing matched under **Fixed**; the console prints `baseline: N known · M fixed`. Known findings carry `"known": { "reason": … }` in `report.json`, which also holds `"baseline": { "fixed": [...] }`. Only findings no entry names count toward the result and the exit code.
+
+With `targetVersion`, the markdown header shows `**Target version**: 1.0.88` (or why it could not be read) and `report.json` holds `"targetVersion": { "value" } | { "error" }`.
+
+### `report.json`
+
+`schemaVersion` comes first in `report.json`, and `matrix.json` states the same version after `kind`. It rises only on a breaking change to a file's shape; an added field is not one. `testing api diff` and `testing api baseline --from` read a report with no `schemaVersion` (0.24) or with `1`, and exit `2` on a later one: `testing api: <file> is report schemaVersion 2; this the API tester reads 1 — upgrade the API tester`.
+
+```json
+{
+	"schemaVersion": 1,
+	"backend": "https://api.example.com",
+	"targetVersion": { "value": "1.0.88" },
+	"generatedAt": "2026-08-15T12:00:00.000Z",
+	"durationMs": 41200,
+	"requests": 842,
+	"entitiesTested": ["store", "product"],
+	"scope": {
+		"mode": "targeted",
+		"requested": { "ops": ["product.update"], "only": [] },
+		"operations": [
+			{
+				"operationId": "product.update",
+				"entity": "product",
+				"status": "failed",
+				"checks": {
+					"passed": ["concurrency.no-lost-update"],
+					"failed": ["patch.minimality"],
+					"suppressed": [],
+					"inconclusive": [],
+					"gaps": []
+				},
+				"findings": 1,
+				"reason": null
+			},
+			{
+				"operationId": "product.create",
+				"entity": "product",
+				"status": "passed",
+				"tier": "verified",
+				"evidence": { "call": 14, "observation": { "seq": 15, "path": "$.items[*].id", "value": "prd_9" } },
+				"checks": {
+					"passed": ["create.persists-submitted-fields"],
+					"failed": [],
+					"suppressed": [],
+					"inconclusive": [],
+					"gaps": []
+				},
+				"findings": 0,
+				"reason": null
+			}
+		],
+		"support": [{ "operationId": "product.create", "calls": 9, "non2xx": 0 }],
+		"excluded": [],
+		"originsSkipped": []
+	},
+	"checksOutOfScope": [{ "check": "filter.in-is-union-of-eq", "entity": "product" }],
+	"checksRun": ["list.read-after-write", "patch.minimality"],
+	"checksSkipped": [{ "check": "async.reaches-terminal-state", "entity": "store", "needs": "…" }],
+	"checksSuppressed": [{ "check": "query.axes-compose", "entity": "store", "because": "list.read-after-write" }],
+	"inconclusive": [{ "check": "filter.and-composes-as-intersection", "entity": "store", "reason": "…" }],
+	"summary": { "BACKEND_BUG": 1 },
+	"coverage": {
+		"neverApplied": ["async.reaches-terminal-state"],
+		"partial": [{ "check": "select.projection-honored", "ran": 1, "skipped": 1 }]
+	},
+	"latency": {
+		"p50": 12,
+		"p95": 90,
+		"max": 400,
+		"slowest": { "method": "GET", "path": "/v1/products" }
+	},
+	"findings": [
+		{
+			"check": "patch.minimality",
+			"verdict": "BACKEND_BUG",
+			"entity": "product",
+			"summary": "PATCH { name } also cleared description",
+			"detail": "…",
+			"evidence": [
+				{
+					"method": "PATCH",
+					"url": "https://api.example.com/v1/products/p1",
+					"status": 200,
+					"requestBody": { "name": "x" },
+					"responseBody": { "name": "x", "description": null }
+				}
+			]
+		}
+	]
+}
+```
+
+Latency is reported, never asserted. The API tester has no baseline for "too slow".
+
+Gate in CI on process exit code, or on `findings` whose `verdict` is not `COVERAGE_GAP` / `BLOCKED`.
+
+### `matrix.json`
+
+```json
+{
+	"kind": "testing.api.matrix",
+	"schemaVersion": 1,
+	"baseUrl": "…",
+	"generatedAt": "…",
+	"thesis": "…",
+	"summary": "…",
+	"index": { "entityCount": 5, "failed": ["product"], "parents": ["store"], "crossClaims": 1, "inbound": {} },
+	"counts": { "failed": 1, "blocked": 0, "passed": 40, "skipped": 14 },
+	"entities": [
+		{
+			"name": "product",
+			"identity": "id",
+			"readSurface": ["GET /v1/products", "GET /v1/products/{id}"],
+			"counts": { "failed": 1, "blocked": 0, "passed": 20, "skipped": 5 },
+			"roots": ["store_id"],
+			"nodes": [
+				{
+					"id": "product/patch.minimality",
+					"group": "product",
+					"layer": "axis",
+					"status": "failed",
+					"verdict": "BACKEND_BUG",
+					"summary": "…"
+				}
+			]
+		}
+	],
+	"invalidate": [
+		{
+			"fromEntity": "product",
+			"fromOp": "product.create",
+			"toEntity": "store",
+			"toRoute": "GET /v1/stores/{id}",
+			"cross": true
+		}
+	],
+	"edges": [{ "from": "product/list.read-after-write", "to": "product/patch.minimality", "kind": "dependsOn" }],
+	"mermaid": "flowchart LR\n…"
+}
+```
+
+Cell `status`: `passed`, `failed`, `blocked`, `skipped`. Edge `kind`: `dependsOn` (cascade) or `uses` (a composition check built from a single-axis check).
+
+### JUnit and the job summary
+
+`--report junit` adds `junit.xml` to the run folder, so `.testing/api/runs/latest/junit.xml` is a stable path for a CI step. It is the common denominator Jenkins, GitLab and the GitHub test-report actions read: one `testsuites`, one `testsuite` per entity, and one `testcase` per operation and check. Each check appears once per operation; when one is listed under several outcomes, the first in this table wins.
+
+| testing api                                         | JUnit                                                                                                                    |
+| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
+| a failing root cause                                | `<failure type="<verdict>" message="<summary>">`: the detail, then up to 5 requests as `METHOD url → status`             |
+| a known finding (baseline)                          | `<skipped message="known (baseline): <reason>">`, since it does not fail the run                                         |
+| a coverage gap                                      | `<skipped message="coverage gap: <summary>">`                                                                            |
+| an inconclusive check                               | `<skipped message="inconclusive: <reason>">`                                                                             |
+| a check blocked by an earlier failure               | `<skipped message="blocked: waiting on <check>">`                                                                        |
+| a passed check                                      | a bare `testcase`                                                                                                        |
+| an operation no check graded                        | one `testcase name="graded"`, skipped with the reason                                                                    |
+| a check that stood down on an entity                | `testcase classname="<entity>" name="<check>"`, skipped with `stood down: <what it needs>`                               |
+| a finding no operation holds (teardown, network, …) | a case in the `testing.api.run` suite, `classname="run"`, named by its check                                             |
+| a run that failed (exit 3) or left records (exit 4) | `<testcase classname="run" name="outcome">` holding `<error type="<outcome>" message="<reason>"/>`, in `testing.api.run` |
+
+Suites are named by entity: `operations` holds an operation with no entity, and a secondary origin's are `<origin>:<entity>`. A case's `classname` is its `operationId`, `<origin>:<operationId>` on a secondary origin. So a gate that reads only the JUnit file fails exactly when the exit code is not `0`. Text goes through the run's redactor, then XML escaping; characters XML 1.0 forbids (most control characters) are dropped, and no CDATA is used.
+
+Inside GitHub Actions, where `GITHUB_STEP_SUMMARY` is set, every `testing api run` also appends a job summary in Markdown, with no flag: the outcome and exit code, the base URL and run size, the operations by status, a table of root causes (failing ones first, then known ones), the count of coverage gaps and blocked findings, a collapsed list of what went untested, and the path to `report.md`. It goes through the same redactor. It stays within 64 KiB: rows are dropped from the end of the untested list, then from the end of the findings, each list ending `… and <n> more, truncated, see <run folder>`. A summary that cannot be written prints `testing api: could not write the step summary: <reason>` and leaves the exit code alone.
+
+### `issue-repro/<entity>-<check>.sh`
+
+Created only when a finding has HTTP evidence. Not created on a clean run.
+
+```bash
+#!/usr/bin/env bash
+# Generated by the API tester. Set TOKEN to a valid credential before running.
+# product — PATCH { name } also cleared description
+set -u
+BASE="${BASE:-https://api.example.com}"
+TOKEN="${TOKEN:?set TOKEN to a valid credential}"
+
+# step 1 — observed 200
+curl -sS -X PATCH "$BASE/v1/products/p1" \
+  -H "authorization: Bearer $TOKEN" \
+  -H "content-type: application/json" \
+  -d '{"name":"x"}'
+```
+
+`authorization`, `cookie`, and `x-api-key` are redacted to `$TOKEN`. Replay needs a live credential; ids in the script are whatever the failing run created (gone if teardown ran). Use `--keep-fixtures` when you want to replay against the same rows.
+
+## Progress logs
+
+`progress.log` starts with a glossary. Fields:
+
+| key                        |                                                                                  |
+| -------------------------- | -------------------------------------------------------------------------------- |
+| `ts`                       | ISO-8601 UTC when the line was written                                           |
+| `status`                   | `ok`, `stall` (`idle_ms` ≥ 15000 after the last call returned), or `in_flight`   |
+| `done` / `total`           | entity index / how many entities                                                 |
+| `phase`                    | `load` \| `auth` \| `seed` \| `test` \| `teardown` \| `done`                     |
+| `entity` / `check`         | current entity and check id                                                      |
+| `msg`                      | phase note. In-flight is `status`, not `msg`                                     |
+| `req` / `find`             | request count, finding count                                                     |
+| `method` / `path` / `http` | last completed call, or the in-flight call when `status=in_flight` (`http=-`)    |
+| `last_ms`                  | duration of the last completed call. `-` while `in_flight`                       |
+| `idle_ms`                  | while `in_flight`: ms since this request started. otherwise ms since it returned |
+| `elapsed_ms`               | wall clock since start                                                           |
+
+`--quiet` keeps the files and drops stderr.
+
+Every HTTP call emits a **start** line (`status=in_flight`, `http=-`, `last_ms=-`) when `fetch` is issued, then a **completed** line when it returns (`http=200`, `last_ms` ≈ wall time, `idle_ms=0`). The 5 s heartbeat keeps the in-flight path and climbs `idle_ms` until the call returns; after return it climbs on the completed call (and becomes `stall` at 15 s). A 300 s `POST /extract` therefore stays `/extract` in the log, not the previous refresh or GET.
+
+Lines are also written when the phase/entity/check/message changes, or every 2 s, or on `load`/`done`. `progress.json` is rewritten about once a second.
+
+If `idle_ms` climbs through a long poll (`x-async`) that is expected. If it climbs on a simple GET while `status` is not `in_flight`, the process or the network is stuck. Live `run` aborts a single `fetch` after `network.requestTimeoutMs` (default **180s**). A hung `text/event-stream` becomes `net.unreachable` (`kind=timeout`), not an unbounded buffer on the transcript. Set `requestTimeoutMs: 0` only if you intend to wait for the socket.
+
+Bun and Node both honor that timeout. If the API tester still runs away with RAM (a bug, or a caller holding the process open), treat the process like any other prefer-kill target for `earlyoom` / systemd — Bun is often **not** in a default `--prefer` list.

package/docs/api/roadmaps/1-freshness-program-20261028T232448Z.md

@@ -0,0 +1,90 @@
+---
+serial: 1
+name: freshness-program
+status: done
+created: 2026-10-28T23:24:48Z
+source: "freshness program"
+---
+
+# freshness-program
+
+## Goal
+
+The API tester 0.26.0 tests a cached API's freshness against the staleness it declares, and anyrow declares every cached read's bound from standard cache presets. Its two security gates no longer rest on caches a revocation cannot clear. A full prod retest proves both sides together.
+
+## Context
+
+- The 2026-10-28 probes on prod (`~/.cache/testing-gates/evidence/`):
+  - profile reads stale 150–450 ms in 195 of 200 rounds;
+  - a deleted template is still served for up to 304 s;
+  - a revoked API key keeps working for about 30 s.
+
+  The database is always right: an `x-fresh` read answered correctly every time.
+
+- anyrow mutations already declare `x-invalidate`, and the frontend reads fresh after its own writes. Background cache writes stay the default; staleness only reaches other viewers, bounded by each cache's soft TTL.
+- anyrow's 16 `TieredCache` definitions use 9 different TTL pairs. The admin gate (`platformAdminCache`, 30 / 60 min) and the TOTP replay guard (`totpReplayCache`, a background KV write) are the two security defects.
+- The API tester today reads once right after a write and assumes strict read-your-writes, so a cached API looks broken and real overruns go unmeasured. The existing three-poll wait rule for async effects is the polling base.
+
+## Decisions
+
+- D1. Freshness is declared per operation as `x-consistency: { staleness: "<n>s", bypass: "<header: value>" }`. The API tester also reads the standard signals: request `Cache-Control: no-cache` (RFC 9111), and response `Cache-Status` (RFC 9211) and `Age`. Rejected: an anyrow-only `x-fresh` convention baked into the API tester.
+- D2. The API tester judges correctness and freshness apart. Verification reads carry the declared bypass. A new `consistency.staleness-within-bound` check measures the window and fails only past the declared bound, and only reports it when nothing is declared. A `consistency.bypass-is-fresh` check confirms the bypass. Rejected: polling every verification read until it converges, which hides real staleness.
+- D3. anyrow standardizes on three presets plus none: `SHORT` (30 s / 60 s, KV's TTL floor), `STANDARD` (2 min / 10 min) and `LONG` (1 h / 24 h). The in-memory tier stays a per-cache flag. Rejected: per-cache hand-picked TTLs.
+- D4. A route's `x-consistency` is generated from the preset of the cache it reads through, so the spec cannot drift from the code. Rejected: hand-written extensions.
+- D5. The admin gate reads `platform_admin` from D1 on every admin request (no cache). The TOTP replay guard moves to a D1 table with an atomic insert. Rejected: a shorter TTL on either, which still leaves a window.
+- D6. Order: security fixes first (independent and urgent), then the API tester 0.26.0, then anyrow's declarations (which need 0.26.0 to verify), then the joint retest. Rejected: anyrow declarations before the API tester that tests them.
+
+## Order
+
+1 [x] Title: anyrow security cache fixes
+Goal: The admin gate reads platform admin from D1 on every request, and TOTP codes are single-use through an atomic D1 insert, deployed to prod with its migration.
+Why: Both are live security defects that need nothing from the API tester. Shipping them first also removes two caches before the presets slice reshapes the rest.
+Location: .tmp/grunt/plans/14-anyrow-security-cache-fixes-20261028T232743Z.md
+2 [x] Title: the API tester 0.26.0 freshness support
+Goal: the API tester 0.26.0 is released, reading `x-consistency` and the standard cache headers, sending the bypass on verification reads, and shipping `consistency.staleness-within-bound` and `consistency.bypass-is-fresh` with reference defects, goldens and docs.
+Why: anyrow's declarations in slice 3 are only provable once a released the API tester can test them.
+Location: .tmp/grunt/plans/15-testing-freshness-support-20261028T234729Z.md
+3 [x] Title: anyrow cache presets and declarations
+Goal: anyrow runs on prod with every cache on a `SHORT`, `STANDARD` or `LONG` preset, a generated `x-consistency` on every cached read, `Cache-Control: no-cache` honored like `x-fresh`, and `Cache-Status` and `Age` on cached answers.
+Why: It depends on slice 2's checks to be verified, and it fixes the template delete's 5-minute staleness by moving it to `STANDARD`.
+Location: .tmp/grunt/plans/16-anyrow-cache-presets-declarations-20261029T042517Z.md
+4 [x] Title: the API tester 0.27.0 attribution and caps
+Goal: the API tester 0.27.0 is released, where a write whose own answer or bypass read drops a submitted field is reported once as `update.persists-submitted-fields` (never as staleness or a lost update), `consistency.*` and `concurrency.no-lost-update` stand down on such fields, and `consistency.staleness-within-bound` caps its watch at 10 minutes whatever the declared bound, reporting "still stale at cap" without failing.
+Why: The joint retest must attribute causes correctly, and a declared 24 h bound must not plan a 24 h watch.
+Location: .tmp/grunt/plans/17-testing-attribution-and-caps-20261029T092754Z.md
+5 [x] Title: anyrow writer freshness and org update
+Goal: anyrow's org.update body documents only the fields it writes, and the generated SDK sends `x-fresh: 1` on reads a mutation's `x-invalidate` marked stale (for the route's declared staleness), turned on in the dashboard.
+Why: It removes the over-claim the new check exposed, and gives writers real read-your-writes, which nothing does today.
+Location: .tmp/grunt/plans/18-anyrow-writer-freshness-org-update-20261029T123256Z.md
+6 [x] Title: joint freshness retest
+Goal: A full anyrow prod acceptance run on the API tester 0.27.0 shows the consistency checks within the declared bounds, a revoked admin and a replayed TOTP refused at once, and `targets.yml` still green.
+Why: It proves the whole program end to end on the real API, the way the 0.25 releases were proven.
+Done 2026-10-29: full run 2026-10-29T18-23-35Z (the API tester 0.27.0, monorepo 72fe73291, 18:23–20:07 UTC, uninterrupted, 38 scenarios) shows 0 consistency.* findings, no organization over-claim, and scenario.mail passing. Security probes on prod: TOTP replay 200→403, revoked admin 200→403 at once. targets.yml 40942174004 15/15 on bb175d5. The earlier attempt (run 14-51-58Z) was interrupted, and its slowness was Gemini upstream latency (p50 1.0 s → 26 s), not anyrow or the API tester. History from before the run:
+
+Paused 2026-10-29 at the user's request. To resume, do items 1 and 2 of "Before the full run" below, then start the slice 6 full run on the deployed shas (`cd $R/api/_tester && CLOUDFLARE_ENV=prod bun run test:the API tester`), and rerun the two security probes alongside it. `targets.yml` 40880998170 (the API tester 7c69a68) is already green.
+
+- Deployed: saas fbba71e6, extract 4f03d51f, gateway ba084099, dashboard e9db42a2 (monorepo e12f383f0), the API tester 0.27.0.
+- The full run also closes slice 5's 6.4: no `update.persists-submitted-fields` on organization, and nothing new against slice 4's run. Then tick plan 18 6.4, set plan 18 done, and mark slice 5 [x].
+- Probe script: scratchpad `freshness-probe.ts`, from slice 1. Its run just before the pause failed at registration under bun (the tester helpers' register call has returned 407 under bun today, but 201 under node with the same headers). Then its cleanup's admin sign-in crashed on a null error body. Prod check afterward: 0 probe `platform_admin` rows and no new probe account. Fix the register path (or run it under node) before rerunning.
+- Before the full run, in this order (testing-4f):
+  1. `scenario.mail` got captcha_required at step 1, at 13:38:23Z, about 67 min into run 12-31-36Z. The deploy is ruled out: 785248a69 is the only saas/common change since slice 3, and it doesn't touch the captcha or the tester gate.
+     - Lead: the harvest queue. Register takes a harvested Turnstile token from KV `spec-tester:harvest:turnstile:<runId>`, which keeps only entries younger than 280 s, and the previous run ended at 66 min. Check whether the testing-harvest Chrome window (pid 4012395) was still solving at 13:38, and whether the queue held a fresh entry then.
+     - Test the bun 407 on its own first, as a local proxy or runtime question, before linking the two.
+     - Prove the cause before any fix: the full run would hit the same thing.
+  2. Run 12-31-36Z left 33 of 179 records unremoved. Confirm each belongs to an account the admin delete removed, so the batched purge took its sole-owned org. Sweep any that survive, under the tester-pattern rule. In the same pass, remove `testing-probe-freshness-1793230634071@anyrow.ai` (usr_01m62dzza0cvava8td2003b6yy), a slice 1 leftover with no admin row.
+  3. The api-key and table update over-claims are written into handoff 7 as an anyrow follow-up, outside this program.
+
+## Watch-outs
+
+- KV's minimum `expirationTtl` is 60 s, so a `SHORT` hard TTL cannot go lower in KV.
+- The TOTP migration changes prod D1 (`anyrow-core-prod`). Apply it before deploying the saas code that writes the table, and keep the old KV guard readable for one release so in-flight codes are not double-accepted during the switch.
+- the API tester's new checks must stay quiet on APIs that declare nothing: no declaration means report the window, never fail on it. Otherwise every target's `expected.json` breaks.
+- honey 1.0 cutover item: an invalidate selector naming another worker's route. honey 0.7.3 refuses one (INVALID_SELECTOR), so saas's template update/delete cannot invalidate extract's `GET /v1/projects/:project_id/table-templates/:template_id`. The dashboard never reads that preview, and its declared 600 s bound still holds server-side (slice 5, 785248a69).
+- The monorepo's repo-wide type-check and ci hook jobs fail in packages outside this program. Skip only those jobs, and prove this program's packages' own gates.
+
+## Verify
+
+7 [x] Verify
+7.1 [x] All six child plans have finished their Verify. Plans 14–18 are status done; slice 6 is this roadmap's run.
+7.2 [x] npm serves `@workerstack/testing@0.26.0` and `@workerstack/testing@0.27.0` with provenance, and each one's CI and `targets.yml` passed on its tag commit.
+7.3 [x] The anyrow prod run on 0.27.0 reports no staleness past a declared bound, and both security probes pass on prod. Run 2026-10-29T18-23-35Z: 0 consistency.* findings. Probes: testing-3f probe6.log.

package/docs/api/steps.md

@@ -0,0 +1,162 @@
+# testing api — Step builders
+
+[← back to the overview](../../README.md)
+
+`@workerstack/testing/api/steps` writes [scenario](configuration/reference.md) steps as typed calls instead of
+object literals. You give it your API's types once. After that, a scenario that names an operation,
+a body field, a path parameter, an error key, an external or a mail kind your API lacks fails at
+`tsc`, before anything is sent. Each builder emits the plain step object a hand-written config
+holds, so the API tester runs a built scenario exactly as it runs the literal.
+
+```ts
+import { steps } from "@workerstack/testing/api/steps"
+
+/* Where a credential goes is your API's to say: here, a session is a bearer token. */
+const { anon, scenario, self, session } = steps<MyTypes>({ session: (name) => ({ authorization: `Bearer {${name}}` }) })
+const rotated = session("rotated")
+
+export const reauth = scenario({
+	id: "reauth",
+	covers: ["auth.reauthenticate", "auth.deleteAccount"],
+	steps: [
+		self.refused("auth.deleteAccount", { status: 403, key: "account_reauth_required" }),
+		self
+			.call("auth.reauthenticate", { body: { password: "{password}" } })
+			.atLeast("$.reauthenticated_until", Date.now()),
+		anon.call("auth.refreshToken", { body: { refresh_token: "{refreshToken}" } }).save({ rotated: "$.access_token" }),
+		rotated.call("auth.deleteAccount").status(204),
+	],
+})
+```
+
+The second step is the literal
+`{ operationId: "auth.reauthenticate", body: { password: "{password}" }, assert: { from: "$.reauthenticated_until", atLeast: … } }`.
+
+## Your types
+
+`steps<T>()` is typed over one `StepTypes`. Only `ops` is required. Write each map as a `type`
+alias, not an `interface`: an interface does not satisfy the index signature `OpMap` declares.
+
+```ts
+type MyTypes = {
+	/* Operation id → its OpShape: body, bind (path parameters), query, and error keys per status. */
+	ops: {
+		"auth.reauthenticate": { body: { password: string }; errors: { 403: "account_reauth_failed" } }
+		"org.delete": { bind: { organization_id: string } }
+	}
+	/* A configured origins[] name → that origin's operations. */
+	origins: { cdn: { "cdn.upload": { bind: { wildcard: string }; body: Uint8Array } } }
+	/* External name → its input; derive it from your registry with ExternalInputs. */
+	externals: ExternalInputs<typeof EXTERNALS>
+	/* The kinds hooks.resolveOutOfBand answers. */
+	mail: "magic-link" | "otp"
+}
+```
+
+`steps<T>(options?)` takes `StepsOptions`: `session`, the headers a session held in the scope as
+`{name}` is sent with (without it, `session(name)` throws), and `origin` (see below).
+
+An `OpShape` part you leave out takes nothing: an operation with no `body` accepts only an
+`unchecked(...)` one, and one with no `bind` accepts no path parameter. `Fillable<T>` lets every
+scalar in a body also take a `Placeholder`, a text holding a `{name}`, so `amount: "{amt}"` passes
+where `amount` is a number. Bytes pass as they are.
+
+## The builder set
+
+`steps<T>()` returns a `Builders<T>` with these names:
+
+| builder                                                | emits                                                                         |
+| ------------------------------------------------------ | ----------------------------------------------------------------------------- |
+| `self`                                                 | the scenario principal's own step (no identity keys)                          |
+| `anon`                                                 | `anonymous: true`                                                             |
+| `session(name)`                                        | `anonymous: true` and the headers `StepsOptions.session(name)` gives          |
+| `principal(name)`                                      | `principal: name`                                                             |
+| `origin(name)`                                         | `self`'s step with `origin: name`, typed over that origin's operations        |
+| `mail(kind, address).as(name)`                         | `outOfBand: { kind, address, as }`; `.sentEarlier()` adds `sentEarlier: true` |
+| `external(name, input?).as(name)`                      | `external`, `input` (left out when the external takes none), `as`             |
+| `scenario({ id, covers, steps, principal?, origin? })` | the plain `Scenario`                                                          |
+| `unchecked(value)`                                     | the value itself, typed `V & Unchecked` past the body check                   |
+
+Each sender is a `Sender` with:
+
+- `call(op, init?)`: `operationId` plus the `CallInit` parts given (`body`, `bind`, `query`,
+  `headers`, `mediaType`). A session's headers merge under the call's, the call winning per key.
+- `refused(op, { status, key, ...init })`: the call, `expect: [status]`, and
+  `assert: { equals: key, from: "$.error_key" }`. `key` must be one the operation declares for
+  that `status`. Its type is `Refusal`.
+- `request(method, path, init?)`: a raw request; `RawInit` takes `body`, `headers`, `query` and
+  `redirect`.
+- `on(origin)`: the same sender with `origin` on every step it builds.
+
+A call returns a `RequestBuilder`. Each chain call returns a new builder, so a builder shared by two
+chains stays as it was:
+
+| chain                                                        | writes                                                               |
+| ------------------------------------------------------------ | -------------------------------------------------------------------- |
+| `.save({ name: "$.path" })`                                  | `saveAs`, merged across calls                                        |
+| `.claims(token, bind)`                                       | `saveClaimsFrom`                                                     |
+| `.status(...codes)`                                          | `expect`                                                             |
+| `.equals` / `.contains` / `.excludes` / `.omits(from, text)` | one assert each                                                      |
+| `.atLeast(from, floor)`                                      | an assert with a number floor                                        |
+| `.observes(...ops)`                                          | `observes` on the assert just before it; with no ops, `observes: []` |
+| `.poll(within, everyMs?)`                                    | `poll`                                                               |
+| `.scheme(name)`                                              | `scheme`                                                             |
+
+One assert is written as an object and several as an array, as in a hand-written step. No
+emitted object holds an own `undefined` key.
+
+A builder is a `BuiltStep`. A scenario's steps are `StepItem`s, which are builders or plain steps,
+so a sign-up flow's own steps mix in. `ScenarioInit` types `covers` over the document's
+operations: it stays an explicit list, because a scenario grades only what it covers.
+
+### A scenario that belongs to an origin
+
+A scenario whose whole flow runs on one configured origin uses that origin's own set:
+
+```ts
+const cdn = steps<{ ops: CdnOps }>({ origin: "cdn" })
+cdn.scenario({ id: "cdn-files", covers: ["cdn.upload"], steps: [cdn.self.call("cdn.upload", { … })] })
+```
+
+Its `scenario` writes `origin: "cdn"` on the scenario and none on the steps. A step's own `origin`
+makes it a support call to another origin, which never counts toward `covers`. That is why
+`origin(name)` and an origin's set are not the same thing.
+
+## Externals
+
+`defineExternals<Ctx, Meta>()(entries)` declares the externals your `hooks.external` answers, one
+`ExternalEntry` each: your `Meta` fields (a `kind`, say) and `resolve(input, ctx)`. It returns the
+entries unchanged, so the hook calls `entries[name].resolve(input, ctx)`, and
+`ExternalInputs<typeof entries>` gives the scenario side each input type.
+
+```ts
+const EXTERNALS = defineExternals<{ stripe: Stripe }, { kind: "computed" | "api" }>()({
+	totp: { kind: "computed", resolve: (input: { secret: string }) => nextTotp(input.secret) },
+})
+```
+
+`externalKinds(entries)` reads each entry's string `kind` into the map `config.externals.kinds`
+takes, so a profile's `excludeExternalKinds` can leave out the scenarios that need one:
+`externals: { kinds: externalKinds(EXTERNALS) }`.
+
+`ExternalDraft`, `MailDraft` and `MailStep` are the types of the drafts and steps those builders return.
+
+## Testing scenarios
+
+`stepsOf(scenarios, id)` returns a `ScenarioView` of one scenario for its unit tests:
+
+```ts
+const arm = stepsOf(scenarios(fakeRun), "offer-arm")
+expect(arm.indexOf("auth.updateNotificationPreferences", { body: { marketing: true } })).toBeLessThan(
+	arm.indexOf("domain.list"),
+)
+expect(stepsOf(scenarios(fakeRun), "account").observes("auth.logout")).toBe(true)
+```
+
+- `indexOf(op, where?)`: the first call of `op` whose fields include `where`, deep, with arrays
+  matched item by item. It returns -1 when no call does.
+- `calls(op)`: every call with its index.
+- `asserts()`: every assert, flattened.
+- `observes(op)`: true when the scenario covers `op`, calls it, and a later assert names it.
+
+An unknown id throws and names the ids there are.