vibecodeagent 0.0.0-stage → 0.1.65

package/dist/resources/skills/builtin/backend-engineering/references/api-contracts.md

@@ -0,0 +1,320 @@
+# API Contracts
+
+This file owns the wire: method semantics, which status code a condition selects, the shape of the error envelope, pagination, versioning and deprecation, conditional requests, and the bulk-response contract. Turning bytes into typed values and classifying a failure into a taxonomy is [validation-and-errors.md](validation-and-errors.md) — the class table there decides *which* status a failure gets, this file decides what that status means and what must ride with it. Idempotency **mechanics** (the key table, the fingerprint, the replay) are [concurrency-and-jobs.md](concurrency-and-jobs.md); the header, which endpoints demand it, and what the client sees are here. `401` vs `403` as a credential question is [auth-and-secrets.md](auth-and-secrets.md). Rate-limit policy and CORS are [security.md](security.md). The index that makes a cursor fast is [data-modelling.md](data-modelling.md). The `trace_id` in the envelope is [observability.md](observability.md).
+
+## Operating Posture
+
+Two failure modes, and the first is worse:
+
+1. **A change that breaks a client you cannot see.** A removed field, a tightened `max`, a new enum value in a response, a changed default. Nothing throws. CI is green, the deploy is clean, and an integrator's nightly job dies at 03:00 — you find out from a support ticket eight days later, and the fix is a rollback of something you have already built on top of.
+2. **A contract that lies about the failure.** `200` with an error body, a `500` on the client's own bad input, a `404` where the row is present but not theirs, a `429` with no `Retry-After`. The client cannot decide whether to retry, so it either hammers you or gives up on a transient fault.
+
+**Never invent a header.** `X-Total-Count`, `X-Page`, `X-Next-Cursor`, `X-Api-Version` are conventions somebody's blog post made up; none is in the IANA HTTP Field Name registry, and RFC 6648 (2012) deprecated the `X-` prefix outright. Use a standard field, or put the value in the body where it is versioned with everything else. Likewise never invent a status code — the `499` / `509` / `418` prohibition in [validation-and-errors.md](validation-and-errors.md) is restated, not re-derived, here.
+
+## 1. Method Semantics
+
+Three properties, defined in RFC 9110 §9.2.1 (safe), §9.2.2 (idempotent), and §9.3 (per method). They are not documentation. Infrastructure you do not control acts on them.
+
+| Method | Safe | Idempotent | Cacheable | What actually breaks when you violate it |
+|---|---|---|---|---|
+| `GET` | Yes | Yes | Yes | A prefetcher replays it. In 2005 Google Web Accelerator prefetched links on pages users opened and silently invoked plain-GET "delete" links across half the Rails apps on the web. Modern equivalent: Outlook Safe Links and every corporate mail scanner fetch every URL in an email — which is exactly why RFC 8058 exists and one-click unsubscribe is a `POST`. |
+| `HEAD` | Yes | Yes | Yes | Same as `GET`, and it must return identical headers with no body. A framework that runs the handler and truncates is sending you a full query for a header check. |
+| `POST` | No | **No** | Only with explicit freshness (RFC 9111 §4) | The client's timeout is not a failure signal — the request may have succeeded. Its retry creates a second row or a second charge. Fixed by `Idempotency-Key`, §8. |
+| `PUT` | No | Yes | No | A `PUT` that *appends* is not idempotent, and a retried request adds the item twice. `PUT` replaces the whole representation, always. |
+| `PATCH` | No | **No** (RFC 5789) | No | `{"op": "increment", "by": 1}` applied twice adds two. Use JSON Merge Patch (RFC 7386), which sets absolute values and is idempotent; JSON Patch (RFC 6902) is not, so it needs `If-Match` or a key. |
+| `DELETE` | No | Yes | No | The retry of a successful delete must not report failure. Return `204` for both the first call and the repeat; `404` only when you genuinely cannot distinguish "already gone" from "never existed", and `410` when you keep a tombstone and want the client to stop asking. |
+| `OPTIONS` | Yes | Yes | No | CORS preflight — [security.md](security.md). Never route business logic through it. |
+
+**The symptom of a mutating `GET` is not an error in your logs.** It is rows disappearing during a crawl, or an "unsubscribe" that fires before the human clicks. If a query parameter changes state, the method is wrong no matter how convenient the link was.
+
+## 2. The Status Code Decision Table
+
+One question per code. Ask them top to bottom and take the first `yes`.
+
+| Code | The one question | Must carry |
+|---|---|---|
+| `200` | Did the work finish, and is there a body describing the result? | The representation |
+| `201` | Did this create a resource that now has its own URL? | `Location:` the new URL. A `201` without it makes the client guess or re-query. |
+| `202` | Did I only *accept* the work, with the outcome decided later? | `Location:` a status resource. See §8. |
+| `204` | Did it finish with nothing worth sending? | **No body.** A `204` with bytes is a protocol violation some proxies will strip and some clients will choke on. |
+| `400` | Could I not parse the request at all? | Bad JSON, bad framing, undecodable query string. Wrong media type is `415`. |
+| `401` | Do I not know who this is? | `WWW-Authenticate` — required by RFC 9110 §11.6.1 and omitted by almost everyone. |
+| `403` | I know who this is, they may not do it, **and it is safe for them to learn the resource exists**? | Generic detail. If the second clause is false, the answer is `404`. |
+| `404` | Is it absent — *or* present, and its existence is private? | A body identical in both cases. |
+| `409` | Is the request valid but incompatible with the resource's **current** state, so the same request could succeed later? | Which invariant, in business terms |
+| `410` | Was it deliberately and permanently removed? | The replacement. Google's crawler treats `410` as a stronger delete signal than `404`; so should your clients. |
+| `412` | Did a **client-supplied precondition** (`If-Match`, `If-Unmodified-Since`) fail? | The current `ETag`. See §7. |
+| `422` | Did it parse, but mean something I cannot accept **no matter when** it is sent? | Field path + rule ([validation-and-errors.md](validation-and-errors.md)) |
+| `429` | Did they exceed a limit that time alone will fix? | `Retry-After` (RFC 9110 §10.2.3) |
+| `500` | Is this my bug, with no idea whether a retry helps? | `code` + `trace_id`, nothing else |
+| `502` | Did an upstream answer, but with a failure or garbage? | Generic — never the upstream's text |
+| `503` | Am I up and deliberately refusing right now — overload, draining, dependency down? | `Retry-After` |
+| `504` | Did an upstream fail to answer within my budget? | Generic |
+
+Three boundaries carry most of the mistakes:
+
+**`403` vs `404` — the existence leak.** If knowing the resource exists is itself information the caller is not entitled to (another tenant's order, a private repository, a user id), return `404`. It must be byte-identical to a real `404`: same `code`, same `detail`, same shape. The symptom of getting this wrong is an **enumeration oracle** — an attacker walks `/orders/1..n`, records `403` vs `404`, and has your customer count and your growth rate without ever reading a row. Use `403` only where the caller could already legitimately know the thing is there: a resource in their own tenant that their role does not cover. Enforcement lives at the data layer, per the hub's gate 1 and [auth-and-secrets.md](auth-and-secrets.md).
+
+**`400` vs `422` — the framing line.** `400` means *I could not turn these bytes into a document*. `422` means *I read the document and its meaning is unacceptable* (RFC 9110 §15.5.21, moved into core HTTP from WebDAV). Bad JSON is `400`. `"currency": "JPY"` against a closed enum is `422`. If your framework returns `400` for both, that is a defect the client feels: it cannot tell "my serializer is broken" from "this order is invalid", and those need opposite responses.
+
+**`409` vs `422` — will time help?** `409` if the same bytes could succeed after the resource changes ("order already confirmed", "email already registered"). `422` if they could never succeed ("quantity must be ≥ 1"). A client with retry logic reads exactly this distinction.
+
+**Never `200` with `{"error": ...}`.** The hub's rule, and here is the operational cost: every load balancer, every uptime check, every 5xx-rate dashboard, and every client's `res.ok` says the request succeeded. Failures become invisible to monitoring and surface as missing data weeks later.
+
+## 3. The Error Envelope — RFC 9457
+
+One shape, every endpoint, `Content-Type: application/problem+json`. RFC 9457 obsoletes RFC 7807; the member set is unchanged.
+
+| Member | Source | Rule |
+|---|---|---|
+| `type` | RFC 9457 | Absolute URI naming the **problem class**. Defaults to `about:blank`. One URI per class, dereferenceable to a docs page. Never per-occurrence, never a UUID. |
+| `title` | RFC 9457 | Short human summary, **constant for a given `type`**. It is prose; treat it as untranslatable-today and translatable-tomorrow. |
+| `status` | RFC 9457 | The same number as the status line. Duplicated on purpose, so the body survives being logged or copied out of the response. |
+| `detail` | RFC 9457 | This occurrence, for a human. Free to change any day. |
+| `instance` | RFC 9457 | URI of this occurrence — the request path, or an error id. |
+| `code` | **Our extension** | Stable `SCREAMING_SNAKE` enum from one file. The only member a client may branch on. |
+| `trace_id` | **Our extension** | Ties the body to the log line ([observability.md](observability.md)). |
+| `errors` | **Our extension** | Field-level failures with JSON Pointer paths — shape in [validation-and-errors.md](validation-and-errors.md). |
+| `retryable`, `retry_after_ms` | **Our extension** | Server-computed from the class table in [validation-and-errors.md](validation-and-errors.md). |
+
+```http
+HTTP/1.1 409 Conflict
+Content-Type: application/problem+json
+Content-Language: en
+
+{
+  "type": "https://api.example.com/problems/order-already-confirmed",
+  "title": "Order already confirmed",
+  "status": 409,
+  "detail": "Order 01J8ZP4K9M7QX2 was confirmed at 2026-04-01T09:12:33Z and cannot be confirmed again.",
+  "instance": "/v1/orders/01J8ZP4K9M7QX2/confirm",
+  "code": "ORDER_ALREADY_CONFIRMED",
+  "retryable": false,
+  "trace_id": "01J8ZQ0RT4V9C1"
+}
+```
+
+**Why `code` and never `title` or `detail`.** RFC 9457 problem details are content-negotiable, so `title` and `detail` are localizable by design — the day someone adds `Accept-Language` support, or a writer fixes a typo, every client matching on the string stops matching. `detail` is worse still: it interpolates per-occurrence data (ids, counts, timestamps), so a substring match on it is a match against data. The symptom is silent and delayed: a client's special-case branch simply never fires again, the fallback path swallows it, and nobody learns until the behaviour it guarded is missed in an audit. `code` is an enum in one file, changed only by adding.
+
+## 4. Pagination
+
+```
+default page size          25   # renders on a phone without the client paginating your pagination
+max page size             100   # identical to the max array length in validation-and-errors.md.
+                                # One number: a 100-item response costs what a 100-item request costs.
+extra row fetched           +1  # LIMIT n+1; the existence of row n+1 IS has_more. Never COUNT(*).
+cursor expiry             none  # a keyset cursor is a position, not a session. It stays valid while
+                                # the sort key exists. Do not attach a clock to it.
+```
+
+| Situation | Answer |
+|---|---|
+| Any list that can grow without bound (orders, events, messages) | **Cursor / keyset.** Non-negotiable, per the hub. |
+| A list with a hard ceiling in the schema (a user's ≤ 20 API keys, 12 months of statements) | **No pagination at all.** Return the array. A cursor on a list that cannot fill one page is ceremony. |
+| An admin table where a human clicks "page 7" | Offset, capped at a page number you enforce (`OFFSET` ≤ 1000), and only over a filtered subset |
+| The client needs a stable snapshot across pages (an export) | Neither — a job with a `202`, §8. Pagination over a moving table is not a snapshot. |
+| Sorting by a column the user picks | Keyset, with the sort spec inside the cursor (below). A cursor for one sort is meaningless under another. |
+
+Offset, and what the database actually does:
+
+```sql
+-- page 401 of 25. Postgres produces 10,025 rows and DISCARDS 10,000 of them.
+SELECT id, created_at FROM orders
+ WHERE tenant_id = $1
+ ORDER BY created_at DESC
+ LIMIT 25 OFFSET 10000;
+-- EXPLAIN ANALYZE shows "Limit (actual rows=25)" over a node with actual rows=10025.
+-- Cost is linear in the page number: page 1 is instant, page 400 is 400x the work.
+```
+
+Keyset, which is constant-time in the page number:
+
+```sql
+SELECT id, created_at FROM orders
+ WHERE tenant_id = $1
+   AND (created_at, id) < ($2, $3)     -- ROW-VALUE comparison, one predicate
+ ORDER BY created_at DESC, id DESC
+ LIMIT 26;                             -- 25 + the has_more probe
+
+CREATE INDEX ON orders (tenant_id, created_at DESC, id DESC);  -- column order: data-modelling.md
+```
+
+**`(created_at, id) < ($2, $3)` is not a style preference.** The hand-expanded version people write instead — `created_at <= $2 AND id < $3` — is wrong: it drops every row with an *earlier* `created_at` whose `id` happens to be larger. The symptom appears only when two rows share a timestamp, so it passes every test with hand-made fixtures and loses rows in production during bulk imports. Postgres and MySQL 8 both evaluate row-value comparisons against a composite index.
+
+**Why an insert during paging shifts an offset page.** `OFFSET` counts rows, and the row set is not frozen between requests. One insert at the head pushes the last row of page N onto page N+1 — the client sees it twice. One delete pulls a row up — the client never sees it. The symptom is the quiet one: an importer that processes 24 of every 25 rows, reports success, and reconciles wrong a quarter later. Keyset compares *values*, so an insert lands where it belongs and a delete removes nothing the client had not already read.
+
+Cursor construction — sort key, unique tiebreak, and the sort spec, base64url so it survives a query string unescaped (RFC 4648 §5):
+
+```
+payload = {"v": 1, "s": "created_at:desc", "k": ["2026-04-01T09:12:33.481Z", "01J8ZP4K9M7QX2"]}
+cursor  = base64url(json(payload))            # optionally: base64url(payload + "." + hmac_sha256)
+```
+
+- **`k` must end in a unique column.** Without the primary-key tiebreak, rows sharing a `created_at` are ordered non-deterministically and the page boundary skips or repeats them.
+- **`v` exists so you can change the encoding later** without every outstanding cursor becoming undefined behaviour.
+- **`s` exists so a changed `sort` parameter is detectable** instead of producing a silently wrong page.
+- **A cursor that is base64 of an offset is an offset wearing a costume.** It has every offset failure and the client can no longer see it happening. Named and forbidden.
+- HMAC it only if the sort key is itself sensitive; opacity is a documentation tool, not a security boundary ([security.md](security.md)).
+
+| Cursor state | Answer |
+|---|---|
+| Undecodable, bad signature, unknown `v` | `400`, `code: "INVALID_CURSOR"`. **Never fall back to page 1** — a client looping "fetch until no cursor" then loops forever over page 1. |
+| Decodes, but `s` ≠ the requested sort | `400`, `code: "CURSOR_SORT_MISMATCH"` |
+| Decodes, and the row it points at was deleted | **Keep going.** Keyset compares values, not row identity; the anchor's values still order correctly. This is a property offset does not have. |
+| Decodes, points past the end | `200`, empty `data`, `next_cursor: null`. Not `404`. |
+
+Response shape — the page metadata is in the body, versioned with everything else:
+
+```json
+{ "data": [ /* … */ ], "page": { "next_cursor": "eyJ2IjoxLCJz…", "has_more": true } }
+```
+
+`Link: <…>; rel="next"` (RFC 8288) is a legitimate addition when you already publish a hypermedia API. **`X-Total-Count` is not.** Beyond not being a standard field, it forces a second `COUNT(*)` over the same predicate on every page — usually the slowest thing in the request, computed to render a number nobody scrolls to. If a total is genuinely required, make it opt-in (`?include=total`) and bound it: `SELECT count(*) FROM (SELECT 1 FROM orders WHERE … LIMIT 1000) t`, rendered as "1000+".
+
+## 5. Versioning and Deprecation
+
+| Situation | Answer |
+|---|---|
+| Public API, unknown clients who will never read your changelog | **URL path**: `/v1/orders`. It shows up in a log line, a curl paste, and a bug report. |
+| One internal consumer that deploys with you | **No version.** Change both sides in one deploy — the hub's internal-vs-public rule. Adding `/v1/` to a service with one caller is a week spent on a promise nobody asked for. |
+| A large surface evolving field-by-field, Stripe-shaped | A **date-pinned header** (`Stripe-Version: 2024-06-20`, `X-GitHub-Api-Version: 2022-11-28`) with a per-account default. This buys per-client migration and costs you a compatibility layer forever. Choose it deliberately. |
+| Someone proposes `Accept: application/vnd.example.v2+json` | Don't. Caches and CDNs need `Vary: Accept` to avoid cross-serving versions, browsers cannot set it, and it is invisible in every log you will read during an incident. |
+
+**Never run two mechanisms at once.** A request carrying both `/v1/` and `X-Api-Version: 2` has no defined answer, and each proxy, SDK, and gateway in the path will pick a different one. Pick one; make the other a `400`.
+
+| Change | Verdict |
+|---|---|
+| Add an optional response field | Additive — *only* if "ignore unknown fields" is written in your docs. Otherwise a strict client parser rejects the whole body. |
+| Add an optional request field with a default | Additive |
+| Accept a **new enum value in a request** | Additive |
+| Return a **new enum value in a response** | **Breaking.** A client with an exhaustive `switch` or a `Literal[…]` parser throws on it. Ship new response enum values in a new version, or document an `"other"` sentinel from day one and reserve the right. |
+| Remove or rename a field | Breaking |
+| Tighten a validation rule (`max 500` → `max 100`) | Breaking |
+| Loosen a validation rule | Additive |
+| Make an optional request field required | Breaking |
+| Change a default value | **Breaking, and the worst kind** — nothing errors anywhere; behaviour silently changes for every client that omitted the field. |
+| Change the status code returned for an existing condition | Breaking |
+| Add a new error `code` | Breaking unless clients have a documented default branch for unknown codes. Publish that rule before you need it. |
+| Change default sort or page size | Breaking — it reorders and re-slices every client's paging |
+
+"No one uses it" is the hub's named claim about logs you have not read. Read them: group request counts by API key and endpoint over a window at least as long as your slowest client's release cycle, and count *distinct keys*, not requests — one integrator making four calls a month is still a broken integration.
+
+Deprecation on the wire. Two RFCs, two date formats in the same response — that is correct, not a typo:
+
+```http
+Deprecation: @1767225600
+    # RFC 9745. A Structured Fields Date item: "@" + seconds since the Unix epoch (2026-01-01T00:00:00Z).
+Sunset: Wed, 01 Jul 2026 00:00:00 GMT
+    # RFC 8594. An IMF-fixdate (RFC 9110 §5.6.7). Different format, different RFC.
+Link: <https://api.example.com/docs/deprecations/orders-v1>; rel="deprecation"; type="text/html"
+```
+
+```
+T+0        Deprecation + Sunset + Link on EVERY response of the old surface. Changelog, docs banner.
+T+0..30d   Direct notice to every API key that called it in the last 30 days.
+T+90d      Brownout #1 — 410 for 5 minutes at a published UTC time. This is the only announcement
+           that reaches a team that does not read email.
+T+120d     Brownout #2 — 410 for 1 hour.
+T+180d     Sunset. 410 Gone permanently, with the replacement URL in the problem detail.
+```
+
+180 days is the floor for a public API. Never shorter than one full release cycle of your slowest known integrator. An internal surface with a known caller list can run this in two weeks — the headers are the same, only the clock changes.
+
+## 6. Conditional Requests
+
+`ETag` is a validator for the *resource's state*, and it does two jobs from one value.
+
+```http
+ETag: "42"          # strong validator: byte-identical representation
+ETag: W/"42"        # weak: semantically equivalent, may differ in bytes (field order, whitespace)
+```
+
+**Derive the `ETag` from the row's version column, never from a hash of the serialized body.** [concurrency-and-jobs.md](concurrency-and-jobs.md) §2 already keeps a monotonic `version` on rows a human edits; `ETag: W/"<version>"` is that value on the wire, and it costs nothing to compute. A body hash instead has a specific, ugly symptom: the first deploy that reorders JSON keys or adds one optional field invalidates every cached validator simultaneously, and every client that would have received `304` gets a full response in the same minute. That is a self-inflicted thundering herd, triggered by a cosmetic change.
+
+Read path:
+
+```http
+GET /v1/orders/01J8Z HTTP/1.1
+If-None-Match: W/"42"
+
+HTTP/1.1 304 Not Modified
+ETag: W/"42"
+Cache-Control: private, max-age=0, must-revalidate
+```
+
+A `304` carries **no body**, and must repeat the `ETag` and the caching headers the `200` would have carried (RFC 9110 §15.4.5). A `304` with a body is a bug that some intermediaries will paper over and one client will not.
+
+Write path — this is optimistic concurrency, free, on any REST resource:
+
+```http
+PATCH /v1/orders/01J8Z HTTP/1.1
+If-Match: W/"42"
+```
+
+| Server state | Answer |
+|---|---|
+| Current version matches `42` | Apply, return `200`/`204` with the **new** `ETag` |
+| Current version is `43` | `412`, `code: "PRECONDITION_FAILED"`, current `ETag` in the problem detail so a UI can offer a merge. **No partial write.** |
+| Resource is gone | `404` (or `410` with a tombstone) — distinguish it, per [concurrency-and-jobs.md](concurrency-and-jobs.md) §2 on ambiguous `rowcount = 0` |
+| No `If-Match` sent, and blind overwrite is never acceptable here | `428 Precondition Required` (RFC 6585 §3) |
+| No `If-Match` sent, conflict detected another way (unique violation, state machine) | `409` — a state conflict, not a failed precondition |
+
+That last pair is the whole `412`/`409` rule: **`412` means the client stated a precondition and it was false; `409` means the client stated nothing and the state said no.** Both siblings agree with this — [validation-and-errors.md](validation-and-errors.md) classes `If-Match` mismatch as `412`, [concurrency-and-jobs.md](concurrency-and-jobs.md) classes a bare version conflict as `409`.
+
+Without any of this, the failure is not an error at all: two editors save, the second silently overwrites the first, both see success, and the lost edit is found a week later with nothing in the logs.
+
+**Never use `Last-Modified` / `If-Unmodified-Since` alone for concurrency control.** HTTP-date has one-second resolution (RFC 9110 §5.6.7), so two writes in the same second are indistinguishable — the identical trap as using `updated_at` as a version. It is a cache validator; it is not a lock.
+
+**Permission to ship none of this:** most endpoints need no `ETag`. Add conditional requests when the resource is human-edited and concurrently editable, or when a client polls something large and rarely-changing. Adding them to a write-once event record is decoration with a maintenance cost.
+
+## 7. Bulk and Partial Success
+
+The atomicity of a bulk endpoint is part of its contract and belongs in its documentation before its first line of code. The decision table (all-or-nothing vs per-item vs a job) and the per-item error shape are in [validation-and-errors.md](validation-and-errors.md); on the wire, three things are non-negotiable:
+
+- **`207 Multi-Status` (RFC 4918 §13) or `200` with a documented `results` array — and the choice is stated, per endpoint, in the docs.** A client that cannot tell from the contract whether `200` means "all applied" must parse the body to find out, which is precisely what a status line exists to prevent.
+- **Every result echoes the client's own item key**, never a positional index — an index breaks the moment a client reorders or retries a subset.
+- **Each item is independently idempotent**, so the client can resubmit *only* the failures ([concurrency-and-jobs.md](concurrency-and-jobs.md)).
+
+The failure mode that makes this worth a section: a `200` whose body reports 40 of 500 items failed is a success to every dashboard you own. The rows are missing; the graph is green.
+
+## 8. Async and Idempotency — the contract half
+
+```http
+POST /v1/exports HTTP/1.1
+Idempotency-Key: 7c1f9a2e-…        # client-generated, opaque, unique per logical operation
+
+HTTP/1.1 202 Accepted
+Location: /v1/exports/01J8ZQ        # a real, GETtable status resource — not a hint
+```
+
+- **Require `Idempotency-Key` on every endpoint that moves money, sends a message, or calls a third party**, and reject the request `400` when it is missing. Optional idempotency is idempotency nobody uses.
+- The client sees exactly three outcomes: the original response replayed verbatim with `Idempotency-Replayed: true`; `409` while the first attempt is still in flight; `422` when the same key arrives with a different body fingerprint. Storage, fingerprinting, and TTL are [concurrency-and-jobs.md](concurrency-and-jobs.md) §1 — keep these three identical to what it states.
+- **A `202` without a `Location` is a dropped request.** The status resource returns `200` with a state field while running, and `303 See Other` to the finished result — so a client polls one URL and never guesses another.
+
+## Never Ship
+
+| Never | Instead |
+|---|---|
+| `200` with an error body | The code from §2 — monitoring reads the status line, not your JSON |
+| A `GET` that mutates | `POST`; prefetchers and mail scanners will replay the `GET` |
+| `403` where existence is private | `404`, byte-identical to a real one |
+| `400` for a semantically invalid but well-formed body | `422` |
+| `429` or `503` without `Retry-After` | `Retry-After` (RFC 9110 §10.2.3) |
+| `201` without `Location`, `202` without a status resource | The URL, in `Location` |
+| A `304` or `204` with a body | No body |
+| `X-Total-Count` on every list response | `page` object in the body; totals opt-in and bounded |
+| `OFFSET` on an unbounded list, or an uncapped `limit` | Keyset `(sort_key, id)` row comparison; default 25, max 100 |
+| A cursor that is base64 of an offset | Sort key + unique tiebreak + sort spec + `v` |
+| A bad cursor that silently restarts at page 1 | `400 INVALID_CURSOR` |
+| `/v1/` **and** a version header | One mechanism; the other is a `400` |
+| Removing a field because "no one uses it" | Distinct-API-key counts, then the T+0…T+180d timeline |
+| A new enum value in a response, in an existing version | New version, or an `"other"` sentinel documented from day one |
+| `ETag` hashed from the serialized body | `W/"<version column>"` |
+| `Last-Modified` as a concurrency check | `If-Match` on a version-derived `ETag` |
+| Clients branching on `title` or `detail` | A stable `code` enum in one file |
+| Bulk `200` hiding per-item failures | `207` / documented `results`, or all-or-nothing |
+| Optional `Idempotency-Key` on a payment endpoint | Required; `400` without it |
+
+## Output
+
+Write the routes and the schemas. Then, briefly: the **contract** (method, request shape, success status + body, and the exact failure statuses this endpoint can return with their `code`s); the **pagination decision** if it returns a list, including why cursor or why none; the **version/compatibility verdict** for anything you changed on an existing surface — additive, or breaking with the deprecation dates; and what you **verified by calling it**, per the hub — the curl and the status code it printed, including one failure path, not only the happy one. If you declined to version an endpoint, declined to paginate a bounded list, or declined to add an `ETag`, say which and why in one line. The code is the deliverable; do not expand this into a report.

package/dist/resources/skills/builtin/backend-engineering/references/auth-and-secrets.md

@@ -0,0 +1,268 @@
+# Auth and Secrets
+
+This file owns three questions: **who the caller is** (authentication, sessions, tokens), **what they may do** (authorization models, multi-tenancy, IDOR), and **where the credentials live** (password hashing, secret storage, key rotation). It is the canonical source for every cookie attribute, token lifetime, and KDF parameter in this skill — when another file disagrees with this one, this one wins.
+
+Neighbours, so you don't half-solve a problem here: injection, SSRF, CORS, CSRF, and rate limiting are [security.md](security.md). Loading and validating config at boot is [deploy-and-config.md](deploy-and-config.md). Where the `401` / `403` / `404` boundary sits in a response body is [validation-and-errors.md](validation-and-errors.md). Adding a `tenant_id` column to a live table is [migrations.md](migrations.md).
+
+## Operating Posture
+
+Two failure modes, and the first is worse:
+
+1. **Authorization that isn't enforced where the data is.** A missing check does not throw. It returns `200` with someone else's row, passes every test that only asserts the happy path, and is discovered by a customer. This is the entire reason the hub's Boundary Checklist puts authorization at gate 1.
+2. **Hand-rolled cryptography.** A homemade token format, a homemade KDF, a homemade comparison. These fail silently and completely — the app works, logins succeed, and the guarantee you thought you had was never there.
+
+**Never invent a security parameter.** Not a cost factor, not a token lifetime, not a key length, not "a reasonable expiry". Every number below is from a published standard or a real default and is checkable. Guessing at a bcrypt cost is the same class of error as inventing a `cubic-bezier` because it looked familiar — it produces a plausible-looking value with no property behind it.
+
+**Producing nothing is often correct here.** If the app has one tenant and four staff users, do not build RBAC — ship `users.is_admin boolean not null default false` and a single check. If the framework has session middleware, do not write a token layer on top of it. The best auth diff of the week is frequently the one that deletes a bespoke login in favour of the framework's. Say so plainly and stop.
+
+## Sessions vs JWT
+
+For a first-party web app — your frontend, your backend, your domain — **a server-side session referenced by an opaque cookie is almost always the right answer.** Reach for a JWT only when you can name the service that must verify a token without calling you.
+
+| | Server-side session | JWT (self-contained) |
+| --- | --- | --- |
+| What the client holds | Opaque random id, meaningless if stolen off the wire | Signed claims, readable by anyone who has it |
+| Verification | One lookup (Redis/Postgres, sub-millisecond, already on the request path) | Signature check, zero I/O, works in a service that cannot reach your DB |
+| **Revocation** | Delete the row. Effective on the next request. | **Not possible before expiry** without the denylist that removes the only reason you chose JWT |
+| Changing a role/permission | Takes effect immediately | Stale until the token expires |
+| Size on every request | ~40 bytes | 400–1200 bytes, on every request, forever |
+| Failure mode when it goes wrong | Session lost, user logs in again | `alg: none` accepted, or RS256 verified with the public key as an HMAC secret — full forgery |
+
+What a JWT actually buys: **stateless verification across a trust boundary.** Three services behind one gateway, a mobile client hitting several backends, a partner validating your token offline. That is the whole list.
+
+What it costs: **revocation.** "Log out everyone", "this employee left ten minutes ago", "rotate after a breach" all become "wait 15 minutes" — or you build a revocation store, at which point you have a session with extra signature math and a bigger cookie.
+
+**A JWT in `localStorage` is a decision to be vulnerable to XSS.** Any script on the page can read it, exfiltrate it, and use it until expiry. A JWT stored in an `HttpOnly` cookie is fine; a JWT stored in JS is a downgrade from the session you rejected. See [security.md](security.md) for the CSRF obligation that comes with any cookie-borne credential.
+
+## Cookie Attributes — Exact
+
+```
+Set-Cookie: __Host-sid=<256 bits of CSPRNG, base64url>; HttpOnly; Secure; SameSite=Lax; Path=/; Max-Age=1209600
+```
+
+| Attribute | What it blocks | When to deviate |
+| --- | --- | --- |
+| `HttpOnly` | `document.cookie` — turns an XSS into a session-theft failure instead of a session-theft success | Never. There is no valid reason for JS to read a session cookie. |
+| `Secure` | Transmission over plaintext HTTP, so a network attacker can't strip TLS and harvest it | Never in production. `http://localhost` is exempt in every modern browser, so local dev needs no deviation. |
+| `SameSite=Lax` | The cookie riding along on cross-site POSTs, iframes, and subresource loads — CSRF's delivery mechanism. Still sent on a top-level GET navigation, so inbound links keep the user logged in. | `Strict` for admin/banking surfaces where arriving from an external link logged-out is acceptable. `None` **only** for a cookie a third-party context genuinely needs, and `None` without `Secure` is rejected by the browser outright. |
+| `Path=/` | Nothing, on its own. It is a scoping declaration, and `__Host-` requires it. | Never narrow it to fake isolation — path is not a security boundary. |
+| `__Host-` prefix | A subdomain (or an XSS on one) writing a cookie your apex trusts. The browser enforces: `Secure`, `Path=/`, and **no `Domain`** — so the cookie is host-locked. | Drop to `__Secure-` only when you genuinely need `Domain=.example.com` for cross-subdomain sessions. |
+| `Max-Age` | Nothing. It's UX. | Server-side expiry is the real one; a client can lie about the cookie's age but not about the row you deleted. |
+
+**Session id rules:** at least 128 bits of entropy (OWASP Session Management); use 256 to stop thinking about it. Generate with the CSPRNG (`crypto.randomBytes`, `secrets.token_urlsafe`, `crypto/rand`) — never `Math.random`, `uuid4` as a secret, or a counter. **Store a SHA-256 hash of the id**, not the id: a leaked DB dump then contains no usable sessions, and the lookup is still one indexed read.
+
+**Regenerate the session id on every privilege change** — login, logout, step-up, role change. Skipping this is session fixation: the attacker plants a known id before login and inherits the authenticated session afterwards.
+
+## Access / Refresh Split
+
+Only when you have decided you need tokens (mobile client, third-party API, cross-service). A first-party web app skips this section entirely.
+
+```
+access token   TTL 15 minutes    JWT or opaque, sent as: Authorization: Bearer <token>
+refresh token  TTL 30 days       opaque random, rotates on every use, one row per token
+```
+
+Why 15 minutes: it is the window in which a stolen access token is useful, and revocation lag is bounded by it. Shorter multiplies refresh traffic for no security gain; longer makes "revoked" a lie for hours.
+
+**Rotation with reuse detection** — the mechanism from RFC 9700 (OAuth 2.0 Security BCP), and the only thing that makes long-lived refresh tokens defensible:
+
+1. Every refresh token belongs to a **family** (one family per login) and stores `family_id`, `used_at`, `replaced_by`.
+2. Redeeming a refresh token issues a new access token *and a new refresh token*, and marks the old one used.
+3. If a token that is **already used** is presented again, that is theft — either the attacker or the legitimate client is replaying. You cannot tell which, and you don't need to: **revoke the entire family immediately.** Both parties get logged out; the legitimate one logs in again.
+
+Without step 3, rotation is decoration: a stolen refresh token rotates happily alongside the real client for thirty days.
+
+**How revocation actually works, per credential:**
+
+| Credential | Revocation | Lag |
+| --- | --- | --- |
+| Server-side session | `DELETE FROM sessions WHERE id = $1` | Next request |
+| Refresh token | Delete the family row / mark revoked | Next refresh |
+| Access token (JWT) | **Nothing revokes it.** It is valid until `exp`. | Up to its full TTL |
+| Access token (opaque, looked up) | Delete the row | Next request — but you have re-invented the session |
+
+Store a `sessions_valid_after` timestamp on the user and reject any token issued before it. That is how "log out all devices" and "force logout after password change" work, and it costs one column.
+
+**Never** `jwt.decode(token, verify=False)` outside a debugger, never accept `alg` from the token header, never a symmetric secret shorter than 32 bytes, and never skip `aud` / `iss` validation — a token minted for a different service of yours is still a valid signature.
+
+## Password Storage
+
+Use the framework's or the language's maintained binding to a real KDF. **Never write your own** — not a salted SHA-256, not a stretched HMAC loop, not "SHA-512 a thousand times". A KDF's security property is memory-hardness against GPU and ASIC attack, and you cannot arrive at that by iterating a fast hash. `md5`, `sha1`, `sha256(password + salt)`, and any construction you can describe in one sentence are all the same answer: no.
+
+```
+Argon2id  — first choice (RFC 9106 / OWASP Password Storage)
+  m = 19456 KiB (19 MiB), t = 2, p = 1     # OWASP minimum
+  m = 65536 KiB (64 MiB), t = 3, p = 4     # RFC 9106 second recommended option; use if the box has RAM
+
+bcrypt    — acceptable fallback where Argon2 is unavailable
+  cost = 12                                 # ~250ms on 2024 server hardware
+  WARNING: bcrypt silently truncates at 72 bytes, and some implementations
+  truncate at a NUL byte. Pre-hash with SHA-256 -> base64 before bcrypt if
+  you accept long passphrases, or you have capped every password at 72 chars
+  without telling anyone.
+
+scrypt    — N = 2^17, r = 8, p = 1
+PBKDF2    — last resort, FIPS only: HMAC-SHA-256, 600,000 iterations
+```
+
+Tune by measurement, not by taste: pick the highest parameters that keep hashing under ~500ms on your production hardware, then write the number in config, not in a comment. The salt is generated by the library and stored in the output string — never one shared application-wide salt, and never a salt you invented.
+
+**Constant-time comparison, everywhere a secret is compared.** `==` on bytes returns early at the first differing byte, and that timing difference is enough to recover an API key or a reset token byte by byte over enough requests. This applies to session ids, API keys, HMAC signatures, webhook signatures, and password reset tokens — not to password hashes, which the KDF's own `verify` already handles.
+
+```js
+crypto.timingSafeEqual(a, b)          // Node — throws if lengths differ; compare hashes of equal length
+```
+```python
+hmac.compare_digest(a, b)             # Python
+```
+```go
+subtle.ConstantTimeCompare(a, b) == 1 // Go
+```
+
+**Symptom, not rule:** the most common password bug is not a weak hash — it is a login endpoint that returns in 8ms for an unknown email and 250ms for a known one, handing out a user enumeration oracle. Run the KDF (against a dummy hash) even when the user does not exist, and return one identical `401` for both cases. The same applies to password reset and signup: "if that address exists, we sent a link."
+
+Other non-negotiables: no maximum length under 64 characters and no composition rules (NIST SP 800-63B), check against a breached-password list, rate-limit by account *and* by IP ([security.md](security.md)), and store reset tokens hashed with a **15-minute, single-use** expiry.
+
+## OAuth 2.0 / OIDC
+
+**Authorization code flow with PKCE. That is the only flow.** RFC 9700 makes PKCE mandatory for every client type — public *and* confidential — and forbids the implicit grant (`response_type=token`) and the password grant outright. If a tutorial hands you a token in a URL fragment, it predates the security BCP.
+
+```
+GET /authorize
+  ?response_type=code
+  &client_id=...
+  &redirect_uri=https://app.example.com/auth/callback   # exact string match, registered
+  &scope=openid+email
+  &state=<128-bit random, bound to the user's session>
+  &code_challenge=<BASE64URL(SHA256(verifier))>
+  &code_challenge_method=S256                            # never "plain"
+  &nonce=<random>                                        # OIDC: binds the ID token to this request
+```
+
+| Parameter | What it prevents | The failure if you skip it |
+| --- | --- | --- |
+| `state` | Login CSRF | An attacker completes the flow with *their* authorization code in *your* user's browser, silently linking the victim's account to the attacker's identity. Generate it server-side, store it in the session, compare on callback, delete after one use. `state` is not a place to stash a return URL unsigned. |
+| `code_challenge` (PKCE, RFC 7636) | Authorization code interception | A stolen code (redirect log, referrer, a malicious app claiming the same URI scheme) is worthless without the verifier. |
+| `nonce` | ID token replay | A previously issued ID token replayed into a new session. |
+| Exact redirect URI match | Open redirect → code theft | The registered URI must match **byte for byte**: no wildcards, no prefix matching, no "any path under this host", no dynamic query params. Prefix matching plus an open redirect on your own domain is a complete account takeover, and it is the single most exploited OAuth misconfiguration. |
+
+On the callback: exchange the code server-side over TLS, validate the ID token's `iss`, `aud`, `exp`, and `nonce` against JWKS (cached, with rotation), **then create your own session** and set your own cookie. The provider's token is not your session. And never trust `email_verified: false` as an identity — that's how account pre-registration takeover works.
+
+## Authorization Models
+
+Pick the smallest model that expresses the rules. Escalating early costs more than escalating late.
+
+| Model | Fits when | Shape | Escalate when |
+| --- | --- | --- | --- |
+| **Boolean / single flag** | One tenant, a handful of staff | `users.is_admin` | You write the third `if is_admin || is_editor`. |
+| **RBAC** | Permissions follow job titles; a user's rights don't depend on *which* object | `role → permissions`, checked as `has("invoice:write")`. Check permissions, never role names — `if role == "admin"` scatters policy across the codebase. | Someone says "editor **on that project**". |
+| **ReBAC** (Zanzibar-style) | Rights come from relationships and inherit through a graph: doc → folder → org | Tuples: `(user:42, editor, doc:9)`; ask "does this subject have this relation to this object". SpiceDB / OpenFGA / Postgres recursive CTE. | Never, really — this is the ceiling for app-level authz. |
+| **Row-level (RLS)** | Isolation is a property of the *row*, not the user's title — tenancy above all | The predicate lives in the database, enforced by the engine | It's a complement to the others, not an alternative. |
+
+### The rule: check at the data layer, not the route
+
+A route guard is a claim about a code path. A predicate in the query is a property of the data. Guards are forgotten on the new endpoint, the background job, the CSV export, the admin tool, and the GraphQL resolver someone added last Tuesday. **A `WHERE` clause is forgotten by nobody, because forgetting it returns no rows.**
+
+```sql
+-- IDOR. The route confirmed SOMEONE is logged in. The query never asked WHO.
+-- GET /invoices/8fa1... returns another tenant's invoice with a 200.
+SELECT id, total_cents, currency, issued_at
+  FROM invoices
+ WHERE id = $1;
+
+-- Correct. Ownership is part of the lookup, so "forgot the check" and
+-- "no such row" collapse into the same code path — and the same 404.
+SELECT id, total_cents, currency, issued_at
+  FROM invoices
+ WHERE id = $1
+   AND tenant_id = $2;   -- $2 comes from the session, NEVER from the request
+```
+
+Enforce it structurally: the repository/query layer takes the actor as a required argument, so a query without a scope does not compile or does not exist. A helper named `invoicesFor(tenantId)` is worth more than a hundred lines of policy documentation.
+
+**IDOR concretely.** The bug is "the identifier in the URL is the only thing deciding which row you get". Three rules:
+
+- **Never take the tenant, user, org, or role from the request.** Not from a body field, not from a query param, not from an `X-Tenant-Id` header, not from a JWT claim the client could have obtained legitimately for a different context. It comes from the server-side session.
+- **Return `404`, not `403`, for a resource the caller may not know exists.** The hub's status table permits exactly this: `404` is "absent, *or* present-but-you-may-not-know". A `403` on a sequential id is a working existence oracle. Use `403` only when the caller is entitled to know the object exists and merely lacks this action on it.
+- **Unguessable ids are not authorization.** A UUIDv7 in the URL still leaks through referrers, browser history, shared links, and logs. It raises the cost of enumeration and does nothing else.
+
+## Multi-Tenancy
+
+**Tenant scoping is a default predicate, not a per-query decision.** A forgotten `WHERE tenant_id = ...` is the single most expensive bug class in a SaaS backend: it does not error, it does not appear in tests seeded with one tenant, it degrades gracefully into a `200`, and by the time it surfaces you owe every affected customer a breach notification. One missing clause on one endpoint is a data breach with a changelog entry attached.
+
+Postgres RLS makes it the engine's problem instead of the author's:
+
+```sql
+ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
+ALTER TABLE invoices FORCE  ROW LEVEL SECURITY;  -- without FORCE, the table OWNER bypasses every policy
+
+CREATE POLICY tenant_isolation ON invoices
+  USING (tenant_id = current_setting('app.tenant_id', true)::uuid);
+```
+
+Two failures that make RLS look like it works when it doesn't:
+
+- **The app connects as the table owner.** Owners (and any `BYPASSRLS` role) ignore policies entirely, so every test passes and production is wide open. Connect as a distinct, non-owning, non-superuser role and prove isolation with a test that asserts tenant B's row is invisible to tenant A's connection.
+- **`SET` instead of `SET LOCAL`.** Under a transaction-pooling proxy (PgBouncer in `transaction` mode) a session-level `SET app.tenant_id` outlives your request and is inherited by the next request on that connection — which may belong to a different customer. Always `SET LOCAL` inside the transaction that runs the query.
+
+RLS is a backstop, not a replacement: keep the explicit predicate in the query too. Belt and engine.
+
+## Secrets
+
+**Environment only, validated at boot, never anywhere else.** Restating the hub's gate 3 in operational terms:
+
+| Rule | The concrete form |
+| --- | --- |
+| Never in the repo | `.env` is gitignored; `.env.example` holds *names and shapes only*, never a real value — not even a "dev" one, because dev keys reach real providers. |
+| Validated at boot | Parse the whole environment into a typed config object at startup and **exit non-zero** on anything missing or malformed. Details in [deploy-and-config.md](deploy-and-config.md). |
+| No fallback defaults for secrets | `process.env.SECRET_KEY \|\| "dev-secret"` is how a signing key of `"dev-secret"` reaches production. A missing secret must crash, loudly, at boot. |
+| Never logged | Never log the config object, the request headers, the OAuth token exchange, or an exception's local variables. Structured loggers serialize whole objects; redact at the logger, by key name, as a filter — see [observability.md](observability.md). |
+| Never in a response | Not in an error body, not in a debug field, not in a `/health` payload, not in a stack trace. Log the detail, return the code. |
+| Never in Memory, an issue, a ticket, or a chat | Once a secret is in a system with different access rules than your secret store, it is out of your control. |
+| Distinct per environment | Dev, staging, and prod share no key. A staging leak must never be a production incident. |
+
+Anything derived from a secret is a secret: a signed URL, a session id in a log line, a webhook signing key, a DB connection string with the password inline.
+
+## When a Key Leaks
+
+**Rotate first. Deleting the commit is not a fix.** By the time you notice, a public-repo key has typically already been exercised — scrapers watch the push firehose and use credentials within minutes, not days. A force-push does not help: the old blob stays reachable by SHA through GitHub's fork network, and the repo is already cloned, mirrored, and in someone's dataset. **Treat the secret as fully compromised from the moment it was committed, and act in this order:**
+
+1. **Issue a new credential at the provider.** New key first, so step 3 has something to deploy.
+2. **Assess blast radius while it deploys.** What can this key do, to whose data, and for how long has it been exposed (commit timestamp, not discovery timestamp)?
+3. **Deploy the new secret** to every environment and worker that used the old one — cron jobs and queue consumers get forgotten and then page you at 3am.
+4. **Revoke the old credential at the provider, and verify.** Call the API with the dead key and confirm you get `401`. "Revoked in the dashboard" is a claim; a `401` is evidence.
+5. **Audit provider logs across the whole exposure window** for calls you cannot attribute — unfamiliar IPs, unusual regions, spikes, resources you did not create. This is where an incident either ends or begins.
+6. **Then clean the repo** — `git filter-repo` / BFG, force-push, invalidate caches, ask the host to purge — and add secret scanning plus a pre-commit hook so the next one is caught before the push. Understand that this step protects nothing that already happened; it stops the leak from being re-discovered later.
+7. **Write it down**: which secret, committed when, discovered how, exposed for how long, rotated when, what the logs showed. Two paragraphs. Skipping this is how the same hook is missing the next quarter too.
+
+If the leaked credential is a signing key (JWT secret, session key, webhook secret), rotation invalidates every outstanding token or session it signed. That is the correct outcome — support a brief dual-verify window (accept old and new, sign with new only) if a mass logout is unacceptable, and close it within hours, not weeks.
+
+## Never Ship
+
+| Never | Instead |
+| --- | --- |
+| Authorization checked only in a route decorator or middleware | A predicate in the query; the actor is a required argument to the repository |
+| Tenant/user/role read from a request body, param, or header | Read from the server-side session |
+| `403` on a resource whose existence is itself private | `404` |
+| A JWT in `localStorage` | `HttpOnly; Secure; SameSite=Lax` cookie |
+| A session cookie without `HttpOnly` or `Secure` | The full attribute set above, `__Host-` prefixed |
+| `SameSite=None` without `Secure` | Browser rejects it anyway — use `Lax` |
+| Refresh tokens that rotate without reuse detection | Token families; a replayed token revokes the family |
+| `jwt.decode(..., verify=False)`, or trusting the token's `alg` | Pin the algorithm and the key server-side; validate `iss`, `aud`, `exp` |
+| Session id from `Math.random`, a UUID, or a counter | 256 bits from the CSPRNG, stored as a SHA-256 hash |
+| Session id kept across login | Regenerate on every privilege change |
+| `sha256(password + salt)` or any hand-rolled KDF | Argon2id at the parameters above; bcrypt cost 12 as fallback |
+| An invented bcrypt cost, token TTL, or key length | The exact values in this file |
+| `==` on a token, API key, or signature | `timingSafeEqual` / `compare_digest` / `subtle.ConstantTimeCompare` |
+| A login that answers faster for unknown emails | Constant work and one identical `401` for both cases |
+| Implicit grant, password grant, or PKCE `method=plain` | Authorization code + PKCE `S256` (RFC 9700) |
+| Wildcard or prefix-matched redirect URIs | Exact registered string match |
+| OAuth callback without `state` verification | Session-bound, single-use `state` |
+| RLS with the app connected as the table owner | A non-owning role plus `FORCE ROW LEVEL SECURITY` |
+| `SET app.tenant_id` under a transaction pooler | `SET LOCAL`, inside the transaction |
+| `env.SECRET \|\| "dev-default"` | Crash at boot on a missing secret |
+| Deleting the commit as the response to a leak | Rotate, verify `401`, audit logs, *then* clean history |
+
+## Output
+
+Write the code. Then, in a few lines: **who the caller is** (the credential and where it is stored), **where the authorization check lives** (name the query or policy, not the middleware), and **what a stolen credential gets an attacker and for how long** — that last one is the honest summary of your revocation story. If you rejected part of the request — a permissions system the app doesn't need, a JWT for a first-party login, an id taken from the request body — say which and why in one line.