@hilbras/keystone 2.5.0 → 3.0.0

package/docs/security/audit.md

@@ -0,0 +1,82 @@
+# Audit and abuse events
+
+Covers SEC-036, SEC-037 and SEC-038.
+
+An event that exists in the vocabulary but is never emitted is worse than one
+that does not exist: it reads as coverage. All three events below were in that
+state and were added in v2.8.0.
+
+## `user_login_failed`
+
+Previously defined, never emitted, on either login route. A wrong password
+produced a `401` and nothing else, so credential guessing left no record beyond
+whatever the rate limiter happened to observe in aggregate.
+
+Deliberately records **no user id**. At the point of failure no credential has
+been proven, and the submitted address may correspond to no account at all —
+attributing the failure to a user id would both misattribute the attempt and
+create a cheap way to probe which addresses exist.
+
+## `rate_limit_triggered`
+
+A refused request previously produced a `429` and nothing else.
+
+The event carries **which limiter decided** — `redis`, `local` or `none`. These
+are three different operational situations, and a degraded in-process control is
+a reason to go and look at Redis. Recording a local refusal in the same shape as
+a healthy distributed one would hide exactly the thing worth knowing.
+
+## `refresh_token_replayed`
+
+Rotation consumes a refresh token, so a second presentation failed in exactly the
+same way as a token that never existed. A stolen token used twice was therefore
+indistinguishable from a typo.
+
+It is now detected by looking the token up and reading its state, and the event
+records which of the two it was:
+
+- `replayed: true` — the token exists and was already spent. This means the
+  token leaked, so the account's remaining credentials are revoked. Answering
+  only that one request would leave everything else it could mint intact.
+- `replayed: false` — the token is unknown. A guess, or a stale client. Nothing
+  is revoked, because an attacker guessing random values must not be able to log
+  a user out.
+
+## A machine principal is not a user
+
+A service account is represented in memory by a user object whose id is the
+sentinel `sa:<uuid>`, so routes expecting `request.user` keep working without a
+matching user row.
+
+That sentinel was being passed straight into `audit_log.user_id`, which is a uuid
+column. Postgres rejected the insert, the subscriber logged a failure, and the
+record was lost. Nothing failed visibly — the request succeeded, and a missing
+audit record is indistinguishable from a request that never happened.
+
+So **every request authenticated by an API key or an mTLS service account left no
+audit trail at all.** The privileged, non-human path was the one that was
+invisible, which is precisely the wrong direction for that gap to point.
+
+The sentinel is now stripped, `user_id` is left null, and the service account is
+recorded in `metadata.serviceAccountId` — the record still identifies who acted,
+without violating the column type. (SEC-046)
+
+## The audit export is opened in a spreadsheet
+
+The CSV export quotes a value containing a delimiter or a quote. It also
+neutralises a value whose **first character** is `=`, `+`, `-` or `@`, because a
+spreadsheet evaluates such a cell as a formula when the file is opened.
+
+Quoting alone is not enough, and several exported columns are attacker-supplied —
+the user agent above all. A `User-Agent` of `=cmd|'/c calc'!A1` reached the export
+intact.
+
+The apostrophe prefix is applied *before* quoting, and a value with no formula
+prefix is left alone so ordinary data is not corrupted.
+
+## Event versioning
+
+Events are stored as `<name>:v1`. A query against the bare name matches nothing
+and returns an empty result, which reads as "no such event occurred" — the same
+vacuous pass that let SEC-037 go unnoticed. Tests that assert on audit output
+must match the versioned name.

package/docs/security/configuration.md

@@ -0,0 +1,60 @@
+# Configuration, secrets and deployment defaults
+
+Covers SEC-026 through SEC-032.
+
+## The configuration endpoint redacts by allowlist
+
+The owner-only configuration endpoint previously matched a **denylist** of
+secret-looking key names. Measured against the real configuration surface,
+**12 of 24** secret-shaped keys were returned unredacted — including the signing
+keys, so the endpoint disclosed the material used to sign tokens.
+
+A denylist is the wrong shape here: it has to predict every name a secret might
+take, and a new secret is unredacted until someone remembers to add it.
+`EXPOSABLE_CONFIG_KEYS` in `src/services/configuration/profiles.ts` is an
+allowlist, so a key that is not explicitly declared exposable is not returned.
+
+## CORS fails closed
+
+An unset or empty `ALLOWED_ORIGINS` was treated as *allow all*, combined with
+credentialed requests. A misconfiguration therefore produced a wildcard CORS
+policy that browsers actually enforce, rather than a closed one.
+
+`isOriginAllowed` in `src/services/trustedProxies.ts` now fails closed on an
+empty allowlist. The policy lives with the rest of the address logic so the
+server and the tests share one implementation — an earlier version of these tests
+asserted against a copy of the rule, which meant they passed whether or not the
+server enforced it.
+
+## Cookies are Secure by default in production
+
+`COOKIE_SECURE` defaulted to `false`, so a production deployment that did not
+set it issued session cookies over plaintext. It now defaults to `true` when
+`NODE_ENV` is `production`.
+
+## The setup server is not exposed
+
+Three separate ways the first-run provisioning endpoint could be reached by
+someone who should not have reached it:
+
+- **Origin.** It was configured with `origin: true` and credentials enabled, so
+  any page in the operator's browser could call it during setup. It is now
+  restricted to the server's own address.
+- **Interface.** It bound `0.0.0.0`, making the unauthenticated endpoint
+  reachable from the network. It binds loopback by default;
+  `KEYSTONE_SETUP_HOST` overrides, and it warns when bound to all interfaces.
+- **The token.** The token granting initial owner access was printed to stdout,
+  so it reached log aggregation and any log shipper — the credential intended to
+  bootstrap trust was the one most widely distributed. Printing now requires an
+  explicit `KEYSTONE_PRINT_SETUP_TOKEN`.
+
+## Webhook secrets are encrypted at rest
+
+Webhook secrets were stored as issued, so a database read, a backup or an admin
+query returned material that lets an attacker forge delivery attempts signed as
+this installation.
+
+They are now stored as AES-256-GCM envelopes. They are **encrypted, not hashed**,
+because Keystone signs outbound deliveries with the secret and must be able to
+recover it. Legacy plaintext values are still readable and are rewritten on next
+rotation.

package/docs/security/enterprise-sso.md

@@ -0,0 +1,193 @@
+# Enterprise SSO Configuration
+
+How to configure SAML and OIDC federation, and what Keystone checks on every
+assertion or token it accepts.
+
+Related: [SECURITY.md](../SECURITY.md),
+[trust-boundaries.md](./trust-boundaries.md).
+
+---
+
+## How federation fits together
+
+```text
+User  →  Keystone /auth/oauth/:provider  →  your IdP  →  callback  →  Keystone
+User  →  Keystone /sso/saml/:connectionId →  your IdP →  ACS        →  Keystone
+```
+
+Keystone is the **relying party**. Your IdP asserts who the user is; Keystone
+decides whether it believes them. Everything below is a check Keystone applies
+on the way back.
+
+A platform owner cannot sign in through tenant SSO, and an account flagged for
+platform review cannot either.
+
+---
+
+## SAML setup
+
+### What you register
+
+| Field | Meaning | Notes |
+| --- | --- | --- |
+| `idpEntityId` | Your IdP's entity ID | **Must** match the `Issuer` in your assertions exactly |
+| `idpSsoUrl` | Single sign-on endpoint | Where Keystone sends the `SAMLRequest` |
+| `idpCertificate` | Your IdP's signing certificate, PEM | Used to verify every assertion |
+| `spEntityId` | Keystone's entity ID | Returned in metadata; must equal your assertion `Audience` |
+| `spAcsUrl` | Keystone's assertion consumer service URL | Must equal the assertion's `Destination` and `Recipient` |
+| `attributeMapping` | Which attributes carry email and name | Falls back to a standard list |
+
+### Obtaining Keystone's metadata
+
+```bash
+curl https://auth.example.com/sso/saml/:connectionId/metadata
+```
+
+Import that into your IdP as a service provider.
+
+### What Keystone verifies on every assertion
+
+| Check | Failure |
+| --- | --- |
+| Signature valid under `idpCertificate` | `SAML validation failed` |
+| Assertion **and** message both signed | `SAML validation failed` |
+| `Issuer` equals `idpEntityId` | `SAML issuer mismatch` |
+| `Audience` contains `spEntityId` | `SAML audience mismatch` |
+| `Destination` is exactly `spAcsUrl` | `SAML response destination mismatch` |
+| Every `SubjectConfirmationData/@Recipient` is `spAcsUrl` | `SAML subject recipient mismatch` |
+| `InResponseTo` matches the stored transaction's request ID | `SAML response request ID mismatch` |
+| `NameID` matches the resolved external identity | `SAML subject mismatch` |
+| `NotBefore` / `NotOnOrAfter` | rejected |
+| RelayState HMAC valid | `Invalid RelayState` |
+| Transaction cookie matches the transaction's browser nonce | `Invalid SAML transaction` |
+| Transaction not already consumed | `SAML transaction already consumed` |
+| Connection is active **and** belongs to the request's organization | `SAML connection not found` |
+
+Matching is exact. A `Destination` of `https://…/acs/extra`, a different case, or
+an added default port are all rejected — as is a `Recipient` that is a prefix or
+superstring of the registered ACS.
+
+### The unsigned Issuer
+
+The response-level `<saml:Issuer>` sits **outside** both signed regions, so
+rewriting it does not invalidate the signature. Keystone therefore compares it
+to `idpEntityId` directly, as SAML 2.0 §2.5.1.5 requires for an unsigned issuer.
+
+If you see `SAML issuer mismatch`, your IdP is emitting a different entity ID
+than the one registered on the connection.
+
+### Certificate rotation
+
+Rotation is a configuration change, not an automatic rollover. To rotate:
+
+1. Add the new certificate to your IdP so it begins signing with it, **keeping
+   the old certificate trusted** for the overlap window.
+2. Update `idpCertificate` on the connection to the new certificate.
+3. Confirm logins succeed.
+4. Remove the old certificate from your IdP.
+
+Step 2 is the point of no return: once Keystone holds only the new certificate,
+assertions signed under the old one are rejected. There is no grace period and no
+support for two certificates at once. Rotate during a quiet period, or accept a
+brief window where some users must retry.
+
+---
+
+## OIDC setup
+
+### What you register
+
+| Field | Meaning |
+| --- | --- |
+| `clientId` / `clientSecret` | Your OAuth client credentials |
+| `issuer` | Expected `iss` claim. Compared exactly. |
+| `authorizationEndpoint`, `tokenEndpoint`, `jwksUri` | Endpoints, or supply `discoveryUrl` and let Keystone resolve them |
+| `userinfoEndpoint` | Optional. Used to enrich a profile, never to establish identity. |
+| `attributeMapping` | Maps IdP claims to Keystone fields |
+
+### What Keystone verifies on every ID token
+
+| Check | Enforcement |
+| --- | --- |
+| Signature | Verified against the JWKS, fetched through the SSRF policy |
+| `iss` | Must equal the configured issuer |
+| `aud` | Must contain the configured client ID |
+| `alg` | Pinned to RS256, ES256, or PS256 — inferred from the key material is not trusted |
+| `exp`, `iat`, `iss`, `aud`, `sub` | **Required**, not merely checked when present |
+| `nonce` | Must equal the value sent in the authorization request |
+
+A token with no `exp` is rejected rather than treated as never-expiring.
+
+### The nonce
+
+Keystone generates a random nonce per authorization request, keeps it in an
+httpOnly `oauth_nonce` cookie, sends it to your IdP, and requires the returned
+ID token to echo it.
+
+Your IdP **must** echo the nonce. If it does not, federation fails with a nonce
+mismatch. Any provider implementing OpenID Connect Discovery does this correctly.
+
+A callback arriving with no `oauth_nonce` cookie is rejected outright — it did
+not come from a flow this browser started.
+
+> The `nonce` option must survive any connector that overrides `exchangeCode`.
+> `GoogleConnector` did not forward it until 2.5.0, which silently disabled nonce
+> validation for the default provider.
+
+### Endpoint policy
+
+All federation endpoints go through one policy:
+
+- HTTPS required in production. HTTP is permitted outside production so a
+  developer can use a local IdP.
+- Loopback and private ranges refused unless `ALLOW_PRIVATE_SSO_ENDPOINTS` is
+  set. This is what stops an IdP URL being pointed at `169.254.169.254` or an
+  internal service.
+- No embedded credentials.
+- Redirects are not followed, and the resolved address is pinned, so a DNS answer
+  that changes mid-request cannot redirect the fetch.
+
+`ALLOW_PRIVATE_SSO_ENDPOINTS` disables the private-address rule. It exists for
+development and for genuinely internal IdPs. Enabling it in production means an
+SSRF target inside your network becomes reachable from a configuration value.
+
+---
+
+## Organization scoping
+
+A membership is identified by **organization and user together**, never the user
+alone. One person can belong to several organizations with a different role in
+each, and no lookup made during federation can see another organization's
+membership.
+
+Membership rows carry a unique constraint on `(org_id, user_id)`, and provisioning
+inserts with `ON CONFLICT DO NOTHING`, so two simultaneous logins cannot create
+duplicate memberships.
+
+An **existing** user cannot be adopted by a new connection just because their
+email matches. They must be invited first, or linked explicitly. Otherwise any
+IdP that can assert an email address could take over an account it does not own.
+
+Connections are resolved by `(connectionId, orgId)`, so a connection belonging to
+one tenant cannot be used to complete a login initiated in another.
+
+---
+
+## Security recommendations
+
+1. **Require signed assertions.** Keystone does. If your IdP offers a choice,
+   turn on signed assertions and message signing.
+2. **Short assertion lifetimes.** Minutes, not hours. `NotOnOrAfter` is enforced.
+3. **Do not reuse assertions across users.** Each login is bound to a
+   single-use transaction.
+4. **Treat the ACS URL as fixed.** Register the exact URL Keystone gives you. No
+   wildcards, no fragments.
+5. **Keep `ALLOW_PRIVATE_SSO_ENDPOINTS` off in production** unless your IdP is
+   genuinely internal, and understand what turning it on permits.
+6. **Prefer OIDC over SAML** where you have the choice: it has fewer moving parts
+   and its validation is stricter by default.
+7. **Rotate certificates deliberately.** There is no dual-certificate support and
+   no grace period.
+8. **Watch the audit log** for `saml_sso_login`, `saml_connection_created`, and
+   `unauthorized_access`. The last one fires on a cross-organization connection
+   attempt.

package/docs/security/mtls.md

@@ -0,0 +1,132 @@
+# mTLS Service Accounts
+
+Mutual-TLS authentication for service accounts: how a caller proves which
+service account it is, and what Keystone does and does not verify.
+
+For the proxy and trust configuration this depends on, see
+[proxy-security.md](./proxy-security.md) and
+[trust-boundaries.md](./trust-boundaries.md).
+
+## The model
+
+```text
+Keystone  ←  trusted proxy  ←  client with a client certificate
+                                    │
+                                    └── proxy validates the chain, then tells
+                                        Keystone the SHA-256 fingerprint
+```
+
+Keystone receives a *fingerprint*, not a certificate. It does not validate the
+certificate, the chain, or the CA — the proxy does that, and Keystone trusts its
+verdict only from a configured trusted peer.
+
+Keystone's own contribution is binding that fingerprint to exactly one service
+account, and refusing to let anything else establish identity.
+
+## Identity sources
+
+A service account authenticates by **either**:
+
+- **A client certificate** whose fingerprint is bound to the account via
+  `service_accounts.cert_fingerprint`, presented through a trusted proxy.
+- **An authenticated credential**: an API key issued to the account, or a
+  session/bearer token belonging to it.
+
+There is no third source. In particular, no header establishes identity on its
+own. Before v2.0.0, `x-service-account-id` alone authenticated as any service
+account named in the header, with no certificate and no credential — anyone who
+could reach Keystone could become any service account. That path is removed.
+
+## Binding a certificate
+
+The fingerprint is a SHA-256 digest, in hex or the colon-separated form AWS ALB
+emits. Keystone stores it canonicalized, so the two spellings of one certificate
+cannot become two bindings.
+
+Get the fingerprint of a client certificate:
+
+```bash
+openssl x509 -in client.pem -noout -fingerprint -sha256
+# SHA256 Fingerprint=AB:CD:...:9F   (colon-separated, uppercase)
+```
+
+```bash
+openssl x509 -in client.pem -outform DER | openssl dgst -sha256 -hex
+# SHA2-256(stdin)= abcd...9f        (plain hex, lowercase)
+```
+
+Then bind it:
+
+```bash
+curl -X PUT https://keystone.example/v1/admin/organizations/$ORG/service-accounts/$ACCOUNT/certificate \
+  -H "Authorization: Bearer $TOKEN" \
+  -H 'Content-Type: application/json' \
+  -d '{"fingerprint":"ABCD...9F"}'
+```
+
+Audits as `service_account_certificate_bound`. Binding a certificate already
+held by another account returns `409` — the unique index on `cert_fingerprint`
+enforces one certificate per account.
+
+Clear the binding with `{"fingerprint": null}` (audits as
+`service_account_certificate_unbound`). The account then authenticates only with
+an API key.
+
+## Revoking
+
+```bash
+curl -X POST https://keystone.example/v1/admin/organizations/$ORG/service-accounts/$ACCOUNT/revoke \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+Sets `is_active = false` and stamps `revoked_at`. A revoked account cannot
+authenticate by certificate or API key, and disappears from reads. Revoking an
+already-revoked account returns `409` rather than a silent success, so a caller
+can tell the difference.
+
+Revocation is separate from `is_active` so that re-enabling an account does not
+quietly restore access — restoring it requires deliberately clearing
+`revoked_at`.
+
+## Responses from `requireMTLS`
+
+| Status | Code | Meaning |
+| --- | --- | --- |
+| 401 | `MTLS_UNTRUSTED_PEER` | Request did not come from a configured trusted proxy. A misconfiguration, not an attack. |
+| 401 | `MTLS_CERTIFICATE_MISSING` | Trusted proxy, but no certificate was forwarded |
+| 401 | `MTLS_FINGERPRINT_MISSING` | Certificate forwarded, but no usable fingerprint header |
+| 403 | `MTLS_CERTIFICATE_UNMAPPED` | Valid fingerprint, but no active service account is bound to it |
+
+401 means the request was not authenticated; 403 means it was identified but is
+not permitted. A malformed fingerprint is a 401 — it never reaches the database
+as a lookup key.
+
+## Using it on a route
+
+`app.requireMTLS` is available as a pre-handler on any route:
+
+```ts
+app.get("/internal/report", { preHandler: [app.requireMTLS] }, async (request) => {
+  // request.serviceAccount is the bound, active account.
+  return buildReport(request.serviceAccount.orgId);
+});
+```
+
+## What Keystone does not verify
+
+- **Certificate chain, expiry, revocation, or CA.** The proxy's responsibility.
+  If the proxy accepts an expired certificate, Keystone accepts its fingerprint.
+- **That the proxy checked anything.** Keystone trusts the peer address, not the
+  proxy's honesty. A compromised trusted host is a full bypass by design.
+- **Certificate-to-account provenance.** Keystone records a fingerprint; it
+  cannot tell you who issued the certificate.
+
+## Operational notes
+
+- Removing a certificate from the proxy's accepted list is **not** sufficient to
+  revoke an account. Revoke the account, or clear the binding.
+- Rotating a client certificate means a new fingerprint. Update the binding
+  before the old certificate expires, or the account stops authenticating at the
+  moment the proxy rolls the certificate.
+- A fingerprint can only ever map to one account. Two accounts cannot share a
+  certificate; give each caller its own.

package/docs/security/proxy-security.md

@@ -0,0 +1,128 @@
+# Proxy Security
+
+How to configure `KEYSTONE_TRUSTED_PROXIES` correctly, and what goes wrong when
+you do not. For the underlying model see [trust-boundaries.md](./trust-boundaries.md).
+
+## Configuration
+
+```bash
+KEYSTONE_TRUSTED_PROXIES="10.0.0.0/8,192.168.1.1,2001:db8::/32"
+```
+
+- Comma-separated. Each entry is an exact IP, an IPv4 CIDR, or an IPv6 prefix.
+- Whitespace around entries is ignored.
+- IPv4-mapped IPv6 peers (`::ffff:10.1.2.3` and the hex form `::ffff:0a01:0203`)
+  are normalized to `10.1.2.3` before matching, so a dual-stack listener cannot
+  slip past an IPv4 allowlist.
+- **Empty or unset means trust nothing.** This is the default and the safe
+  posture. Forwarded headers are stripped and the peer address is used.
+
+Unrecognized input fails closed: a malformed address or prefix never matches, so
+a typo disables trust rather than widening it.
+
+## How the setting is applied
+
+The value drives two things at startup:
+
+1. **Fastify's `trustProxy`.** Derived from the same list. When the list is
+   empty, `trustProxy` is `false`, so Fastify's own `request.ip` is the peer
+   address and never `x-forwarded-for`.
+2. **The header-sanitization hook.** Registered before every plugin and route.
+
+`request.ip` is *not* used to decide whether a peer is trusted, because with
+`trustProxy` enabled it is derived from the attacker-controlled header. The
+decision always uses the socket peer address.
+
+## What each header is used for
+
+| Header | Trusted-proxy requests | Direct requests |
+| --- | --- | --- |
+| `x-forwarded-for` | Client address for rate limiting and logs | Stripped; peer address used |
+| `x-real-ip` | Fallback when `x-forwarded-for` is absent | Stripped |
+| `x-client-cert-fingerprint` | Service account identity | Stripped |
+| `x-forwarded-client-cert` | Certificate subject evidence | Stripped |
+| `x-service-account-id` | Hint, cross-checked against the certificate | Stripped |
+
+Because direct requests have the headers removed, a client that reaches Keystone
+bypassing the proxy gains nothing — it is treated as an untrusted peer.
+
+## Rate limiting
+
+The limiter keys on `clientAddress()`:
+
+- **Behind a trusted proxy:** the left-most `x-forwarded-for` entry, which is
+  the real client. Distinct clients get distinct budgets.
+- **From any other peer:** the peer address. Rotating `x-forwarded-for` grants
+  no additional budget.
+
+This closes a bypass present before v2.0.0, where `trustProxy: true` combined
+with an unconditional read of `x-forwarded-for` let any client present a fresh
+address on every request and never be limited.
+
+### Fail-open behavior
+
+The limiter fails open when Redis is unavailable. That is a deliberate
+availability choice, not part of the trust model, but it means rate limiting is
+not an authentication control and must not be relied on as one.
+
+## Required proxy configuration
+
+Your proxy must meet all of these. Each one has caused a real bypass.
+
+```nginx
+# 1. Overwrite, never append: `$proxy_add_x_forwarded_for` would let a client
+#    prepend a forged address. Use only $remote_addr, or the real-client module.
+proxy_set_header X-Forwarded-For $remote_addr;
+
+# 2. Strip inbound identity headers before setting your own, so a client's copy
+#    cannot survive. `underscores_in_headers off` also rejects them outright.
+proxy_set_header X-Forwarded-Client-Cert "";
+proxy_set_header X-Client-Cert-Fingerprint "";
+proxy_set_header X-Service-Account-Id "";
+proxy_set_header X-Real-IP "";
+
+# 3. Do not expose Keystone directly. It should be unreachable except from the
+#    proxy, or the trusted range does not mean what you think.
+```
+
+For AWS ALB, the same requirements in Terraform:
+
+```hcl
+resource "aws_lb_listener" "https" {
+  protocol            = "HTTPS"
+  certificate_arn     = var.certificate_arn
+  ssl_policy          = "ELBSecurityPolicy-TLS13-1-2-2021-06"
+  # Client certificate validation for mTLS:
+  mutual_tls {
+    mode = "verify"
+  }
+}
+```
+
+ALB sets `$ssl_client_fingerprint` in colon-separated uppercase hex. Map it to
+`X-Client-Cert-Fingerprint`; Keystone canonicalizes the form.
+
+## Verifying a deployment
+
+From a host that is **not** in the trusted range:
+
+```bash
+# Must not be believed: expect the peer address in logs, not 198.51.100.9.
+curl -sS -H 'X-Forwarded-For: 198.51.100.9' https://keystone.example/health
+
+# Must be stripped: expect 401, never 200.
+curl -sS -H 'X-Service-Account-Id: <any-uuid>' https://keystone.example/some/protected/route
+```
+
+From **inside** the trusted range, forwarded addresses should be honoured and
+separate clients should have separate rate-limit budgets.
+
+## Troubleshooting
+
+| Symptom | Cause |
+| --- | --- |
+| Every client shares one rate-limit budget | Proxy address not in the list, so all requests key on the proxy's peer address |
+| Legitimate mTLS requests get `401 MTLS_UNTRUSTED_PEER` | Proxy not in the list, so certificate headers are stripped |
+| `401 MTLS_CERTIFICATE_UNMAPPED` | Certificate presented correctly, but no active service account has that fingerprint bound |
+| `403` with no certificate header reaching handlers | Fingerprint malformed; must be 64 hex characters, optionally colon-separated |
+| Works locally, fails in production | Development ran with an empty list; production needs the list set |

package/docs/security/rate-limiting.md

@@ -0,0 +1,79 @@
+# Rate limiting and abuse prevention
+
+Covers SEC-033 through SEC-039.
+
+## Redis is the primary limiter, and there is a fallback
+
+`isAllowed()` returned `true` whenever Redis was unavailable. During an outage,
+`login`, `mfa/verify`, `sms-otp/verify` and the OAuth token exchange had **no
+limit at all**.
+
+That is backwards. An outage is exactly when unlimited attempts are worth
+having — it is the moment a burst of guessing no longer looks like a burst,
+because there is no single event stream showing the ramp.
+
+Sensitive endpoints now set `emergencyLocalLimit: true`, which falls back to
+`src/services/localRateLimit.ts` when Redis is down or errors mid-request.
+
+The fallback is strictly weaker than the distributed limiter: a client gets one
+budget **per instance**, so a fleet multiplies the allowance. That is a
+degradation worth having. Unbounded is not.
+
+The store is capped at 10,000 keys. An unbounded map keyed by client address is
+itself a denial-of-service vector — rotating addresses would grow it without
+limit — so expired windows are swept and the oldest are evicted at the cap.
+Eviction only ever forgets an already-expired window, so it grants no extra
+budget; there is a test for that specifically.
+
+Endpoints that limit without `emergencyLocalLimit` still fail open. That is
+deliberate: the fallback is per-instance, and applying it to high-volume
+low-value endpoints would trade a real availability problem for a small
+reduction in abuse resistance.
+
+## What each budget is keyed on
+
+The key matters more than the number.
+
+| Endpoint | Keyed on | Why |
+| --- | --- | --- |
+| `login` | address **and** submitted address | stops repeated guesses at one account |
+| `login-per-address` | address only | bounds spraying, which the first budget cannot see |
+| `mfa-verify` | address **and** challenge | the challenge is one login attempt |
+| `totp-*` | user | a TOTP code is checked against one account's secret |
+| `scim` | credential | one noisy IdP cannot exhaust everyone else's budget |
+
+Two of these were wrong before v2.8.0, and both were found by a test suite that
+had been passing for the wrong reason:
+
+- `mfa-verify` included `body.email`, which that endpoint does not carry. So
+  every second-factor verification from one address shared a budget of 20. An
+  attacker got 20 guesses; so did an office behind a single NAT, where ordinary
+  traffic could lock out every legitimate second-factor login.
+- `totp-*` allowed 10 attempts keyed on the address alone, on **authenticated**
+  routes, with the same consequence.
+
+## Credential spraying
+
+The login budget is keyed on address *and* submitted address, so it stops
+repeated guesses at one account and does nothing about an attacker who varies
+the address on every request and guesses across a thousand accounts from one
+host. A second, address-keyed budget bounds that independently.
+
+## A refused request is recorded
+
+A rate-limit trip produced a `429` and nothing else. Sustained guessing at
+`login` or `mfa/verify` was invisible except in aggregate — the requests that
+most warranted attention were the only ones absent from the log.
+
+`rate_limit_triggered` now records the endpoint, the client address, and
+**which limiter decided**. `redis`, `local` and `none` are three distinct
+operational situations, and a degraded in-process control is a reason to go and
+look at Redis. Recording it as if it were the healthy path would hide exactly
+the thing worth knowing.
+
+## Related
+
+A failed login is audited as `user_login_failed`, and a refresh token presented
+twice is detected as `refresh_token_replayed` — see
+[audit.md](./audit.md). A session surviving a password change is covered in
+[trust-boundaries.md](./trust-boundaries.md).