sanook-cli 0.4.0 → 0.5.1

package/skills/ingest-webhook-secure/SKILL.md

@@ -0,0 +1,120 @@
+---
+name: ingest-webhook-secure
+description: Builds secure inbound webhook receivers that verify HMAC/asymmetric signatures over the raw body, reject replays via signed-timestamp windows and seen-id stores, dedup idempotently on provider event id, and fast-ack within timeout before processing async. Use when receiving callbacks from an external service that must be authentic, non-replayed, and handled exactly once.
+when_to_use: When standing up or debugging an inbound webhook/callback endpoint that must reject spoofed, replayed, or duplicate events and survive retry storms. Distinct from auth-jwt-session (verifies your own users' identity, not a provider's request signature), message-queue-jobs (the async worker you hand off to), and rate-limiting (caps request rate, not authenticity).
+---
+
+## When to Use
+
+Reach for this skill when an **external service POSTs to you** and you must trust, deduplicate, and reliably process those events:
+
+- "Stripe/GitHub/Slack/Twilio/Shopify webhook — verify the signature before acting"
+- "We're getting duplicate webhook deliveries / charged twice / sent the email twice"
+- "Provider says our endpoint timed out and they're hammering us with retries"
+- "Someone is POSTing fake events to our `/webhooks` URL"
+- "Signature verification fails intermittently" (almost always raw-body mangling)
+- Designing one intake endpoint for several providers with different header/encoding quirks
+
+NOT this skill:
+- Verifying *your own* logged-in user (session/JWT/cookie) → auth-jwt-session
+- The background worker/queue that does the slow processing → message-queue-jobs
+- Capping how many requests a caller may send → rate-limiting
+- Where the signing secret is stored/rotated at rest → secrets-management
+- Metrics/traces/dashboards for the endpoint → observability-instrument
+
+## Steps
+
+1. **Verify BEFORE parsing — over the RAW bytes, not re-serialized JSON.** Capture the exact body as received (`bytes`/`Buffer`) and sign *that*. Any JSON round-trip (`json.loads`→`json.dumps`, framework body-parser, pretty-printer, key reorder, trailing-newline strip) changes the bytes and breaks HMAC. Disable the framework's auto JSON parse for this route and read the raw stream first.
+
+   | Provider style | Signature scheme | What is signed |
+   |---|---|---|
+   | Stripe | HMAC-SHA256, header `Stripe-Signature: t=…,v1=…` | `"{t}.{rawbody}"` |
+   | GitHub | HMAC-SHA256, header `X-Hub-Signature-256: sha256=…` | raw body |
+   | Slack | HMAC-SHA256, header `X-Slack-Signature: v0=…` | `"v0:{ts}:{rawbody}"` |
+   | Shopify | HMAC-SHA256, **base64**, header `X-Shopify-Hmac-Sha256` | raw body |
+   | Svix/Standard Webhooks | HMAC-SHA256 base64, `webhook-signature` | `"{id}.{ts}.{rawbody}"` |
+   | GitHub App / Apple / some payment rails | **asymmetric** Ed25519 or RSA-SHA256, public key | raw body (you hold only the public key) |
+
+2. **Constant-time compare, support multiple/rotating secrets.** Never `==` on signatures — that leaks timing. Compute the digest and use a constant-time check. Iterate over *all* currently-valid secrets so rotation has zero-downtime overlap (old + new accepted during the window).
+
+   ```python
+   import hmac, hashlib, time
+   # header_sig MUST already be the parsed hex digest, NOT the raw header:
+   #   GitHub "sha256=<hex>" -> strip "sha256="; Stripe "t=..,v1=<hex>" -> the v1 value.
+   def verify(raw: bytes, header_sig: str, ts: str, secrets: list[bytes], tol=300) -> bool:
+       try:                                          # malformed/missing ts -> reject, never 500
+           skew = abs(time.time() - int(ts))
+       except (TypeError, ValueError):
+           return False
+       if skew > tol:                                # replay window FIRST (cheap reject)
+           return False
+       signed = f"{ts}.".encode() + raw              # STRIPE-SHAPED ("{ts}.{rawbody}"); swap per Step 1:
+                                                     #   GitHub -> signed = raw
+                                                     #   Slack  -> signed = b"v0:" + ts.encode() + b":" + raw
+                                                     #   Svix   -> signed = id.encode() + b"." + ts.encode() + b"." + raw
+       for secret in secrets:                        # accept any active secret (rotation overlap)
+           expected = hmac.new(secret, signed, hashlib.sha256).hexdigest()
+           if hmac.compare_digest(expected, header_sig):
+               return True
+       return False
+   ```
+   For asymmetric schemes, swap the body for `nacl.signing.VerifyKey(pub).verify(...)` (Ed25519) or `cryptography` `public_key.verify(...)` (RSA-PSS/PKCS1v15) — you never hold a shared secret. For base64 providers (Shopify, Svix) compare base64 digests, not hex.
+
+3. **Reject replays — two layers.** (a) Tolerance window on the **signed** timestamp (default **±300 s**); a captured-but-stale request fails the window even with a valid signature. (b) Store the provider event id with a TTL ≥ the window and reject a second sighting. The timestamp must be the one *inside the signature*, not a client header you didn't authenticate.
+
+4. **Idempotency — dedup on the provider's event id, atomically.** Use `SETNX webhook:{provider}:{event_id} 1 EX 86400` (or a UNIQUE column + `INSERT … ON CONFLICT DO NOTHING`). First writer proceeds; a `0`/conflict means already-seen → return `200` immediately (acknowledge, do nothing). TTL/retention ≥ the provider's max retry horizon (Stripe ~3 days, others up to weeks — check the table in step 7).
+
+5. **Respond 2xx fast, then process async — never do slow work inline.** The handler's only inline job: verify → persist the verified raw event → enqueue → return `200`. Hand the actual processing (DB writes, emails, downstream calls) to a worker/queue (→ message-queue-jobs). Most providers retry on >~5–10 s; slow inline work causes a retry storm that multiplies load. Return `200`/`202` within ~2 s.
+
+   | Outcome | Status | Why |
+   |---|---|---|
+   | Verified + enqueued (or duplicate) | `200`/`202` | Ack; stops retries |
+   | Bad/missing signature, failed asymmetric verify | `401` | Not authentic — do **not** 200 |
+   | Replay outside window / malformed timestamp | `400` | Authentic-looking but stale/garbage |
+   | Body too large / not the expected content-type | `413` / `415` | Reject before reading fully |
+   | Your DB/queue down (verified but can't persist) | `500`/`503` | Let the provider retry — do NOT 200 and drop |
+
+6. **Handle out-of-order delivery by resource version, not arrival order.** Retries and parallel deliveries mean `updated` can land before `created`. Reconcile on a monotonic field the provider gives (`sequence`, resource `version`, `updated_at`, Stripe object `created`): apply an event only if its version > the version you've stored; otherwise drop it as stale. When in doubt, treat the webhook as a *signal to re-fetch* the resource from the provider's API and use that as truth.
+
+7. **Lock down the surface + ship a safe replay tool.** Cap body size (`413` past e.g. 1 MB) before reading the whole stream. Reject unsigned/missing-header requests with `401` — never fall through to processing. Optionally pin source IPs to the provider's published CIDR allowlist (defense in depth, not a substitute for the signature). Some providers require a one-time **handshake/challenge** (Slack `url_verification` echo, Stripe/Meta GET with a `hub.challenge`, EventSub `webhook_callback_verification`) — answer it verbatim or you'll never receive events. Store the verified raw payload so you can re-drive processing later; the replay tool must re-run the *worker*, never re-accept an unverified HTTP request.
+
+   | Provider | Signature header | Encoding | Handshake | Notes |
+   |---|---|---|---|---|
+   | Stripe | `Stripe-Signature` | hex, `t=`/`v1=` | none | tolerance 300 s; secret per-endpoint (`whsec_…`) |
+   | GitHub | `X-Hub-Signature-256` | hex | ping event | also legacy SHA-1 header — ignore it, use 256 |
+   | Slack | `X-Slack-Signature` + `X-Slack-Request-Timestamp` | hex, `v0=` | `url_verification` echo | reject ts older than 5 min |
+   | Shopify | `X-Shopify-Hmac-Sha256` | **base64** | none | sign raw body, compare base64 not hex |
+   | Twilio | `X-Twilio-Signature` | base64 over **URL + sorted POST params** | none | not raw-body — concat full URL + params |
+   | Svix/Standard Webhooks | `webhook-id`/`webhook-timestamp`/`webhook-signature` | base64, `v1,` | none | id+ts+body signed; multiple space-sep sigs |
+
+## Common Errors
+
+- **Signing re-serialized JSON instead of raw bytes.** The #1 "works in Postman, fails in prod" bug. Read and sign the exact received bytes; never let a body-parser touch the route before verification.
+- **Plain `==` / string compare on signatures.** Timing oracle. Use `hmac.compare_digest` / `crypto.timingSafeEqual` (and length-check first since it throws on mismatched length).
+- **Comparing against the raw header instead of the parsed digest.** `X-Hub-Signature-256` is `sha256=<hex>`; `Stripe-Signature` is `t=…,v1=<hex>`. Extract the digest field first, then constant-time compare — comparing the whole header always fails.
+- **Reconstructing the signed string wrong (right secret, still rejects).** Each provider signs a different preimage (raw body vs `"{ts}.{body}"` vs `"v0:{ts}:{body}"`). Build it byte-for-byte from the Step 1 table; a generic `"{ts}.{body}"` silently works only for Stripe-shaped schemes.
+- **Crashing on a malformed/missing timestamp.** `int(ts)` on a non-numeric or absent header throws → `500` (provider retries forever). Catch and treat a bad timestamp as a hard reject (`400`/`401`), not an exception.
+- **Parsing the JSON before verifying.** Hands attacker-controlled bytes to your parser and downstream logic pre-trust. Verify first, parse second.
+- **Trusting an unsigned timestamp/IP header for replay defense.** Use the timestamp *inside the signed payload*; anyone can set a raw header. IP allowlists are spoofable behind misconfigured proxies — keep them as defense in depth only.
+- **No idempotency, or dedup that isn't atomic.** "Check then insert" in two steps lets two concurrent retries both pass → double processing. Use `SETNX`/`INSERT … ON CONFLICT` as one atomic op on the event id.
+- **Doing the work inline, returning 200 after.** Causes timeouts → provider retries → storm. Persist + enqueue + 200 fast; process in a worker.
+- **Returning 200 when persistence/enqueue failed.** Swallows the event forever — the provider thinks it's delivered and stops retrying. On internal failure return `5xx` so the retry redelivers.
+- **Applying events in arrival order.** Out-of-order retries overwrite newer state with older. Gate on resource version/sequence, or re-fetch the resource.
+- **One global secret, no rotation path.** Rotating means downtime or dropped events. Accept a *list* of active secrets; remove the old one after the overlap window.
+- **Ignoring the handshake/challenge.** Endpoint silently never activates; you debug "missing events" that were never sent. Implement the provider's verification echo.
+- **No body-size cap.** A multi-GB POST OOMs the process before you ever check the signature. Enforce a max length and `413` early.
+
+## Verify
+
+1. **Happy path:** Replay a captured real delivery with its original headers and raw body → `200`, event persisted once, worker ran exactly once.
+2. **Tampered body:** Flip one byte of the body, keep the signature → `401`, nothing persisted, worker never invoked.
+3. **Tampered/forged signature:** Random or empty signature header → `401`. Missing header entirely → `401` (not a 500, not a 200).
+4. **Raw-body integrity:** Send a payload whose `json.dumps` re-serialization differs from the bytes (extra whitespace, reordered keys) → still `200`. Proves you verify the raw bytes, not a re-encode.
+5. **Replay window:** Valid signature with a timestamp older than tolerance (e.g. ts−600 s) → `400`/`401`; same request within tolerance → `200`.
+6. **Duplicate delivery:** POST the identical valid event twice (and concurrently, in parallel) → both return `200` but the worker side-effect happens **exactly once**. This catches non-atomic dedup.
+7. **Out-of-order:** Deliver `version=2` then `version=1` for the same resource → final stored state reflects v2; the v1 arrival is dropped/ignored.
+8. **Fast-ack:** Make downstream processing sleep; the HTTP response still returns 2xx within the provider timeout (assert response latency, not just status).
+9. **Persistence failure:** Force the store/queue to fail on a verified event → endpoint returns `5xx` (so the provider retries), not `200`.
+10. **Oversized / wrong type:** POST > size cap → `413`; wrong `Content-Type` → `415`; both reject before full read.
+
+Done = a tampered or unsigned request gets `401`, a stale one `400`, a duplicate (including concurrent) is accepted but processed exactly once, a valid one is acked 2xx within timeout and processed via the worker, and raw-body verification survives a JSON re-serialization that would have broken a naïve implementation.

package/skills/integrate-oauth-oidc/SKILL.md

@@ -0,0 +1,126 @@
+---
+name: integrate-oauth-oidc
+description: Integrates a THIRD-PARTY identity provider via OpenID Connect — "Log in with Google/GitHub/Microsoft/Apple" or acting as an OAuth client to a third-party API. Uses the Authorization Code flow with PKCE (S256) everywhere (SPA, native, server); mandatory state (CSRF) + nonce (replay); exact-match redirect_uri; server-side code→token exchange (no client_secret in public clients); strict ID-token validation against JWKS; safe email_verified account linking; refresh rotation with reuse detection; system-browser-only native flows.
+when_to_use: Adding "Sign in with <provider>", consuming a third-party OAuth API, validating an ID token, linking accounts across providers, or fixing a broken OAuth callback/redirect. Distinct from auth-jwt-session (that ISSUES and validates YOUR app's own session/JWT after this handshake completes — this skill is the third-party handshake itself) and design-authorization-model (what a user may DO — permissions — not who they ARE).
+---
+
+## When to Use
+
+Reach for this skill when you are talking to an identity provider you do not own:
+
+- "Add Log in with Google / GitHub / Microsoft / Apple"
+- "Call the Stripe/Slack/Notion API on a user's behalf" (you are the OAuth client)
+- "Validate this ID token / id_token / JWT from Google" — check signature + claims
+- "A user signed up with Google but already has a password account — merge them"
+- "My OAuth callback redirects but the token exchange / state check fails"
+- "Refresh the access token / our Google refresh stopped working"
+- "The native app login opens a webview and Google blocks it with disallowed_useragent"
+
+NOT this skill:
+- Issuing, signing, or verifying YOUR app's OWN session cookie / JWT AFTER login succeeds, refresh rotation of YOUR token, RP-initiated logout clearing YOUR session → **auth-jwt-session** (this skill ends when you have a validated set of claims; minting your session from them is that skill)
+- "Which users can edit vs view", roles, multi-tenant isolation, per-resource rules → **design-authorization-model** (authZ — what they may do — not authN — who they are)
+- Where to STORE the `client_secret` (Vault/Secrets Manager, OIDC-to-cloud, rotation, leak remediation) → **secrets-management**
+- Auditing an existing diff for vulns by severity → **security-review**
+
+## Steps
+
+**1. Pick the flow — Authorization Code + PKCE, for every client type.**
+- The implicit flow (`response_type=token`) is dead (deprecated by OAuth 2.1 / Security BCP) — never use it. So is ROPC (password grant). Use `response_type=code` always.
+- PKCE (`code_challenge` + `code_verifier`) is mandatory for ALL clients now, including confidential server apps — not just SPA/native.
+
+| Client | client_secret? | PKCE | Token exchange runs |
+|---|---|---|---|
+| Server / web app (BFF) | yes (server-only) | yes | server |
+| SPA (React/Vue) | **no** | yes | **server (BFF)** — never the browser |
+| Native / mobile | **no** | yes | server, or native via AppAuth |
+| CLI | no | yes | local loopback or device code |
+
+**2. Build the authorize request with state + nonce + PKCE.**
+- `code_verifier` = 43–128 random chars; `code_challenge = BASE64URL(SHA256(verifier))`, send `code_challenge_method=S256` (never `plain`).
+- `state` = random, server-stored, tied to the user's session → verify on callback. This is your **CSRF** defense; a missing/unchecked state lets an attacker inject their own auth code.
+- `nonce` (OIDC) = random, stored, sent on authorize → **must equal** the `nonce` claim in the returned ID token. This is your **ID-token replay** defense.
+- `redirect_uri` must **exactly** match a value pre-registered with the provider (scheme, host, port, path, trailing slash — byte-for-byte). No wildcards; "almost matches" = error or open redirect.
+
+**3. Do the code→token exchange SERVER-SIDE. Never ship a secret to a public client.**
+- POST `code` + `code_verifier` (+ `client_secret` only if confidential) to the token endpoint from your backend.
+- A `client_secret` in SPA JS, mobile binary, or a public repo IS published — anyone can extract it. SPA/mobile use **PKCE without a secret** (it replaces the secret) behind a Backend-for-Frontend (BFF) that holds any secret and sets an httpOnly session cookie.
+- Store the `client_secret` per **secrets-management** (env/Vault), never in source.
+
+**4. VALIDATE the ID token — this is where most integrations are silently broken.**
+- Fetch the provider's **JWKS** (`jwks_uri` from `/.well-known/openid-configuration`), select the key by the token's `kid`, **verify the signature**. Cache JWKS; refresh on unknown `kid`.
+- **alg allowlist:** accept only what you expect (`["RS256"]` / `["ES256"]`). **Reject `alg:none`** and reject `HS256` when expecting RS — the RS→HS confusion attack signs with the public key as an HMAC secret. Never let the library read `alg` from the token.
+- Check claims: `iss` == provider's exact issuer; `aud` == **your** `client_id` (reject tokens minted for another app); `exp` not past, `iat` not absurdly future (small clock skew ok); `nonce` == the one you sent.
+- Only AFTER the token validates may you trust its claims or call `userinfo`. The `userinfo` response itself is not signed — trust comes from the validated ID token / the access token used to fetch it.
+
+**5. Read verified claims, then hand off to YOUR app.**
+- Standard OIDC claims: `sub` (the provider's STABLE user id — your join key, not email), `email`, `email_verified`, `name`, `picture`.
+- Match users on `sub`, never on email alone (email is reassignable and provider-controlled). Now mint your own session/JWT — that is **auth-jwt-session**'s job; this skill is done at "validated claims".
+
+**6. ACCOUNT LINKING — get this wrong and you enable account takeover.**
+- Link an OAuth identity to an existing local account by email **only if `email_verified == true`** AND the provider is one you trust to verify email. If you auto-link on an unverified email, an attacker registers `victim@example.com` at a sloppy IdP and takes over the victim's account.
+- Safer default: if an account with that email exists, require the user to **log in with the existing method first**, THEN link (first-party confirmation), instead of silently merging.
+- Model identities as a separate table: one user → many `(provider, sub)` rows. A user with Google + GitHub + password is normal. Unique-constrain `(provider, sub)`.
+
+**7. Refresh tokens — rotation, reuse detection, secure storage.**
+- Request `offline_access` / `access_type=offline` only if you actually need long-lived access. Google returns a refresh token **only on the first consent** (or with `prompt=consent`) — capture and store it then.
+- Rotate: each refresh use issues a new refresh token and invalidates the old. If an already-used (rotated) refresh token reappears → it was stolen → revoke the whole token family. (Mechanics overlap **auth-jwt-session**.)
+- Storage: server-side or httpOnly `Secure` cookie; native → **Keychain (iOS) / Keystore (Android)**. **Never `localStorage`** (XSS reads it).
+
+**8. Logout.**
+- RP-initiated logout: redirect to the provider's `end_session_endpoint` with `id_token_hint` + `post_logout_redirect_uri` to end the provider session, and **revoke** the refresh token at the provider's revocation endpoint.
+- Clearing YOUR app's own session/cookie is **auth-jwt-session**. Logging out of your app does NOT log the user out of Google unless you hit `end_session`.
+
+**9. Scopes & incremental consent.**
+- Request the **minimum** scopes at login (`openid profile email`). Ask for sensitive/extra scopes later, at the moment you need them (incremental consent) — broad upfront scopes scare users and over-privilege your token.
+
+**10. NATIVE / mobile — system browser only, never a webview.**
+- Use **ASWebAuthenticationSession** (iOS) / **Custom Tabs** (Android) via **AppAuth**. These share the system cookie jar (SSO) and isolate credentials from your app.
+- **Never an embedded `WKWebView`/`WebView`**: Google (and others) block it (`disallowed_useragent`), it defeats SSO, and an embedded webview CAN read the user's IdP credentials — that is the whole point of avoiding it.
+- PKCE is mandatory; redirect via a custom scheme or App Link/Universal Link that exact-matches registration.
+
+**11. Apple Sign In quirks (and other provider gotchas).**
+- Apple returns the user's **name only on the FIRST authorization** — persist it then or it's gone forever. Email may be a **private relay** (`@privaterelay.appleid.com`) the user can disable later — handle bounces.
+- Provider table:
+
+| Provider | Watch out for |
+|---|---|
+| Apple | name first-auth only; relay email; `client_secret` is a short-lived **JWT you sign** (ES256), not a static string — must regenerate |
+| GitHub | OAuth, **not full OIDC** — no id_token; call `/user` + `/user/emails` with the access token; pick the `primary`+`verified` email |
+| Microsoft (Entra) | `iss` varies per tenant; validate against `https://login.microsoftonline.com/{tid}/v2.0`; `v1.0` vs `v2.0` endpoints differ |
+| Google | refresh token only on first consent / `prompt=consent`; `email_verified` reliable |
+
+**12. Use a vetted library — do not hand-roll JWT validation or the flow.**
+
+| Stack | Library |
+|---|---|
+| Node | `openid-client` |
+| Python | `Authlib` |
+| Java/Spring | Spring Security OAuth2 Client |
+| Next.js / full-stack JS | NextAuth / Auth.js |
+| iOS / Android | AppAuth |
+
+These handle discovery, JWKS caching, PKCE, state/nonce, and clock skew correctly. Rolling your own ID-token verifier is the single most common source of `alg:none`/audience-confusion bugs.
+
+## Common Errors
+
+- **No PKCE / `code_challenge_method=plain`** — auth code interceptable. Always S256.
+- **Skipping or not comparing `state`** — CSRF / code injection. Store server-side, compare on callback.
+- **Trusting the ID token without checking `aud`** — a token minted for a DIFFERENT app of the same provider passes signature but is not for you. Require `aud == your client_id`.
+- **`alg:none` / RS→HS confusion accepted** — verifier reads `alg` from the token. Hardcode an allowlist; reject `none` and unexpected algs.
+- **`client_secret` shipped in SPA/mobile/repo** — it's public. PKCE replaces it; secret lives only server-side.
+- **Auto-linking on unverified email** — account takeover. Link only when `email_verified` AND trusted IdP, or require existing-login confirmation.
+- **Refresh token in `localStorage`** — XSS-readable. httpOnly cookie / Keychain.
+- **Embedded webview for native login** — provider blocks it and it can steal IdP creds. System browser (ASWebAuthenticationSession / Custom Tabs).
+- **redirect_uri "close enough"** — provider rejects, or a loose registration becomes an open redirect. Exact match, pre-registered.
+- **Lost Apple name / dropped Google refresh token** — both arrive once. Persist on first response.
+
+## Verify
+
+1. Tamper one byte of the ID-token signature → validation rejects. Craft `alg:none` → rejected. Swap to `HS256` signed with the public key → rejected.
+2. Token with wrong `aud` (another client_id) → rejected; expired `exp` → rejected; mismatched `nonce` → rejected.
+3. Callback with a wrong/missing `state` → rejected. Token exchange with a wrong `code_verifier` → fails.
+4. `grep` the SPA/mobile bundle for the `client_secret` → not present.
+5. Account-link test: register `victim@x.com` at a provider that does NOT verify email → your app refuses to auto-link to the existing local account.
+6. Refresh rotation: use a refresh token, replay the old one → family revoked, refresh fails.
+7. Native: confirm login opens the system browser (ASWebAuthenticationSession / Custom Tabs), not an in-app webview.
+8. Logout: after RP-initiated logout, the refresh token no longer mints access tokens at the provider.

package/skills/load-stress-test/SKILL.md

@@ -0,0 +1,129 @@
+---
+name: load-stress-test
+description: Designs and runs load, stress, soak, and spike tests against an HTTP/gRPC service using an open arrival-rate model — driving a realistic endpoint mix with think-time past the saturation knee and reporting latency percentiles, throughput ceiling, and breaking point against machine-checkable SLO thresholds.
+when_to_use: Before a launch/scale event, for capacity planning, or to validate an SLO — when the question is sustained req/s, where p99 degrades, or whether the service survives a soak/spike. Distinct from performance-profiling (explains why one already-measured request is slow) and optimize-sql-query (tunes one query's plan); this skill finds the limit, those explain it.
+---
+
+## When to Use
+
+Reach for this skill when the question is **"how much can it take, and where does it break"** — a capacity/SLO question, not a code question:
+
+- "How many req/s can this hold before p99 blows past 500ms?"
+- "Will checkout survive Black Friday / the launch spike?"
+- "Find the breaking point — ramp until error rate crosses 1%."
+- "Does it leak / degrade over an 8-hour soak at steady load?"
+- "Validate the SLO: p95 < 300ms, p99 < 800ms, errors < 0.5% at 2k RPS."
+- "Gate CI so a PR can't regress p95 by >10%."
+
+NOT this skill:
+- *Why* one endpoint is slow when you already know it is (flame graph, allocations) → performance-profiling
+- A specific slow SQL query's plan/indexes → optimize-sql-query
+- A prod incident already happening (this is a planned test, not a live outage) → incident-response-sre, or debug-root-cause for a reproducible failure
+- Adding the metrics/traces you watch during the run → observability-instrument (a prerequisite, not this)
+- Wiring the gate into the pipeline mechanics → cicd-pipeline-author (this defines the threshold; that plumbs it in)
+
+## Steps
+
+1. **Write the goal as numbers before touching a tool.** A test with no pass/fail line is just a graph. Fix four things:
+   - **Objective + scenario** (drives the load shape):
+
+     | Scenario | Question it answers | Shape | Duration |
+     |---|---|---|---|
+     | **Smoke** | Does the script even work? | 1–5 VUs | 1 min |
+     | **Load** | Holds at *expected peak*? | ramp to target RPS, hold | 10–30 min |
+     | **Stress** | Where's the knee / breaking point? | ramp **past** target until SLO breaks | until failure |
+     | **Soak** | Leak/degradation over time? | steady moderate load | 2–8 hr |
+     | **Spike** | Survives a sudden surge + recovers? | flat → instant 5–20×, then drop | 1–5 min spike |
+
+   - **SLO thresholds** as concrete inequalities: e.g. `p95 < 300ms`, `p99 < 800ms`, `error_rate < 0.5%`, `throughput ≥ 2000 req/s`. These become the exit code.
+   - **Target intensity** in **arrival rate (RPS)**, not just VUs — VU count without think-time is meaningless (see Common Errors). Derive VUs from Little's Law: `VUs ≈ target_RPS × (avg_response_time + think_time)`.
+   - **Environment**: a prod-like staging box (same instance class, DB size, cache warm, autoscaling either off or explicitly in-scope). Never load-test prod blind.
+
+2. **Model a realistic workload, not a hammer on one URL.** A single hot endpoint at 100% gives a fantasy number.
+   - **Endpoint mix** weighted to real traffic (read from access logs / APM): e.g. 70% `GET /feed`, 20% `GET /item/:id`, 8% `POST /cart`, 2% `POST /checkout`.
+   - **Think-time** between steps (`sleep(rand 1..3)`) so each VU models a user, not a tight loop.
+   - **Parameterized + correlated data**: unique users/items per iteration from a CSV/SharedArray (no caching by accident); capture a token/ID from response N and feed request N+1 (login → use `access_token`; create order → reuse `order_id`).
+   - **Auth**: log in once per VU and reuse the token; don't re-auth every iteration unless that's the scenario under test.
+
+3. **Pick the tool by team + need, encode thresholds as exit-code gates.** Default to **k6** for code-first, CI-friendly tests — it has RPS-precise arrival-rate executors and native threshold gates, so it covers most cases. Reach for the others only for the listed reason:
+
+   | Tool | Script lang | Reach for it when | Native threshold gate |
+   |---|---|---|---|
+   | **k6** (default) | JS | CI, scripted, RPS-precise (`constant-arrival-rate`) | `thresholds` → exit 99 on breach |
+   | Locust | Python | dynamic per-user logic, Python shop | `--exit-code-on-error` + custom |
+   | Gatling | Scala/Java DSL | JVM teams, rich HTML report | `assertions` → non-zero exit |
+   | Artillery | YAML/JS | quick YAML scenarios, serverless | `ensure` plugin |
+   | JMeter | XML/GUI | legacy/enterprise, protocol breadth | clunky; prefer above for CI |
+
+   k6 with an **open model** (arrival rate — the correct way to fix RPS and dodge coordinated omission) and SLOs as code:
+
+   ```js
+   import http from 'k6/http';
+   import { check, sleep } from 'k6';
+   import { SharedArray } from 'k6/data';
+   const users = new SharedArray('u', () => JSON.parse(open('./users.json')));
+
+   export const options = {
+     scenarios: {
+       ramp_to_knee: {
+         executor: 'ramping-arrival-rate',   // open model: fixed RPS, k6 adds VUs as needed
+         startRate: 100, timeUnit: '1s',
+         preAllocatedVUs: 200, maxVUs: 2000,
+         stages: [
+           { target: 500,  duration: '2m' },  // warm-up — exclude from SLO judgment
+           { target: 2000, duration: '5m' },  // hold at target peak
+           { target: 4000, duration: '5m' },  // push PAST to find the knee
+         ],
+       },
+     },
+     thresholds: {                            // breach → process exits non-zero → CI fails
+       http_req_duration: ['p(95)<300', 'p(99)<800'],
+       http_req_failed:   ['rate<0.005'],
+       http_reqs:         ['rate>1800'],      // throughput floor
+     },
+   };
+   export default function () {
+     const u = users[Math.floor(Math.random() * users.length)];
+     const r = http.get(`https://staging.internal/feed?u=${u.id}`);
+     check(r, { 'status 200': (res) => res.status === 200 });
+     sleep(Math.random() * 2 + 1);            // think-time 1–3s
+   }
+   ```
+   Run: `k6 run --summary-trend-stats="avg,p(95),p(99),max" test.js`.
+
+4. **Run staged, and watch the server while the client pushes.** Escalate; don't jump to max:
+   1. **Smoke** (1–5 VUs) — fix the script/correlation before scaling.
+   2. **Baseline** at low steady load — record reference percentiles.
+   3. **Ramp to target** — confirm SLO holds at expected peak.
+   4. **Push past** — keep ramping until a threshold breaks; the load just below that is the **breaking point / knee**.
+
+   The client number alone is half the picture. Capture **server-side** metrics over the same window (Grafana/Prometheus/APM): CPU%, memory (RSS trend for soak), **DB/connection-pool saturation**, thread/worker queue depth, GC pauses, downstream latency. The first resource to hit ~100% (CPU, pool exhaustion, disk I/O, a downstream rate limit) **is the bottleneck** — that's the finding. Always confirm the **client isn't the bottleneck** (load-gen box CPU/network not saturated, file descriptors raised) before trusting a ceiling.
+
+5. **Report the four numbers + the saturated resource, then gate.** A useful report states: **(a)** latency percentiles (p50/p95/p99/max) at target load, **(b)** sustained throughput ceiling (max RPS where SLO still holds), **(c)** breaking point (load where it broke + how — errors, timeouts, or latency cliff), **(d)** the saturated resource at that point. For soak, add the RSS/latency trend over time (flat = healthy; rising = leak). For CI: store the baseline summary, fail the build when p95/p99/error-rate regress beyond an allowed delta.
+
+## Common Errors
+
+- **Coordinated omission.** A closed-model loop that waits on each slow response stops *issuing* new requests during a stall, so the slowest requests are undercounted and p99 looks great. Fix: use an **open/arrival-rate model** (k6 `*-arrival-rate`, Gatling `constantUsersPerSec`, wrk2) that schedules requests on a fixed clock regardless of in-flight latency.
+- **No warm-up.** First requests hit cold JIT, empty caches, unconnected pools, and cold autoscalers — folding them in poisons percentiles. Run a warm-up stage and **exclude it** from the SLO judgment window.
+- **VUs as the target, no think-time.** "500 VUs" in a tight loop is an unrealistic, immeasurable arrival rate. Specify **RPS**; add think-time so a VU models a user. Convert via Little's Law.
+- **Single-VU extrapolation.** "1 user got 50ms, so 1000 users = 50ms each" — ignores contention, queueing, and pool limits, the entire point of the test. Latency is non-linear past the knee; you must actually ramp.
+- **Client is the bottleneck.** A maxed-out load-gen box (CPU, NIC, ephemeral ports, `ulimit -n`) caps *your* throughput, not the server's. Raise FD limits, distribute across machines (k6 cloud / multiple agents), and verify the generator is under ~70% before believing any ceiling.
+- **Testing a non-prod-like env.** Tiny DB, no cache, debug logging, a shared box — numbers don't transfer. Match instance class, data volume, and config; disable verbose logging.
+- **One endpoint at 100%.** Over-caches and misses cross-endpoint contention (shared pool, locks). Use a weighted mix from real traffic.
+- **Reusing the same record every iteration.** One user ID hits a hot cache row and reports impossibly low latency. Parameterize from a dataset of unique keys.
+- **Reporting only the average.** A 40ms mean can hide a 4s p99. Averages lie under load — always report **p95/p99/max**.
+- **Load-testing production unannounced.** Real users, real bills, real pages. Use staging; if prod is mandatory, schedule it, cap blast radius, and tell the on-call.
+- **Ignoring server metrics.** Client-only results tell you *that* it broke, never *why*. Without CPU/mem/pool/DB you can't name the bottleneck or fix it.
+
+## Verify
+
+1. **Threshold gate is real:** intentionally set an impossible threshold (`p(95)<1`) → the run **exits non-zero**. Proves the SLO is machine-checked, not eyeballed.
+2. **Open model confirmed:** the actual issued RPS tracks the configured arrival rate even as latency rises (not throttled by in-flight count) — no coordinated omission.
+3. **Warm-up excluded:** reported percentiles come from the steady window, and the first-stage cold numbers are visibly separated, not blended in.
+4. **Breaking point is named with a cause:** report states "broke at ~N RPS — `http_req_failed` crossed 1% / p99 hit the cliff" **and** the saturated resource (e.g. "DB pool at 100%, CPU 95%"), not just a latency graph.
+5. **Client wasn't the limiter:** load-gen CPU/network stayed below ~70% and FD limits weren't hit at the reported ceiling — otherwise the number is the generator's, not the service's.
+6. **Realism holds:** endpoint mix ≈ production weights, data was parameterized (cache-hit ratio sane, not artificially 100%), think-time present.
+7. **Soak (if run):** memory RSS and p95 are **flat** across the full duration — a rising slope is a leak/degradation finding, not a pass.
+8. **Reproducible:** the script, dataset, env spec, and exact command are committed so the run can be replayed and CI-gated.
+
+Done = the scenario ran on a prod-like env with an open arrival-rate model and excluded warm-up, every SLO threshold is enforced by a non-zero exit code, and the report states latency percentiles, the sustained throughput ceiling, the breaking point with its cause, and the saturated server-side resource — with the load generator proven not to be the bottleneck.

package/skills/map-privacy-data-gdpr/SKILL.md

@@ -0,0 +1,146 @@
+---
+name: map-privacy-data-gdpr
+description: Implements privacy/data-protection engineering — personal-data inventory/mapping (RoPA), lawful-basis and versioned consent capture, DSAR machine-readable export and right-to-erasure cascades across derived data/logs/backups, TTL/scheduled retention purge, and PII minimization/pseudonymization — for GDPR/CCPA-style compliance.
+when_to_use: A product stores personal data and needs consent capture, data export, deletion/erasure, or retention controls. Distinct from auth-jwt-session and design-authorization-model (who may access), build-audit-logging (tamper-evident action trail), and security-review (vulnerability audit).
+---
+
+## When to Use
+
+Reach for this skill when the requirement is **what happens to a person's data**, not who may touch it:
+
+- "A user requested their data / file a DSAR export endpoint"
+- "Implement account deletion / right to be forgotten / erasure that actually removes them everywhere"
+- "Record consent for marketing/analytics — granular, withdrawable, with proof"
+- "We keep PII forever — add retention limits / a purge job"
+- "Map where personal data lives for our DPIA / Article 30 record of processing"
+- "Stop collecting/storing PII we don't need; pseudonymize the rest"
+
+NOT this skill:
+- Logging in users, sessions, OAuth, refresh rotation → **auth-jwt-session**
+- Deciding which role/tenant may read a record (RBAC/ABAC/row scoping) → **design-authorization-model**
+- The tamper-evident trail of *who did what* (including who ran an erasure) → **build-audit-logging**
+- Hunting injection/SSRF/access-control bugs in changed code → **security-review**
+- Design-level threat enumeration over a system handling PII → **threat-model-stride**
+- Backup/PITR/retention *of the datastore itself* (RPO/RTO, WAL archiving) → **design-backup-dr-recovery**
+
+## Steps
+
+1. **Build the data inventory first — you cannot delete or export what you haven't mapped.** Produce a machine-readable record of processing (RoPA, GDPR Art. 30). One row per (data element × store). Drive everything downstream — export, erasure, retention — off this file, not off tribal knowledge.
+
+   ```yaml
+   # privacy/inventory.yaml — source of truth for DSAR + purge + DPIA
+   - element: email
+     category: contact          # contact | identifier | special-category | behavioral | financial
+     store: postgres.users.email
+     purpose: account login, transactional mail
+     lawful_basis: contract     # see step 2 table
+     retention: account_lifetime_plus_30d
+     subject_key: users.id
+     export: true
+     erase: anonymize           # delete | anonymize | retain-with-basis
+   - element: ip_address
+     category: identifier
+     store: postgres.request_logs.ip
+     purpose: fraud/abuse detection
+     lawful_basis: legitimate_interest
+     retention: 90d
+     subject_key: request_logs.user_id
+     export: true
+     erase: delete
+   - element: clickstream
+     category: behavioral
+     store: bigquery.analytics.events
+     purpose: product analytics
+     lawful_basis: consent
+     retention: 14m
+     subject_key: events.user_pseudo_id    # NOT the raw user id — see step 6
+     export: true
+     erase: delete
+   ```
+   Enumerate **every** store: primary DB, replicas, search index (Elasticsearch), caches (Redis), object storage (S3 uploads), data warehouse, application logs, third-party processors (Stripe, Segment, Intercom, Sentry), and **backups**. A store missing from this file is a store your erasure silently skips — that is the #1 compliance gap.
+
+2. **Pin a lawful basis to every element before you collect it.** No basis = you may not process it. Pick the *narrowest* basis that fits and record it in the inventory; consent is the weakest because it's revocable and must be audited.
+
+   | Lawful basis (GDPR Art. 6) | Use for | On erasure request |
+   |---|---|---|
+   | **Consent** | marketing, non-essential analytics, optional cookies | must delete; consent withdrawable any time |
+   | **Contract** | login email, order/shipping data, billing | retain while contract active, then purge |
+   | **Legal obligation** | tax invoices, AML/KYC records | **keep** for statutory period; refuse erasure with reason |
+   | **Legitimate interest** | fraud/abuse, basic security logs | keep if LIA balancing holds; honor objection |
+   | **Vital / public interest** | rare; safety-of-life | document case-by-case |
+
+   Default new fields to **no collection** until a basis is assigned. CCPA framing differs (opt-out of "sale/share", not opt-in consent) — model both as flags on the same consent record.
+
+3. **Capture consent as granular, versioned, withdrawable, time-stamped records — never a single boolean.** A `marketing_opt_in = true` column proves nothing and can't show *which* policy version they agreed to. Append-only consent ledger:
+
+   ```sql
+   CREATE TABLE consent_records (
+     id             bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+     subject_id     uuid NOT NULL,
+     purpose        text NOT NULL,      -- 'marketing_email','analytics','third_party_share'
+     granted        boolean NOT NULL,   -- false row = explicit withdrawal
+     policy_version text NOT NULL,      -- 'privacy-policy@2026-03-01'
+     source         text NOT NULL,      -- 'signup_form','preference_center','cookie_banner'
+     evidence       jsonb NOT NULL,     -- {ip, user_agent, banner_choice_ids}
+     created_at     timestamptz NOT NULL DEFAULT now()
+   );
+   CREATE INDEX ON consent_records (subject_id, purpose, created_at DESC);
+   -- current state = latest row per (subject_id, purpose); NEVER UPDATE/DELETE rows
+   ```
+   Withdrawal is a new `granted=false` row, not a mutation — you must be able to prove the full history. Check consent at point of use (`SELECT … WHERE subject_id=$1 AND purpose=$2 ORDER BY created_at DESC LIMIT 1`), and gate consent-based processing on it. Cookie banner: no non-essential cookies/tags fire before a `granted=true` row exists.
+
+4. **DSAR export: assemble a complete, machine-readable package keyed off the inventory.** Iterate every row with `export: true`, pull the subject's data by `subject_key`, emit structured JSON (GDPR Art. 20 portability requires machine-readable). Include data from processors via their APIs. Authenticate the requester *hard* (re-auth + verification) — handing one user's data to another is itself a breach. Respond within the statutory window (GDPR 30 days; CCPA 45). Don't include other people's PII caught in the subject's rows (e.g. recipient emails) — redact third parties.
+
+5. **Right-to-erasure must cascade to every store, including derived data, caches, logs, and backups — or document the backup-expiry path.** A `DELETE FROM users WHERE id=…` that leaves the subject in the search index, Redis cache, analytics warehouse, and last night's snapshot is **not** erasure. Drive a cascade from the inventory's `erase:` column:
+
+   ```python
+   def erase_subject(subject_id):
+       for row in inventory:                      # one source of truth
+           store = connect(row.store)
+           match row.erase:
+               case "delete":    store.delete(row, subject_id)
+               case "anonymize": store.anonymize(row, subject_id)   # see step 6
+               case "retain-with-basis":
+                   log_retained(subject_id, row, reason=row.lawful_basis)  # legal hold
+       invalidate_cache(subject_id)               # Redis keys, CDN signed URLs
+       delete_from_search_index(subject_id)       # Elasticsearch/OpenSearch
+       enqueue_processor_deletions(subject_id)    # Stripe, Segment, Intercom, Sentry APIs
+       add_to_suppression_list(subject_id)        # tombstone — see below
+       record_erasure(subject_id)                 # write to audit log (build-audit-logging)
+   ```
+   **Backups can't be selectively edited** — the defensible approach is: (a) restrict restored-backup access, (b) re-apply the erasure to any data restored from backup, and (c) let backups age out under a bounded retention (e.g. 35 days), documented in your privacy policy. Maintain a **suppression/tombstone list** so a restore or a late-arriving event for an erased subject is re-deleted, not resurrected. Erasure under a legal-hold basis (tax/AML) is refused with a recorded reason, not silently ignored.
+
+6. **Minimize and pseudonymize — the cheapest data to protect is data you don't hold.** Per element ask: do we *need* it? Don't collect optional PII "just in case". Replace direct identifiers with a per-subject pseudonym key in analytics/derived stores so erasing the key map breaks linkage (`erase: anonymize` then becomes deleting the mapping, not rewriting a warehouse). True anonymization (irreversible, no re-identification via combination) takes data **out of GDPR scope** — prefer it for analytics/ML training sets. Tokenize or keyed-hash (HMAC with a secret salt) and delete the mapping; **never** use a plain unsalted hash you call "anonymized". Drop high-cardinality quasi-identifiers (full IP → /24, exact DOB → birth year) where the purpose allows.
+
+7. **Enforce retention with TTL or scheduled purge jobs — retention written in a policy but not enforced in code is a fiction.** Translate each `retention:` value into a real mechanism:
+   - Native TTL where the store has one: MongoDB TTL index, DynamoDB TTL attribute, Redis `EXPIRE`, BigQuery partition expiration, S3 lifecycle rules, Elasticsearch ILM.
+   - A scheduled job (cron / Airflow / pg_cron) that `DELETE`s rows past `created_at + retention` for stores without TTL — run daily, log counts purged, alert on zero-purged-when-expected.
+   Logs and analytics are the usual offenders (PII-laden, retained forever). Cap them explicitly.
+
+8. **Document cross-border transfers and processor agreements.** Any element flowing to a processor outside the data's region needs a transfer mechanism (SCCs / adequacy decision) and a signed DPA. List sub-processors. Record this alongside the inventory — auditors ask for it, and a new third-party integration that isn't in the list is an unmapped data egress.
+
+## Common Errors
+
+- **Erasure that hits the primary DB only.** The subject survives in the search index, cache, warehouse, logs, and backups. Drive deletion from the inventory across *every* store; assert absence afterward.
+- **Consent as one boolean column.** Can't prove which policy version, when, or how; an `UPDATE` erases the prior state. Use an append-only versioned ledger; withdrawal is a new row.
+- **No suppression list.** A backup restore or a delayed event re-creates an erased subject ("data resurrection"). Keep a tombstone list and re-apply erasure on restore/ingest.
+- **Reversible "anonymization".** A plain SHA-256 of an email is re-identifiable by dictionary attack — still personal data, still in scope. Keyed-hash/tokenize and delete the mapping, or aggregate so individuals can't be singled out.
+- **Treating backups as out of scope entirely.** Ignoring them fails erasure; trying to surgically edit them corrupts them. Use bounded retention + restore-time re-erasure, and document it.
+- **Weak DSAR identity check.** Emailing an export to whoever asks lets an attacker harvest a victim's data. Re-authenticate and verify before export or erasure.
+- **Retention policy with no enforcement job.** "We keep logs 90 days" while the table grows unbounded. Wire a TTL or a scheduled purge and verify it actually runs.
+- **Logging full PII.** Request/error logs capturing emails, tokens, full bodies become an uncontrolled PII store with infinite retention. Redact at the logger; set log retention.
+- **Forgetting processors.** Deleting locally but leaving the subject in Stripe/Segment/Intercom/Sentry. Call each processor's deletion API as part of the cascade.
+- **Hardcoding the store list in deletion code instead of the inventory.** A new table added without touching the deletion code is silently skipped forever. Single source of truth, generated cascade.
+
+## Verify
+
+- **Erasure completeness:** run `erase_subject(id)`, then query **every** store in the inventory for that subject's `subject_key` → zero rows (or only anonymized/legal-hold-retained rows with a recorded reason). This is the test that catches the forgotten store; automate it per store.
+- **Resurrection resistance:** restore a backup taken before an erasure (or replay a late event) → the subject is re-suppressed, not present. Suppression list is consulted on restore/ingest.
+- **Export completeness:** for a seeded subject with data in N stores, the DSAR package contains all N, is valid JSON/machine-readable, and contains **no other** subject's PII.
+- **Consent gate:** withdrawing consent (new `granted=false` row) stops the gated processing on the next check; granted/withdrawn history is fully reconstructable; no non-essential tag fires before consent.
+- **Retention enforcement:** advance a record past its retention window (or wait/seed) → the TTL/purge job removes it on the next scheduled run; purge job logs counts and alerts on anomalies.
+- **Lawful basis coverage:** every element in the inventory has a basis; legal-obligation elements correctly *survive* an erasure request with a recorded reason.
+- **Minimization:** no element is collected/stored without a row in the inventory; derived/analytics stores key on a pseudonym, not the raw identifier.
+- **Transfers:** every cross-border element maps to a transfer mechanism + DPA; processor list matches actual integrations.
+
+Done = an erasure request provably removes or anonymizes the subject across every inventoried store (with backups handled by bounded retention + restore-time re-erasure and a suppression list), the DSAR export is complete and machine-readable with no third-party PII leakage, consent is versioned/withdrawable with reconstructable history, and every retention window is enforced by a TTL or scheduled purge that demonstrably runs.