sanook-cli 0.4.0 → 0.5.0

package/skills/file-upload-object-storage/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: file-upload-object-storage
+description: Implements secure file/image/video upload to object storage via short-lived presigned URLs or POST policies, with content-type + size validation, magic-byte verification, non-guessable tenant-scoped key namespacing, multipart/resumable transfer, private buckets with signed-URL access, and post-upload scan/transcode + lifecycle cleanup.
+when_to_use: User is adding file/image/video upload, generating presigned/direct upload URLs, handling large/resumable/multipart transfers, validating uploads, controlling object access, or serving via signed URLs/CDN. Distinct from auth-jwt-session (who the caller is — this consumes that identity) and secrets-management (storing the bucket credentials themselves).
+---
+
+## When to Use
+
+Reach for this skill when bytes flow **from a client into a bucket** and back out under access control:
+
+- "Let users upload an avatar / document / video to object storage (S3, GCS, R2, Azure Blob)"
+- "Generate a presigned URL / POST policy so the browser uploads direct-to-bucket"
+- "Handle large/resumable uploads, multipart with retry"
+- "Validate uploaded files — block executables, cap size, check the real type"
+- "Keep these files private and serve them with a time-limited signed URL / behind a CDN"
+- "Scan/resize/transcode after upload; clean up orphaned objects when a record is deleted"
+
+NOT this skill:
+- Deciding *who* may request an upload URL or read an object → auth-jwt-session (this skill consumes the authenticated identity; it does not establish it)
+- Where the bucket's own access keys / service-account JSON live → secrets-management
+- Broad storage-class/egress cost modeling across services → cloud-cost-optimize (this covers only per-object lifecycle/tiering for uploads)
+- Front-end image perf (responsive `srcset`, lazy-load, format) → optimize-core-web-vitals
+- Validating the *other* form fields around the file → build-form-validation
+- App-response caching to cut load → caching-strategy (CDN here is for object delivery, not API responses)
+
+## Steps
+
+1. **Default: client uploads direct-to-bucket via a short-lived presigned credential — never stream bytes through your app server.** Proxying uploads burns app memory/bandwidth and adds a hop. Your server only mints a credential and records metadata. Pick the credential type:
+
+   | Mechanism | Use when | Constrains |
+   |---|---|---|
+   | **Presigned POST policy** (S3 `create_presigned_post`, R2 same) | Browser/HTML form, single file, **want server-enforced size + type** | `Content-Type`, `content-length-range`, exact key or key prefix, expiry |
+   | Presigned PUT URL | Simple programmatic single-shot PUT | content-type + expiry only (no size cap pre-upload) |
+   | **Multipart presigned** (`create_multipart_upload` + per-part `upload_part` URLs) | File > ~100 MB, flaky networks, resumable | per-part, parallel, retryable |
+   | GCS resumable session (`x-goog-resumable:start`) / Azure Block Blob (`PutBlock`+`PutBlockList`) | GCS/Azure large or resumable | session URL + range |
+
+   Default for web uploads = **presigned POST policy** because it is the only one that enforces a size ceiling *before* bytes land. Expiry: **5 minutes** (`Expires=300`). Generate per-upload, single-use.
+
+2. **Validate on the boundary — twice. Never trust client-supplied MIME or extension.** A `.jpg` can be a polyglot HTML/JS or a zip bomb. Enforce in two places:
+   - **In the policy (pre-upload, hard ceiling):** size via `content-length-range` and a `Content-Type` allowlist condition. Example POST policy conditions:
+     ```python
+     fields = {"Content-Type": content_type, "x-amz-meta-owner": str(user_id)}
+     conditions = [
+         {"bucket": BUCKET},
+         ["starts-with", "$key", f"tenants/{tenant_id}/uploads/"],
+         {"Content-Type": content_type},          # must equal the one you signed
+         ["content-length-range", 1, 10 * 1024 * 1024],  # 1 B .. 10 MiB
+     ]
+     post = s3.generate_presigned_post(BUCKET, key, Fields=fields,
+                                       Conditions=conditions, ExpiresIn=300)
+     ```
+     S3 rejects with `403 EntityTooLarge` / `AccessDenied` if the upload violates the policy — the server never sees the oversized body.
+   - **Server-side after upload, by magic bytes:** on the upload event (step 6), read the **first 256–512 bytes** and sniff the real type (`file --mime-type -`, `python-magic`, Go `net/http.DetectContentType`, Node `file-type`). If the sniffed type is not in your allowlist, delete the object and mark the record `rejected`. Allowlist concrete types (`image/jpeg image/png image/webp image/gif application/pdf video/mp4`) — never a denylist.
+
+3. **Design the key/namespace: non-guessable, tenant-scoped, no user-controlled path segments.** Pattern: `tenants/{tenant_id}/{kind}/{uuidv4}{ext}` — e.g. `tenants/9f3.../avatars/0b1e7d2a-...-.webp`.
+   - Use a **server-generated UUIDv4/ULID** as the object id; never the raw filename. This blocks enumeration *and* path traversal (`../../etc/passwd`, leading `/`, `%2e%2e`).
+   - Sanitize/extension only: strip everything but a known-good extension derived from the **validated** type, not the client name.
+   - Store the real filename, owner, sniffed content-type, size, and `status` in **your DB** — the bucket is a blob store, not a database. The DB row is the source of truth; the object is referenced by key.
+   - Same prefix discipline lets one IAM policy / lifecycle rule target `tenants/*/uploads/`.
+
+4. **Large files: multipart/resumable with part retry + an abort-incomplete lifecycle rule.** Parts: **8–16 MiB** each (S3 min 5 MiB except last; max 10,000 parts). Upload parts in parallel (3–4 at a time), retry a failed part by re-PUTting just that part number, then `complete_multipart_upload` with the `{PartNumber, ETag}` list. **Failed/abandoned multipart uploads keep billable orphaned parts forever** — add the lifecycle rule:
+   ```json
+   { "Rules": [{ "ID": "abort-incomplete-mpu", "Status": "Enabled",
+       "Filter": { "Prefix": "tenants/" },
+       "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 1 } }] }
+   ```
+   GCS resumable / Azure uncommitted blocks need the equivalent (resumable sessions expire in 7 days; Azure uncommitted blocks GC after 7 days).
+
+5. **Access control: private buckets by default; serve via time-limited signed URLs.** Block all public access (`PublicAccessBlockConfiguration` all-true on S3; Uniform bucket-level access + no `allUsers` on GCS). Two-bucket split:
+   - **`public-assets`** bucket/prefix → genuinely public, immutable, cacheable (logos, released static media) → fronted by CDN, long `Cache-Control: public, max-age=31536000, immutable`.
+   - **`private-data`** bucket → user docs, originals → **never public**. Read access = a signed GET URL minted **per request after an authz check**, short expiry (**60–300 s**). For many objects on one page (galleries) use **signed cookies** (CloudFront) / signed-URL prefix to avoid signing each object.
+   The authz check (does this user own/may-read this key?) lives in **your** code before signing — the signature only proves the URL wasn't tampered with, not that the requester is entitled. That ownership decision is auth-jwt-session territory; this skill just enforces it at sign time.
+
+6. **Post-upload pipeline: react to the upload event, go `pending → ready`.** Insert the DB row as `status=pending` when you mint the URL. Fire on the storage event (**S3 → EventBridge/SNS/SQS or Lambda; GCS → Pub/Sub; Azure → Event Grid**) — do **not** trust a client "I'm done" callback as the only signal. In the handler: (a) magic-byte validate (step 2), (b) AV scan (e.g. ClamAV / `clamdscan`) for any user-shared file, (c) derive — resize/strip-EXIF for images, transcode for video (`ffmpeg` → HLS/MP4), (d) on success flip `status=ready`, on failure delete object + `status=rejected`. Clients only ever see/serve `ready` objects.
+
+7. **Lifecycle & cost: expire temp uploads, tier cold objects, CDN the hot reads.** Separate a `tmp/` prefix for unconfirmed uploads with a **1-day expiry** lifecycle rule (the orphan from an abandoned form never lingers). Transition originals not read in 30/90 days to Infrequent-Access / Nearline / Cool. Serve hot public reads through a CDN (CloudFront/Cloudflare/Fastly) with an origin-access identity so the bucket stays private to the world but readable by the CDN.
+
+8. **Cleanup orphaned objects on record delete.** Deleting the DB row must enqueue a delete of its object key(s) — including derived renditions/thumbnails. Do it **transactionally-ish**: delete the row, then on commit enqueue an idempotent delete job (retry-safe; a missing key is success). A nightly **reconcile** sweep (list bucket prefix vs DB keys) catches drift in both directions — objects with no row, rows with no object.
+
+## Common Errors
+
+- **Proxying upload bytes through the app server.** OOMs on large files, doubles bandwidth. Mint a presigned credential; let the client PUT/POST straight to the bucket.
+- **Trusting `Content-Type`/extension from the client.** Spoofable; enables stored-XSS via a `.jpg` that's really HTML served inline. Verify magic bytes server-side and serve user files with `Content-Disposition: attachment` + `X-Content-Type-Options: nosniff`.
+- **Putting the user's filename in the key.** Invites path traversal and enumeration. Key on a server UUID; keep the display name in the DB.
+- **Public bucket / public ACL "just to make it work."** Leaks every object and lets anyone overwrite. Block public access; use signed URLs. A `?`-less object URL that loads in incognito is a finding.
+- **Presigned URL with hours/days expiry.** A leaked long-lived URL is a permanent backdoor. Cap at minutes; mint per request.
+- **No `content-length-range` in the POST policy.** Client uploads a 5 GB file and you pay for it. Always set a size ceiling in the policy, not just a client-side JS check.
+- **Authz only at URL-mint time, never re-checked.** Object IDs leak in logs/referers; a stale signed URL outlives the user's permission. Keep expiry short and re-authorize on every mint.
+- **Forgetting `AbortIncompleteMultipartUpload`.** Failed multipart uploads accrue invisible, billable parts indefinitely. Add the 1-day abort rule on day one.
+- **Marking `ready` before the scan/transcode finishes.** Serves unscanned malware or a half-written object. Flip to `ready` only from the post-upload handler.
+- **Deleting the DB row but not the object (or vice-versa).** Orphans cost money and leak data; missing objects 404 live records. Enqueue an idempotent object delete on row delete + run a reconcile sweep.
+- **CORS not configured on the bucket.** Browser direct-PUT/POST fails preflight. Set `AllowedMethods` (`PUT POST`), `AllowedOrigins` (your exact origins, not `*`), and expose `ETag` for multipart.
+- **Same bucket for public assets and private originals.** One misconfig exposes everything. Split public-asset and private-data buckets with different policies.
+
+## Verify
+
+1. **Direct-to-bucket works:** client obtains a presigned POST/PUT and uploads with no bytes touching the app server (confirm app logs show only the mint call, not the body).
+2. **Size ceiling enforced server-side:** upload `max+1` bytes → bucket rejects (`403 EntityTooLarge`/policy violation); the body never reaches you. A client that disables the JS size check still cannot exceed the policy.
+3. **Type spoof blocked:** upload an HTML/EXE file renamed `.png` with `Content-Type: image/png` → passes the policy but the magic-byte check deletes it and sets `status=rejected`; it is never served `ready`.
+4. **Key is non-guessable + traversal-proof:** the stored key is a server UUID under `tenants/{id}/...`; a key containing `../`, a leading `/`, or the raw filename is rejected/never produced.
+5. **Privacy:** the raw object URL (no signature) returns `403` in an incognito session; a freshly signed URL returns `200`; the **same** URL after `Expires` returns `403`.
+6. **Cross-tenant denied:** user A's signed-URL request for user B's key fails the authz check at mint time (no URL issued), not just at read time.
+7. **Resumable:** kill the network mid-multipart, resume → only missing parts re-upload, `complete` succeeds, final object bytes match the source checksum (`s3 cp` then `sha256sum`).
+8. **Orphan hygiene:** an abandoned multipart upload is gone after the abort window; a `tmp/` object expires per its 1-day rule; deleting a record removes its object + renditions (verify via `aws s3 ls`/reconcile sweep shows no drift).
+9. **Post-upload state machine:** a fresh upload is `pending`, becomes `ready` only after scan+derive complete, and a malicious/corrupt file ends `rejected` with the object deleted.
+
+Done = uploads go direct-to-bucket under a ≤5-min single-use credential with a server-enforced size cap and magic-byte type check; objects are private, tenant-scoped, non-guessable, and readable only via short-lived signed URLs after an ownership check; large uploads resume and abandoned parts auto-abort; and every record delete (plus a reconcile sweep) leaves zero orphaned objects.

package/skills/fuzz-dynamic-security-test/SKILL.md

@@ -0,0 +1,111 @@
+---
+name: fuzz-dynamic-security-test
+description: Sets up dynamic security testing — coverage-guided fuzzing of parsers and input handlers (libFuzzer/cargo-fuzz/AFL++/go test -fuzz/atheris) and DAST scanning of a running app (OWASP ZAP/nuclei) — wired into CI with seed corpora, crash minimization, baseline suppression, and regression-corpus commits.
+when_to_use: Hardening code that parses untrusted input, or a running web app, with active runtime testing that drives real inputs to provoke crashes/vulns. Distinct from write-tests (functional-correctness tests), security-review (static code audit), remediate-web-vulnerabilities (fixing a known vuln), and load-stress-test (performance under load).
+---
+
+## When to Use
+
+Reach for this skill when you want to **actively drive inputs at code or a running app** to provoke crashes/vulns, not reason about them statically:
+
+- "Fuzz this parser / deserializer / protocol decoder / image or PDF loader for crashes"
+- "Set up cargo-fuzz / libFuzzer / `go test -fuzz` / atheris / AFL++ with a seed corpus and run it in CI"
+- "An input crashes / hangs / OOMs — minimize it and add a regression test"
+- "Run OWASP ZAP / nuclei against staging, authenticated, and triage the findings"
+- "Wire short fuzz on PR + long nightly fuzz, and DAST on every staging deploy"
+
+NOT this skill:
+- Functional correctness / example-based unit tests → write-tests
+- Reading code by eye for injection/authz/secret bugs (no execution) → security-review
+- Fixing a *specific known* SQLi/XSS/SSRF you already found → remediate-web-vulnerabilities
+- Measuring latency/throughput/breaking point under concurrency → load-stress-test
+- Reviewing an authorization *design* rather than testing it at runtime → design-authorization-model
+
+If a finding is confirmed, hand the fix to remediate-web-vulnerabilities; this skill *finds and reproduces*, it does not patch app logic.
+
+## Steps
+
+1. **Pick the right tool for the target — do not write a fuzzer by hand.** Coverage-guided engines mutate toward new code paths; random byte-spray finds nothing. Match the language:
+
+   | Target | Engine | Harness entry | Sanitizers |
+   |---|---|---|---|
+   | C/C++ | **libFuzzer** (clang `-fsanitize=fuzzer`) | `LLVMFuzzerTestOneInput(const uint8_t*, size_t)` | ASan + UBSan (+ MSan separately) |
+   | Rust | **cargo-fuzz** (libFuzzer under the hood) | `fuzz_target!(\|data: &[u8]\| { ... })` | ASan on by default |
+   | Go | **native `go test -fuzz`** | `func FuzzX(f *testing.F)` + `f.Fuzz(...)` | race + built-in checks |
+   | Python | **atheris** (libFuzzer bindings) | `atheris.Setup` + `TestOneInput(data)` | native-ext ASan optional |
+   | JS/TS | **Jazzer.js** | `module.exports.fuzz = (data) => {...}` | n/a (catches throws) |
+   | Out-of-process C binary | **AFL++** (`afl-fuzz -i in -o out`) | feed stdin/file | persistent mode + cmplog |
+
+   Default to the **in-process libFuzzer-family** engine for the language; reach for AFL++ only when you can't instrument the target (closed binary, weird build).
+
+2. **Fuzz the smallest deterministic boundary, structure-aware.** Target one pure `bytes → parsed value` function — the deserializer, the codec/protocol decode, the template/expression parser — not the whole HTTP handler. Make it deterministic (no clock/network/RNG/global state). For structured formats, decode the byte buffer into typed inputs with `arbitrary` (Rust) / `FuzzedDataProvider` (C++/atheris) so mutations stay valid-ish and reach deep logic instead of dying at the length check. Rust example:
+
+   ```rust
+   #![no_main]
+   use libfuzzer_sys::fuzz_target;
+   use arbitrary::Arbitrary;
+
+   #[derive(Arbitrary, Debug)]
+   struct Input { name: String, depth: u8, body: Vec<u8> }
+
+   fuzz_target!(|inp: Input| {
+       // never unwrap() inside a harness on the parser's own error path —
+       // a clean Err is correct, only a panic/abort is a finding.
+       let _ = my_parser::parse(&inp.name, inp.depth, &inp.body);
+   });
+   ```
+
+3. **Seed the corpus and add a dictionary — this multiplies coverage.** Drop real, valid sample files into `corpus/<target>/` (one input per file). Add a `.dict` of format tokens/magic bytes (`"PDF"`, `"\xFF\xD8"`, keywords) and pass `-dict=tokens.dict`. Without seeds the fuzzer wastes hours rediscovering the file header. Keep the corpus in the repo so CI starts warm.
+
+4. **Run, then minimize every crash before committing it.** Run locally first (`cargo fuzz run target -- -max_total_time=300` / `go test -fuzz=FuzzX -fuzztime=5m`). On a crash, **minimize the input** (`cargo fuzz tmin`, libFuzzer `-minimize_crash=1 -runs=100000`, AFL++ `afl-tmin`) so the repro is small and the root cause is obvious. Add `-rss_limit_mb`, `-timeout=`, and `-max_len=` so OOMs and hangs are reported as findings, not killed silently.
+
+5. **Commit each crash as a regression seed — this is the deliverable.** Copy the minimized input to `corpus/<target>/crash-<hash>` (or Go's `testdata/fuzz/FuzzX/`). It now re-runs on every fuzz invocation, so the bug can't silently return. This is what turns a one-off crash into a permanent test. Open a finding (step 7) linking the seed.
+
+6. **For a running app, run DAST against staging — never prod.** Stand up a disposable staging instance with seeded test data, then:
+   - **nuclei** for known-CVE/misconfig templates: `nuclei -u https://staging.app -severity medium,high,critical -rl 50`.
+   - **ZAP** for app-aware crawling: baseline first, then authenticated active scan. Authenticate (ZAP context + auth script, or pass a session cookie/Bearer) so the scanner reaches logged-in routes — an unauthenticated scan misses ~80% of the surface.
+   - **Baseline-suppress accepted findings** instead of muting the whole rule: keep a `zap-baseline.conf` / nuclei exclude list of triaged-and-accepted IDs so the gate only fails on *new* findings. Tune `-rl`/throttle so the active scan doesn't DoS staging.
+
+7. **Triage every finding: reproduce → severity → dedupe → file.** Re-run the exact input/request to confirm it's real (drop scanner false positives — reflected param that's actually encoded, "missing header" on an internal-only route). Rate by realistic impact (RCE/memory-corruption/authn-bypass = Critical; reflected-but-encoded = noise). Dedupe by crash stack / vuln class, not by input bytes — 500 inputs hitting one `parse()` panic are one bug. File with the minimized repro and the committed seed path.
+
+8. **Wire into CI in two tiers + a DAST stage.** Cheap on every PR, deep on a schedule:
+
+   ```yaml
+   # PR: smoke-fuzz only the changed/corpus seeds — must finish in <2 min, gates merge
+   pr-fuzz:
+     run: cargo fuzz run parser -- -runs=0 corpus/parser   # replay corpus, no mutation
+   # Nightly: long mutation run, upload new crashes as artifacts, file on failure
+   nightly-fuzz:
+     run: cargo fuzz run parser -- -max_total_time=3600 -timeout=10 -rss_limit_mb=2048
+   # Per staging deploy: DAST gate
+   dast:
+     run: nuclei -u $STAGING_URL -severity high,critical -ed <accepted.txt> -ni
+   ```
+
+   PR job replays the corpus (deterministic, fast, catches regressions); nightly does the expensive mutation. Never put an unbounded mutation run on the PR critical path.
+
+## Common Errors
+
+- **`unwrap()`/`expect()` in the harness on the parser's own error path.** Every malformed input then "crashes" — pure noise. A returned `Err` is correct behavior; only a panic/abort/sanitizer-trip in the *code under test* is a finding.
+- **No seed corpus and no dictionary.** The fuzzer burns the whole budget rediscovering the file magic/header and never reaches real logic. Seed with valid samples; add a token `.dict`.
+- **Non-deterministic harness.** Reading the clock, network, RNG, or mutating global state makes crashes non-reproducible and corrupts coverage feedback. The harness must be a pure function of `data`.
+- **Committing the raw crashing input, not the minimized one.** A 4 MB repro hides the root cause and bloats the corpus. Always `tmin`/`-minimize_crash` first.
+- **Fuzzing the whole HTTP handler instead of the parser.** Network/auth/DB setup dwarfs the parse step, so mutations rarely reach it — throughput collapses to a few execs/sec. Target the pure decode boundary in-process.
+- **No `-rss_limit_mb`/`-timeout`/`-max_len`.** OOMs and infinite loops get OS-killed and look like a hung job instead of a reported memory/hang bug. Set explicit limits.
+- **Sanitizers off (release build).** Use-after-free, OOB read, and integer-UB pass silently without ASan/UBSan — you only catch hard segfaults. Build the fuzz target with sanitizers on.
+- **Running ZAP/nuclei against production.** Active scans send malicious payloads, mutate data, and can take the service down. Always a disposable staging instance with test data.
+- **Unauthenticated DAST scan.** Misses every logged-in route — the high-value surface. Configure auth (context/script/session token) and verify the scanner is actually inside a session.
+- **Muting a whole scanner rule to clear noise.** Hides future real hits of that class. Suppress the specific accepted finding ID in a baseline file so only *new* findings fail the gate.
+- **Unbounded mutation fuzz on the PR job.** Blocks every merge for an hour or times out. PR replays the corpus; the long mutation run goes nightly.
+
+## Verify
+
+1. **Engine actually mutates and gains coverage:** a short run shows rising `cov:`/`ft:` and `exec/s` counters (libFuzzer) — not flat. Flat coverage means the harness rejects inputs at the door (wrong shape, missing `arbitrary`/`FuzzedDataProvider`).
+2. **Planted-bug catch:** add a deliberate `assert!`/OOB/`panic!` on a specific byte pattern (or use a target with a known-CVE-style flaw), run the fuzzer, and confirm it finds and minimizes the input within minutes. A fuzzer that can't catch a planted bug catches nothing.
+3. **Every crash yields a committed seed:** for each crash found, the minimized input lives in the corpus/`testdata` and is tracked in git. Re-running the harness over the corpus reproduces the crash deterministically.
+4. **Regression gate works:** with the seed committed but the bug *un*fixed, the PR corpus-replay job fails; after the fix it passes — proving the seed actually guards the regression.
+5. **DAST reproduced + authenticated:** at least one scanner finding is independently re-sent (curl/HTTP client) and reproduces; the scan log shows it traversed authenticated routes (logged-in paths visited), not just the login page.
+6. **Baseline suppression is scoped:** a previously-accepted finding is silenced by its specific ID, while a freshly introduced vuln of a *different* class still fails the gate (suppression didn't blanket the rule).
+7. **CI tiers honored:** PR fuzz finishes under its time cap (corpus replay only); nightly runs the bounded mutation budget and uploads any new crash as an artifact + files it.
+
+Done = the engine provably mutates toward new coverage, a planted bug or known-CVE pattern is caught and minimized, every crash is committed as a regression seed that fails-then-passes across the fix, and DAST runs authenticated against staging with scoped baseline suppression and a two-tier CI wiring.

package/skills/harden-llm-app-reliability/SKILL.md

@@ -0,0 +1,126 @@
+---
+name: harden-llm-app-reliability
+description: Hardens LLM API calls for production with per-call timeouts and cancellation, exponential-backoff-plus-full-jitter retries on 429/500/529 that honor Retry-After, model fallback, one-round structured-output repair, refusal/stop_reason handling, and a circuit-breaker degraded mode so a flaky provider never breaks the feature.
+when_to_use: Shipping an LLM feature where provider errors, timeouts, rate limits, or refusals must not crash the UX. Distinct from optimize-llm-cost-latency (speed/spend), defend-llm-prompt-injection (security of inputs), and rate-limiting (protecting your own API from callers, not surviving a provider's limits).
+---
+
+## When to Use
+
+Reach for this skill when the failure mode you fear is **the provider**, not your code or your callers:
+
+- "The model call sometimes hangs / times out and the request just spins forever"
+- "We get 429s / 529s / 500s in bursts and the feature errors out"
+- "Wrap the LLM call so a bad response or refusal degrades gracefully instead of throwing"
+- "Add fallback to a cheaper/other model when the primary is down or refuses"
+- "JSON-mode output is occasionally malformed and crashes the parser"
+- "Mid-stream the connection drops and the user sees half an answer"
+
+NOT this skill:
+- Making calls *cheaper or faster* (model routing for cost, prompt caching, token trimming) → optimize-llm-cost-latency
+- Defending the prompt against injection / untrusted-content attacks → defend-llm-prompt-injection
+- Limiting how often *your callers* hit *your* API (token bucket, quotas, your own 429s) → rate-limiting
+- Designing the prompt + structured-output schema itself → prompt-engineering
+- Measuring output quality across prompt/model changes → llm-eval-harness
+- Offloading the whole LLM job to a durable background queue with DLQ → message-queue-jobs
+
+This skill is the resilience wrapper *around* one logical LLM call. It assumes the prompt is already written.
+
+## Steps
+
+1. **Wrap every call in a timeout + cancellation token. No naked `await`.** A hung socket must die on a deadline you own, not the SDK default (often 600s+). Two clocks: a per-attempt timeout (the request) and a total deadline (all retries combined). Stream long calls so the per-attempt timeout measures *time-to-first-byte*, not total generation.
+
+   ```ts
+   const TOTAL_DEADLINE_MS = 30_000;   // whole operation, retries included
+   const PER_ATTEMPT_MS    = 12_000;   // one HTTP attempt (TTFB for streams)
+
+   // remainingMs = total deadline left for this attempt (computed by the caller, step 2)
+   async function callWithDeadline(fn, remainingMs) {
+     const ctrl = new AbortController();
+     const budget = Math.max(0, Math.min(PER_ATTEMPT_MS, remainingMs));
+     const t = setTimeout(() => ctrl.abort(), budget);
+     try { return await fn(ctrl.signal); }
+     finally { clearTimeout(t); }
+   }
+   ```
+   Pass `signal` into the SDK (`client.messages.create({...}, { signal })`). On the user side wire the inbound request's abort signal through so a closed browser tab cancels the upstream call instead of burning tokens.
+
+2. **Retry only what's retryable, with exponential backoff + full jitter, and honor `Retry-After`.** Classify the error before you retry — retrying a 400 is just slower failure.
+
+   | Status / condition | Retry? | Wait |
+   |---|---|---|
+   | 429 rate-limited | Yes | `Retry-After` header if present, else backoff |
+   | 529 overloaded (Anthropic) / 503 | Yes | backoff + jitter |
+   | 500 / 502 / 504 / gateway | Yes | backoff + jitter |
+   | Network reset / timeout / ECONNRESET | Yes | backoff + jitter |
+   | 408 request timeout | Yes | backoff |
+   | 400 / 422 bad request | **No** | fix the request, not the retry |
+   | 401 / 403 auth | **No** | rotate key / fix scope |
+   | 413 too large | **No** | trim input |
+   | Refusal / `stop_reason` | **No retry — fall back** (step 4) | — |
+
+   Defaults: **max 4 attempts**, base 500ms, cap 8s, **full jitter** (`sleep = random(0, min(cap, base * 2**attempt))`). Full jitter beats fixed/equal backoff because synchronized clients (a 429 storm) otherwise retry in lockstep and re-stampede. Always clamp the wait to the remaining total deadline — never sleep past it.
+
+   ```ts
+   const start = Date.now();
+   const elapsed = () => Date.now() - start;
+   for (let attempt = 0; attempt < 4; attempt++) {
+     try { return await callWithDeadline(fn, TOTAL_DEADLINE_MS - elapsed()); }
+     catch (e) {
+       if (!isRetryable(e) || attempt === 3 || elapsed() > TOTAL_DEADLINE_MS) throw e;
+       const ra = retryAfterMs(e);                       // parse header, seconds or http-date
+       const backoff = Math.random() * Math.min(8000, 500 * 2 ** attempt);
+       await sleep(Math.min(ra ?? backoff, TOTAL_DEADLINE_MS - elapsed()));
+     }
+   }
+   ```
+   LLM calls are **non-idempotent and billed** — a retry after a *partial* success double-charges. Only retry attempts that demonstrably failed before producing a usable response (connection error, non-2xx, timeout-before-first-byte). Never retry a call that already streamed a full body.
+
+3. **Validate structured output; repair once; then fail safe — never crash on malformed JSON.** When you asked for JSON, do not feed the raw model string straight into `JSON.parse` + a schema and let it throw to the user.
+   - Parse → validate against the schema (Zod / Pydantic / JSON Schema).
+   - On failure, **one** repair round: send the model the broken output + the validator error, ask for corrected JSON only. Strip code fences and prose first.
+   - Still invalid → return a typed safe default (e.g. `{ status: "unavailable" }`) or route to degraded mode. Log the raw output. Do **not** loop repairs (cost + latency blowup).
+
+   Prefer the SDK's native enforcement (tool/`tool_choice` forcing, strict JSON mode) over free-text + regex — it eliminates most repairs. Repair is the safety net, not the plan.
+
+4. **Fall back to another model on persistent failure or refusal.** When the primary is exhausted (retries spent, circuit open) or returns a refusal, try a fallback before giving up. Order by capability-then-availability, e.g. primary Sonnet → fallback Haiku, or cross-provider if you run multi-vendor.
+   - A **refusal** (`stop_reason: "refusal"`, or the model declining) is not a transport error — do not retry the same model; either fall back or return the refusal as a first-class result.
+   - Treat `stop_reason: "max_tokens"` as a *truncated* (not failed) result: the JSON is incomplete — repair or raise `max_tokens` and retry once, don't ship the cutoff.
+   - Cap fallback depth at 1–2 models. Record which model actually served the response.
+
+5. **Stream with a heartbeat; discard partials on mid-stream error.** Long generations should stream so the user sees progress and you detect stalls. Set an **inter-chunk idle timeout** (e.g. 20s with no new token → abort) — a stream can hang open without erroring. If the stream errors or aborts mid-way, **discard the accumulated partial** and either retry from scratch (step 2 rules) or degrade; never persist or render a half-message as if complete. Buffer to a scratch variable and only commit on the terminal `message_stop`.
+
+6. **Circuit-breaker around the provider → degraded mode.** Per-provider breaker: after N consecutive failures (e.g. 5) or a failure rate over a window, **open** the circuit and stop calling for a cooldown (e.g. 30s), then **half-open** one probe. While open, skip the doomed call and serve degraded mode immediately: a cached previous answer, a canned/templated response, or a clear "this feature is temporarily unavailable" — chosen per feature, decided *before* the incident. This stops a provider outage from turning into 30s timeouts on every request and exhausting your own connection pool.
+
+7. **Never lose user input on failure.** Before the call, persist the user's prompt/turn so any failure path (timeout, all-retries-exhausted, circuit open) returns a retryable state, not a black hole. The user should be able to resend with one tap, or the system auto-resumes — input is never silently dropped. For expensive multi-step agent runs, checkpoint so you resume from the failed step, not step 1.
+
+## Common Errors
+
+- **Relying on the SDK default timeout.** It's often minutes. A spike of hung sockets exhausts your connection pool and takes the whole service down. Set an explicit per-attempt timeout you own.
+- **Retrying non-retryable errors.** Looping on a 400/401/413 wastes the deadline and (for auth) can lock the key. Classify first; only retry 408/429/5xx/network.
+- **Fixed or equal-jitter backoff.** All clients that got 429'd retry at the same instant and re-stampede the provider. Use full jitter: `random(0, min(cap, base·2^n))`.
+- **Ignoring `Retry-After`.** The provider told you exactly when to come back; backoff math that retries sooner just earns another 429. Parse the header (seconds *or* HTTP-date) and prefer it.
+- **Retrying a partially-streamed call.** It already cost tokens and may have half-applied a side effect; the retry double-charges and can double-act. Only retry failures that occurred before a usable response.
+- **`JSON.parse` straight onto the response.** One malformed token throws an unhandled exception to the user. Always validate, repair once, then fail to a typed default.
+- **Infinite repair loop.** Re-asking the model until JSON is valid can run forever and 10x the bill. Exactly one repair round, then degrade.
+- **Treating a refusal as a 5xx.** Retrying the identical prompt on the same model just refuses again. Fall back or surface it; don't burn retries.
+- **Shipping a `max_tokens` cutoff as complete.** Truncated JSON silently corrupts downstream. Check `stop_reason`; repair or re-call with higher limit.
+- **Rendering the mid-stream partial.** A dropped stream leaves a half-answer the user reads as final. Buffer and only commit on `message_stop`; discard on error.
+- **No circuit breaker.** During a provider outage every request pays the full timeout × retries before failing — your latency and pool collapse. Trip the breaker and serve degraded mode fast.
+- **Dropping user input on the failure path.** The user retypes everything. Persist the turn before the call; make every failure resumable.
+- **Sharing one breaker/timeout budget across unrelated features.** A flaky batch job opens the circuit for your latency-critical chat path. Scope breakers per provider+route.
+
+## Verify
+
+Prove resilience with **fault injection**, not hope. Force each failure and assert the wrapper holds — don't wait for prod to hit them.
+
+1. **Forced 429 storm:** Stub the client to return `429` with `Retry-After: 2` for the first 3 calls, then `200`. Assert: exactly 4 attempts, waits honor `Retry-After` (≈2s, not the backoff curve), final result returned, total stays under the deadline.
+2. **Forced timeout:** Stub a response slower than `PER_ATTEMPT_MS`. Assert: the attempt aborts at the deadline (not the SDK default), the `AbortController` fired, and either a retry or a clean degraded response — never a hang.
+3. **Non-retryable:** Stub a `400`. Assert: **zero** retries, immediate failure, deadline barely consumed.
+4. **Malformed JSON:** Stub output that fails the schema, then valid on the repair call. Assert: exactly one repair round, valid object returned. Then stub it invalid twice → assert the typed safe default, no thrown exception.
+5. **Refusal / cutoff:** Stub `stop_reason: "refusal"` → assert fallback model is tried (no same-model retry). Stub `stop_reason: "max_tokens"` → assert truncation is detected, not shipped as complete.
+6. **Mid-stream drop:** Start a stream, kill the connection after 2 chunks. Assert: the partial is discarded (not rendered/persisted), and retry-or-degrade fires.
+7. **Circuit breaker:** Force N consecutive failures → assert the circuit opens, subsequent calls return degraded mode **immediately** (no timeout wait), then half-open probes and closes on recovery.
+8. **Input preservation:** Trigger total failure → assert the user's input is still retrievable/resumable, returned as retryable state, never silently lost.
+9. **Idempotency/billing:** Assert a fully-streamed-then-errored response is **not** retried (no double charge).
+
+Done = fault-injection tests 1–9 pass, every LLM call has an explicit per-attempt timeout + total deadline, retries use full-jitter backoff that honors `Retry-After` and never fires on non-retryable or already-served calls, malformed/refused/truncated output degrades to a typed safe path instead of throwing, the circuit breaker serves degraded mode under a forced outage without paying timeouts, and no failure path loses user input.

package/skills/i18n-localization-setup/SKILL.md

@@ -0,0 +1,113 @@
+---
+name: i18n-localization-setup
+description: Externalizes user-facing text into message catalogs keyed by stable IDs and wires locale-correct rendering — ICU MessageFormat plurals/gender/select, named-placeholder interpolation, Intl/CLDR number/date/list/relative-time formatting, RTL/bidi via logical CSS, and an extract→translate→compile pipeline with pseudo-localization.
+when_to_use: Making a product support multiple languages/locales, or auditing existing i18n — hardcoded UI strings, sentence concatenation, English-only `if(n===1)` plurals, missing RTL, locale-blind number/date formatting, or wiring i18next/react-intl/gettext/Rails i18n/Fluent. Distinct from style-responsive-tailwind (visual layout) and audit-accessibility-wcag (a11y conformance — i18n only owns translatable a11y *attribute text*).
+---
+
+## When to Use
+
+Reach for this when text must render correctly in **more than one language/locale**, not just look right:
+
+- "Add Spanish/Arabic/Japanese — what's the right way to externalize strings?"
+- "Our plurals break in Polish/Russian" or "we do `count === 1 ? 'item' : 'items'` everywhere"
+- "Dates show as `6/15/2026` for everyone" / numbers use `.` for thousands in `de-DE`
+- "Arabic/Hebrew layout is broken — everything's still left-to-right"
+- "Translators can't reorder words — we concatenate `'Deleted ' + n + ' files'`"
+- "Set up the extraction pipeline: extract → PO/XLIFF/JSON → translate → compile" + catch missing keys before ship
+- Auditing an app that's "already i18n'd" for the traps below
+
+NOT this skill:
+- Visual responsive layout, breakpoints, container sizing → **style-responsive-tailwind** (i18n owns *logical* CSS props + `dir`, not the design system)
+- WCAG conformance, screen-reader semantics, contrast → **audit-accessibility-wcag** (i18n only owns making `aria-label`/`alt`/`title` *translatable*)
+- `hreflang`, localized URLs, sitemap per-locale, canonical → **audit-technical-seo**
+- Validating/parsing user-entered locale data (phone, postal) → **build-form-validation**
+- Wrapping a single component's copy as you build it → **build-react-component** (use this skill when standing up the *catalog system*)
+- UTC storage, DST, IANA conversion math behind a displayed timestamp → **datetime-timezone-correctness** (i18n only *formats* the instant per locale; it doesn't compute it)
+
+## Steps
+
+1. **Externalize every user-facing string into a catalog keyed by a stable ID — kill concatenation.** A string is translatable if a human ever reads it: labels, buttons, errors, emails, `alt`/`aria-label`/`title`/`placeholder`, `<title>`, push/toast text. Key by **semantic ID**, never by English source (English changes → key shouldn't). Co-locate by feature: `checkout.cart.empty`, not `string_447`.
+
+   ```jsonc
+   // en.json — one message = one full sentence with named placeholders
+   { "checkout.items": "{count, plural, one {# item} other {# items}}",
+     "profile.greeting": "Welcome back, {name}!" }
+   ```
+   Never build sentences from fragments. `t('deleted') + ' ' + n + ' ' + t('files')` is **untranslatable** — word order, plural agreement, and gender all vary by language. One key = one whole sentence.
+
+2. **Pluralize with ICU MessageFormat / CLDR categories — never `if (n === 1)`.** English has 2 forms; Arabic has **6** (zero/one/two/few/many/other), Polish/Russian have 4. Provide every category the *target* locale's CLDR rules require; `other` is the mandatory fallback. Same mechanism for gender/choice via `select`. Use `#` for the count (auto-formatted per locale), not `{count}` re-interpolated.
+
+   | Need | ICU construct | Anti-pattern it replaces |
+   |---|---|---|
+   | Count agreement | `{n, plural, one {…} few {…} many {…} other {…}}` | `n === 1 ? 'x' : 'xs'` |
+   | Ordinals (1st/2nd) | `{n, selectordinal, one {#st} two {#nd} few {#rd} other {#th}}` | string-suffix hacks |
+   | Gender / enum | `{g, select, female {…} male {…} other {…}}` | branching in code, concatenating |
+   | Money/percent inside text | `{amt, number, ::currency/EUR}` | manual `$` + `toFixed(2)` |
+
+   Translators supply the categories *their* language needs — don't hardcode the English set into the message shape.
+
+3. **Interpolate with named placeholders so translators can reorder.** `"{count} {unit} remaining"` lets a translator emit `"quedan {count} {unit}"`. Positional `{0}`/`%s`/`printf` ordering is fixed and breaks under reordering — use named only. Pass an explicit values object: `t('checkout.items', { count })`. Escape literal braces per ICU (`'{'`). Auto-escape interpolated values for the sink (HTML) to avoid injection.
+
+4. **Format numbers/dates/lists/units via `Intl` (CLDR) in the user's locale — never roll your own.** Locale decides separators (`1,234.5` vs `1.234,5`), date order, AM/PM vs 24h, RTL digit shaping, list conjunctions. Always pass the resolved locale explicitly; relying on the host default is non-deterministic.
+
+   ```js
+   new Intl.NumberFormat(locale, { style: 'currency', currency: 'JPY' }).format(1234)   // ¥1,234
+   new Intl.DateTimeFormat(locale, { dateStyle: 'long' }).format(d)                       // 15 de junio de 2026
+   new Intl.RelativeTimeFormat(locale, { numeric: 'auto' }).format(-1, 'day')             // "yesterday"
+   new Intl.ListFormat(locale, { type: 'conjunction' }).format(['a','b','c'])             // "a, b, and c"
+   ```
+   `currency` is data, not locale — `de` user paying USD shows `1.234,00 $`. Never store formatted strings; format at render time from raw numbers + ISO/epoch timestamps. Default time storage to **UTC**, convert to the user's IANA timezone for display (the conversion/DST math itself lives in **datetime-timezone-correctness**).
+
+5. **Make layout direction-agnostic: logical CSS + `dir` + bidi isolation.** Set `<html dir="rtl" lang="ar">` from the locale (RTL set: ar, he, fa, ur). Replace physical properties with logical ones so one stylesheet serves both directions:
+
+   | Physical (breaks RTL) | Logical (correct both) |
+   |---|---|
+   | `margin-left` / `padding-right` | `margin-inline-start` / `padding-inline-end` |
+   | `left` / `right` | `inset-inline-start` / `inset-inline-end` |
+   | `text-align: left` | `text-align: start` |
+   | `border-left` | `border-inline-start` |
+
+   Wrap user/dynamic content of unknown direction in `<bdi>` or `unicode-bidi: isolate` so an Arabic username doesn't scramble surrounding LTR punctuation. Mirror directional icons (back/forward arrows) via `[dir=rtl]` or transform; don't mirror logos.
+
+6. **Stand up the pipeline: extract → catalog → translate → compile, gated by pseudo-loc + missing-key detection.** Source code is the single source of truth for *keys*; translators own values. Pick the format by toolchain:
+
+   | Format | Use with | Plurals |
+   |---|---|---|
+   | **JSON / ICU** | i18next, react-intl/FormatJS | native ICU |
+   | **PO/POT** (gettext) | Rails (`gettext`), Python, PHP, C | `nplurals` header |
+   | **XLIFF** | Angular, enterprise TMS handoff | ICU or `<plural>` |
+   | **FTL** (Fluent) | Mozilla stack, attribute-rich UI | built-in selectors |
+
+   Pipeline: (1) `extract` keys from source (`i18next-parser`, `formatjs extract`, `xgettext`) → POT/template; (2) merge into per-locale catalogs without dropping existing translations; (3) translate / push to TMS; (4) `compile` to runtime bundles (`formatjs compile`, `msgfmt`). Generate a **pseudo-locale** (`en-XA`: `[!!! Ŝéàŕçĥ ~~~]`) — accent + ~40% length padding + bracket markers — to surface hardcoded strings, truncation, and concatenation in CI before any human translates. Fail the build on missing keys / unknown ICU vars.
+
+7. **Negotiate locale, fall back, and support runtime switching.** Resolve in priority order: explicit user setting → URL/cookie → `Accept-Language` → app default. Match with BCP-47 lookup (`fr-CA` → `fr` → default); never 404 on an unsupported region — fall back to base language, then to source locale. Lazy-load the active locale's bundle (don't ship all 30); switching locale re-renders messages **and** updates `lang`/`dir` on `<html>`. Use a real BCP-47 matcher — `@formatjs/intl-localematcher` (`match()`) or `accept-language-parser` for the header, canonicalized via `Intl.getCanonicalLocales` — never naive string equality (there is no `Intl.LocaleMatcher` global; locale matching is the `localeMatcher` *option* on `Intl` constructors or a library).
+
+## Common Errors
+
+- **`count === 1 ? x : xs`.** Breaks every language with ≠2 plural forms (Arabic, Polish, Russian, Welsh). Use ICU `plural` with CLDR categories.
+- **Sentence concatenation** (`t('sent') + name + t('a_msg')`). Word order/agreement/gender vary; translators can't fix it. One key = one full sentence with named placeholders.
+- **Keying by English source text.** Editing the copy silently orphans the translation. Key by stable semantic ID.
+- **Hand-formatted numbers/dates** (`'$' + n.toFixed(2)`, `MM/DD/YYYY`). Wrong separators/order/currency per locale. Use `Intl.NumberFormat`/`DateTimeFormat` with an explicit locale.
+- **Conflating locale with currency/timezone.** A `de` user can pay in USD in `America/New_York`. Format with the user's *locale* but the transaction's *currency* and the event's *timezone*; store UTC + ISO currency code.
+- **Physical CSS** (`margin-left`, `float: right`). Layout breaks in RTL. Use logical properties + `dir`.
+- **No bidi isolation.** An RTL name/number injected into LTR text reorders adjacent punctuation/brackets. Wrap unknown-direction content in `<bdi>`/`unicode-bidi: isolate`.
+- **Forgetting non-`textContent` text.** `alt`, `aria-label`, `title`, `placeholder`, `<title>`, email subjects, validation messages are all translatable — and untranslated `aria-label` regresses a11y.
+- **No length budget.** German/Finnish run ~35% longer than English; pseudo-loc padding exposes truncation/overflow before translators do.
+- **Locale-blind sort/case.** JS `.sort()` is code-point order (`Ä` after `Z`); Turkish `i`↔`İ`/`ı` breaks `toUpperCase()`. Use `Intl.Collator(locale)` for sorting and `toLocaleUpperCase(locale)` for case.
+- **Inventing `Intl.LocaleMatcher`.** No such global exists — locale matching is the `localeMatcher` option on `Intl` constructors or a library (`@formatjs/intl-localematcher`). Don't string-compare BCP-47 tags.
+- **Shipping all locales eagerly / hard 404 on unknown region.** Lazy-load active locale; fall back `region → language → source`, never error.
+- **Pluralizing the count with `#` but re-interpolating `{count}` raw.** `#` is locale-formatted (`1,000`); a separate `{count}` isn't. Use `#` inside `plural`.
+
+## Verify
+
+1. **No hardcoded strings:** lint (`eslint-plugin-formatjs`, `i18next` no-literal rule) reports zero user-facing literals outside the catalog.
+2. **Pseudo-loc pass:** run UI in `en-XA` — every visible string is accented+bracketed (no bare English = no missed key), nothing truncates or overflows, no concatenated fragments appear.
+3. **Plural matrix:** render the count message at `n = 0,1,2,5,11,100` in `en`, `pl` (4 forms), and `ar` (6 forms); each picks the CLDR-correct category. `if(n===1)` cannot pass this.
+4. **Reordering:** a target locale that reverses placeholder order renders correctly (proves named, not positional, interpolation).
+5. **Formatting:** the same number/date/currency/list renders per-locale separators/order (`1,234.5`↔`1.234,5`, `06/15`↔`15/06`, currency symbol placement) — assert against `Intl` golden strings.
+6. **RTL:** load `ar`/`he` → `<html dir="rtl">`, layout mirrors via logical props, directional icons flip, bidi-isolated names don't scramble punctuation.
+7. **Missing-key gate:** delete a key from a non-source catalog → CI fails (or falls back to source) — it must never render a raw key like `checkout.items` to a user.
+8. **Negotiation:** `Accept-Language: fr-CA` with only `fr` available resolves to `fr` (not default/404) via a real matcher; switching locale at runtime updates messages **and** `lang`/`dir`.
+9. **Sort/case:** a localized list sorts via `Intl.Collator(locale)` (e.g. Swedish `å/ä/ö` last); Turkish case round-trips with `toLocaleUpperCase('tr')`.
+
+Done = zero hardcoded user-facing strings, pseudo-loc clean, the plural matrix passes for a 4-form and a 6-form locale, RTL renders with logical CSS + bidi isolation, all formatting goes through `Intl` with explicit locale, locale negotiation uses a real BCP-47 matcher, and CI fails on any missing key or unknown ICU variable.

package/skills/idempotency-keys/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: idempotency-keys
+description: Makes operations safe to repeat so retries and at-least-once delivery don't double-charge or double-create — idempotency by design first (PUT/upsert, conditional writes with version/ETag/If-Match, natural deterministic keys, set-don't-increment) and by key second (client Idempotency-Key header, a dedup table keyed unique on the key that stores request fingerprint + status + response and replays the SAME response, 409 in-progress lock for concurrent duplicates, 422 on key-reuse-with-different-body), plus consumer-side dedup (processed-event-id store / dedup window), the outbox pattern for atomic write+publish, and DB mechanics (ON CONFLICT, SELECT FOR UPDATE / advisory locks). Effectively-once via dedup, because exactly-once delivery is a myth.
+when_to_use: An operation can run more than once and must not have double effects — a POST that creates/charges behind a client/proxy/SDK retry, an at-least-once queue or webhook consumer that may redeliver, a job that may run twice, or you're adding an Idempotency-Key header or a dedup table. Distinct from resilience-timeouts-retries (decides WHEN/how to retry; this skill makes the target safe to retry into) and deliver-webhooks (the sender side — at-least-once delivery + signed retries; this skill is what makes the receiver safe under that redelivery).
+---
+
+## When to Use
+
+Reach for this skill when the same operation may execute more than once and a second execution must NOT produce a second effect:
+
+- "A POST timed out, the client retried, and we charged/created twice"
+- "Our SDK/proxy/load balancer retries — make the create idempotent"
+- "Add an Idempotency-Key header so replays return the original response"
+- "The queue/webhook is at-least-once; the consumer ran the same event twice"
+- "Make this job safe to run twice" / "dedup redelivered events"
+- "Atomically write a row AND publish an event without dual-write loss" (outbox)
+
+NOT this skill:
+- Deciding the retry policy itself — backoff, jitter, retry budget, circuit breaker, which errors are retryable → resilience-timeouts-retries (it generates the duplicate calls; this skill absorbs them safely)
+- The webhook *sender*: at-least-once dispatch, signing, retry schedule, DLQ for failed deliveries → deliver-webhooks
+- The webhook *receiver's* signature/replay-window verification (HMAC over raw body, timestamp window) → ingest-webhook-secure (this skill is the dedup-on-event-id half it hands off to)
+- Building the queue/worker, DLQ, poison-message handling → message-queue-jobs (this skill specifies the *idempotent consumer* it needs)
+- Idempotent PSP charges + subscription/proration/ledger reconciliation → payments-billing-integration (it owns billing state and calls this skill's key pattern for money-mutating calls)
+- The rounding/allocation math of the amounts → money-decimal-arithmetic
+
+## Steps
+
+1. **Make it idempotent BY DESIGN before reaching for a key — that's cheaper and self-healing.** A surprising amount of "double effect" disappears if the operation is naturally repeatable:
+
+   | Technique | How | Why it's idempotent |
+   |---|---|---|
+   | **PUT / upsert** to a client-chosen id | `PUT /orders/{client_uuid}` → `INSERT ... ON CONFLICT (id) DO NOTHING/UPDATE` | second call hits the same row, no new row |
+   | **Conditional write** (optimistic concurrency) | `If-Match: <etag>` / `WHERE version = N` → bump version | stale retry's precondition fails → no double-apply |
+   | **Natural / deterministic key** | derive id from stable inputs (`hash(order_id+sku)`, not `uuid()`) | same inputs → same id → conflict, not insert |
+   | **Set, don't increment** | `balance = 100` not `balance += 10`; `status = 'paid'` | reapplying the same set is a no-op |
+   | **DELETE / "ensure absent"** | delete-by-id, "cancel if active" | already-gone is success, not error |
+
+   Increments, "append a row", and server-generated ids on POST are the *non*-idempotent shapes that force you to step 2.
+
+2. **For non-idempotent POSTs, use a client-supplied Idempotency-Key.** The client (not the server, not per-retry) generates ONE key for a logical operation and sends it on the original request AND every retry — header `Idempotency-Key: <opaque-uuid>`. The key must be **stable across retries and unique per operation**: generate it once before the first send, store it with the in-flight request, reuse it on retry. This is the Stripe model and the reference semantics to copy.
+
+3. **Persist the key with a dedup table — fingerprint, status, and the stored response.** One row per key:
+
+   ```sql
+   CREATE TABLE idempotency_keys (
+     id_key        text NOT NULL,
+     scope         text NOT NULL,            -- e.g. (user_id || ':' || endpoint)
+     request_hash  text NOT NULL,            -- SHA-256 of canonical request body+route
+     status        text NOT NULL,            -- 'in_progress' | 'completed'
+     response_code int,
+     response_body jsonb,
+     created_at    timestamptz NOT NULL DEFAULT now(),
+     expires_at    timestamptz NOT NULL,     -- TTL: now() + 24h..7d
+     PRIMARY KEY (scope, id_key)              -- UNIQUE on (scope, key)
+   );
+   ```
+   Scope the key per `(user/tenant, endpoint)` so one client's key can't collide with or replay another's. Retention 24h–7d (Stripe = 24h); a TTL/cron purges expired rows so storage is bounded.
+
+4. **Server flow — claim, execute, store, replay — atomically.** On each request:
+   1. Compute `request_hash` from the canonical (sorted/normalized) body + method + route.
+   2. **Atomically claim** the key: `INSERT (scope, id_key, request_hash, status='in_progress') ON CONFLICT (scope, id_key) DO NOTHING`. The insert *is* the lock.
+   3. **If the insert won** (0 conflicts): run the real operation, then `UPDATE ... SET status='completed', response_code, response_body` and return the response.
+   4. **If it conflicted**, read the existing row:
+      - `status='completed'` **and** `request_hash` matches → return the **stored** `response_code`/`response_body` verbatim (the replay path — same result, no re-execution).
+      - `status='in_progress'` → a concurrent duplicate is still running → return **409 Conflict** (or `425`-style "in progress"); the client should retry-after, not re-execute.
+      - `request_hash` **differs** (same key, different body) → return **422 Unprocessable Entity** — the key was reused for a different operation; never run it.
+
+5. **Hold a lock for the in-flight window so concurrent duplicates don't both execute.** The `INSERT ... ON CONFLICT DO NOTHING` claim handles most of it, but if you read-then-write, take a row lock: `SELECT ... FOR UPDATE` on the key row, or a Postgres advisory lock `pg_advisory_xact_lock(hashtext(scope||id_key))` around the whole claim+execute. Without this, two parallel retries can both see "no row" and both run. Wrap claim + business write + result-store in **one transaction** (or make the business write itself idempotent via step 1) so a crash between execute and store doesn't lose the recorded response.
+
+6. **Consumer-side dedup for at-least-once queues and webhooks — "exactly-once delivery" is a myth; you get effectively-once.** Brokers (SQS, Kafka, RabbitMQ) and webhook senders redeliver on ack timeout, so the *consumer* must be idempotent. Two patterns:
+   - **Processed-event-id store:** a `processed_events(event_id PRIMARY KEY, processed_at)` table. Before handling, `INSERT ... ON CONFLICT DO NOTHING`; if 0 rows inserted, it's a duplicate → ack and skip. Dedup on the **provider's stable event id** (not your own per-receipt uuid). Pairs with ingest-webhook-secure / message-queue-jobs.
+   - **Dedup window:** a bounded TTL set (Redis `SET key NX EX <window>`) when full history is too large — only safe if redelivery is bounded within the window.
+
+   Best of all: make the *handler* naturally idempotent (step 1: upsert by event-derived key, set-don't-increment) so even a missed dedup is harmless.
+
+7. **Atomic write + publish → outbox pattern (no dual-write).** Writing the DB row and publishing the event as two separate calls can crash between them (row saved, event lost — or vice versa). Instead, in **one transaction** write the business row AND an `outbox` row; a separate relay polls/CDC-tails the outbox and publishes (at-least-once → consumers dedup per step 6). The transaction guarantees the event is recorded iff the state changed.
+
+8. **DB mechanics cheat-sheet.**
+   - Postgres/SQLite: `INSERT ... ON CONFLICT (cols) DO NOTHING` (claim/dedup) or `DO UPDATE SET ...` (upsert). MySQL: `INSERT ... ON DUPLICATE KEY UPDATE`. The **UNIQUE constraint/index is what makes it safe** — `ON CONFLICT` without a matching unique index silently doesn't dedup.
+   - Serialize the in-flight window with `SELECT ... FOR UPDATE` (row) or `pg_advisory_xact_lock` (cross-row/logical) — released at transaction end.
+   - Check the *result* of the upsert (rows affected / `RETURNING xmax = 0`) to know whether you inserted or hit an existing row.
+
+## Common Errors
+
+- **Generating the key per-retry (`uuid()` / `now()` inside the retry loop).** Every attempt gets a fresh key → zero dedup → still double-charges. Fix: generate ONCE before the first send; reuse the identical key on every retry.
+- **No request-fingerprint check.** Same key replayed with a *different* body silently returns the old response (or runs the new op). Fix: store `request_hash`; on mismatch return 422, never execute.
+- **Racing duplicates with no lock.** Two parallel retries both `SELECT` (no row), both execute, both insert. Fix: atomic `INSERT ... ON CONFLICT DO NOTHING` as the claim, or `FOR UPDATE` / advisory lock around read-modify-write.
+- **`ON CONFLICT` / upsert without a UNIQUE index on the key.** No conflict ever fires → no dedup, duplicate rows. Fix: enforce a unique constraint on `(scope, id_key)` (or the natural key).
+- **Unbounded key storage.** The dedup table grows forever. Fix: `expires_at` + a purge job; pick 24h–7d retention.
+- **Treating a non-idempotent op as idempotent.** Retrying `balance += 10` or "append row" doubles the effect even *with* a key if you don't replay the stored response. Fix: replay the stored response on hit; or redesign to set-don't-increment (step 1).
+- **Recording the result in a separate step from the business write.** Crash in between → next retry re-executes a completed op. Fix: same transaction, or idempotent business write so re-execution is a no-op.
+- **Believing the broker gives exactly-once.** "Exactly-once delivery" doesn't exist over a network; redelivery happens. Fix: idempotent consumer + processed-event-id dedup = effectively-once.
+- **Dual-write (DB then publish, two calls).** A crash loses one side. Fix: outbox in the same transaction + a relay.
+- **Acking before the work is durable.** Ack-then-process loses the message on a crash. Fix: process (idempotently) and commit, *then* ack.
+
+## Verify
+
+1. **Duplicate POST is a no-op:** send the same request with the same `Idempotency-Key` twice → exactly one effect (one charge/row) and the second response is byte-identical to the first.
+2. **Concurrent duplicates:** fire N parallel requests with the same key → exactly one executes; the rest get the stored response or `409 in-progress`, never a second effect. (This is the race test — run it against the real shared store.)
+3. **Key reuse, different body:** same key + changed payload → `422`, and no operation runs.
+4. **Per-retry-key bug guard:** confirm the client generates the key once and reuses it (grep the retry path for `uuid()`/`now()` *inside* the loop).
+5. **Consumer redelivery:** deliver the same event id to the queue/webhook consumer twice → handled once (processed-events insert conflicts on the second); effect is identical to single delivery.
+6. **By-design ops:** issue the same `PUT`/upsert / conditional write twice → one row, version advances once; a stale `If-Match` retry is rejected, not double-applied.
+7. **Outbox atomicity:** kill the process between the business write and publish → on restart the relay still publishes (event recorded iff state changed); no orphan event, no lost event.
+8. **Retention bounded:** expired keys are purged; an old key past TTL behaves as a fresh request (documented), and the table doesn't grow without bound.
+
+Done = duplicate and concurrent requests produce exactly one effect with an identical replayed response, same-key/different-body returns 422, the in-flight window is locked, consumers dedup at-least-once delivery on stable event ids, write+publish is atomic via the outbox, and key storage is TTL-bounded — all proven by the parallel/redelivery tests in checks 1–7.