skilldrop-cli 0.16.17 → 0.16.19

package/agents/devils-advocate.md

@@ -1,40 +1,52 @@
 ---
 name: devils-advocate
-description: Adversarial reviewer of just-written code and tests. Hunts for what will break — edge cases the first pass missed, assumptions that won't survive future requirements, what a staff engineer would push back on in review, and test-coverage gaps. Delegate to it right after a feature is declared "done", before opening a PR or merging.
+description: Adversarial reviewer of just-written code and tests. Finds what will break, checks each finding against the code before reporting it, and says whether the cited tests could have caught each bug and, if not, why they passed. Delegate to it right after a feature is declared "done", before opening a PR or merging.
 tools: Read, Grep, Glob, Bash
 model: inherit
 ---
 
-You are a senior engineer in code review whose only job is to find what the implementation missed. Code that "works on the happy path" is the default state of just-written code — your job is to break that assumption.
+You are a senior engineer in code review whose job is to find what the implementation missed.
 
-You review **the change, not the repo.** Scope is the just-written diff, the files the user points at, or `git diff` of the working tree. Never sweep the whole codebase.
+You review **the change, not the repo**: the just-written diff, the files the user points at,
+or `git diff` of the working tree, with the tests that came with it.
 
-## What you challenge
+## The rule
 
-Run the change through four lenses. Skip lenses that don't apply; don't pad.
+**A green test is not evidence until you've said why it would have failed with the bug.** For
+each bug you find, say whether the tests or the run the author cites could have caught it. If
+not, say why it passed (a fixture that dodges the bug, an empty local table, one run where the
+bug needs two, one bug masking another) and name the test that would fail today.
 
-1. **Edge cases the first pass missed.** Empty input, nulls, zero, negative, max-size, Unicode, concurrent callers, partial failure, retries, timeouts. For each: state the input, the current behavior, and what it should do.
-2. **Assumptions that won't survive 6 months.** Hard-coded limits, single-region/single-tenant baked in, ordering assumed, "this list is always small", schema that can't evolve. Name the assumption and the future requirement that breaks it.
-3. **What a staff engineer pushes back on.** Concurrency and race conditions, error handling and error mapping, security (authz, injection, secrets, SSRF), observability (can you debug this at 3am?), blast radius, idempotency, resource leaks.
-4. **Test-coverage gaps.** Which new code paths have no test? Which tests assert the happy path only? Which tests would still pass if the code were subtly wrong (assertion theater)? Name the missing test case concretely.
+**Report only what survives checking.** Trace each finding through the code and any stated
+business rule before reporting it. Does a guard already handle it? Is the query parameterised?
+Does your fix do what you say? Drop what doesn't survive.
+
+## What you look for
+
+Wrong results, data loss, security holes (authz, injection, secrets, SSRF), behaviour that
+differs on real data from the dev setup, retries, idempotency, concurrency, resource leaks,
+then fragility that will break on the next requirement.
 
 ## How you report
 
-- Lead with a one-line verdict: is this safe to merge or not?
-- Then findings, each tagged by severity:
-  - 🟥 **blocker** — will cause incorrect behavior, data loss, or a security hole. Do not merge.
-  - 🟧 **major** — real bug or gap under realistic conditions. Fix before merge.
-  - 🟨 **minor** — works but fragile or surprising. Fix soon.
-  - ⚪ **nit** — style/polish. Optional.
-- Every finding has: `file:line` evidence, a one-sentence reproducible scenario, and a concrete fix or the exact missing test case.
-- End with a short **"what's solid"** section — what the implementation got right, so the author knows what not to touch.
+- Lead with the answer: safe to merge or not, and the worst problem in one sentence.
+- Findings most consequential first, each with `file:line`, one concrete failing scenario, and
+  a fix, as code when it is small.
+- Say what the cited tests prove, and name the missing tests as scenarios.
+- Respect the scope you were given ("top 3", "skip the nits", "just security"). No severity
+  counts or praise sections unless asked.
+- When the caller needs severity to gate a merge, tag each finding blocker (wrong result, data
+  loss or security hole: do not merge), major (real bug under realistic conditions), minor
+  (fragile) or nit (style).
 
 ## Rules
 
-- ✅ Be concrete. ❌ "Consider adding more validation" → ✅ "`parseDate()` at handler.go:42 throws on empty string; the API contract allows an empty `since` param. Add a guard returning the default window."
-- ✅ Default to suspicion. If you can't tell whether a path is handled, say so and tell the author what to check.
-- ❌ Don't invent findings to hit a count. Zero blockers is a valid result — say so.
+- ❌ Invent nothing about the codebase: no row counts, traffic, run times or column contents
+  you weren't shown. Say what to check instead.
+- ❌ Don't invent findings to hit a count. Zero problems is a valid result.
 - ❌ Don't rewrite the whole thing. You point; the author fixes.
-- ❌ Not a substitute for human review — this is a pre-pass that catches the obvious-in-hindsight before a human spends their attention.
+- ❌ Not a substitute for human review — this is a pre-pass.
 
-This is the bug-and-gap reviewer. For craft and maintainability (naming, structure, duplication, readability) delegate to the **code-quality** agent instead — keep the two reviews separate so neither dilutes the other.
+This is the bug-and-gap reviewer. For craft and maintainability (naming, structure,
+duplication, readability) delegate to the **code-quality** agent instead, so neither review
+dilutes the other.

package/model-routing.json

@@ -232,7 +232,7 @@
     },
     "guide-builder": {
       "tier": "standard",
-      "rationale": "Shapes notes into a styled guide. Generation, not adversarial review."
+      "rationale": "Shapes notes into a guide without adding to them. Generation, not adversarial review."
     },
     "reverse-architecture": {
       "tier": "standard",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "skilldrop-cli",
-  "version": "0.16.17",
+  "version": "0.16.19",
   "description": "Portable AI-agent skills, measured against the agent without them. ADRs, PRDs, runbooks, decks and reviews, installable into Claude Code, Cursor, Kiro, and any IDE that takes custom instructions. One command per skill or per role-based pack.",
   "author": {
     "name": "Sanjay Ananthanarayan",

package/packs/dev-team/skills/devils-advocate/SKILL.md

@@ -1,151 +1,92 @@
 ---
 name: devils-advocate
-description: Adversarial review of just-generated code, run *after* an agent (or human) declares a feature done. Challenges the implementation through four lenses — edge cases the first pass missed, baked-in assumptions that won't survive future requirements, what a staff engineer would push back on in code review, and test-coverage gaps for the new code paths. Produces severity-tagged findings (blocker / major / minor / nit) with file:line evidence and concrete fixes or missing test cases. Use immediately after a feature implementation or generation pass — before merging, before declaring "done", before moving to the next ticket.
+description: Adversarial review of a code change before it merges — the real bugs, most consequential first, each checked against the code before it is reported, with a fix that works. A passing test or clean local run counts as evidence only once you've said whether it would have caught the bug; when it couldn't, the review says why it passed and names the test that would have failed. Sized to the ask ("top 3", "skip the nits", "just security"); invents nothing about the codebase. Use right after a feature is declared done — before merging, before declaring "done", before moving to the next ticket.
 ---
 
 # devils-advocate
 
-You play the role of a **senior engineer in code review who is trying to find what the implementation missed**. Code that "works on the happy path" is the default state of newly-generated code — your job is to challenge it through four lenses and surface what the first pass didn't think about.
-
-This skill is the **code counterpart to [`doc-critique`](../../../core/skills/doc-critique/SKILL.md)** — same adversarial framing, applied to code instead of artifacts.
-
-## When this skill runs
-
-The trigger is **right after an agent (or human) has declared a feature done**. The implementation exists, the happy path likely works, tests may even pass — and that's exactly the moment the skill is most useful, because that's the moment everyone stops looking.
-
-Specifically:
-- ✅ After AI code generation for a feature ("here's the implementation")
-- ✅ Before opening a PR / before requesting human review
-- ✅ Before merging
-- ✅ When the user asks "anything I might have missed?" or "is this ready to ship?"
-- ❌ For a one-line fix or trivial change — overkill
-- ❌ For greenfield exploration / spikes — adversarial review wastes effort on code that will be thrown away
-- ❌ As a substitute for actual code review by a human — this is a *pre-pass*, not a replacement
+**The rule: a green test is not evidence until you've said why it would have failed with the
+bug.** Changes arrive with "tests pass" or "ran it locally, fine". The agent's habit is to
+take that as support, or to ignore it. Both miss the most useful thing a reviewer can say:
+*why* the test passed anyway. A fixture that happens to dodge the bug, a table that was empty
+on the laptop, a test that runs the job once when the bug only shows on the second run, an
+assertion that one bug cancels out. For each bug you find, say whether the cited tests or run
+could have caught it, and if not, why it passed and the one test that would have failed:
+✅ *"`test_refill` passes because it waits 2 s against a rate of 1/s, so it never sees the
+partial-token case. Assert `allow()` is false at 0.5 s."*
+
+A second rule: **report only what survives checking.** Before a finding goes in, trace it
+through the code. Does an earlier guard already handle the case? Is the query actually
+parameterised? Does the stated business rule make this behaviour correct? Does your own fix
+do what you say it does? A false finding costs the reader more than a missed nit.
 
 ## How to respond
 
-**Four rules come before the steps and outrank them:**
-
-- **Answer what was asked, first.** Open with the answer, the decision or the artifact, in plain words. Scores, matrices, frameworks and tags come after it, and anything that doesn't change the answer is cut.
-- **Use only what you were given.** Don't add facts, names, numbers, incidents, history, steps or sections the input doesn't contain. What you need and don't have is left out of the artifact and listed once at the end under "To confirm".
-- **Deliver from what you have.** When the request gives you something to work on, state your assumptions in a line and produce the result. When it gives you nothing to work on, ask for it in one or two plain sentences and say what you will do once you have it.
-- **Write for someone who has never heard of this skill.** No skill names, no paths or scripts from this folder, no internal terms, and nothing about how the run was set up. A next step is one plain sentence at the end that describes the work.
-- **For this skill:** With no code or document to review, ask for it in one sentence and name the three or four things you will check in it, taken from what the user said it does.
-
-1. **Identify the change under review.** In order of preference:
-   - The agent / user has just shown you a diff — review that diff.
-   - Git working tree has unstaged changes — `git diff` is the scope.
-   - User points to specific files / a PR / a recent commit — that's the scope.
-   - User says "the feature I just built" with no diff — ask which files, or run `git diff HEAD~1` if a commit was just made.
-
-   **Don't review the whole repo.** The scope is the *just-generated* code. Touched files only.
-
-2. **Classify the code shape.** What you challenge depends on what was built. Map to one or more:
-
-   | Shape | Lens emphasis |
-   |---|---|
-   | HTTP / RPC endpoint, API handler | edge-cases (input validation, auth), adversarial (rate limit, error mapping) |
-   | Async handler / queue consumer | edge-cases (retries, idempotency, poison messages), future-proofing (ordering) |
-   | DB migration / schema change | future-proofing (rollback, online-safe), adversarial (locking, big-table risk) |
-   | State machine / workflow | edge-cases (impossible transitions, concurrent updates) |
-   | Parser / data transformation | edge-cases (malformed, encoding, huge inputs) |
-   | CLI tool / script | edge-cases (missing flags, env, partial runs) |
-   | UI component | edge-cases (loading/empty/error states, a11y) |
-   | Background job / cron | edge-cases (overlap, missed runs, partial completion) |
-   | Library / pure function | edge-cases (boundary inputs), future-proofing (API stability) |
-   | Config / infra change | adversarial (blast radius), future-proofing (drift) |
-
-   Multiple shapes is common — a PR can be "endpoint + migration + UI." Apply each lens to the right slice.
-
-3. **Sweep the four lenses in order.** Each lens has its own checklist — read it, apply it, write findings. Don't merge lenses; each surfaces a different category of risk.
-
-   | Lens | File | Asks |
-   |---|---|---|
-   | 1. Edge cases | [`lenses/edge-cases.md`](lenses/edge-cases.md) | What inputs / states / timings break this? |
-   | 2. Future-proofing | [`lenses/future-proofing.md`](lenses/future-proofing.md) | What assumptions won't survive the next 6 months? |
-   | 3. Adversarial review | [`lenses/adversarial-review.md`](lenses/adversarial-review.md) | What would a staff engineer push back on in review? |
-   | 4. Test coverage | [`lenses/test-coverage.md`](lenses/test-coverage.md) | Which new code paths have no test, and which scenarios are missing tests? |
-
-4. **Categorize every finding by severity:**
-
-   | Severity | Meaning |
-   |---|---|
-   | 🟥 **Blocker** | This will ship a bug or a security issue. Examples: SQL injection, unhandled `null`, race condition on shared state, migration that locks a 10M-row table, missing auth check, infinite retry loop. |
-   | 🟧 **Major** | This will hurt within months. Examples: hard-coded limits that 2× growth will hit, no idempotency on a retryable operation, error swallowed silently, no observability for a load-bearing path, magic constants. |
-   | 🟨 **Minor** | Good craft. Examples: missing edge-case test, suboptimal log level, naming that won't age well, a comment that explains *what* instead of *why*. |
-   | ⚪ **Nit** | Style, formatting, micro-preference. Optional. |
-
-   When in doubt between two severities, **downgrade**. Inflated severity makes the report unreadable. If your output has more blockers than major findings, you're inflating.
-
-5. **For every finding, write four parts:**
-
-   - **Where:** `file:line` (e.g. `src/api/checkout.ts:47`) — and quote 5–15 chars of the offending line.
-   - **What's wrong:** one sentence — which lens fired and why.
-   - **Concrete scenario:** ✅ "When `discount=null` reaches line 47, `applyDiscount` throws `TypeError`." — ❌ "Doesn't handle null." A scenario the first pass can reproduce.
-   - **Fix:** ✅ "Add `if (discount == null) return 0;` at the top of `applyDiscount` — or thread the null through to skip the line item entirely (preferred — see how `applyTax` handles it at line 102)." — ❌ "Add null handling."
-
-6. **Open with a verdict, not the findings.** First thing the user sees:
-
-   ```
-   Verdict: SHIP WITH FIXES — 2 blockers, 5 major, 4 minor, 3 nits.
-
-   The endpoint validates auth and the happy path but doesn't bound the request body (blocker — DoS via 1GB payload), and the new migration adds a NOT NULL column without a default on a table the migration script doesn't backfill (blocker — will fail on first deploy). Of the 5 major findings, 3 are around idempotency on the retry path. Test coverage misses the new error branches at api/checkout.ts:71–88 entirely.
-   ```
+1. **Answer the question first.** Open with a plain answer to what they asked, usually
+   whether it's safe to merge, and the worst problem in one sentence.
 
-   Verdicts: `SHIP IT` · `SHIP WITH FIXES` · `DO NOT MERGE` · `RECONSIDER APPROACH`. The last is for when the implementation works but solved the wrong problem (e.g. building a custom retry queue when the framework already has one).
+2. **Review the change, not the repo.** The diff, the files they named, or `git diff` of the
+   working tree. Read the tests that came with it.
 
-7. **Output structure** — use [`templates/challenge.md`](templates/challenge.md):
+3. **Find what would bite, most consequential first.** Wrong results, data loss, security
+   holes, failures on real data that the dev setup hides, retries and concurrency, then
+   fragility. For a large or unfamiliar change, the checklists in
+   [`lenses/edge-cases.md`](lenses/edge-cases.md),
+   [`lenses/future-proofing.md`](lenses/future-proofing.md),
+   [`lenses/adversarial-review.md`](lenses/adversarial-review.md) and
+   [`lenses/test-coverage.md`](lenses/test-coverage.md) are a sweep to run privately; they are
+   not sections of the reply.
 
-   ```markdown
-   # Devil's advocate: {feature name / PR title}
-   _Scope: {N files, M lines changed} · Reviewed: {YYYY-MM-DD}_
+4. **Each finding:** where (`file:line`), what goes wrong in one concrete scenario ("when
+   `tz` is `None`, line 47 raises `TypeError`"), and the fix. When the fix is small,
+   give it as code they can paste. When the change is a script or migration, give the
+   corrected version.
 
-   ## Verdict
-   {one-liner} — {N} blocker · {N} major · {N} minor · {N} nit
-   {3-sentence summary, lead with the worst}
+5. **Say what the tests prove.** Apply the rule to every test or run the user cites, and name
+   the missing tests as scenarios: ✅ *"No test runs the job twice; add one asserting the reminder
+   email is sent once."*
 
-   ## Findings by lens
+6. **Size to the ask.** "Top 3" gets three. "Skip the nits" gets none. "Just security" gets
+   security. Without a limit, report what would actually bite; leave style out unless asked.
+   No severity counts, verdict labels or praise sections unless they asked.
 
-   ### 1. Edge cases
-   ### 2. Future-proofing
-   ### 3. Adversarial review
-   ### 4. Test coverage
+7. **Invent nothing about the codebase.** No row counts, traffic, run times, callers,
+   column contents or config you weren't shown. Where a finding depends on one, say what to
+   check: "if `orders` already has rows in production, this migration fails".
 
-   ## What's solid
-   - {2–4 things the implementation got right — not flattery, real patterns to preserve}
+8. **Deliver from what you have.** With code to review, review it. With nothing, ask for the
+   diff in one sentence and name the three or four things you'll check in it.
 
-   ## Suggested order to address
-   1. Fix blockers
-   2. Add the missing tests in §4 (they double as regression catches)
-   3. Major findings in priority order
-   ```
+## Useful references in this skill
 
-8. **End with "what's solid"**, not more criticism. The user (or the next agent) often needs to know what to *keep* and what *patterns to reuse* elsewhere in the change.
+- [`examples/rate-limiter-review.md`](examples/rate-limiter-review.md) — a change with "tests pass", and a review that says why they passed
 
 ## Quality bar
 
-- **Every finding has a reproducible scenario.** A senior engineer reading the finding should be able to reproduce the bug or violation in their head. "Doesn't handle null" is not a finding; "When `discount=null` reaches line 47, …" is.
-- **Severity is calibrated.** More blockers than major ⇒ inflating ⇒ demote.
-- **Findings cite `file:line`.** Reviewers must be able to jump to the spot. Section names aren't enough for code.
-- **Fixes are concrete and paste-able where possible.** When you can suggest a one-line patch, do; when you can't, suggest the *approach* in one sentence — not "improve error handling."
-- **Don't redesign.** This isn't your chance to push a different architecture. Stay in the lane of "given this approach, what's missing." If the approach itself is wrong, that's a single `RECONSIDER APPROACH` verdict, not a wholesale rewrite suggestion.
-- **Test-coverage findings reference scenarios, not files.** ❌ "No test for `applyDiscount`." ✅ "No test covers `applyDiscount(null)` — add a case that asserts the function returns 0 (or skips) without throwing."
+- **Every cited test or run is judged:** could it have caught each bug, and if not, why did it
+  pass and what test would fail.
+- **Every finding survives a trace through the code and the stated rules.** No false
+  positives.
+- **The answer to "can this merge?" comes first;** findings are ordered by consequence.
+- **Fixes work,** and are code when the change is small.
+- **The scope asked for is respected.**
+- **Nothing about the codebase is invented.**
 
 ## When NOT to use this skill
 
-- ❌ For greenfield prototype / spike code that's about to be thrown away.
-- ❌ For one-line / trivial changes — overkill, and the noise drowns out real signal.
-- ❌ As a substitute for human code review — it's a pre-pass, not a replacement.
-- ❌ For *evaluating the choice of approach* before code exists. That's a design conversation, not a code challenge. Use `design-doc` + `doc-critique` for that loop.
-- ❌ For generated code in a language you can't reason about line-by-line. Better to refuse than fabricate findings.
+- ❌ For greenfield prototype or spike code that's about to be thrown away.
+- ❌ For one-line or trivial changes.
+- ❌ As a substitute for human code review — it's a pre-pass.
+- ❌ For *evaluating the choice of approach* before code exists. That's a design conversation: `design-doc`, then `doc-critique`.
+- ❌ For code in a language you can't reason about line by line. Refuse rather than fabricate findings.
 
 ## Anti-patterns to avoid
 
-- ❌ **Severity inflation.** If everything is a blocker, the report is unreadable. Most findings are major or minor.
-- ❌ **Vague findings.** "Handle errors better" / "Consider edge cases" — the whole point of this skill is to *name* the missed cases. If you can't name a specific scenario, drop the finding.
-- ❌ **Restating the lens.** ❌ "Code should be future-proof." ✅ "Hard-coded 100ms timeout on line 53 won't survive p99 growth — make it configurable."
-- ❌ **Lecturing on style** when there's a real bug to flag. Style notes go in the Nits section, not on top.
-- ❌ **Reviewing untouched code.** Out-of-scope. The skill challenges the *change*, not the codebase.
-- ❌ **Demanding tests for *everything* the change touches.** Only the new logic and new error paths. Don't insist on backfilling tests for previously-untested neighbouring code.
-- ❌ **Skipping "what's solid".** Trains the user (or downstream agent) to dread the skill. End on signal that's actionable in the other direction.
-- ❌ **Showing the machinery.** The reply and the artifact are for the person who asked. Don't mention this skill, its files, templates, caps or internal terms, or that the run is non-interactive. Name another skill once, at the end, as a suggested next step, never inside the artifact.
+- ❌ **"Tests pass" taken at face value,** or ignored. Say what they prove.
+- ❌ **The plausible false positive.** SQL injection in a parameterised query; a bug in
+  behaviour the business rule asks for; a CRLF issue text mode already handles.
+- ❌ **Padding to fill sections.** Marginal findings are where the errors are.
+- ❌ **A count that disagrees with the list.** Don't print counts.
+- ❌ **Describing a fix instead of giving it** when it fits in a few lines.
+- ❌ **Facts about production you weren't given.** Row counts, lock times, what a column holds.
+- ❌ **Redesigning.** Stay on what's missing given this approach.

package/packs/dev-team/skills/devils-advocate/evals/evals.json

@@ -72,6 +72,33 @@
         "It does not state the row count of orders, how long the migration will take, or what customers.region contains as fact",
         "It leads with three main concerns as asked, and anything beyond the three is kept to a brief mention"
       ]
+    },
+    {
+      "id": 6,
+      "held_out": "2026-10-17",
+      "prompt": "I'm the reviewer on PR #412 for the Fernlocket loyalty service and it's meant to merge this afternoon. The change is in redeem.py, the tests are in test_redeem.py, and the author's description and test run are in PR_DESCRIPTION.md. Find what's wrong with it before it ships. Give me the three issues that matter most, most serious first, and skip style and naming nits entirely.",
+      "assertions": [
+        "It finds that quantity is only checked against the upper limit, so a negative quantity gives a negative cost that passes the balance check and adds points to the member's balance, and puts this among its top issues",
+        "It finds that the points are deducted and committed before fulfillment.ship() is called, so a failed shipment leaves the member charged with no refund, and that the \"Please try again\" message invites a second deduction for the same reward",
+        "It does not report the f-string query in list_redemptions as SQL injection, since the ORDER BY text can only come from the fixed SORT_COLUMNS mapping and the values are bound parameters, and it does not report the balance read-then-update as a double-spend race, since the row is locked with FOR UPDATE inside the same transaction",
+        "It says test_fulfillment_failure_returns_502 would pass with the bug in place, because it checks only the status and call count and never the member's balance, so the 6-passed run does not show that #398 is fixed",
+        "Its fixes would work: quantity must be required to be at least 1 (a tighter upper limit is not enough), and on a failed shipment the points must be rolled back or credited back, or not committed until the shipment succeeds; logging the failure or changing the status code alone does not count",
+        "It does not state as fact anything the files don't show: how fulfillment.ship() or the transaction helper behave beyond what the code and comments say, whether the fulfillment service rejects a negative quantity, how many members or redemptions there are, or what else calls redeem()",
+        "It leads with three issues, most serious first, and makes no style or naming comments; anything beyond the three is at most a brief mention"
+      ]
+    },
+    {
+      "id": 7,
+      "held_out": "2026-10-17",
+      "prompt": "hey quick one, can you check this before I push? it sends the weekly digest to all subscribers, max 8 at a time. we're on go 1.23. go test ./... is green and I ran it on staging against a handful of test subscribers, all delivered fine. just tell me if anything's actually broken, I don't care about style\n\n```go\nfunc SendAll(ctx context.Context, subs []Subscriber, d Digest) error {\n\tsem := make(chan struct{}, 8) // at most 8 pushes in flight\n\tvar wg sync.WaitGroup\n\tvar errs []error\n\n\tfor _, s := range subs {\n\t\twg.Add(1)\n\t\tsem <- struct{}{}\n\t\tgo func() {\n\t\t\tdefer wg.Done()\n\t\t\tdefer func() { <-sem }()\n\t\t\tif err := push(ctx, s, d); err != nil {\n\t\t\t\terrs = append(errs, fmt.Errorf(\"subscriber %s: %w\", s.ID, err))\n\t\t\t}\n\t\t}()\n\t}\n\n\twg.Wait()\n\treturn errors.Join(errs...)\n}\n```",
+      "assertions": [
+        "It finds that the goroutines append to the shared errs slice with no lock, a data race that can lose errors or corrupt the slice when more than one push fails",
+        "It does not flag the closure capturing the loop variable s as a bug, since on Go 1.22 and later each loop iteration gets its own s",
+        "It says the green test run and the staging run don't show the code is safe: with every push succeeding the append never runs, and a plain go test without -race wouldn't report the race anyway",
+        "Its fix would work: a mutex around the append, or a per-subscriber slot in a slice sized to len(subs); if it suggests errgroup it says that this changes the behaviour (only the first error comes back, and with a group context the rest of the sends are cancelled)",
+        "It does not state as fact what push() does inside (retries, timeouts, how it uses ctx), how many subscribers there are, or that this has already failed in production",
+        "It keeps to what's actually broken, as asked: a short reply with no style or naming suggestions, not padded out with a long list of minor or hypothetical concerns"
+      ]
     }
   ]
 }

package/packs/dev-team/skills/devils-advocate/evals/files/6/PR_DESCRIPTION.md

@@ -0,0 +1,20 @@
+## Synchronous reward redemption (#412)
+
+Members currently wait for the nightly batch to find out whether a redemption went through.
+This moves redemption onto a synchronous endpoint so they get an answer straight away.
+
+- `redeem()` backs `POST /members/{member_id}/redemptions`
+- `list_redemptions()` backs the history screen, with a `sort` option
+- The balance row is locked with `SELECT ... FOR UPDATE`, so a double tap on "Redeem" can't spend the same points twice
+- A failed shipment returns 502 and asks the member to try again; this closes #398 and is covered by `test_fulfillment_failure_returns_502`
+- Per-order cap of 10 so nobody can clear out a reward's stock in one request
+
+### Test run
+
+```
+$ pytest tests/test_redeem.py -q
+......                                                                   [100%]
+6 passed in 0.41s
+```
+
+Ready for review. I'd like to get this into Thursday's release.

package/packs/dev-team/skills/devils-advocate/evals/files/6/redeem.py

@@ -0,0 +1,102 @@
+"""Reward redemption for the Fernlocket loyalty programme.
+
+PR #412: move redemption off the nightly batch onto a synchronous endpoint.
+"""
+import logging
+import uuid
+
+from .db import transaction
+from .fulfillment import FulfillmentClient, FulfillmentError
+
+log = logging.getLogger(__name__)
+
+MAX_PER_ORDER = 10
+
+SORT_COLUMNS = {
+    "newest": "created_at DESC",
+    "oldest": "created_at ASC",
+    "points": "points_spent DESC",
+}
+
+
+class RedemptionError(Exception):
+    def __init__(self, status: int, message: str):
+        super().__init__(message)
+        self.status = status
+        self.message = message
+
+
+def redeem(conn, fulfillment: FulfillmentClient, member_id: str, payload: dict) -> dict:
+    """Spend a member's points on a reward and ship it.
+
+    Called by POST /members/{member_id}/redemptions with the JSON body as `payload`.
+    """
+    reward_id = payload["reward_id"]
+    quantity = int(payload.get("quantity", 1))
+    if quantity > MAX_PER_ORDER:
+        raise RedemptionError(400, f"at most {MAX_PER_ORDER} of a reward per order")
+
+    with transaction(conn) as cur:
+        cur.execute(
+            "SELECT points_cost, active FROM rewards WHERE id = %s",
+            (reward_id,),
+        )
+        row = cur.fetchone()
+        if row is None or not row[1]:
+            raise RedemptionError(404, "reward not available")
+        cost = row[0] * quantity
+
+        # Lock the member's balance row so two taps on "Redeem" can't both spend it.
+        cur.execute(
+            "SELECT balance FROM point_accounts WHERE member_id = %s FOR UPDATE",
+            (member_id,),
+        )
+        balance = cur.fetchone()[0]
+        if balance < cost:
+            raise RedemptionError(409, "not enough points")
+
+        redemption_id = str(uuid.uuid4())
+        cur.execute(
+            "UPDATE point_accounts SET balance = %s WHERE member_id = %s",
+            (balance - cost, member_id),
+        )
+        cur.execute(
+            "INSERT INTO redemptions (id, member_id, reward_id, quantity, points_spent, status)"
+            " VALUES (%s, %s, %s, %s, %s, 'pending')",
+            (redemption_id, member_id, reward_id, quantity, cost),
+        )
+    # transaction commits on leaving the block
+
+    try:
+        fulfillment.ship(redemption_id, reward_id, quantity)
+    except FulfillmentError:
+        log.exception("fulfillment failed for redemption %s", redemption_id)
+        raise RedemptionError(502, "We couldn't send your reward. Please try again.")
+
+    with transaction(conn) as cur:
+        cur.execute(
+            "UPDATE redemptions SET status = 'shipped' WHERE id = %s",
+            (redemption_id,),
+        )
+
+    return {
+        "redemption_id": redemption_id,
+        "points_spent": cost,
+        "balance": balance - cost,
+    }
+
+
+def list_redemptions(conn, member_id: str, sort: str = "newest", limit: int = 20) -> list:
+    """Redemption history for the member's account screen."""
+    order_by = SORT_COLUMNS.get(sort)
+    if order_by is None:
+        raise RedemptionError(400, "unknown sort")
+    limit = max(1, min(int(limit), 100))
+
+    with transaction(conn) as cur:
+        cur.execute(
+            "SELECT id, reward_id, quantity, points_spent, status, created_at"
+            f" FROM redemptions WHERE member_id = %s ORDER BY {order_by} LIMIT %s",
+            (member_id, limit),
+        )
+        return cur.fetchall()

package/packs/dev-team/skills/devils-advocate/evals/files/6/test_redeem.py

@@ -0,0 +1,54 @@
+import pytest
+
+from rewards.fulfillment import FulfillmentError
+from rewards.redeem import RedemptionError, list_redemptions, redeem
+from tests.fakes import FakeFulfillment, make_db
+
+
+@pytest.fixture
+def db():
+    conn = make_db()
+    conn.seed_member("m-100", balance=500)
+    conn.seed_reward("r-mug", points_cost=120, active=True)
+    conn.seed_reward("r-old-tote", points_cost=50, active=False)
+    return conn
+
+
+def test_happy_path(db):
+    result = redeem(db, FakeFulfillment(), "m-100", {"reward_id": "r-mug", "quantity": 2})
+    assert result["points_spent"] == 240
+    assert db.balance("m-100") == 260
+
+
+def test_not_enough_points(db):
+    with pytest.raises(RedemptionError) as err:
+        redeem(db, FakeFulfillment(), "m-100", {"reward_id": "r-mug", "quantity": 5})
+    assert err.value.status == 409
+    assert db.balance("m-100") == 500
+
+
+def test_inactive_reward(db):
+    with pytest.raises(RedemptionError) as err:
+        redeem(db, FakeFulfillment(), "m-100", {"reward_id": "r-old-tote"})
+    assert err.value.status == 404
+
+
+def test_quantity_limit(db):
+    with pytest.raises(RedemptionError) as err:
+        redeem(db, FakeFulfillment(), "m-100", {"reward_id": "r-mug", "quantity": 11})
+    assert err.value.status == 400
+
+
+def test_fulfillment_failure_returns_502(db):
+    """Regression for #398: a failed shipment must not cost the member their points."""
+    fulfillment = FakeFulfillment(fail_with=FulfillmentError("carrier timeout"))
+    with pytest.raises(RedemptionError) as err:
+        redeem(db, fulfillment, "m-100", {"reward_id": "r-mug", "quantity": 1})
+    assert err.value.status == 502
+    assert fulfillment.calls == 1
+
+
+def test_history_rejects_unknown_sort(db):
+    with pytest.raises(RedemptionError) as err:
+        list_redemptions(db, "m-100", sort="points; DROP TABLE redemptions")
+    assert err.value.status == 400