skilldrop-cli 0.16.31 → 0.16.33

package/packs/dev-team/skills/dependency-upgrade-review/SKILL.md

@@ -1,11 +1,11 @@
 ---
 name: dependency-upgrade-review
-description: Give a merge verdict on a dependency-upgrade pull request (Dependabot, Renovate or a hand-made bump) across npm, pip, Go, Cargo and Maven lockfiles and GitHub Actions pins. Reads the release notes for every skipped version, lists what the lockfile changes beyond the PR title (transitive bumps, new and removed packages, install scripts, registry changes), checks that each SHA pin is the commit its release tag points to, says whether CI ran the changed code, finds the repo's uses of changed APIs and writes the rollback, ending in MERGE, MERGE AFTER CHECKS or DO NOT MERGE. Use when the user says "review this Dependabot PR", "is this Renovate bump safe to merge", "can I merge this dependency upgrade", "check this lockfile diff", or "is this action SHA pin right".
+description: Give a merge verdict on a dependency-upgrade pull request (Dependabot, Renovate or a hand-made bump) across npm, pip, Go, Cargo and Maven lockfiles and GitHub Actions pins. Reads the release notes for every skipped version, lists what the lockfile changes beyond the PR title (transitive bumps, new and removed packages, install scripts, registry changes), checks each SHA pin against its release tag and against the PR's other pins, checks raised runtime floors against the images and runners that install it, says whether CI ran the changed code, traces what each caller of a changed API now does and writes the rollback. The reply leads with MERGE, MERGE AFTER CHECKS or DO NOT MERGE and the reasons that decide it. Use when the user says "review this Dependabot PR", "is this Renovate bump safe to merge", "can I merge this dependency upgrade", "check this lockfile diff", or "is this action SHA pin right".
 ---
 
 # dependency-upgrade-review
 
-A dependency bump is someone else's code entering the repo under a one-line title. This skill reads it the way a careful maintainer does: every release between the old and new version, every line of the lockfile, every pin, and the tests that would have caught a break. The verdict follows from those checks, not from a green badge. Reviewing the repo's own code changes is `pre-merge-review`; writing the notes for a release of this project is `release-notes`; planning a framework migration across many PRs is `migration-plan`.
+A dependency bump is someone else's code entering the repo under a one-line title. This skill reads it the way a careful maintainer does: every release between the old and new version, every line of the lockfile, every pin, every place the code installs and runs, and the tests that would have caught a break. The verdict follows from those checks, not from a green badge. The reply is short: the verdict and what decides it come first, and the checks that found nothing take one line. Reviewing the repo's own code changes is `pre-merge-review`; writing the notes for a release of this project is `release-notes`; planning a framework migration across many PRs is `migration-plan`.
 
 ## How to respond
 
@@ -15,11 +15,13 @@ A dependency bump is someone else's code entering the repo under a one-line titl
 - **Use only what you were given.** Don't add facts, names, numbers, incidents, history, steps or sections the input doesn't contain. What you need and don't have is left out of the artifact and listed once at the end under "To confirm".
 - **Deliver from what you have.** When the request gives you something to work on, state your assumptions in a line and produce the result. When it gives you nothing to work on, ask for it in one or two plain sentences and say what you will do once you have it.
 - **Write for someone who has never heard of this skill.** No skill names, no paths or scripts from this folder, no internal terms, and nothing about how the run was set up. A next step is one plain sentence at the end that describes the work.
-- **For this skill:** A check nobody ran is an open check, written as the command that runs it. It is never reported as passed, and "CI is green" closes only the checks CI actually ran.
+- **For this skill:** A check nobody ran is an open check, written as the command that runs it. It is never reported as passed, and "CI is green" closes only the checks CI actually ran. Say what was not checked and the command that checks it; never narrate the session (what was blocked, denied or unavailable).
 
-1. **Get the diff, the notes and the code, asking once.** Three inputs: the PR diff covering manifests, lockfiles and workflow files (`gh pr diff 212` or `git diff origin/main...HEAD`); the release notes or changelog for every version between old and new; the repo, or at least the files that import the package and the CI workflow files. Ask for what is missing in one message. With network access, fetch missing notes from the package's releases page or changelog; without it, each missing release becomes an open check.
+Steps 1 to 10 are the review. Step 11 is the reply, and most of what the review turns up stays out of it.
 
-2. **Run the diff script first.** It lists every changed dependency with old → new, semver class, direct or transitive, and flags added and removed packages, install hooks, native-code hints, registry changes, licence changes and pins, then prints a floor verdict.
+1. **Get the diff, the notes and the code, asking once.** Three inputs: the PR diff covering manifests, lockfiles and workflow files (`gh pr diff 212` or `git diff origin/main...HEAD`); the release notes or changelog for every version between old and new; the repo, or at least the files that import the package and the CI workflow files. Also read what decides where the code installs and runs: the Dockerfile or base image, every workflow that installs dependencies (release, deploy and scheduled ones too, not only the PR's), `.nvmrc` / `.python-version` / `engines`, and any note in the PR body of the runtime the lockfile was generated on. When the PR has more than one commit, note who made each; a hand-made commit on a bot's PR is read first. Ask for what is missing in one message. With network access, fetch missing notes; without it, each missing release becomes an open check.
+
+2. **Run the diff script first.** It lists every changed dependency with old → new, semver class, direct or transitive, and flags added and removed packages, install hooks, native-code hints, registry changes, licence changes, runtime requirement changes, pins, and one tag pinned to different commits, then prints a floor verdict.
    ```bash
    # Claude Code
    python3 "${CLAUDE_SKILL_DIR}/scripts/upgrade_diff.py" pr.diff --title "Bump quillmark from 3.4.1 to 4.1.0"
@@ -28,50 +30,65 @@ A dependency bump is someone else's code entering the repo under a one-line titl
    ```
    It reads stdin with `-` (`gh pr diff 212 | python3 scripts/upgrade_diff.py -`). If it reports version lines out of context, re-run on `git diff -U12`. If no script can run, read every lockfile hunk by hand with the per-ecosystem notes in [`reference.md`](reference.md).
 
-3. **List everything the lockfile changes beyond the title.** The title names one package; the lockfile is the truth. Report each of: transitive bumps (with class), packages added, packages removed, new install scripts (`hasInstallScript: true`, a `preinstall` / `install` / `postinstall` / `prepare` hook, pnpm `requiresBuild: true`), new native or binary dependencies (`prebuild-install`, `node-gyp`, a Cargo `-sys` crate), a changed `resolved` / `source` / index URL, and a maintainer or publisher change when the input shows one. ✅ *"Also changes: `entity-codec` 2.1.0 → 3.0.2 (major, licence MIT → MPL-2.0), adds `quill-native-highlight` 1.0.0 with an install script, removes `html-escape-lite`."* ❌ *"Bumps quillmark to 4.1.0."*
+3. **List everything the lockfile changes beyond the title.** The title names one package; the lockfile is the truth. Find each of: transitive bumps (with class), packages added, packages removed, new install scripts (`hasInstallScript: true`, a `preinstall` / `install` / `postinstall` / `prepare` hook, pnpm `requiresBuild: true`), new native or binary dependencies (`prebuild-install`, `node-gyp`, a Cargo `-sys` crate, wheels for only some platforms), a changed `resolved` / `source` / index URL, a raised `engines` or `python-versions`, and a maintainer or publisher change when the input shows one. ✅ *"Also changes: `entity-codec` 2.1.0 → 3.0.2 (major, licence MIT → MPL-2.0), adds `quill-native-highlight` 1.0.0 with an install script, removes `html-escape-lite`."* ❌ *"Bumps quillmark to 4.1.0."*
 
 4. **Class each jump.** Major, minor or patch from the version numbers. Below 1.0 the minor is the breaking number: 0.8.3 → 0.9.0 is a breaking jump, and below 0.1 so is the patch. A Go module path gaining `/v2` is a new module, not a bump. A prerelease target (`-rc.1`, `-beta`) or a downgrade is never a plain MERGE. A moving major tag (`@v2` → `@v3`) spans every v2.x after the one in use plus v3.0.0 to the current v3.
 
-5. **Read every release between old and new, not only the target.** For each direct dependency, list every release after the old version up to and including the new one, and pull four things from each: breaking changes, deprecations, security fixes, licence changes. Transitive majors, 0.x breaking jumps and new packages get the same reading; transitive patch and minor bumps are listed, not read. Name each release whose notes are missing. ✅ *"3.5.0 fixes an attribute-injection bug in link titles and deprecates `setOptions()`; 4.0.0 removes `renderInline()` and turns `sanitize` off by default; 4.0.1 and 4.1.0 are fixes and an optional highlighter."* ❌ *"4.1.0 adds a native highlighter. Looks fine."* A security fix is a reason to take the upgrade; when the full jump is blocked, name the smallest version that carries the fix as the alternative.
+5. **Read every release between old and new, not only the target.** For each direct dependency, read every release after the old version up to and including the new one, and pull five things from each: breaking changes, dropped runtime versions, deprecations, security fixes, licence changes. Transitive majors, 0.x breaking jumps and new packages get the same reading; transitive patch and minor bumps are listed, not read. Name each release whose notes are missing. Copy version numbers exactly as the notes give them: "deprecated since 2.2, warns from 2.5.0" is two facts. Note any migration aid an intermediate release offers (an opt-in flag, a compatibility mode); it is the staged path. A security fix is a reason to take the upgrade; when the full jump is blocked, name the smallest version that carries the fix as the alternative.
+
+6. **Search for every changed API, then trace each caller.** For each removed or renamed function, changed signature, changed default, changed return value and changed error, search and report `file:line` for each hit, or "no hits". Give the search (`git grep -n -E "renderInline|setOptions|render\(" -- '*.js' '*.ts'`); with no repo in hand, give that command as an open check and claim neither hits nor no hits. Then read the calling function and say what it does under the new behaviour, and the fix in this repo's code:
+   - A return value that became an exception skips every `if result is None` / `== -1` branch and escapes the retry or fallback built on it.
+   - A tuple or dict that became an object breaks the unpacking or the key access.
+   - A changed default hits every call that doesn't pass the option.
+   - An import inside a function fails only when that function runs, never at import time or test collection.
+   ✅ *"`load_profile` (`app/cache.py:31`) falls back to the database when `cache.get()` returns `None`; from 5.0.0 `get()` raises `CacheMiss`, so every miss escapes as an error and the fallback never runs. Catch `CacheMiss` there, or move to 4.6 first with `strict_miss=True`."* ❌ *"`get()` behaviour changed in 5.0.0; check usages."*
 
-6. **Search the repo for every API the notes change.** For each removed or renamed function, changed signature and changed default, search and report `file:line` for each hit, or "no hits". A changed default hits every call that doesn't pass the option. ✅ *"`src/render/note.js:7` calls `quillmark.render(markdown)` with no `sanitize`, so from 4.0.0 note bodies pass raw HTML through."* ❌ *"Check whether the code uses `renderInline`."* Give the search you ran (`git grep -n -E "renderInline|setOptions|render\(" -- '*.js' '*.ts'`); with no repo in hand, give that command as an open check and claim neither hits nor no hits.
+7. **Check every runtime floor against where the code runs.** A raised minimum (Python `>=3.10`, Node `engines >=20`, a Go directive, an MSRV, wheels built only for some interpreter versions, an action needing a newer runner) is compared with the Dockerfile `FROM`, each workflow's `setup-*` version and matrix, `.nvmrc` / `.python-version`, self-hosted runner versions, and the runtime the lockfile was generated on. CI and the lock tool running a newer runtime than production is how a green PR breaks the image build. ✅ *"`deploy/Dockerfile:1` builds on `node:18-alpine` and 6.0 needs Node >=20; CI runs Node 22 (`ci.yml:14`), so the PR passed and the image build will fail at `npm ci`."*
 
-7. **Verify every pin; an unverified SHA is an open check.** For a SHA-pinned action (or any dependency pinned to a commit), the pinned SHA must be the commit the release tag points to:
+8. **Verify every pin the PR changes; an unverified SHA is an open check.** A SHA-pinned action (or any dependency pinned to a commit) must be the commit its release tag points to:
    ```bash
    git ls-remote https://github.com/<owner>/<repo> refs/tags/<tag> 'refs/tags/<tag>^{}'
    ```
-   For an annotated tag the `^{}` line is the commit; the other line is the tag object. For a lightweight tag the single line is the commit. Not run means `UNVERIFIED`, never "looks right": a SHA from a fork of the repo also resolves under the upstream path. A SHA with no version comment is an unknown release. A tag-only pin (`@v3`) moved to another tag is weaker than a SHA, because the tag can be moved after review; say so and give the command that finds the SHA to pin. Say what the job hands the action: a `secrets.*` token, `id-token: write` or `contents: write` raises the stakes of every pin in that job.
+   For an annotated tag the `^{}` line is the commit; for a lightweight tag the single line is. Not run means unverified, never "looks right": a SHA from a fork also resolves under the upstream path. Compare the PR's pins with each other: one action and tag pinned to two different SHAs means at most one is the tag's commit, and the suspect is the one changed by hand or in the job with secrets. A SHA with no version comment is an unknown release. A tag-only pin (`@v3`) is weaker than a SHA, because the tag can be moved after review. Say what the job hands the action: a `secrets.*` token, `id-token: write` or `contents: write` raises the stakes. Pins the PR does not change are not part of this review.
 
-8. **Say whether CI ran the changed code.** A green check proves what it ran and nothing else. For each changed package or action, name the workflow job and test file that exercise it, or say nothing does. The usual gaps: an action or tool used only in a workflow triggered by a tag push, a release, a schedule or `workflow_dispatch`, which the PR never ran; a library used only in a file no test imports (a step 6 hit with no test); a test that runs the code but not the changed behaviour. ✅ *"`ci / test` runs `test/note.test.js`, which renders Markdown with no raw HTML, so the `sanitize` change is untested; nothing imports `src/render/comment.js`."* ❌ *"All checks pass."*
+9. **Say whether CI ran the changed code.** A green check proves what it ran and nothing else. For each changed package or action, name the workflow job and test that exercise it, or say nothing does. The usual gaps: a workflow triggered only by a tag push, a release, a schedule or `workflow_dispatch`; a file no test imports (a step 6 hit with no test); a mock that returns the old behaviour (a fake client that never times out tests the old contract); a PR job that installs differently from the release job (`npm ci --ignore-scripts` on the PR, plain `npm ci` in release means a new install script first runs in the job holding the secrets). ✅ *"`ci / test` runs `test/note.test.js`, which renders Markdown with no raw HTML, so the `sanitize` change is untested; nothing imports `src/render/comment.js`."* ❌ *"All checks pass."*
 
-9. **Write the rollback.** The exact revert: `git revert` of the merge commit, or the pin-back for the ecosystem (`npm install quillmark@3.4.1`, `poetry add tidyparse@2.2.0`, `go get <module>@v0.8.3 && go mod tidy`, `cargo update -p <crate> --precise <old>`, the old `uses:` line) with the lockfile regenerated and committed. Then name what a revert does not undo: a lockfile format migration, a data, cache or schema format the new version wrote, an artifact built or published with it (a registry version cannot be republished), and a security fix that rolling back reopens. When the old version sits inside a fixed advisory's range, the pin-back target is the first fixed version, not the old one: ✅ *"pin back to 3.5.0"* ❌ *"pin back to 3.4.1"* when 3.5.0 fixed a bug affecting 3.0.0 to 3.4.x. Per-ecosystem commands are in [`reference.md`](reference.md).
-
-10. **Derive the verdict from the checks.** Exactly one of:
-    - **DO NOT MERGE** when any holds: a removed or changed API is still used and the PR does not fix it; a new install script or native binary download nobody has confirmed is expected; a dependency source or registry moved to an unexplained host; a SHA that does not match its tag; a pin weakened (SHA to tag, tag to branch); a licence the project cannot ship; a target version an advisory names as vulnerable; a red CI check.
-    - **MERGE AFTER CHECKS** when none of those holds but any of these does: an unverified SHA; a major or 0.x breaking jump; a release whose notes were not read; changed code no CI job runs; a deprecated API still used; a new package or transitive major; a licence change that needs sign-off; a moved tag-only pin; no repo to search. Each check is a command or a concrete action.
+10. **Write the rollback, then derive the verdict.** Rollback: `git revert` of the merge commit, or the pin-back for the ecosystem (`npm install quillmark@3.4.1`, `poetry add tidyparse@2.2.0`, `go get <module>@v0.8.3 && go mod tidy`, the old `uses:` line and SHA) with the lockfile regenerated, plus what a revert does not undo (a lockfile format migration, data or cache the new version wrote, an artifact already published or deployed, a security fix that rolling back reopens; then pin back to the first fixed version). Per-ecosystem commands are in [`reference.md`](reference.md). Then exactly one verdict per PR:
+    - **DO NOT MERGE** when any holds: a removed or changed API is still used and the PR does not fix it; a runtime floor above what production or the release job runs; a new install script or native binary download nobody has confirmed is expected; a dependency source moved to an unexplained host; a SHA that does not match its tag, or two SHAs for one tag; a pin weakened (SHA to tag, tag to branch); a licence the project cannot ship; a target version an advisory names as vulnerable; a red CI check.
+    - **MERGE AFTER CHECKS** when none of those holds but any of these does: an unverified SHA; a major or 0.x breaking jump; a release whose notes were not read; changed code no CI job runs; a deprecated API still used; a new package or transitive major; a licence change that needs sign-off; a moved tag-only pin; no repo to search.
     - **MERGE** only when every check above is closed, with the evidence in the reply.
-    The script's floor is a minimum. Raise it on what steps 5 to 8 find. Lower DO NOT MERGE only when the user confirms the install script or new source is expected, and say so in the reply.
-
-11. **Reply in this order, in one message:** the verdict and the one reason that decides it; 🟥 blockers, 🟧 checks before merge and 🟨 notes, each with `file:line` or the command; a short table of what the PR changes, including everything beyond the title; the rollback; "To confirm". A routine PR fits in about 300 words. Lead with what blocks, not with the package history.
+    The script's floor is a minimum; raise it on what steps 5 to 9 find. Lower DO NOT MERGE only when the user confirms the install script or new source is expected, and say so.
+
+11. **Reply short, verdict first, in one message.** Only what changes the decision or the fix goes in.
+    - **First line:** the verdict and the one to three reasons that decide it, in plain words. With two PRs, one line each. ✅ *"DO NOT MERGE. 6.0 needs Node 20 and the production image runs 18, and two call sites break on removed APIs."* ❌ *"Here is my review of the quillmark upgrade."*
+    - **🟥 Blockers, numbered, the one that breaks first on top.** Each: what breaks, `file:line`, why the green check missed it, and the fix in this repo, in two to four lines. When an intermediate release offers a migration aid, one line after the blockers gives the staged path. The count in the first line matches the list.
+    - **🟧 Before merge:** each open check as the command or action that closes it. An unverified SHA is written as the command and the SHA it must equal.
+    - **Also in the lockfile:** one line naming every package added, removed or moved beyond the title, with install scripts and source changes called out. Not a table.
+    - **Rollback:** one or two lines, the command and what a revert leaves behind.
+    - **To confirm** only for what only the user can answer (is this install script expected?); omit it when empty.
+    - **Last line:** one sentence offering the next piece of work (the code fixes, the pin change).
+    No tables, no recited changelog, no pins or files the PR does not touch, no empty sections. A clean MERGE fits in about 120 words; a blocked PR with three blockers in about 300.
 
 **Non-interactive runs** (subagent, CI, headless): with no diff, emit `BLOCKED: need the PR diff covering the manifests, lockfiles and workflow files (gh pr diff <number>)`. Missing release notes, repo or CI files do not block: each becomes a named open check, the verdict is at best MERGE AFTER CHECKS, and the gap is stated first as `[assumption] notes for <versions> not read` or `[assumption] CI coverage unknown, treated as none`.
 
 ## Useful references in this skill
 
-- [`reference.md`](reference.md): reading each ecosystem's lockfile diff, where install scripts and sources show up, how to list the releases between two versions, SHA verification details, and per-ecosystem rollback commands with what a revert leaves behind
+- [`reference.md`](reference.md): reading each ecosystem's lockfile diff, where install scripts, sources and runtime floors show up, how to list the releases between two versions, SHA verification details, tracing behaviour changes, and per-ecosystem rollback commands with what a revert leaves behind
 - [`scripts/upgrade_diff.py`](scripts/upgrade_diff.py): stdlib parser for a unified diff of manifests, lockfiles and workflow files; prints changes, flags, pin checks and a floor verdict (exit 2 on bad input)
-- [`examples/ledgerline-tidyparse.md`](examples/ledgerline-tidyparse.md): worked example, a Poetry patch bump with a transitive bump, real script output and the review that reaches MERGE
+- [`examples/ledgerline-tidyparse.md`](examples/ledgerline-tidyparse.md): worked example, a Poetry patch bump that earns MERGE, and a blocked reply in the same shape
 
 ## Quality bar
 
-- **Exactly one verdict, first, and it follows from the checks.** Any unverified SHA, added install script, or removed API still in use rules out MERGE.
-- **Every release between old and new is accounted for:** read, with its breaking changes, deprecations, security fixes and licence changes named, or listed as unread.
-- **Everything the lockfile changes beyond the title is listed:** transitive bumps, added and removed packages, install scripts, native code, source changes.
-- **Every changed API has a search result:** `file:line` hits, "no hits", or the command to run when there is no repo.
-- **Every SHA pin is verified or marked UNVERIFIED with the `git ls-remote` command;** tag-only pins are called weaker.
-- **CI coverage is named per changed package:** the job and test that exercise it, or "nothing does".
+- **The first line is the verdict and the reasons that decide it,** one verdict per PR, following from the checks. Any unverified SHA, added install script, or removed API still in use rules out MERGE.
+- **Every release between old and new is accounted for:** read, with its breaking changes, runtime drops, deprecations, security fixes and licence changes, or named as unread.
+- **Everything the lockfile changes beyond the title is listed,** in one line: transitive bumps, added and removed packages, install scripts, native code, source and runtime changes.
+- **Every changed API has a search result and every hit is traced:** what the caller does now and the fix, or the search command when there is no repo.
+- **Every raised runtime floor is compared** with the base image, the CI runtime, the runners and the lockfile tool's runtime.
+- **Every changed SHA pin is verified or given as the `git ls-remote` command,** and the PR's pins are compared with each other.
+- **CI coverage is named per changed package:** the job and test that exercise it, or "nothing does", and how each workflow installs.
 - **A rollback with the exact command** and what a revert does not undo.
-- **No invented versions, advisories, APIs, files or test names.** Everything cited appears in the diff, the notes or the repo supplied.
+- **Nothing reads as checked that was not run,** and nothing describes the session's tools or network.
+- **No invented versions, advisories, APIs, files or test names;** version numbers are copied from the notes.
 
 ## When to use this skill
 
@@ -91,11 +108,12 @@ A dependency bump is someone else's code entering the repo under a one-line titl
 
 ## Anti-patterns to avoid
 
+- ❌ **The report reply.** A package table, a rollback essay and an empty "To confirm" around a verdict the reader has to dig for. Verdict and reasons first; the rest in a line each.
 - ❌ **Reading only the target version's notes.** The breaking change usually lives in a release the PR skipped.
-- ❌ **Trusting the title.** "Bump X from 1 to 2" says nothing about the twelve transitive packages that moved with it.
-- ❌ **Calling a SHA pin fine because it is a SHA.** A SHA proves nothing until it matches the release tag.
-- ❌ **"CI is green, ship it."** Release workflows, publish steps and untested code paths never ran on the PR.
-- ❌ **"No breaking changes affect us" without a search.** Show the grep and its hits, or say it was not run.
-- ❌ **A rollback that is only "revert the PR".** Name the lockfile regeneration and what a revert leaves behind.
-- ❌ **Reciting the changelog.** Report the releases that change the verdict; one line covers the rest.
+- ❌ **Trusting the title.** "Bump X from 1 to 2" says nothing about the transitive packages that moved with it.
+- ❌ **Finding the change but not the caller.** "`fetch()` now raises" is half a finding; "so the retry loop never retries" is the finding.
+- ❌ **Trusting CI's runtime.** CI on Python 3.11 says nothing about an image built on 3.9.
+- ❌ **Calling a SHA pin fine because it is a SHA.** A SHA proves nothing until it matches the release tag, and two SHAs for one tag cannot both match.
+- ❌ **"CI is green, ship it."** Release workflows, publish steps, lazy imports and mocked paths never ran on the PR.
+- ❌ **Explaining the session.** "The network was blocked, so I couldn't check" is about the run, not the PR; give the command.
 - ❌ **Hedged verdicts.** "Probably safe" is not a verdict; pick one of the three and give the reason.

package/packs/dev-team/skills/dependency-upgrade-review/evals/evals.json

@@ -12,6 +12,7 @@
         "Says the green CI does not cover the change: the only test renders Markdown with no raw HTML, and no test imports src/render/comment.js",
         "Accounts for the skipped releases, not only 4.1.0: names the 3.5.0 link-title security fix and the setOptions() deprecation that src/render/note.js still uses",
         "Gives a rollback that pins quillmark back and regenerates package-lock.json (or reverts the PR), and names taking 3.5.x as the way to get the security fix without the breaking changes",
+        "Opens with the verdict and the decisive reasons (the removed renderInline() call and the sanitize default) in plain words before any list, keeps the lockfile changes to a line or a short list rather than a table, and does not describe the session's network or tool access",
         "Does not invent versions, advisory IDs, APIs, files or tests that are not in the supplied input"
       ]
     },
@@ -26,6 +27,7 @@
         "Reports that release.yml still passes the deprecated password input (release.yml line 22) and that verify-metadata now defaults to true, so the next release can fail on an invalid long description",
         "Classes the jumps: checkout-lite v3.1.0 to v3.2.0 as minor, pypi-publish v2 to v3 as major",
         "Gives a rollback (revert the PR or restore the old uses: lines) and notes a revert cannot undo a release already published to PyPI with v3",
+        "Does not report either SHA or the tag move as verified, and does not explain the gap by describing the session's network or tool access; it gives the command and the SHA it must equal instead",
         "Does not invent action versions, SHAs, inputs or workflow steps that are not in the supplied files"
       ]
     },
@@ -40,6 +42,7 @@
         "Flags github.com/lanternhq/clock v1.2.0 as a new indirect module the PR title does not mention",
         "Says the passing CI only covers this if a test exercises the burst-0 Wait path, and lists that as a check rather than assuming it",
         "Gives a rollback command such as go get github.com/lanternhq/ratelimit@v0.8.3 followed by go mod tidy, or reverting the PR",
+        "Stays short, as the user asked for a quick verdict: the verdict and its reasons come first, and there is no package table and no empty section",
         "Does not invent file names, call sites, tests or release versions that are not in the prompt"
       ]
     },
@@ -72,6 +75,36 @@
         "Explains that the green checks never ran the changed code. ci.yml runs on pull_request with `npm ci --ignore-scripts` and never runs scripts/release.mjs. release.yml runs plain `npm ci`, so notary-telemetry's install script would first execute in the production environment, with RELEASE_SIGNING_KEY and NPM_TOKEN in the job and contents: write / id-token: write permissions. Mentions NOTARY_TELEMETRY=0 and/or keeping 1.8.x as mitigations.",
         "Gives a rollback for each PR: revert #412 to restore the c8aee2ae521bdb1ef1b97761673ebbe92c953b0f (v3.2.0) pins; revert #413, or pin release-notary back to 1.8.2 and regenerate package-lock.json. Does not invent versions, flags or APIs that are not in the files, and does not claim to have run npm install, the workflows, or the release script."
       ]
+    },
+    {
+      "id": 6,
+      "held_out": "2026-10-20",
+      "prompt": "Dependabot opened PR #412 on our payments-gateway service bumping github.com/fernwire/httpkit from 1.8.1 to 1.8.4. It's only a patch bump and every check is green, so I'd like to merge it before the release cut this afternoon. Can I merge this dependency upgrade? The PR body is in PR.md, the diff is pr.diff, and I pasted the upstream changelog into vendor-notes/httpkit-CHANGELOG.md. The repo source is checked out here.",
+      "assertions": [
+        "Picks out the v1.8.3 entry in vendor-notes/httpkit-CHANGELOG.md (a version the Dependabot release notes in PR.md leave out): NewClient no longer applies a default 30-second Timeout, and the zero value now means no client-level timeout. It treats this as the key finding even though the bump looks like a patch.",
+        "Points to internal/payments/client.go NewClient, which calls httpkit.NewClient with only WithBaseURL and WithRetries and whose comment says it relies on the 30s default (INC-2291), and to internal/payments/settle.go, which calls Charge with context.Background() and no deadline. So after the upgrade a hung gateway can block the settlement batch indefinitely. It also notes that internal/notify/client.go sets WithTimeout(5*time.Second) explicitly and is unaffected.",
+        "Explains that the green CI does not test this change: client_test.go injects fakeDoer, so the real httpkit client is never exercised, and ci.yml runs `go test -race -short ./...` without `-tags integration`, so client_integration_test.go (TestChargeHungGatewayTimesOut) did not run on this PR. It runs only nightly or on manual dispatch via integration.yml.",
+        "Gives a verdict of DO NOT MERGE as-is, or MERGE AFTER CHECKS on the specific condition that payments NewClient gains an explicit httpkit.WithTimeout(30*time.Second) (or a per-call context deadline in SettleBatch) and the integration test passes. It is never plain MERGE.",
+        "Lists the go.mod/go.sum changes the PR title does not mention: indirect github.com/fernwire/backoff v0.3.0 -> v0.4.0 and a newly added indirect github.com/fernwire/otelshim v0.1.0. It ties them to the v1.8.2 (backoff, full-jitter) and v1.8.4 (WithTracer) changelog entries rather than calling them unexplained.",
+        "Gives a concrete rollback: revert the merge commit, or `go get github.com/fernwire/httpkit@v1.8.1 && go mod tidy`, which also returns backoff to v0.3.0 and drops otelshim. It notes that adding the explicit WithTimeout is safe to keep on either version.",
+        "Does not claim to have run tests or verified the go.sum hashes or upstream tags. It gives the network or execution steps as commands to run (for example `go mod verify`, `go test -tags integration -run TestChargeHungGatewayTimesOut ./internal/payments/`, or triggering integration.yml via workflow_dispatch).",
+        "Invents no httpkit versions, options or behaviours beyond those in the files (for example no made-up v1.8.5, no claimed retry-on-POST change, no options other than WithBaseURL, WithRetries, WithTimeout and WithTracer)."
+      ]
+    },
+    {
+      "id": 7,
+      "held_out": "2026-10-20",
+      "prompt": "Renovate grouped this week's dev dependency updates into PR #88 for our invoice-widgets package: five npm packages in one go (details in PR.md, diff in pr.diff). After last quarter's incident with a grouped bump I'm nervous about these. Is this Renovate bump safe to merge, or should I split it up? Repo files are here.",
+      "assertions": [
+        "Ends with a verdict of MERGE. It does not downgrade to MERGE AFTER CHECKS or DO NOT MERGE over problems the files do not contain (for example an install script, a registry change, a maintainer change, or a breaking change that affects this repo), and it does not insist on splitting the PR without a concrete reason from the files.",
+        "Checks bundlecrate 6.1.0's new Node >=18.18 minimum against the repo and finds it harmless: .nvmrc is 20.11.1, package.json engines is node >=20, and ci.yml uses node-version-file: .nvmrc.",
+        "Checks the repo's uses of the changed APIs and finds nothing that needs action. vellum-test 2.8.0 deprecates vt.legacyTimers(), but tests/due-date.test.ts uses only vt.fakeTimers/vt.restoreTimers. lintwright 4.3.0's new no-floating-promise rule is only in lintwright:strict, while .lintwright.json extends lintwright:recommended.",
+        "Lists the lockfile changes beyond the five headline bumps: @bundlecrate/resolver-core 1.0.0 added, bundlecrate-legacy-resolver 2.3.4 removed, and the transitive picolabel 0.9.1 -> 0.9.2. It ties each to a changelog entry (bundlecrate 6.1.0 resolver extraction, lintwright 4.3.0 picolabel update).",
+        "States that no lockfile entry in pr.diff has \"hasInstallScript\" and that every resolved URL is on https://registry.npmjs.org/, which matches .npmrc. So there are no new install-time scripts and no registry switch.",
+        "Says CI did exercise the changed code, mapping the packages to jobs in .github/workflows/ci.yml: lintwright -> lint, prettyquill -> format, tsforge-types -> typecheck, vellum-test -> test, bundlecrate -> build-and-smoke (which runs test:dist against the built bundle). It says the pass/fail status comes from PR.md and was not observed directly.",
+        "Gives a rollback: revert the merge commit (restoring package.json and package-lock.json together) and run `npm ci`. It notes these are all devDependencies, so only the bundlecrate-built dist/ could reach users, and that build is covered by the smoke step.",
+        "Does not claim to have verified integrity hashes, provenance or upstream tags offline. It invents no versions, rules or APIs not in the files, and frames anything network-dependent (for example `npm audit signatures`, `npm ci` locally) as an optional command, not a blocker."
+      ]
     }
   ]
 }

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/.github/workflows/ci.yml

@@ -0,0 +1,39 @@
+name: ci
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d1f0a9c7e5b2846f0c1a7d9e3b5f8c2a6d4e017 # v4.2.2
+      - uses: actions/setup-go@8e2c4a6f0b1d3957a2c8e4f6b0d1a3c5e7f9b204 # v5.1.0
+        with:
+          go-version: "1.23"
+      - run: go vet ./...
+
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d1f0a9c7e5b2846f0c1a7d9e3b5f8c2a6d4e017 # v4.2.2
+      - uses: actions/setup-go@8e2c4a6f0b1d3957a2c8e4f6b0d1a3c5e7f9b204 # v5.1.0
+        with:
+          go-version: "1.23"
+      # Unit tests only. Integration tests (-tags integration) run nightly in
+      # integration.yml because they need a long timeout.
+      - run: go test -race -short ./...
+
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d1f0a9c7e5b2846f0c1a7d9e3b5f8c2a6d4e017 # v4.2.2
+      - uses: actions/setup-go@8e2c4a6f0b1d3957a2c8e4f6b0d1a3c5e7f9b204 # v5.1.0
+        with:
+          go-version: "1.23"
+      - run: go build ./...

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/.github/workflows/integration.yml

@@ -0,0 +1,20 @@
+name: integration
+
+on:
+  schedule:
+    - cron: "30 2 * * *"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  integration:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@3d1f0a9c7e5b2846f0c1a7d9e3b5f8c2a6d4e017 # v4.2.2
+      - uses: actions/setup-go@8e2c4a6f0b1d3957a2c8e4f6b0d1a3c5e7f9b204 # v5.1.0
+        with:
+          go-version: "1.23"
+      - run: go test -tags integration -timeout 10m ./...

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/PR.md

@@ -0,0 +1,39 @@
+# Bump github.com/fernwire/httpkit from 1.8.1 to 1.8.4 #412
+
+**dependabot** wants to merge 1 commit into `main` from `dependabot/go_modules/github.com/fernwire/httpkit-1.8.4`
+
+Bumps [github.com/fernwire/httpkit](https://github.com/fernwire/httpkit) from 1.8.1 to 1.8.4.
+
+<details>
+<summary>Release notes</summary>
+<p><em>Sourced from <a href="https://github.com/fernwire/httpkit/releases">github.com/fernwire/httpkit's releases</a>.</em></p>
+<blockquote>
+<h2>v1.8.4</h2>
+<ul>
+<li>Add optional OpenTelemetry hooks via <code>WithTracer</code> (backed by the new <code>fernwire/otelshim</code> module). No behaviour change unless the option is passed.</li>
+<li>Fix a data race in connection-pool metrics (#240).</li>
+</ul>
+</blockquote>
+</details>
+<details>
+<summary>Commits</summary>
+<ul>
+<li><code>9c1e4b7</code> release v1.8.4</li>
+<li><code>3fa02d1</code> pool: fix metrics race (#240)</li>
+<li><code>b81d6e0</code> add WithTracer option (#238)</li>
+<li>Additional commits viewable in <a href="https://github.com/fernwire/httpkit/compare/v1.8.1...v1.8.4">compare view</a></li>
+</ul>
+</details>
+<br />
+
+Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself.
+
+---
+
+### Checks
+
+- ci / lint — passed (41s)
+- ci / test — passed (1m 52s)
+- ci / build — passed (38s)
+
+All checks have passed.

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/go.mod

@@ -0,0 +1,17 @@
+module example.internal/payments-gateway
+
+go 1.23
+
+require (
+	github.com/fernwire/httpkit v1.8.4
+	github.com/quillstack/zlogx v0.11.2
+	github.com/stretchr/testify v1.9.0
+)
+
+require (
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/fernwire/backoff v0.4.0 // indirect
+	github.com/fernwire/otelshim v0.1.0 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/notify/client.go

@@ -0,0 +1,35 @@
+// Package notify sends merchant webhooks.
+package notify
+
+import (
+	"bytes"
+	"context"
+	"net/http"
+	"time"
+
+	"github.com/fernwire/httpkit"
+)
+
+type Sender struct {
+	hc *httpkit.Client
+}
+
+func NewSender() *Sender {
+	return &Sender{hc: httpkit.NewClient(
+		httpkit.WithTimeout(5*time.Second),
+		httpkit.WithRetries(3),
+	)}
+}
+
+func (s *Sender) Send(ctx context.Context, url string, payload []byte) error {
+	req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(payload))
+	if err != nil {
+		return err
+	}
+	req.Header.Set("Content-Type", "application/json")
+	resp, err := s.hc.Do(ctx, req)
+	if err != nil {
+		return err
+	}
+	return resp.Body.Close()
+}

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/client.go

@@ -0,0 +1,66 @@
+// Package payments talks to the upstream card-processing gateway.
+package payments
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"strings"
+
+	"github.com/fernwire/httpkit"
+)
+
+// doer is the subset of *httpkit.Client we use. Tests substitute a fake.
+type doer interface {
+	Do(ctx context.Context, req *http.Request) (*http.Response, error)
+}
+
+// Client calls the gateway's /v2/charges API.
+type Client struct {
+	http    doer
+	baseURL string
+}
+
+// NewClient builds a gateway client.
+//
+// We deliberately do not set a timeout here: httpkit's default 30s client
+// timeout is what protects the settlement worker from a hung gateway
+// connection (see INC-2291).
+func NewClient(baseURL string) *Client {
+	hc := httpkit.NewClient(
+		httpkit.WithBaseURL(baseURL),
+		httpkit.WithRetries(2),
+	)
+	return &Client{http: hc, baseURL: baseURL}
+}
+
+type ChargeResult struct {
+	ID     string `json:"id"`
+	Status string `json:"status"`
+}
+
+// Charge submits a capture for a previously authorised payment.
+func (c *Client) Charge(ctx context.Context, authID string, amountMinor int64, idemKey string) (*ChargeResult, error) {
+	body := fmt.Sprintf(`{"authorization":%q,"amount":%d}`, authID, amountMinor)
+	req, err := http.NewRequest(http.MethodPost, c.baseURL+"/v2/charges", strings.NewReader(body))
+	if err != nil {
+		return nil, err
+	}
+	req.Header.Set("Content-Type", "application/json")
+	req.Header.Set("Idempotency-Key", idemKey)
+
+	resp, err := c.http.Do(ctx, req)
+	if err != nil {
+		return nil, fmt.Errorf("gateway charge: %w", err)
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode >= 300 {
+		return nil, fmt.Errorf("gateway charge: status %d", resp.StatusCode)
+	}
+	var out ChargeResult
+	if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
+		return nil, fmt.Errorf("gateway charge: decode: %w", err)
+	}
+	return &out, nil
+}

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/client_integration_test.go

@@ -0,0 +1,38 @@
+//go:build integration
+
+package payments
+
+import (
+	"context"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/require"
+)
+
+// TestChargeHungGatewayTimesOut guards INC-2291: a gateway that accepts the
+// connection and never responds must not block the settlement worker forever.
+func TestChargeHungGatewayTimesOut(t *testing.T) {
+	block := make(chan struct{})
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		<-block
+	}))
+	defer srv.Close()
+	defer close(block)
+
+	c := NewClient(srv.URL)
+	done := make(chan error, 1)
+	go func() {
+		_, err := c.Charge(context.Background(), "auth_1", 100, "idem-hang")
+		done <- err
+	}()
+
+	select {
+	case err := <-done:
+		require.Error(t, err)
+	case <-time.After(2 * time.Minute):
+		t.Fatal("Charge did not time out against a hung gateway")
+	}
+}

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/client_test.go

@@ -0,0 +1,53 @@
+package payments
+
+import (
+	"context"
+	"errors"
+	"io"
+	"net/http"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+)
+
+type fakeDoer struct {
+	status int
+	body   string
+	err    error
+	got    *http.Request
+}
+
+func (f *fakeDoer) Do(_ context.Context, req *http.Request) (*http.Response, error) {
+	f.got = req
+	if f.err != nil {
+		return nil, f.err
+	}
+	return &http.Response{
+		StatusCode: f.status,
+		Body:       io.NopCloser(strings.NewReader(f.body)),
+	}, nil
+}
+
+func TestChargeOK(t *testing.T) {
+	fd := &fakeDoer{status: 201, body: `{"id":"ch_1","status":"captured"}`}
+	c := &Client{http: fd, baseURL: "https://gw.test"}
+	res, err := c.Charge(context.Background(), "auth_1", 1250, "idem-1")
+	require.NoError(t, err)
+	require.Equal(t, "ch_1", res.ID)
+	require.Equal(t, "idem-1", fd.got.Header.Get("Idempotency-Key"))
+}
+
+func TestChargeGatewayError(t *testing.T) {
+	fd := &fakeDoer{err: errors.New("boom")}
+	c := &Client{http: fd, baseURL: "https://gw.test"}
+	_, err := c.Charge(context.Background(), "auth_1", 1250, "idem-1")
+	require.Error(t, err)
+}
+
+func TestChargeNon2xx(t *testing.T) {
+	fd := &fakeDoer{status: 502, body: ``}
+	c := &Client{http: fd, baseURL: "https://gw.test"}
+	_, err := c.Charge(context.Background(), "auth_1", 1250, "idem-1")
+	require.ErrorContains(t, err, "status 502")
+}

package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/settle.go

@@ -0,0 +1,30 @@
+package payments
+
+import (
+	"context"
+
+	"github.com/quillstack/zlogx"
+)
+
+// Pending is a capture waiting to be settled.
+type Pending struct {
+	AuthID      string
+	AmountMinor int64
+	IdemKey     string
+}
+
+// SettleBatch is run by the nightly settlement worker. It processes captures
+// one at a time, in order; a single stuck call blocks the rest of the batch.
+func SettleBatch(c *Client, batch []Pending, log *zlogx.Logger) (settled int) {
+	for _, p := range batch {
+		// No per-call deadline: we rely on the client's own timeout.
+		res, err := c.Charge(context.Background(), p.AuthID, p.AmountMinor, p.IdemKey)
+		if err != nil {
+			log.Warn("settle failed", "auth", p.AuthID, "err", err)
+			continue
+		}
+		log.Info("settled", "auth", p.AuthID, "charge", res.ID)
+		settled++
+	}
+	return settled
+}