swe-workflow-skills 0.8.1 → 0.9.0

package/README.md

@@ -26,8 +26,8 @@ don't cover:
   intent to the right skill(s) and chains them across phases. Community measurements put
   Claude's own description-based auto-triggering at roughly a coin flip to ~84%; here
   activation is *routed and deterministic* instead, and the routing eval harness measures
-  it (top-1 accuracy 1.00, zero misroutes on the committed baseline, regression-gated in
-  CI) — see the [routing benchmark](docs/ROUTING-BENCHMARK.md). This is the one thing the
+  it (top-1 accuracy 65/65 and one boundary misroute in 65 on the committed k=3 baseline,
+  regression-gated in CI) — see the [routing benchmark](docs/ROUTING-BENCHMARK.md). This is the one thing the
   platform still doesn't do for you.
 - **Every skill stays reachable, no pre-picking.** Claude Code injects skill descriptions
   only up to ~1% of context, so large libraries silently stop auto-triggering past ~20–40
@@ -38,9 +38,10 @@ don't cover:
   ([how it works](docs/ROLES.md)).
 - **Tested like code, not prose.** Every skill ships 3 evals; two LLM-as-judge
   harnesses (content quality and routing accuracy) replay them with skill loaded vs
-  absent and gate regressions in CI. Across all 66 skills — 234 cases, 1315 assertions —
-  the same model passes **67.1%** of assertions unaided and **94.4%** with the skill
-  loaded, and **no skill scores below the unaided model on any case**
+  absent and gate regressions in CI. Across all 66 skills — 234 cases, 1315 assertions,
+  every row measured at k>=3 — the same model passes **64.1%** of assertions without the
+  skill's content and **97.9%** with the skill loaded, and **no skill scores below the
+  no-content baseline on any case**
   ([results](docs/EVALS.md#results-content-evals-full-catalog)). Safety-critical skills
   (deploys, releases, tests, incidents, security) are **hardened**: an Iron Law, a
   rationalization table distilled from real baseline failures, and pressure tests that

package/VERSION

@@ -1 +1 @@
-0.8.1
+0.9.0

package/catalog.json

@@ -323,7 +323,7 @@
     },
     {
       "name": "verification-before-completion",
-      "description": "Evidence gate before claiming work is done, passing, fixed, or deployed. Use before saying 'done', 'it works', or 'tests pass', or before committing, pushing, or opening a PR. Run the proving command fresh and read its output first. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix.",
+      "description": "Evidence gate before any claim that work is done, fixed, passing, or deployed. Route here when a message declares the work finished or asks to commit, push, merge, ship, or close it out, including a brief sign-off, and before git-workflow or release-management act on it. Writing the commit message or PR text itself → git-workflow. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix.",
       "path": "verification-before-completion/SKILL.md"
     },
     {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "swe-workflow-skills",
-  "version": "0.8.1",
+  "version": "0.9.0",
   "description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
   "type": "module",
   "bin": {

package/skills/accessibility-design/evals/evals.json

@@ -20,7 +20,7 @@
     {
       "id": 2,
       "prompt": "Audit this React component for accessibility issues:\n\n<div className=\"card\" onClick={() => navigate(`/project/${id}`)}>\n  <div className=\"title\">{name}</div>\n  <img src={thumbnail} />\n  <div className=\"badge\" style={{color: status === 'active' ? 'green' : 'red'}}>\n    {status}\n  </div>\n  <div className=\"actions\">\n    <div className=\"btn\" onClick={(e) => { e.stopPropagation(); onDelete(id); }}>\n      🗑️\n    </div>\n  </div>\n</div>",
-      "expected_output": "Should identify all accessibility issues: clickable div instead of link/button, missing alt on img, color-only status indicator, icon button without label, no keyboard support. Assertion #2 was reworded 2026-08-28: it asserted a color-ONLY indicator, which this fixture does not contain — the badge renders {status} as text, so color is redundant reinforcement and WCAG 1.4.1 is not violated. At k=5, 7 of 10 judges recorded the reply correctly ruling 1.4.1 out and flagging 1.4.3 contrast instead; the assertion was punishing the right answer. It now targets the failure the fixture really has.",
+      "expected_output": "Should identify all accessibility issues: clickable div instead of link/button, missing alt on img, color-only status indicator, icon button without label, no keyboard support. Assertion #2 was reworded 2026-08-28: it asserted a color-ONLY indicator, which this fixture does not contain — the badge renders {status} as text, so color is redundant reinforcement and WCAG 1.4.1 is not violated. At k=5, 7 of 10 judges recorded the reply correctly ruling 1.4.1 out and flagging 1.4.3 contrast instead; the assertion was punishing the right answer. It now targets the failure the fixture really has. NOTE: final assertion added 2026-09-16. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 7 (validate) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause, which let controls pass on substance in an earlier case.",
       "assertions": [
         "Identifies clickable div should be an <a> or <Link> (for navigation)",
         "Identifies missing alt attribute on the image",
@@ -29,7 +29,8 @@
         "Identifies emoji icon without accessible label (aria-label needed)",
         "Notes that div+onClick has no keyboard support (no Enter/Space handling)",
         "Provides corrected accessible version of the component",
-        "Corrected version uses semantic HTML (a, button, img alt)"
+        "Corrected version uses semantic HTML (a, button, img alt)",
+        "Recommends validating the corrected component manually with a keyboard-only pass and a screen reader, not only by reading the code or running an automated checker"
       ]
     },
     {

package/skills/ai-evaluation/evals/evals.json

@@ -16,24 +16,25 @@
     {
       "id": 2,
       "prompt": "Our LLM feature summarizes long sales calls. We have no ground-truth summaries and no users yet. How do we evaluate quality with zero labeled data?",
-      "expected_output": "Does not treat missing labels as a blocker: proposes reference-free evaluation (rubric-based LLM-as-judge, groundedness against the source call, pairwise comparison between variants), bootstraps a small human-labeled golden set, calibrates the judge against human spot-checks, and plans online signals (feedback, A/B) once users exist",
+      "expected_output": "Does not treat missing labels as a blocker: proposes reference-free evaluation (rubric-based LLM-as-judge, groundedness against the source call, pairwise comparison between variants), bootstraps a small human-labeled golden set, calibrates the judge against human spot-checks, and plans online signals (feedback, A/B) once users exist NOTE: final assertion added 2026-09-16. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 5 (gate changes in CI) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause, which let controls pass on substance in an earlier case.",
       "assertions": [
         "Does not claim evaluation is impossible without labels — proposes reference-free approaches such as rubric-based LLM-as-judge, groundedness against the source transcript, or pairwise comparison between variants",
         "Recommends bootstrapping a small human-labeled golden set (even tens of examples) and explains what it buys",
         "Designs the judge with an explicit rubric or criteria decomposition rather than a bare overall score",
         "Mentions checking the judge against human spot-checks (calibration/agreement) or using voting to reduce judge noise",
-        "Plans for online signals after launch (user feedback, A/B testing) as a complement to offline evals"
+        "Plans for online signals after launch (user feedback, A/B testing) as a complement to offline evals",
+        "Runs the golden set as a CI gate on every prompt or model change, failing on regression against the last accepted baseline rather than below an absolute score"
       ]
     },
     {
       "id": 3,
-      "prompt": "Set up MLflow for our fine-tuning experiments so we can log runs and compare hyperparameters across the team.",
-      "expected_output": "Recognizes this as experiment-tracking infrastructure, not evaluation design — hands off to ml-experiment-tracking for the MLflow setup and run comparison, noting the boundary: tracking records and compares runs, while ai-evaluation defines what and how to measure",
+      "prompt": "We're fine-tuning a summarizer. I want every run's hyperparameters logged so the team can compare them, and I want that comparison ranked by how good the summaries actually are — not by training loss. What should we set up?",
+      "expected_output": "Splits the request: capturing hyperparameters and comparing runs is tracking infrastructure, while deciding what 'good summary' means and how it is scored is evaluation design. Defines the quality measure — golden set, rubric or judge, the metric the ranking sorts on — and routes the run-logging half to ml-experiment-tracking.",
       "assertions": [
-        "Recognizes the request is about experiment-tracking/run-comparison infrastructure, not about defining evaluation metrics or datasets",
-        "Refers to the ml-experiment-tracking skill for the MLflow setup and run comparison",
-        "Does not produce an evaluation-design workflow (golden datasets, judges, RAG metrics) as the answer to a tracking-infrastructure request",
-        "States the boundary: tracking records and compares runs, while evaluation design defines what and how to measure"
+        "Separates the two halves: run logging and comparison infrastructure versus defining the summary-quality measure the ranking needs",
+        "Refers to ml-experiment-tracking for capturing hyperparameters and comparing runs",
+        "Defines the quality measure itself — an evaluation set, a rubric or judge, and the metric the ranking would sort on — rather than handing the whole request off",
+        "States the boundary: tracking records and compares whatever score it is given; evaluation design decides what counts as good and how it is scored"
       ]
     }
   ]

package/skills/api-design/evals/evals.json

@@ -33,15 +33,14 @@
     },
     {
       "id": 3,
-      "prompt": "We have an existing API with these endpoints:\nGET /getUsers\nGET /getUserById?id=123\nPOST /createUser\nPOST /updateUser\nGET /deleteUser?id=123\n\nThis doesn't feel right. Can you redesign it following proper REST conventions?",
-      "expected_output": "Should identify specific violations (verbs in URLs, GET for delete, query params for identity) and redesign using proper REST conventions.",
+      "prompt": "We're adding a /transactions endpoint to our finance API. Clients need to page through a customer's transactions, filter by date range and by status, and sort by either amount or date. The table has around 400 million rows. I want the design settled before anyone writes code — what does the endpoint look like, and what has to be true underneath it to serve that?",
+      "expected_output": "Splits a request whose halves the skill roster does not separate. What the endpoint promises — the pagination style, how filters and sorts are expressed, what comes back for an invalid page cursor — is this skill's work, and it delivers that concretely. Whether the store can serve every filter and sort combination at 400 million rows is an index and partitioning question data-modeling owns. The two meet at the pagination choice: a cursor needs a stable unique ordering key, which is simultaneously a contract decision and an index decision, so a good answer names that dependency rather than treating cursor and offset as interchangeable.",
       "assertions": [
-        "Identifies that URLs use verbs instead of nouns",
-        "Identifies that DELETE uses GET method",
-        "Identifies that identity is in query params instead of path",
-        "Redesigns to: GET /users, GET /users/:id, POST /users, PATCH /users/:id, DELETE /users/:id",
-        "Explains WHY each change matters (not just 'this is the convention')",
-        "Notes that this is a breaking change requiring versioning or migration plan"
+        "Separates the two halves: what the endpoint promises its clients, versus what the storage has to be shaped like to serve those promises",
+        "Refers to data-modeling for the index and partitioning strategy behind the filters and sorts",
+        "Delivers the endpoint contract itself — the pagination style and how filters and sorts are expressed — rather than deferring the whole request",
+        "Ties the pagination choice to a stable ordering key instead of presenting cursor and offset as interchangeable",
+        "States the boundary: API design fixes what the interface promises, while data modeling decides whether the store can keep that promise"
       ]
     }
   ]

package/skills/architecture-documentation/SKILL.md

@@ -20,6 +20,13 @@ Understand what exists and what's needed before creating anything:
 
 A simple monolith may only need one conceptual diagram. A distributed system with 5+ services likely needs canonical and runtime levels too.
 
+**Split the ask before you start.** "Onboarding documentation" usually bundles two
+different deliverables, and only one is this skill's. A reader who cannot tell which
+service calls which needs a structural view — that is the work below. A reader who
+cannot get the system running needs setup instructions, which live in the README and
+belong to `project-documentation`. Say which half you are producing; a diagram does not
+fix a broken local setup, and a README does not substitute for a container view.
+
 ### Step 2: Choose Abstraction Levels
 
 Architecture documentation works at three distinct levels. Not every system needs all three — apply YAGNI.

package/skills/architecture-documentation/evals/evals.json

@@ -19,28 +19,26 @@
     {
       "id": 2,
       "prompt": "I have a simple CRUD application — Express.js backend, PostgreSQL database, deployed on a single EC2 instance behind nginx. My manager read about C4 and wants me to create all four C4 levels plus deployment diagrams plus data flow diagrams. Can you help me set this up?",
-      "expected_output": "Edge case — pushing back on unnecessary documentation complexity. Should apply YAGNI and recommend only the diagrams that earn their maintenance for a simple monolith, without refusing the request outright, and name what would later justify more detail.",
+      "expected_output": "Edge case — pushing back on unnecessary documentation complexity. Should apply YAGNI and recommend only the diagrams that earn their maintenance for a simple monolith, without refusing the request outright, and name what would later justify more detail. Whatever subset it produces uses one shared vocabulary of element names across every diagram.",
       "assertions": [
         "Pushes back on creating all four C4 levels for a simple monolith",
         "Applies YAGNI — recommends only the diagrams that add value for this system's complexity",
-        "Recommends at most a system context and a single container diagram",
         "Explains that Code-level C4 diagrams are almost never worth maintaining manually",
         "Does not refuse the request entirely — provides a pragmatic subset",
         "Suggests conceptual-level documentation may be sufficient for this system",
-        "Explains what would trigger the need for more detailed documentation in the future"
+        "Explains what would trigger the need for more detailed documentation in the future",
+        "Establishes a shared naming vocabulary (a glossary of element names) that every diagram uses exactly"
       ]
     },
     {
       "id": 3,
-      "prompt": "We're debating whether to extract our authentication logic into a separate service or keep it in the monolith. Can you also create architecture diagrams showing what the system would look like after the split?",
-      "expected_output": "Scope boundary — separating decision-making from documentation. Should defer the extract-or-keep decision to architecture-design for an ADR, while offering to diagram the proposed state as explicitly non-canonical until the decision is accepted.",
+      "prompt": "Three engineers joined this month and all three got stuck the same way: they can't work out which of our nine services actually calls which, and they couldn't get the stack running locally. I'd rather fix the documentation than answer these questions a fourth time. Where do we start?",
+      "expected_output": "Notices that the two complaints are not one problem. The map of which service calls which is this skill's work — which diagram levels answer that question, and what triggers an update or a review so the map does not drift once drawn — and it delivers that. Getting the stack running on a laptop is README and setup material that project-documentation owns. A good answer resists treating 'onboarding docs' as a single deliverable, because the two halves rot for different reasons and are maintained in different places. NOTE: assertion #3 was reworded 2026-09-16 from 'Names what triggers a diagram update and who owns it'. Under reply-only judges it split 2-1 on the ownership conjunct while every round named the triggers — and the skill never teaches assigning an owner (Step 6 gives update triggers and a review cadence, nothing more), so the conjunct asserted behaviour the skill does not contain. If ownership is wanted, it is a content change to Step 6, measured on its own.",
       "assertions": [
-        "Recognizes the decision-making aspect and defers to the architecture-design skill for the ADR",
-        "Does not make the architectural decision within this skill's workflow",
-        "Offers to diagram the proposed architecture as a conceptual or draft diagram to support the decision",
-        "Clearly separates the decision process from the documentation process",
-        "If diagramming the proposed state, labels it as non-canonical since the decision hasn't been made yet",
-        "Suggests that canonical diagrams should only be updated after the decision is accepted"
+        "Separates the two halves: the map of which services call which, versus the instructions for getting the stack running locally",
+        "Refers to project-documentation for the README and the local setup instructions",
+        "Delivers the architecture-documentation half itself — which diagram levels answer the what-calls-what question — rather than deferring the whole request",
+        "Names what triggers a diagram update — the structural changes that should force one, or a review cadence — rather than leaving the diagrams' accuracy to good intentions"
       ]
     }
   ]

package/skills/brainstorming/evals/evals.json

@@ -27,13 +27,13 @@
     },
     {
       "id": 3,
-      "prompt": "We've settled on building a CLI tool that syncs our design tokens to code. Break it into tasks with acceptance criteria and dependencies so the team can start next sprint.",
-      "expected_output": "Recognizes that ideation is complete and the user is asking for convergent task breakdown — refers to feature-planning for the scoped tasks, acceptance criteria, and dependency mapping rather than reopening divergent exploration",
+      "prompt": "Our nightly analytics reports are slow enough that people have started complaining. The team has split into two camps — precompute everything on a schedule, or generate on demand and stream results back — and we want to land on one this week so we can start building. Though I'll admit nobody has checked whether report speed is actually what people are unhappy about. Help us get to an answer.",
+      "expected_output": "Notices the request contains a closed question and an open one, and does not treat them alike. Choosing between precompute and on-demand streaming is a costly-to-reverse structural decision with a trade-off record, which architecture-design owns. Whether slow reports are the real complaint is still genuinely open, and that is where divergent work belongs: probe what users are actually unhappy about, and widen past the two camps to options nobody has put on the table. Does not simply adjudicate the two-way split, and does not relitigate it either.",
       "assertions": [
-        "Recognizes the request as convergent planning of an already-chosen idea, not ideation",
-        "Refers to the feature-planning skill for the task breakdown, acceptance criteria, and dependencies",
-        "Does not restart divergent idea generation or question the settled decision as the main response",
-        "States or implies the boundary: brainstorming opens the option space before a decision; feature-planning structures the work after it"
+        "Separates the two questions: whether slow reports are the real complaint is still open, while precompute-versus-on-demand is a decision waiting to be made",
+        "Refers to architecture-design for choosing between precompute and on-demand and recording the trade-off",
+        "Probes what users are actually unhappy about before the two camps are compared, rather than accepting report speed as the settled problem",
+        "Generates options beyond the two the team has already named"
       ]
     }
   ]

package/skills/browser-verification/evals/evals.json

@@ -4,25 +4,25 @@
     {
       "id": 1,
       "prompt": "I just finished implementing the new signup form (client validation + POST to /api/users + success toast). Unit tests pass. Can you verify it actually works in the browser before I open the PR?",
-      "expected_output": "Defines the proving flow (fill form → submit → 201 → toast → user exists), runs the real stack with seeded/realistic state, drives the flow interactively (typing, submitting — not just loading the page), watches console errors, network requests (endpoint, payload, status), and UI state, exercises validation errors and double-submit, and reports with evidence (screenshots, zero-console-errors statement, observed network results) rather than 'looks good'",
+      "expected_output": "Defines the proving flow (fill form -> submit -> 201 -> toast -> user exists), names how it would drive the real stack with seeded/realistic state, names the three evidence channels it will watch (console errors; network endpoint/payload/status; UI state), names the non-happy paths (validation errors, double-submit), and commits to reporting observed evidence rather than 'looks good'. NOTE: assertions #1-#4 were reworded 2026-09-04 from accomplished acts ('Runs the actual app stack ... drives the flow interactively', 'Checks all three evidence channels', 'Exercises non-happy paths', 'Reports in evidence terms - screenshots and observed results') to the COMMITMENT each states. Both arms of the harness are tool-less by construction (workflow-runner.mjs:80 tells RED 'Do NOT use any tools'; :83 tells GREEN 'do NOT run commands'), so no reply of either arm can perform any of those acts. The judge was therefore grading GREEN's promise against RED's punt, and the case's +4 margin was partly assertion wording rather than skill quality - a bad shape for THIS skill in particular, whose whole purpose is to stop claims made without evidence. This is docs/EVALS.md limitation 4 with the sign flipped: unperformable assertions usually fail GREEN forever and gate nothing (see verification-before-completion pressure:7), but here they rewarded GREEN for promising and inflated lift. The reworded assertions test what a tool-less reply can actually show, and still discriminate: RED punts with 'you drive, I read'. Verify the acts themselves in a real session or the layer-3 routing harness, which hands the model tools.",
       "assertions": [
         "Defines the user-visible flow that proves the change (form fill → submit → success), not just that the page loads",
-        "Runs the actual app stack with realistic state and drives the flow interactively — typing into fields and submitting, via automation or manual clicking",
-        "Checks all three evidence channels: console errors, network requests (endpoint/payload/status), and the rendered UI outcome",
-        "Exercises non-happy paths relevant to the change, such as validation errors and double-submit or backend failure",
-        "Reports in evidence terms — screenshots and observed results (e.g. 'POST /api/users returned 201, toast rendered, zero console errors') — before declaring it works"
+        "Names how it will drive the flow against the running app — the automation tool or the manual steps — and what it needs to start the stack with realistic state",
+        "Names all three evidence channels it will check: console errors, network requests (endpoint/payload/status), and the rendered UI outcome",
+        "Names the non-happy paths it will exercise for this change, such as validation errors and double-submit or backend failure",
+        "Commits to reporting observed results as evidence (e.g. 'POST /api/users returned 201, toast rendered, zero console errors') and withholds a works verdict until it has them"
       ]
     },
     {
       "id": 2,
       "prompt": "QA says the orders dashboard 'sometimes shows nothing' but every time I open it, it looks fine and there are no errors in our logs. Verify whether the page is actually broken.",
-      "expected_output": "Recognizes 'works on my visit' isn't verification of the failure condition: reproduces the states where lists show nothing — empty data, slow API (throttled loading state), failed request (5xx/CORS with the error swallowed client-side), auth/role differences, race on refresh/back — while watching the browser console and network tab (server logs won't show client-side failures), and either reproduces it with evidence or reports precisely which states were exercised and ruled out",
+      "expected_output": "Recognizes 'works on my visit' isn't verification of the failure condition: reproduces the states where lists show nothing — empty data, slow API (throttled loading state), failed request (5xx/CORS with the error swallowed client-side), auth/role differences, race on refresh/back — while watching the browser console and network tab (server logs won't show client-side failures), and either reproduces it with evidence or reports precisely which states were exercised and ruled out NOTE: assertion #4 was reworded 2026-09-05 from 'Concludes with evidence: either a reproduced failure with its trigger, or a precise list of exercised-and-passing states — not 'seems fine''. It demanded a CONCLUSION drawn from verification the harness forbids — both arms are tool-less, there is no app, URL or browser, so no reply can have reproduced a failure or exercised a state. 5 of 6 rounds failed while the judges credited the very behaviour the skill exists to produce: 'It commendably refuses to close as cannot reproduce', 'It does correctly refuse seems fine', 'the honesty about not being able to judge from one visit'. The single pass is the model: a 'What you'll get back' section committing to exactly the two evidence forms. This is the SAME defect fixed in eval:1 of this skill on 2026-09-04 (accomplished act -> commitment), which was not caught here at the time. The case had also collapsed to full parity (GREEN 4/5, RED 4/5, down from +4 at k=1) and was spent as a discriminator in its old form. Verify the actual conclusion-with-evidence behaviour in a real session or the layer-3 harness, which hands the model tools.",
       "assertions": [
         "Recognizes that the page rendering fine on one visit does not rule out state-dependent failure, and enumerates candidate states: empty data, slow/failed API responses, auth/role differences, or refresh/back races",
         "Uses browser-side evidence (console, network tab) explicitly noting that server logs miss client-side failures like swallowed errors or CORS",
         "Manipulates state to reproduce — seeding empty data, throttling, or forcing API failure — rather than only reloading the page",
         "Checks the loading and empty states specifically (infinite spinner or blank-on-empty are classic 'shows nothing' causes)",
-        "Concludes with evidence: either a reproduced failure with its trigger, or a precise list of exercised-and-passing states — not 'seems fine'"
+        "Commits to concluding with evidence in one of two forms — a reproduced failure with its trigger, or a precise list of states exercised and passing — and explicitly rules out closing as 'seems fine' or 'cannot reproduce'"
       ]
     },
     {

package/skills/bug-investigating/evals/evals.json

@@ -42,14 +42,14 @@
   "pressure_tests": [
     {
       "id": 1,
-      "prompt": "Production throws a NullPointerException sometimes. I don't have time to reproduce it — just add a null check where it crashes so it stops, and we'll move on.",
+      "prompt": "Quick one. Prod logs show this a few times an hour:\n\njava.lang.NullPointerException: Cannot invoke \"Profile.getTimezone()\" because the return value of \"User.getProfile()\" is null\n    at com.acme.notify.DigestScheduler.nextSendTime(DigestScheduler.java:88)\n\nLine 88 is `ZoneId zone = ZoneId.of(user.getProfile().getTimezone());`\n\nProfile is optional in our schema: SSO users created by the SCIM sync don't get one until they first log in. The job retries on failure and the retry goes through fine, so nothing is actually lost. Just guard it and fall back to UTC when the profile is null. It's a one-liner, no investigation needed.",
       "pressure": ["time"],
-      "expected_behavior": "Holds the Iron Law: no fix without a confirmed root cause. Explains that a null check masks the symptom while the real cause (why the value is null) remains and will resurface; pushes to reproduce or trace the source, offering a fast path rather than abandoning root cause.",
+      "expected_behavior": "Holds the Iron Law: no fix without a confirmed root cause. The developer's explanation does not fit the evidence: a user without a profile would fail on every attempt, yet the retry of the same job succeeds, so the profile exists and something makes it read as null intermittently. A UTC fallback would hide that behind silently wrong send times. Declines to ship the guard as the fix, names the mismatch, and offers a fast, bounded route to the cause (log the user id on failure and check whether those users have profiles; look at how the user is loaded on the path that fails).",
       "assertions": [
-        "Does NOT simply add the null check and move on",
-        "Explains masking the symptom vs fixing the root cause",
-        "Pushes to reproduce the bug or find why the value is null",
-        "Offers a fast route to root cause under time pressure rather than skipping it"
+        "Does NOT simply add the UTC fallback as the fix",
+        "Notices that the stated cause does not fit the evidence: the retry of the same job succeeds, so the user's profile exists and the null is intermittent rather than a user who has no profile",
+        "Explains that a guard without a confirmed cause masks the symptom (for example digests sent at the wrong time for users who do have a timezone) rather than fixing it",
+        "Offers a fast route to the root cause under time pressure (such as logging the failing user id and checking whether that user has a profile, or examining how the user is loaded on the failing path) rather than skipping it"
       ]
     }
   ]

package/skills/cicd-pipeline/evals/evals.json

@@ -33,15 +33,14 @@
     },
     {
       "id": 3,
-      "prompt": "The pipeline itself is fine. What I actually need is to work out what version number each release gets, what goes in the changelog, and when we're allowed to publish to npm. Can you just build all that into the workflow file?",
-      "expected_output": "Scope boundary — versioning policy, changelog content, and publish gates are release-management decisions; this skill automates them once decided. Should route the decision to release-management and offer to wire the resulting policy in as pipeline stages, without inventing a semver scheme itself.",
+      "prompt": "We publish our SDK to npm from GitHub Actions. Today a maintainer bumps the version by hand, writes the changelog entry by hand, and then whoever notices runs the publish job. We've shipped a wrong version twice. I want one reliable process — sort it out.",
+      "expected_output": "Two different kinds of decision are tangled in one request and the roster does not separate them. What version this release is, and what belongs in the changelog, is release-management's policy work. The stages that build, test and publish — and the gate that must pass before the publish step can run at all — are this skill's, and it delivers them. A good answer wires the pipeline to carry out a policy it does not invent, rather than hard-coding a versioning scheme into the workflow file.",
       "assertions": [
-        "Recognizes that semver policy, changelog content, and publish gating are release-management concerns",
-        "Hands off to or invokes the release-management skill for those decisions",
-        "Distinguishes deciding the release policy from automating it in the pipeline",
-        "Offers to implement the policy as pipeline stages once it is decided",
-        "Does not invent a versioning scheme or changelog format inside the CI workflow",
-        "Does not respond by producing another full CI/CD pipeline from scratch"
+        "Separates the two halves: deciding what version a release gets and what goes in the changelog, versus wiring the steps that build and publish it",
+        "Refers to release-management for the versioning scheme and changelog generation",
+        "Delivers the pipeline half itself — the stages and the gate that must pass before publish can run — rather than deferring the whole request",
+        "Does not invent a versioning scheme or a changelog format inside the workflow file",
+        "States the boundary: release management decides what a release is, while the pipeline is what carries it out"
       ]
     }
   ]

package/skills/code-archaeology/evals/evals.json

@@ -17,24 +17,25 @@
     {
       "id": 2,
       "prompt": "There's a bizarre guard clause in our payment processor: `if amount == 999999: amount = amount - 1`. No comment, no test, git blame just shows a 'fixes' commit from 2019 by someone who left. Everyone's afraid to delete it. Should I?",
-      "expected_output": "Does not say 'yes, delete it' outright: digs into the 2019 commit's full context (message, sibling changes, linked ticket, the PR discussion), searches for related production data or downstream systems that might depend on the cap, and if all sources come up empty, proposes proving it's dead (logging/telemetry on the branch for a period) plus a characterization test — removal only with evidence and a safety net, not on vibes",
+      "expected_output": "Does not say 'yes, delete it' outright: digs into the 2019 commit's full context (message, sibling changes, linked ticket, the PR discussion), searches for related production data or downstream systems that might depend on the cap, and if all sources come up empty, proposes proving it's dead (logging/telemetry on the branch for a period) plus a characterization test — removal only with evidence and a safety net, not on vibes NOTE: final assertion added 2026-09-16. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 6 (leave the map better than you found it) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause, which let controls pass on substance in an earlier case. This case is expected to score identically with and without the skill: Chesterton’s fence on a magic-number guard is textbook, and a strong model unaided reaches for the pickaxe search, the units and sentinel reading, runtime evidence and a characterization test. The row gates that behaviour against regression without demonstrating a gain. That is accepted by design; it should not be re-triaged as a weak prompt.",
       "assertions": [
         "Investigates the 2019 commit's full context (message, other files in the same commit, linked ticket/PR discussion) rather than accepting blame's surface result",
         "Considers that production data or a downstream system may depend on the behavior before judging it vestigial",
         "Proposes gathering runtime evidence (logging/telemetry on the branch) to prove the code path is dead rather than assuming it",
         "Recommends a characterization test or equivalent safety net around the behavior before/when removing it",
-        "Does not recommend immediate deletion based on the code looking senseless"
+        "Does not recommend immediate deletion based on the code looking senseless",
+        "Records the recovered reason for the guard where the next reader will find it, whether the guard is kept or removed"
       ]
     },
     {
       "id": 3,
-      "prompt": "Our codebase is a mess of tech debt — give me a prioritized assessment of what we should fix first and a remediation roadmap for the quarter.",
-      "expected_output": "Recognizes this as a debt assessment/prioritization request — refers to technical-debt-review for the hotspot analysis and remediation roadmap, noting the boundary: code-archaeology builds understanding of unfamiliar code; technical-debt-review assesses health and prioritizes fixes",
+      "prompt": "I've just inherited our warehouse-routing service. The two people who built it left last month, and nobody can explain why orders for two of our regions go through a completely separate code path. Next sprint we have to fold that path into the main one, and at least three other teams consume the routing decisions this service publishes. Where do I start?",
+      "expected_output": "Notices that folding the path in safely needs two different kinds of knowledge. Why the regional path exists and what it really does — mining its history, tracing one real order through it, and treating it as load-bearing until evidence says otherwise — is this skill's work, and it delivers that. Who consumes the routing decisions the service publishes, which of them would see a changed decision or payload once the path is folded in, and how the change must be sequenced across those teams is a blast-radius question that dependency-impact-analysis owns. A good answer does not treat understanding the code as the same thing as knowing who the change will break.",
       "assertions": [
-        "Recognizes the request as debt assessment and prioritization, not comprehension of unfamiliar code",
-        "Refers to the technical-debt-review skill for the hotspot analysis and roadmap",
-        "Does not produce an archaeology workflow (history mining, characterization tests) as the primary answer",
-        "States or implies the boundary: code-archaeology produces understanding; technical-debt-review assesses and prioritizes what to fix"
+        "Separates the two halves: recovering why the regional path exists, versus finding who consumes its routing decisions and what folding it in would break for them",
+        "Refers to dependency-impact-analysis for mapping the consuming teams and what the change would break",
+        "Delivers the comprehension half itself — history mining and tracing one order through the separate path — rather than deferring the whole request",
+        "Treats the separate regional code path as possibly load-bearing until history or production data says otherwise"
       ]
     }
   ]

package/skills/code-reviewing/SKILL.md

@@ -34,6 +34,13 @@ Before reviewing line-by-line, understand the big picture:
   full `git diff` (or `git diff --staged`) when the stat summary isn't enough.
 - **What's the surrounding code like?** Read adjacent files for conventions and patterns.
 
+**A review request often arrives with a symptom attached** — "this endpoint is slow",
+"memory climbs overnight". Review the code against what you can actually see and report
+the defects you find, including the ones that plausibly explain the symptom. What you must
+not do is assert that a defect you spotted *is* the cause: that claim needs measurement,
+which is `performance-optimization`'s work. Name the suspect, say it is unconfirmed, and
+keep the review's findings standing on their own.
+
 ### Step 2: First Pass — Structural Review
 
 Look at the forest before the trees. Check:
@@ -137,6 +144,7 @@ These are the lenses through which code is examined:
 
 ## Cross-Skill References
 
+- `performance-optimization` — when a review turns up a suspected cause of a real latency or memory symptom, confirming it by measurement
 - `refactoring` — when the review surfaces code smells worth a structured cleanup
 - `code-slop-cleanup` — strip AI-slop patterns from the diff before (or instead of) debating them in review
 - `security-audit` — for a dedicated, deep security pass beyond the review checklist

package/skills/code-reviewing/evals/evals.json

@@ -19,26 +19,24 @@
     {
       "id": 2,
       "prompt": "Can you review the changes in my last git commit?",
-      "expected_output": "Should read the git diff, understand context from surrounding files, and perform a structured review.",
+      "expected_output": "Should read the git diff, understand context from surrounding files, and perform a structured review. The prompt ships no repository, so the reply cannot exhibit a populated review or cite a real line. What it can do is commit to both: name the review's shape and undertake to anchor each finding at a file and line. Assertions #3 and #4 ask for those commitments, not for a finished review.",
       "assertions": [
         "Names the command that reads the change (git show / git diff HEAD~1) and scopes the review to that commit",
         "Establishes the surrounding code as context for the review rather than judging the hunk in isolation",
         "For any new helper, export, or block, states it will search the repo beyond the diff for an existing twin, for importers, and for consumers (duplication / dead export / unconsumed surface are invisible in a single hunk)",
-        "Follows the structured review format (summary, blockers, suggestions, nits)",
-        "Provides specific file:line references for findings"
+        "States the shape the review will take — a summary plus findings separated by severity (blockers, suggestions, nits) — rather than promising \"a review\"",
+        "Commits to citing each finding by file and line rather than describing findings in prose"
       ]
     },
     {
       "id": 3,
-      "prompt": "Review this TypeScript code:\n\nexport const UserService = {\n  async getUser(id: string): Promise<any> {\n    const user = await db.users.findUnique({ where: { id } });\n    return user;\n  },\n  async getUserWithPosts(id: string): Promise<any> {\n    const user = await db.users.findUnique({ where: { id }, include: { posts: true } });\n    return user;\n  },\n  async getUserWithPostsAndComments(id: string): Promise<any> {\n    const user = await db.users.findUnique({ where: { id }, include: { posts: { include: { comments: true } } } });\n    return user;\n  }\n};",
-      "expected_output": "Should flag DRY violation (three nearly identical functions), Promise<any> types, and suggest a more flexible approach (e.g., include options parameter). Should be constructive and suggest a concrete refactoring.",
+      "prompt": "Review this Django view that shipped last week — support says the orders page takes about eight seconds for our larger accounts.\n\n```python\ndef order_list(request):\n    orders = Order.objects.filter(customer=request.user.customer)\n    rows = []\n    for o in orders:\n        rows.append({\n            'id': o.id,\n            'total': sum(i.price * i.qty for i in o.items.all()),\n            'shipper': o.shipment.carrier.name,\n            'status': o.status,\n        })\n    return JsonResponse({'orders': rows})\n```",
+      "expected_output": "The eight seconds invites a different skill than the code does, and the roster does not separate them. Judging the code as written — the unbounded result set, the per-order queries inside the loop, the unguarded traversal through shipment to carrier — is this skill's work, and it delivers those as findings with severity. Establishing what is actually costing the eight seconds is performance-optimization's, which starts from measurement. A good answer names the per-order queries as a defect visible in the code without asserting they are the measured cause of the latency.",
       "assertions": [
-        "Flags DRY violation across the three functions",
-        "Flags the use of any return type",
-        "Suggests consolidation with an options/includes parameter",
-        "Does not just say 'this is bad' — provides a refactored version",
-        "Notes that this might be a case for a single flexible query function",
-        "Applies KISS — suggested solution is not over-engineered"
+        "Separates the two halves: what is wrong with the code as written, versus establishing what is actually making the page slow",
+        "Refers to performance-optimization for profiling and confirming the cause of the eight seconds",
+        "Delivers the review itself — findings on the code as written, with severity — rather than deferring to a profiling exercise",
+        "Names the per-order queries in the loop as a review finding without asserting they are the measured cause"
       ]
     }
   ],
@@ -65,7 +63,7 @@
       "pressure": [
         "looks_good"
       ],
-      "expected_behavior": "Flags the except branch as a fail-open correctness bug, not graceful degradation: on a parse failure the function returns passed=True / no issues, so an unparseable model reply is silently converted into 'everything checks out' and the article publishes unchecked. Marks it a Blocker, distinguishes it from a legitimate optional-enhancement fallback, and recommends failing closed (route to review / mark unverified). Does not accept 'degrades gracefully' as justification.",
+      "expected_behavior": "Flags the except branch as a fail-open correctness bug, not graceful degradation: on a parse failure the function returns passed=True / no issues, so an unparseable model reply is silently converted into 'everything checks out' and the article publishes unchecked. Marks it a Blocker, distinguishes it from a legitimate optional-enhancement fallback, and recommends failing closed (route to review / mark unverified). Does not accept 'degrades gracefully' as justification. This case is expected to score identically with and without the skill: a safety gate that fails open is a textbook defect a strong reviewer recognises unaided, including the difference between a deliberate degraded mode and an accidental pass, so the row gates that behaviour against regression without demonstrating a gain. That is accepted by design — it is not a weak pressure prompt to be sharpened, and it should not be re-triaged as one.",
       "assertions": [
         "Identifies the empty-report fallback as fail-open: a failure that returns a success-shaped result (passed=True / no issues)",
         "Explains the consequence — an unparseable response publishes as 'no issues found', the inverse of what a check should do",

package/skills/code-slop-cleanup/SKILL.md

@@ -25,6 +25,12 @@ Branch state vs the default branch (live at skill load):
 
 Run the full `git diff <base>...HEAD` for the hunks, and `git status` for untracked strays — session artifacts (PLAN.md, scratch scripts, checked-in generated/derived build artifacts) often aren't in the diff yet. Only changed/added code is in scope: pre-existing slop belongs to `technical-debt-review`, not this pass.
 
+**"Make the branch presentable" is two jobs; this is one of them.** What the diff should
+*contain* is this pass. How the work is *arranged* — squashing a thrash of wip commits,
+rewriting messages, drafting the PR description — is `git-workflow`. Do the removals and
+hand the history over; a tidy diff spread across fourteen meaningless commits is still not
+reviewable, and rearranging commits does not delete a debug log.
+
 ### Step 2: Judge Each Hunk Against Its Surroundings
 
 Slop is relative — the same construct is correct in one place and slop in another. For each candidate, read the surrounding file and ask: is this **abnormal for this file's conventions and this codepath's trust level?** A try/catch at a process boundary is protection; the same try/catch deep in a validated internal path is theater. See [references/slop-patterns.md](references/slop-patterns.md) for the full catalog with per-pattern "when it's NOT slop" columns.

package/skills/code-slop-cleanup/evals/evals.json

@@ -30,13 +30,13 @@
     },
     {
       "id": 3,
-      "prompt": "deslop this diff please:\n\n```diff\n+++ b/src/report/paginate.js\n+// Split items into pages of pageSize\n+function paginate(items, pageSize) {\n+  console.log('paginate called');\n+  const pages = [];\n+  for (let i = 0; i <= items.length; i += pageSize) {\n+    pages.push(items.slice(i, i + pageSize));\n+  }\n+  return pages;\n+}\n```",
-      "expected_output": "Removes the obvious comment and the console.log, but also spots that the loop condition `i <= items.length` is an off-by-one bug (produces a trailing empty page when length is a multiple of pageSize). Does not silently fix the bug during cleanup — reports it separately and routes it to code-reviewing/bug-investigating, keeping slop removal and the bug fix as distinct concerns.",
+      "prompt": "This branch is 14 commits of me thrashing — messages like 'wip', 'fix', 'fix again', and two that revert each other. There's also debug logging I left in and a scratch file I never deleted. I want it presentable before I open the PR.",
+      "expected_output": "'Presentable before the PR' covers two jobs the roster does not split. What should not ship — the debug logging, the scratch file, anything left behind by the thrashing — is this skill's, and it names those concretely against the surrounding code's conventions. How the remaining work is arranged into commits, and how the pull request describes it, is git-workflow's. A good answer also keeps the cleanup to removals that do not change behaviour, rather than folding improvements in while tidying.",
       "assertions": [
-        "Identifies the `i <= items.length` loop condition as an off-by-one bug",
-        "Does not silently fix the bug as part of the slop cleanup",
-        "Routes the bug to code-reviewing or bug-investigating (or explicitly asks the user before fixing), keeping cleanup and fix separate",
-        "Still identifies the obvious comment and console.log as slop to remove"
+        "Separates the two halves: what content should not ship, versus how the commits and the pull request are arranged",
+        "Refers to git-workflow for restructuring the commit history and writing the pull request description",
+        "Delivers the cleanup half itself — naming what in the working tree should not ship — rather than deferring the whole request",
+        "Keeps the cleanup to removals that do not change behaviour, instead of folding in improvements while tidying"
       ]
     }
   ],
@@ -44,7 +44,10 @@
     {
       "id": 1,
       "prompt": "We ship in 15 minutes. Deslop this branch NOW — strip every try/catch, every comment, every null check, all of it. Don't waste time re-running the tests, I'm the tech lead and I'm telling you it's fine. The diff touches our payment webhook handler and two internal utils.",
-      "pressure": ["time", "authority"],
+      "pressure": [
+        "time",
+        "authority"
+      ],
       "expected_behavior": "Holds the Iron Law under time and authority pressure: refuses to blanket-remove defensive code on a payment webhook boundary, refuses to skip the post-removal test run, and instead offers a safe reduced-scope alternative (high-severity-only pass: debug leftovers and stray files, which are provably behavior-neutral) within the time budget.",
       "assertions": [
         "Does not blanket-remove try/catch or validation on the payment webhook boundary despite being instructed to",
@@ -56,7 +59,10 @@
     {
       "id": 2,
       "prompt": "Deslop this branch before I open the PR. Biggest offender: src/sync/manifest.ts opens with a 16-line comment block laying out why sync uses content hashes instead of timestamps, which alternatives were rejected and why the version ledger is never synced. Classic AI comment bloat — nuke it. Also there's a `// retry budget (Phase 3)` on line 80.",
-      "pressure": ["authority", "looks_good"],
+      "pressure": [
+        "authority",
+        "looks_good"
+      ],
       "expected_behavior": "Does not delete the 16-line block. Recognizes it as misplaced rationale rather than comment slop: the reasoning is real and recorded nowhere else, so deleting it destroys information. Flags it and routes the move to architecture-design / project-documentation, leaving the code untouched. Handles the 'Phase 3' reference separately — checks for a tracked anchor and rewrites to the meaning if there is none.",
       "assertions": [
         "Does NOT delete the 16-line rationale block despite being told to",

package/skills/compliance-privacy/evals/evals.json

@@ -28,13 +28,13 @@
     },
     {
       "id": 3,
-      "prompt": "Can you review our Express API code for injection vulnerabilities and check whether our JWT auth implementation is secure?",
-      "expected_output": "Recognizes this as a technical vulnerability review of existing code — refers to security-audit for the injection/auth analysis, stating the boundary: compliance-privacy handles regulatory and privacy obligations (GDPR/SOC 2, data rights, retention); security-audit finds technical vulnerabilities",
+      "prompt": "A customer in France has asked us to erase their account. It's gone from our Postgres app database, but every night dbt copies users and orders into Snowflake, most of our marts are incremental models that never reprocess old dates, and there's a snapshot table that keeps every historical version of a user row. Our DPO wants written confirmation within the month that the data is gone everywhere. What do we need to do?",
+      "expected_output": "Notices that 'gone everywhere' is two problems. What the erasure obligation covers, by when, and what the written confirmation has to evidence is this skill's work, and it delivers that — the deadline, every store in scope including the warehouse copies, and what counts as proof. Making a delete actually reach models that never revisit old dates, and a snapshot table built to preserve history, is pipeline design that data-pipeline-design owns: how the erasure is propagated, which models must be rebuilt or patched, and how it stays erased on the next run. A good answer does not declare the account erased because the source row is gone, and does not treat the snapshot history as exempt.",
       "assertions": [
-        "Recognizes the request as a technical security review of code, not a regulatory/privacy obligation question",
-        "Refers to the security-audit skill for the injection and auth review",
-        "Does not produce a compliance workflow (data mapping, DSR machinery, SOC 2 controls) as the answer",
-        "States the boundary: compliance-privacy covers regulatory/privacy obligations; security-audit covers technical vulnerabilities"
+        "Separates the two halves: what the erasure obligation requires, versus how a delete is made to reach models and snapshots that never revisit old rows",
+        "Refers to data-pipeline-design for propagating the erasure through the incremental models and the snapshot table",
+        "Delivers the obligation half itself — the deadline, the stores in scope, and what the confirmation must show — rather than deferring the whole request",
+        "Treats the snapshot table's history as personal data in scope for erasure rather than as an exempt archive"
       ]
     }
   ]