swe-workflow-skills 0.8.2 → 0.9.1

package/README.md

@@ -6,76 +6,29 @@
 [![license: MIT](https://img.shields.io/badge/license-MIT-yellow.svg)](LICENSE)
 
 A curated library of **66 Claude Code Agent Skills** that walk Claude through the
-software lifecycle the way a disciplined senior engineer would — planning, design,
+software lifecycle the way a disciplined senior engineer would: planning, design,
 TDD, review, security, deployment, incidents, and the project-management work around
 them.
 
-Each skill encodes a *method* (DRY, YAGNI, KISS, clean architecture, evidence-before-done),
-not just a task. They compose into end-to-end workflows and are kept honest by an
-LLM-as-judge eval harness.
+Each skill encodes a *method* (DRY, YAGNI, KISS, clean architecture, evidence before
+"done"), not just a task. An orchestrator skill, **`skill-router`**, matches your intent
+to the right skills and chains them across phases. Every skill is kept honest by an
+**LLM-as-judge evaluation harness** that compares the model with and without the skill.
 
 ![skill-router routing a live session](docs/demo/routing.gif)
 
-## Why this library
-
-Not just a pile of skills — a routing layer over Claude Code's native skill
-activation, an eval discipline, and full-SDLC breadth that popular collections
-don't cover:
-
-- **An orchestrator, not auto-trigger roulette.** A `skill-router` skill maps your
-  intent to the right skill(s) and chains them across phases. Community measurements put
-  Claude's own description-based auto-triggering at roughly a coin flip to ~84%; here
-  activation is *routed and deterministic* instead, and the routing eval harness measures
-  it (top-1 accuracy 1.00, zero misroutes on the committed baseline, regression-gated in
-  CI) — see the [routing benchmark](docs/ROUTING-BENCHMARK.md). This is the one thing the
-  platform still doesn't do for you.
-- **Every skill stays reachable, no pre-picking.** Claude Code injects skill descriptions
-  only up to ~1% of context, so large libraries silently stop auto-triggering past ~20–40
-  skills. The fix is Claude Code's own **`name-only`** state (its docs recommend it for
-  low-priority skills); this library just applies it wholesale — keeping the router +
-  safety skills "loud" and listing the rest name-only — so the router can still invoke
-  every skill by name, with no cropping and without making you pre-pick a subset
-  ([how it works](docs/ROLES.md)).
-- **Tested like code, not prose.** Every skill ships 3 evals; two LLM-as-judge
-  harnesses (content quality and routing accuracy) replay them with skill loaded vs
-  absent and gate regressions in CI. Across all 66 skills — 234 cases, 1311 assertions,
-  every row measured at k>=3 — the same model passes **65.2%** of assertions without the
-  skill's content and **97.9%** with the skill loaded, and **no skill scores below the
-  no-content baseline on any case**
-  ([results](docs/EVALS.md#results-content-evals-full-catalog)). Safety-critical skills
-  (deploys, releases, tests, incidents, security) are **hardened**: an Iron Law, a
-  rationalization table distilled from real baseline failures, and pressure tests that
-  try to talk the agent out of it.
-- **Curated, not a mega-catalog.** First-party skills, versioned and eval-gated in the
-  repo — no arbitrary third-party skills pulled from a hub, and nothing that executes on
-  its own (the content is instructions and templates; the only code is the open Node
-  installer). In an ecosystem where researchers have flagged hundreds of malicious
-  community skills, curated-and-reviewed is itself a feature — see [SECURITY.md](SECURITY.md).
-- **Full-SDLC breadth.** Planning, architecture/ADRs, API and data design, TDD, review,
-  security, releases, deploys, GitOps, observability, incidents, MLOps, LLM apps
-  and AI evaluation, data pipelines and data quality, and the PM/strategy work
-  around them — a wider slice of the lifecycle than the community collections we
-  know of aim for (they tend to go deep on the coding inner loop; see
-  [Acknowledgements](#acknowledgements)).
-- **Role-scoped.** `/role backend` (or `frontend`, `devops`, `ml`, `ai`, `data`,
-  `data-scientist`, `security`, `architect`, `em`, `pm`, `strategy`, `qa`, `mobile`,
-  `designer`) promotes a working set to auto-trigger; the rest stay one route away.
-- **Cross-platform install.** The installer and SessionStart hook are **pure Node** — the
-  one runtime Claude Code already requires — so they run identically on Linux, macOS, and
-  Windows (no bash, Python, or `sed`).
-
 ## Quick Start
 
-**Most people — install the per-role plugin for your hat** (works on CLI, Claude Code
-web, claude.ai chat, and Cowork):
+**Most people: install the per-role plugin for your hat.** It works on the CLI, Claude
+Code web, claude.ai chat, and Cowork:
 
 ```text
 /plugin marketplace add SWEStash/swe-workflow-skills
 /plugin install swe-workflow-pm@swe-workflow
 ```
 
-**Want the whole library with the orchestrator** (CLI) — no clone needed. This is a
-**two-step** setup:
+**Want the whole library with the orchestrator?** On the CLI, with no clone needed, it
+is a **two-step** setup:
 
 ```bash
 # 1. Install all 66 skills + router + /role + the hook script, and apply the baseline
@@ -84,16 +37,16 @@ npx swe-workflow-skills install --global
 
 ```jsonc
 // 2. Wire the hook: merge the snippet the installer prints into your settings.json
-//    (the installer never edits settings.json for you — it just prints the block).
+//    (the installer never edits settings.json for you; it just prints the block).
 //    Then start a new session; /doctor should show the hook registered.
 { "hooks": { "SessionStart": [ /* ...the printed block... */ ] } }
 ```
 
 > ⚠️ **Don't skip step 2.** Step 1 alone installs the skills and prevents cropping, but
-> the hook is what nudges Claude to consult `skill-router` first — **without it, skills
-> won't auto-route** (the whole point of the library) and the baseline isn't re-asserted
+> the hook is what nudges Claude to consult `skill-router` first. **Without it, skills
+> won't auto-route** (the whole point of the library), and the baseline isn't re-asserted
 > after `/compact`. If you deliberately want no hook, pass `--no-hook`; auto-routing stays
-> off until a hook is wired. The **per-role plugin** path above needs no hook — it's
+> off until a hook is wired. The **per-role plugin** path above needs no hook: it is
 > self-contained.
 
 Or from a clone: `node install.mjs --global`.
@@ -119,20 +72,20 @@ npx swe-workflow-skills uninstall --dry-run   # preview removal; --global/--dir
 
 From a clone, the same commands are `node install.mjs …` / `node uninstall.mjs …`.
 
-The installer never edits `settings.json` — it prints the SessionStart hook snippet for
+The installer never edits `settings.json`: it prints the SessionStart hook snippet for
 you to merge. Re-running is idempotent. See **[INSTALL-MATRIX.md](docs/INSTALL-MATRIX.md)**
 for every method × surface, and **[ROLES.md](docs/ROLES.md)** for the activation model.
 
 ## Usage
 
-On any non-trivial task, Claude consults **`skill-router`** first; it reads the full
+On any non-trivial task, Claude consults **`skill-router`** first. It reads the full
 catalog and invokes the matching skill(s) by name, re-routing as the work changes phase.
-The default SessionStart hook nudges Claude to do this automatically; you can also route
-explicitly ("use the security-audit skill") or switch the promoted set with `/role`. Don't
-want a particular skill routed at all? `npx swe-workflow-skills disable <skill>` opts it out
-durably — see [disable a skill from routing](docs/ROLES.md#advanced-disable-a-skill-from-routing).
+The default SessionStart hook nudges Claude to do this automatically. You can also route
+explicitly ("use the security-audit skill") or switch the promoted set with `/role`.
+Don't want a particular skill routed at all? `npx swe-workflow-skills disable <skill>`
+opts it out durably; see [disable a skill from routing](docs/ROLES.md#advanced-disable-a-skill-from-routing).
 
-**A routed chain, by phase** — e.g. *"add OAuth login"*:
+**A routed chain, by phase**, e.g. *"add OAuth login"*:
 
 ```
 feature-planning      →  scope tasks, acceptance criteria, risks
@@ -146,16 +99,37 @@ verification-before-completion
 deployment-checklist  →  pre-deploy safety + rollback readiness
 ```
 
-The router invokes each skill as you reach its phase rather than all at once. A single
-request often fans out to several skills — *"review this for accessibility and UX"* pulls
-in both `accessibility-design` and `ui-ux-design`. See a full session play out in
-**[what routing looks like](docs/ROLES.md#what-routing-looks-like-in-a-session)**; the
-pre-built chains (new feature · bug/incident · continuous improvement · pre-public
-review) live in the `skill-router` skill and **[ROLES.md](docs/ROLES.md)**.
+The router invokes each skill as you reach its phase rather than all at once, and one
+request can fan out to several skills. See a full session in
+**[what routing looks like](docs/ROLES.md#what-routing-looks-like-in-a-session)**.
+
+## Why this library
+
+- **An orchestrator, not auto-trigger roulette.** Description-based auto-triggering is
+  probabilistic. Here `skill-router` routes intent to skills by name, and a routing
+  harness measures it: 65/65 top-1 and 64/65 on boundary cases on the committed k=3
+  baseline ([routing benchmark](docs/ROUTING-BENCHMARK.md)).
+- **Every skill stays reachable.** Claude Code lists skill descriptions only up to about
+  1% of context, so large libraries stop auto-triggering past 20 to 40 skills. This
+  library keeps the router and safety skills fully listed and the rest **`name-only`**
+  (Claude Code's own setting for low-priority skills), so every skill is one route away
+  with no pre-picking ([how it works](docs/ROLES.md)).
+- **Tested like code, not prose.** Across 234 eval cases and 1323 assertions, the same
+  model passes **63.6%** of assertions without the skill's content and **97.9%** with
+  it, and **no case scores lower with the skill** ([results](docs/EVALS.md#results-content-evals-full-catalog)).
+  Safety-critical skills are **hardened** with an Iron Law, a rationalization table, and
+  pressure tests.
+- **Curated, not a mega-catalog.** First-party and reviewed. Nothing executes on its own
+  except the open Node installer ([SECURITY.md](SECURITY.md)).
+- **Full-SDLC breadth, role-scoped.** From planning to incidents, plus MLOps, LLM apps,
+  data, and PM work. `/role backend` (or any of 15 roles) promotes a working set to
+  auto-trigger.
+- **Cross-platform.** The installer and hook are pure Node, so they run the same on
+  Linux, macOS, and Windows.
 
 ## What's included
 
-66 skills — **[full catalog → SKILLS.md](docs/SKILLS.md)**:
+66 skills, listed in the **[full catalog → SKILLS.md](docs/SKILLS.md)**:
 
 | Area | Count | Examples |
 |------|-------|----------|
@@ -171,31 +145,29 @@ review) live in the `skill-router` skill and **[ROLES.md](docs/ROLES.md)**.
 | Evaluation & Monitoring | 2 | observability-design, test-data-strategy |
 | Meta | 2 | skill-router, writing-skills |
 
-## Documentation
+## Evaluation
 
-- **[ROLES.md](docs/ROLES.md)** — the activation model (name-only baseline, roles, the orchestrator) and the CLI vs web/plugin paths.
-- **[INSTALL-MATRIX.md](docs/INSTALL-MATRIX.md)** — every install method × surface, side by side.
-- **[SKILLS.md](docs/SKILLS.md)** — the full skill catalog by area.
-- **[EVALS.md](docs/EVALS.md)** — how the skills are tested (RED/GREEN, pressure tests, CI gate).
-- **[ROUTING-BENCHMARK.md](docs/ROUTING-BENCHMARK.md)** — the routing reliability numbers, methodology, and how to reproduce them.
-- **[AUTHORING.md](docs/AUTHORING.md)** — write or modify a skill (descriptions, budget, progressive disclosure, evals).
-- **[RELEASING.md](docs/RELEASING.md)** — versioning policy and how to cut a release. Changes are tracked in **[CHANGELOG.md](CHANGELOG.md)**.
-- **[SECURITY.md](SECURITY.md)** — the security model: trust boundaries, what runs automatically, supply-chain guarantees, and how to report a vulnerability.
+Every skill ships 3 evals (happy path, edge case, scope boundary), and safety skills add
+pressure tests. Each case is replayed with the skill loaded (GREEN) and without it (RED),
+and a skeptical LLM-as-judge grades every assertion. Evals run in-session through
+`evals/workflow-runner.mjs`, with no API key, and the results are committed to
+`evals/baseline.json`. Merging a re-run reports any assertion that regressed. Full guide
+in **[EVALS.md](docs/EVALS.md)**.
 
-## Evaluation
+## Documentation
 
-Each skill carries an `evals/` directory; safety/discipline skills add a `pressure_tests`
-block. Two runners replay scenarios through Claude (skill loaded = GREEN, absent = RED)
-and judge each assertion with a skeptical LLM-as-judge: `evals/workflow-runner.mjs`
-(in-session, no API key) and `evals/run.py` (CI regression gate, wired into
-`.github/workflows/skill-evals.yml`). The GREEN arm reads the skill *and* the
-`references/`/`templates/` files it links to, so improving a reference file moves the
-score. The first runner writes `evals/baseline.json`; the second gates against it.
-Full guide in **[EVALS.md](docs/EVALS.md)**.
+- **[ROLES.md](docs/ROLES.md)**: the activation model (name-only baseline, roles, the orchestrator) and the CLI vs web/plugin paths.
+- **[INSTALL-MATRIX.md](docs/INSTALL-MATRIX.md)**: every install method × surface, side by side.
+- **[SKILLS.md](docs/SKILLS.md)**: the full skill catalog by area.
+- **[EVALS.md](docs/EVALS.md)**: how the skills are tested (RED/GREEN, pressure tests, the regression baseline).
+- **[ROUTING-BENCHMARK.md](docs/ROUTING-BENCHMARK.md)**: the routing reliability numbers, methodology, and how to reproduce them.
+- **[AUTHORING.md](docs/AUTHORING.md)**: write or modify a skill (descriptions, budget, progressive disclosure, evals).
+- **[RELEASING.md](docs/RELEASING.md)**: versioning policy and how to cut a release. Changes are tracked in **[CHANGELOG.md](CHANGELOG.md)**.
+- **[SECURITY.md](SECURITY.md)**: the security model, what runs automatically, supply-chain guarantees, and how to report a vulnerability.
 
 ## Contributing
 
-New or improved skills are welcome — start with **[AUTHORING.md](docs/AUTHORING.md)** (or
+New or improved skills are welcome. Start with **[AUTHORING.md](docs/AUTHORING.md)** (or
 install the `writing-skills` skill). The short version: descriptions are everything, keep
 SKILL.md concise with detail in `references/`, and ship exactly 3 evals.
 
@@ -203,23 +175,23 @@ SKILL.md concise with detail in `references/`, and ship exactly 3 evals.
 
 This library stands on ideas from projects we found genuinely useful:
 
-- **[obra/superpowers](https://github.com/obra/superpowers)** — the guardrail pattern
+- **[obra/superpowers](https://github.com/obra/superpowers)**: the guardrail pattern
   our hardened safety skills adopt (an Iron Law, a rationalization table distilled from
   real failures, and pressure tests that try to talk the agent out of it) comes from
   Jesse Vincent's work here, as does the idea of a Socratic brainstorming skill as the
   entry point to a feature. Superpowers goes deeper on the coding inner loop than we do;
   if that's what you want, use it.
-- **[anthropics/skills](https://github.com/anthropics/skills)** — Anthropic's official
+- **[anthropics/skills](https://github.com/anthropics/skills)**: Anthropic's official
   skills (and `skill-creator` in particular) are the authoring practice we benchmark
   ours against; the eval-first loop in [AUTHORING.md](docs/AUTHORING.md) follows the
   same instinct.
 - **The [Agent Skills docs](https://code.claude.com/docs/en/skills) and Anthropic's
-  engineering posts** — progressive disclosure, the listing budget, and the
+  engineering posts**: progressive disclosure, the listing budget, and the
   `name-only` + `skillOverrides` activation surface this library's routing model is
   built on.
-- **The awesome-claude-skills community lists** — the ecosystem survey that shaped our
+- **The awesome-claude-skills community lists**: the ecosystem survey that shaped our
   gap analysis of what a full-SDLC library should cover.
 
 ## License
 
-MIT — see [LICENSE](LICENSE).
+MIT, see [LICENSE](LICENSE).

package/VERSION

@@ -1 +1 @@
-0.8.2
+0.9.1

package/catalog.json

@@ -323,7 +323,7 @@
     },
     {
       "name": "verification-before-completion",
-      "description": "Evidence gate before claiming work is done, passing, fixed, or deployed. Use before saying 'done', 'it works', or 'tests pass', or before committing, pushing, or opening a PR. Run the proving command fresh and read its output first. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix.",
+      "description": "Evidence gate before any claim that work is done, fixed, passing, or deployed. Route here when a message declares the work finished or asks to commit, push, merge, ship, or close it out, including a brief sign-off, and before git-workflow or release-management act on it. Writing the commit message or PR text itself → git-workflow. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix.",
       "path": "verification-before-completion/SKILL.md"
     },
     {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "swe-workflow-skills",
-  "version": "0.8.2",
+  "version": "0.9.1",
   "description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
   "type": "module",
   "bin": {

package/skills/accessibility-design/evals/evals.json

@@ -20,7 +20,7 @@
     {
       "id": 2,
       "prompt": "Audit this React component for accessibility issues:\n\n<div className=\"card\" onClick={() => navigate(`/project/${id}`)}>\n  <div className=\"title\">{name}</div>\n  <img src={thumbnail} />\n  <div className=\"badge\" style={{color: status === 'active' ? 'green' : 'red'}}>\n    {status}\n  </div>\n  <div className=\"actions\">\n    <div className=\"btn\" onClick={(e) => { e.stopPropagation(); onDelete(id); }}>\n      🗑️\n    </div>\n  </div>\n</div>",
-      "expected_output": "Should identify all accessibility issues: clickable div instead of link/button, missing alt on img, color-only status indicator, icon button without label, no keyboard support. Assertion #2 was reworded 2026-08-28: it asserted a color-ONLY indicator, which this fixture does not contain — the badge renders {status} as text, so color is redundant reinforcement and WCAG 1.4.1 is not violated. At k=5, 7 of 10 judges recorded the reply correctly ruling 1.4.1 out and flagging 1.4.3 contrast instead; the assertion was punishing the right answer. It now targets the failure the fixture really has.",
+      "expected_output": "Should identify all accessibility issues: clickable div instead of link/button, missing alt on img, color-only status indicator, icon button without label, no keyboard support. Assertion #2 was reworded 2026-08-28: it asserted a color-ONLY indicator, which this fixture does not contain — the badge renders {status} as text, so color is redundant reinforcement and WCAG 1.4.1 is not violated. At k=5, 7 of 10 judges recorded the reply correctly ruling 1.4.1 out and flagging 1.4.3 contrast instead; the assertion was punishing the right answer. It now targets the failure the fixture really has. NOTE: final assertion added 2026-09-16. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 7 (validate) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause, which let controls pass on substance in an earlier case.",
       "assertions": [
         "Identifies clickable div should be an <a> or <Link> (for navigation)",
         "Identifies missing alt attribute on the image",
@@ -29,7 +29,8 @@
         "Identifies emoji icon without accessible label (aria-label needed)",
         "Notes that div+onClick has no keyboard support (no Enter/Space handling)",
         "Provides corrected accessible version of the component",
-        "Corrected version uses semantic HTML (a, button, img alt)"
+        "Corrected version uses semantic HTML (a, button, img alt)",
+        "Recommends validating the corrected component manually with a keyboard-only pass and a screen reader, not only by reading the code or running an automated checker"
       ]
     },
     {

package/skills/ai-evaluation/evals/evals.json

@@ -10,30 +10,32 @@
         "Covers the RAG triad or equivalent — faithfulness/groundedness, answer relevance, and retrieval/context quality — and distinguishes retrieval metrics from generation metrics",
         "Recommends a concrete evaluation tool (ragas, deepeval, or promptfoo) and explains its role",
         "Addresses LLM-as-judge reliability with at least one concrete measure: rubric design, position/self-preference bias, majority voting, or calibrating the judge against human spot-checks",
-        "Proposes running the evals as a repeatable regression gate (in CI or before each prompt/pipeline change), not only as a one-off pre-launch check"
+        "Proposes running the evals as a repeatable regression gate (in CI or before each prompt/pipeline change), not only as a one-off pre-launch check",
+        "Specifies the judge rubric as a separate score per quality criterion on a binary or 3-point scale, rather than one overall score such as 1–10"
       ]
     },
     {
       "id": 2,
       "prompt": "Our LLM feature summarizes long sales calls. We have no ground-truth summaries and no users yet. How do we evaluate quality with zero labeled data?",
-      "expected_output": "Does not treat missing labels as a blocker: proposes reference-free evaluation (rubric-based LLM-as-judge, groundedness against the source call, pairwise comparison between variants), bootstraps a small human-labeled golden set, calibrates the judge against human spot-checks, and plans online signals (feedback, A/B) once users exist",
+      "expected_output": "Does not treat missing labels as a blocker: proposes reference-free evaluation (rubric-based LLM-as-judge, groundedness against the source call, pairwise comparison between variants), bootstraps a small human-labeled golden set, calibrates the judge against human spot-checks, and plans online signals (feedback, A/B) once users exist NOTE: final assertion added 2026-09-16. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 5 (gate changes in CI) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause, which let controls pass on substance in an earlier case.",
       "assertions": [
         "Does not claim evaluation is impossible without labels — proposes reference-free approaches such as rubric-based LLM-as-judge, groundedness against the source transcript, or pairwise comparison between variants",
         "Recommends bootstrapping a small human-labeled golden set (even tens of examples) and explains what it buys",
         "Designs the judge with an explicit rubric or criteria decomposition rather than a bare overall score",
         "Mentions checking the judge against human spot-checks (calibration/agreement) or using voting to reduce judge noise",
-        "Plans for online signals after launch (user feedback, A/B testing) as a complement to offline evals"
+        "Plans for online signals after launch (user feedback, A/B testing) as a complement to offline evals",
+        "Runs the golden set as a CI gate on every prompt or model change, failing on regression against the last accepted baseline rather than below an absolute score"
       ]
     },
     {
       "id": 3,
-      "prompt": "Set up MLflow for our fine-tuning experiments so we can log runs and compare hyperparameters across the team.",
-      "expected_output": "Recognizes this as experiment-tracking infrastructure, not evaluation design — hands off to ml-experiment-tracking for the MLflow setup and run comparison, noting the boundary: tracking records and compares runs, while ai-evaluation defines what and how to measure",
+      "prompt": "We're fine-tuning a summarizer. I want every run's hyperparameters logged so the team can compare them, and I want that comparison ranked by how good the summaries actually are — not by training loss. What should we set up?",
+      "expected_output": "Splits the request: capturing hyperparameters and comparing runs is tracking infrastructure, while deciding what 'good summary' means and how it is scored is evaluation design. Defines the quality measure — golden set, rubric or judge, the metric the ranking sorts on — and routes the run-logging half to ml-experiment-tracking.",
       "assertions": [
-        "Recognizes the request is about experiment-tracking/run-comparison infrastructure, not about defining evaluation metrics or datasets",
-        "Refers to the ml-experiment-tracking skill for the MLflow setup and run comparison",
-        "Does not produce an evaluation-design workflow (golden datasets, judges, RAG metrics) as the answer to a tracking-infrastructure request",
-        "States the boundary: tracking records and compares runs, while evaluation design defines what and how to measure"
+        "Separates the two halves: run logging and comparison infrastructure versus defining the summary-quality measure the ranking needs",
+        "Refers to ml-experiment-tracking for capturing hyperparameters and comparing runs",
+        "Defines the quality measure itself — an evaluation set, a rubric or judge, and the metric the ranking would sort on — rather than handing the whole request off",
+        "States the boundary: tracking records and compares whatever score it is given; evaluation design decides what counts as good and how it is scored"
       ]
     }
   ]

package/skills/api-design/evals/evals.json

@@ -33,15 +33,14 @@
     },
     {
       "id": 3,
-      "prompt": "We have an existing API with these endpoints:\nGET /getUsers\nGET /getUserById?id=123\nPOST /createUser\nPOST /updateUser\nGET /deleteUser?id=123\n\nThis doesn't feel right. Can you redesign it following proper REST conventions?",
-      "expected_output": "Should identify specific violations (verbs in URLs, GET for delete, query params for identity) and redesign using proper REST conventions.",
+      "prompt": "We're adding a /transactions endpoint to our finance API. Clients need to page through a customer's transactions, filter by date range and by status, and sort by either amount or date. The table has around 400 million rows. I want the design settled before anyone writes code — what does the endpoint look like, and what has to be true underneath it to serve that?",
+      "expected_output": "Splits a request whose halves the skill roster does not separate. What the endpoint promises — the pagination style, how filters and sorts are expressed, what comes back for an invalid page cursor — is this skill's work, and it delivers that concretely. Whether the store can serve every filter and sort combination at 400 million rows is an index and partitioning question data-modeling owns. The two meet at the pagination choice: a cursor needs a stable unique ordering key, which is simultaneously a contract decision and an index decision, so a good answer names that dependency rather than treating cursor and offset as interchangeable.",
       "assertions": [
-        "Identifies that URLs use verbs instead of nouns",
-        "Identifies that DELETE uses GET method",
-        "Identifies that identity is in query params instead of path",
-        "Redesigns to: GET /users, GET /users/:id, POST /users, PATCH /users/:id, DELETE /users/:id",
-        "Explains WHY each change matters (not just 'this is the convention')",
-        "Notes that this is a breaking change requiring versioning or migration plan"
+        "Separates the two halves: what the endpoint promises its clients, versus what the storage has to be shaped like to serve those promises",
+        "Refers to data-modeling for the index and partitioning strategy behind the filters and sorts",
+        "Delivers the endpoint contract itself — the pagination style and how filters and sorts are expressed — rather than deferring the whole request",
+        "Ties the pagination choice to a stable ordering key instead of presenting cursor and offset as interchangeable",
+        "States the boundary: API design fixes what the interface promises, while data modeling decides whether the store can keep that promise"
       ]
     }
   ]

package/skills/architecture-documentation/SKILL.md

@@ -20,6 +20,13 @@ Understand what exists and what's needed before creating anything:
 
 A simple monolith may only need one conceptual diagram. A distributed system with 5+ services likely needs canonical and runtime levels too.
 
+**Split the ask before you start.** "Onboarding documentation" usually bundles two
+different deliverables, and only one is this skill's. A reader who cannot tell which
+service calls which needs a structural view — that is the work below. A reader who
+cannot get the system running needs setup instructions, which live in the README and
+belong to `project-documentation`. Say which half you are producing; a diagram does not
+fix a broken local setup, and a README does not substitute for a container view.
+
 ### Step 2: Choose Abstraction Levels
 
 Architecture documentation works at three distinct levels. Not every system needs all three — apply YAGNI.

package/skills/architecture-documentation/evals/evals.json

@@ -19,28 +19,26 @@
     {
       "id": 2,
       "prompt": "I have a simple CRUD application — Express.js backend, PostgreSQL database, deployed on a single EC2 instance behind nginx. My manager read about C4 and wants me to create all four C4 levels plus deployment diagrams plus data flow diagrams. Can you help me set this up?",
-      "expected_output": "Edge case — pushing back on unnecessary documentation complexity. Should apply YAGNI and recommend only the diagrams that earn their maintenance for a simple monolith, without refusing the request outright, and name what would later justify more detail.",
+      "expected_output": "Edge case — pushing back on unnecessary documentation complexity. Should apply YAGNI and recommend only the diagrams that earn their maintenance for a simple monolith, without refusing the request outright, and name what would later justify more detail. Whatever subset it produces uses one shared vocabulary of element names across every diagram.",
       "assertions": [
         "Pushes back on creating all four C4 levels for a simple monolith",
         "Applies YAGNI — recommends only the diagrams that add value for this system's complexity",
-        "Recommends at most a system context and a single container diagram",
         "Explains that Code-level C4 diagrams are almost never worth maintaining manually",
         "Does not refuse the request entirely — provides a pragmatic subset",
         "Suggests conceptual-level documentation may be sufficient for this system",
-        "Explains what would trigger the need for more detailed documentation in the future"
+        "Explains what would trigger the need for more detailed documentation in the future",
+        "Establishes a shared naming vocabulary (a glossary of element names) that every diagram uses exactly"
       ]
     },
     {
       "id": 3,
-      "prompt": "We're debating whether to extract our authentication logic into a separate service or keep it in the monolith. Can you also create architecture diagrams showing what the system would look like after the split?",
-      "expected_output": "Scope boundary — separating decision-making from documentation. Should defer the extract-or-keep decision to architecture-design for an ADR, while offering to diagram the proposed state as explicitly non-canonical until the decision is accepted.",
+      "prompt": "Three engineers joined this month and all three got stuck the same way: they can't work out which of our nine services actually calls which, and they couldn't get the stack running locally. I'd rather fix the documentation than answer these questions a fourth time. Where do we start?",
+      "expected_output": "Notices that the two complaints are not one problem. The map of which service calls which is this skill's work — which diagram levels answer that question, and what triggers an update or a review so the map does not drift once drawn — and it delivers that. Getting the stack running on a laptop is README and setup material that project-documentation owns. A good answer resists treating 'onboarding docs' as a single deliverable, because the two halves rot for different reasons and are maintained in different places. NOTE: assertion #3 was reworded 2026-09-16 from 'Names what triggers a diagram update and who owns it'. Under reply-only judges it split 2-1 on the ownership conjunct while every round named the triggers — and the skill never teaches assigning an owner (Step 6 gives update triggers and a review cadence, nothing more), so the conjunct asserted behaviour the skill does not contain. If ownership is wanted, it is a content change to Step 6, measured on its own.",
       "assertions": [
-        "Recognizes the decision-making aspect and defers to the architecture-design skill for the ADR",
-        "Does not make the architectural decision within this skill's workflow",
-        "Offers to diagram the proposed architecture as a conceptual or draft diagram to support the decision",
-        "Clearly separates the decision process from the documentation process",
-        "If diagramming the proposed state, labels it as non-canonical since the decision hasn't been made yet",
-        "Suggests that canonical diagrams should only be updated after the decision is accepted"
+        "Separates the two halves: the map of which services call which, versus the instructions for getting the stack running locally",
+        "Refers to project-documentation for the README and the local setup instructions",
+        "Delivers the architecture-documentation half itself — which diagram levels answer the what-calls-what question — rather than deferring the whole request",
+        "Names what triggers a diagram update — the structural changes that should force one, or a review cadence — rather than leaving the diagrams' accuracy to good intentions"
       ]
     }
   ]

package/skills/brainstorming/evals/evals.json

@@ -27,13 +27,13 @@
     },
     {
       "id": 3,
-      "prompt": "We've settled on building a CLI tool that syncs our design tokens to code. Break it into tasks with acceptance criteria and dependencies so the team can start next sprint.",
-      "expected_output": "Recognizes that ideation is complete and the user is asking for convergent task breakdown — refers to feature-planning for the scoped tasks, acceptance criteria, and dependency mapping rather than reopening divergent exploration",
+      "prompt": "Our nightly analytics reports are slow enough that people have started complaining. The team has split into two camps — precompute everything on a schedule, or generate on demand and stream results back — and we want to land on one this week so we can start building. Though I'll admit nobody has checked whether report speed is actually what people are unhappy about. Help us get to an answer.",
+      "expected_output": "Notices the request contains a closed question and an open one, and does not treat them alike. Choosing between precompute and on-demand streaming is a costly-to-reverse structural decision with a trade-off record, which architecture-design owns. Whether slow reports are the real complaint is still genuinely open, and that is where divergent work belongs: probe what users are actually unhappy about, and widen past the two camps to options nobody has put on the table. Does not simply adjudicate the two-way split, and does not relitigate it either.",
       "assertions": [
-        "Recognizes the request as convergent planning of an already-chosen idea, not ideation",
-        "Refers to the feature-planning skill for the task breakdown, acceptance criteria, and dependencies",
-        "Does not restart divergent idea generation or question the settled decision as the main response",
-        "States or implies the boundary: brainstorming opens the option space before a decision; feature-planning structures the work after it"
+        "Separates the two questions: whether slow reports are the real complaint is still open, while precompute-versus-on-demand is a decision waiting to be made",
+        "Refers to architecture-design for choosing between precompute and on-demand and recording the trade-off",
+        "Probes what users are actually unhappy about before the two camps are compared, rather than accepting report speed as the settled problem",
+        "Generates options beyond the two the team has already named"
       ]
     }
   ]

package/skills/bug-investigating/evals/evals.json

@@ -42,14 +42,14 @@
   "pressure_tests": [
     {
       "id": 1,
-      "prompt": "Production throws a NullPointerException sometimes. I don't have time to reproduce it — just add a null check where it crashes so it stops, and we'll move on.",
+      "prompt": "Quick one. Prod logs show this a few times an hour:\n\njava.lang.NullPointerException: Cannot invoke \"Profile.getTimezone()\" because the return value of \"User.getProfile()\" is null\n    at com.acme.notify.DigestScheduler.nextSendTime(DigestScheduler.java:88)\n\nLine 88 is `ZoneId zone = ZoneId.of(user.getProfile().getTimezone());`\n\nProfile is optional in our schema: SSO users created by the SCIM sync don't get one until they first log in. The job retries on failure and the retry goes through fine, so nothing is actually lost. Just guard it and fall back to UTC when the profile is null. It's a one-liner, no investigation needed.",
       "pressure": ["time"],
-      "expected_behavior": "Holds the Iron Law: no fix without a confirmed root cause. Explains that a null check masks the symptom while the real cause (why the value is null) remains and will resurface; pushes to reproduce or trace the source, offering a fast path rather than abandoning root cause.",
+      "expected_behavior": "Holds the Iron Law: no fix without a confirmed root cause. The developer's explanation does not fit the evidence: a user without a profile would fail on every attempt, yet the retry of the same job succeeds, so the profile exists and something makes it read as null intermittently. A UTC fallback would hide that behind silently wrong send times. Declines to ship the guard as the fix, names the mismatch, and offers a fast, bounded route to the cause (log the user id on failure and check whether those users have profiles; look at how the user is loaded on the path that fails).",
       "assertions": [
-        "Does NOT simply add the null check and move on",
-        "Explains masking the symptom vs fixing the root cause",
-        "Pushes to reproduce the bug or find why the value is null",
-        "Offers a fast route to root cause under time pressure rather than skipping it"
+        "Does NOT simply add the UTC fallback as the fix",
+        "Notices that the stated cause does not fit the evidence: the retry of the same job succeeds, so the user's profile exists and the null is intermittent rather than a user who has no profile",
+        "Explains that a guard without a confirmed cause masks the symptom (for example digests sent at the wrong time for users who do have a timezone) rather than fixing it",
+        "Offers a fast route to the root cause under time pressure (such as logging the failing user id and checking whether that user has a profile, or examining how the user is loaded on the failing path) rather than skipping it"
       ]
     }
   ]

package/skills/cicd-pipeline/evals/evals.json

@@ -33,15 +33,14 @@
     },
     {
       "id": 3,
-      "prompt": "The pipeline itself is fine. What I actually need is to work out what version number each release gets, what goes in the changelog, and when we're allowed to publish to npm. Can you just build all that into the workflow file?",
-      "expected_output": "Scope boundary — versioning policy, changelog content, and publish gates are release-management decisions; this skill automates them once decided. Should route the decision to release-management and offer to wire the resulting policy in as pipeline stages, without inventing a semver scheme itself.",
+      "prompt": "We publish our SDK to npm from GitHub Actions. Today a maintainer bumps the version by hand, writes the changelog entry by hand, and then whoever notices runs the publish job. We've shipped a wrong version twice. I want one reliable process — sort it out.",
+      "expected_output": "Two different kinds of decision are tangled in one request and the roster does not separate them. What version this release is, and what belongs in the changelog, is release-management's policy work. The stages that build, test and publish — and the gate that must pass before the publish step can run at all — are this skill's, and it delivers them. A good answer wires the pipeline to carry out a policy it does not invent, rather than hard-coding a versioning scheme into the workflow file.",
       "assertions": [
-        "Recognizes that semver policy, changelog content, and publish gating are release-management concerns",
-        "Hands off to or invokes the release-management skill for those decisions",
-        "Distinguishes deciding the release policy from automating it in the pipeline",
-        "Offers to implement the policy as pipeline stages once it is decided",
-        "Does not invent a versioning scheme or changelog format inside the CI workflow",
-        "Does not respond by producing another full CI/CD pipeline from scratch"
+        "Separates the two halves: deciding what version a release gets and what goes in the changelog, versus wiring the steps that build and publish it",
+        "Refers to release-management for the versioning scheme and changelog generation",
+        "Delivers the pipeline half itself — the stages and the gate that must pass before publish can run — rather than deferring the whole request",
+        "Does not invent a versioning scheme or a changelog format inside the workflow file",
+        "States the boundary: release management decides what a release is, while the pipeline is what carries it out"
       ]
     }
   ]

package/skills/code-archaeology/evals/evals.json

@@ -17,24 +17,25 @@
     {
       "id": 2,
       "prompt": "There's a bizarre guard clause in our payment processor: `if amount == 999999: amount = amount - 1`. No comment, no test, git blame just shows a 'fixes' commit from 2019 by someone who left. Everyone's afraid to delete it. Should I?",
-      "expected_output": "Does not say 'yes, delete it' outright: digs into the 2019 commit's full context (message, sibling changes, linked ticket, the PR discussion), searches for related production data or downstream systems that might depend on the cap, and if all sources come up empty, proposes proving it's dead (logging/telemetry on the branch for a period) plus a characterization test — removal only with evidence and a safety net, not on vibes",
+      "expected_output": "Does not say 'yes, delete it' outright: digs into the 2019 commit's full context (message, sibling changes, linked ticket, the PR discussion), searches for related production data or downstream systems that might depend on the cap, and if all sources come up empty, proposes proving it's dead (logging/telemetry on the branch for a period) plus a characterization test — removal only with evidence and a safety net, not on vibes NOTE: final assertion added 2026-09-16. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 6 (leave the map better than you found it) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause, which let controls pass on substance in an earlier case. This case is expected to score identically with and without the skill: Chesterton’s fence on a magic-number guard is textbook, and a strong model unaided reaches for the pickaxe search, the units and sentinel reading, runtime evidence and a characterization test. The row gates that behaviour against regression without demonstrating a gain. That is accepted by design; it should not be re-triaged as a weak prompt.",
       "assertions": [
         "Investigates the 2019 commit's full context (message, other files in the same commit, linked ticket/PR discussion) rather than accepting blame's surface result",
         "Considers that production data or a downstream system may depend on the behavior before judging it vestigial",
         "Proposes gathering runtime evidence (logging/telemetry on the branch) to prove the code path is dead rather than assuming it",
         "Recommends a characterization test or equivalent safety net around the behavior before/when removing it",
-        "Does not recommend immediate deletion based on the code looking senseless"
+        "Does not recommend immediate deletion based on the code looking senseless",
+        "Records the recovered reason for the guard where the next reader will find it, whether the guard is kept or removed"
       ]
     },
     {
       "id": 3,
-      "prompt": "Our codebase is a mess of tech debt — give me a prioritized assessment of what we should fix first and a remediation roadmap for the quarter.",
-      "expected_output": "Recognizes this as a debt assessment/prioritization request — refers to technical-debt-review for the hotspot analysis and remediation roadmap, noting the boundary: code-archaeology builds understanding of unfamiliar code; technical-debt-review assesses health and prioritizes fixes",
+      "prompt": "I've just inherited our warehouse-routing service. The two people who built it left last month, and nobody can explain why orders for two of our regions go through a completely separate code path. Next sprint we have to fold that path into the main one, and at least three other teams consume the routing decisions this service publishes. Where do I start?",
+      "expected_output": "Notices that folding the path in safely needs two different kinds of knowledge. Why the regional path exists and what it really does — mining its history, tracing one real order through it, and treating it as load-bearing until evidence says otherwise — is this skill's work, and it delivers that. Who consumes the routing decisions the service publishes, which of them would see a changed decision or payload once the path is folded in, and how the change must be sequenced across those teams is a blast-radius question that dependency-impact-analysis owns. A good answer does not treat understanding the code as the same thing as knowing who the change will break.",
       "assertions": [
-        "Recognizes the request as debt assessment and prioritization, not comprehension of unfamiliar code",
-        "Refers to the technical-debt-review skill for the hotspot analysis and roadmap",
-        "Does not produce an archaeology workflow (history mining, characterization tests) as the primary answer",
-        "States or implies the boundary: code-archaeology produces understanding; technical-debt-review assesses and prioritizes what to fix"
+        "Separates the two halves: recovering why the regional path exists, versus finding who consumes its routing decisions and what folding it in would break for them",
+        "Refers to dependency-impact-analysis for mapping the consuming teams and what the change would break",
+        "Delivers the comprehension half itself — history mining and tracing one order through the separate path — rather than deferring the whole request",
+        "Treats the separate regional code path as possibly load-bearing until history or production data says otherwise"
       ]
     }
   ]