swe-workflow-skills 0.6.3 → 0.8.0

package/README.md

@@ -38,9 +38,13 @@ don't cover:
   ([how it works](docs/ROLES.md)).
 - **Tested like code, not prose.** Every skill ships 3 evals; two LLM-as-judge
   harnesses (content quality and routing accuracy) replay them with skill loaded vs
-  absent and gate regressions in CI. Safety-critical skills (deploys, releases, tests,
-  incidents, security) are **hardened**: an Iron Law, a rationalization table distilled
-  from real baseline failures, and pressure tests that try to talk the agent out of it.
+  absent and gate regressions in CI. Across all 66 skills — 234 cases, 1315 assertions —
+  the same model passes **67.1%** of assertions unaided and **94.4%** with the skill
+  loaded, and **no skill scores below the unaided model on any case**
+  ([results](docs/EVALS.md#results-content-evals-full-catalog)). Safety-critical skills
+  (deploys, releases, tests, incidents, security) are **hardened**: an Iron Law, a
+  rationalization table distilled from real baseline failures, and pressure tests that
+  try to talk the agent out of it.
 - **Curated, not a mega-catalog.** First-party skills, versioned and eval-gated in the
   repo — no arbitrary third-party skills pulled from a hub, and nothing that executes on
   its own (the content is instructions and templates; the only code is the open Node
@@ -183,7 +187,10 @@ Each skill carries an `evals/` directory; safety/discipline skills add a `pressu
 block. Two runners replay scenarios through Claude (skill loaded = GREEN, absent = RED)
 and judge each assertion with a skeptical LLM-as-judge: `evals/workflow-runner.mjs`
 (in-session, no API key) and `evals/run.py` (CI regression gate, wired into
-`.github/workflows/skill-evals.yml`). Full guide in **[EVALS.md](docs/EVALS.md)**.
+`.github/workflows/skill-evals.yml`). The GREEN arm reads the skill *and* the
+`references/`/`templates/` files it links to, so improving a reference file moves the
+score. The first runner writes `evals/baseline.json`; the second gates against it.
+Full guide in **[EVALS.md](docs/EVALS.md)**.
 
 ## Contributing
 

package/VERSION

@@ -1 +1 @@
-0.6.3
+0.8.0

package/catalog.json

@@ -218,7 +218,7 @@
     },
     {
       "name": "plan-execution",
-      "description": "Execute an already-approved plan with discipline — batch tasks into verifiable checkpoints, verify each with fresh evidence before marking it done, log drift, and stop to re-plan when reality diverges from the plan's assumptions. Triggers: execute the plan, implement the approved plan, work through this plan, continue/resume the plan, next phase, checkpoint. NOT for creating plans — 'plan this' / 'break this down' → feature-planning; this skill starts only after a plan exists and is approved.",
+      "description": "Execute an already-approved plan with discipline — batch tasks into verifiable checkpoints, verify each with fresh evidence before marking it done, log drift, and stop to re-plan when reality diverges from the plan's assumptions. NOT for creating plans — 'plan this' / 'break this down' → feature-planning; this skill starts only after a plan exists and is approved. Triggers: execute the plan, implement the approved plan, work through this plan, continue/resume the plan, next phase, checkpoint.",
       "path": "plan-execution/SKILL.md"
     },
     {
@@ -323,7 +323,7 @@
     },
     {
       "name": "verification-before-completion",
-      "description": "Evidence gate before claiming work is done, passing, fixed, or deployed. Use before saying 'done', 'it works', or 'tests pass', or before committing, pushing, or opening a PR. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix. Run the proving command fresh and read its output first.",
+      "description": "Evidence gate before claiming work is done, passing, fixed, or deployed. Use before saying 'done', 'it works', or 'tests pass', or before committing, pushing, or opening a PR. Run the proving command fresh and read its output first. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix.",
       "path": "verification-before-completion/SKILL.md"
     },
     {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "swe-workflow-skills",
-  "version": "0.6.3",
+  "version": "0.8.0",
   "description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
   "type": "module",
   "bin": {

package/skills/architecture-design/evals/evals.json

@@ -4,15 +4,14 @@
     {
       "id": 1,
       "prompt": "We're building a SaaS platform and need to decide whether to use a multi-tenant single database or database-per-tenant approach. We have about 50 customers now, expecting 500 in a year. It's a Django app on PostgreSQL. Some customers are enterprise with strict data isolation requirements.",
-      "expected_output": "A structured analysis with at least 3 options (shared DB with row-level security, schema-per-tenant, DB-per-tenant), trade-off analysis against quality attributes, a clear recommendation with rationale, and an ADR document.",
+      "expected_output": "A structured analysis with at least 3 options (shared DB with row-level security, schema-per-tenant, DB-per-tenant), trade-off analysis against quality attributes, a clear recommendation with rationale, and an ADR document. NOTE: an assertion requiring the ADR to be 'saved to the filesystem' was removed — both eval arms generate a reply and cannot write files, so it could never pass and contributed no gate signal. Verify ADR file creation in a real session instead.",
       "assertions": [
         "Presents at least 3 distinct options",
         "Analyzes trade-offs for each option (not just pros/cons lists)",
         "Considers the enterprise data isolation requirement explicitly",
         "Applies KISS and YAGNI (doesn't over-engineer for 500 customers)",
         "Makes a clear recommendation with primary reason stated",
-        "Includes revisit triggers for when to reconsider",
-        "Produces an ADR document saved to the filesystem"
+        "Includes revisit triggers for when to reconsider"
       ]
     },
     {

package/skills/architecture-documentation/evals/evals.json

@@ -1,44 +1,47 @@
-[
-  {
-    "name": "Microservices architecture documentation from scratch",
-    "description": "Happy path: documenting a moderately complex distributed system for new hires",
-    "prompt": "We have a system with 8 microservices: a React frontend, a BFF API gateway, a user service, an order service, a payment service, a notification service, a search service, and a recommendation engine. We use PostgreSQL, Redis, RabbitMQ, and Elasticsearch. Everything runs on Kubernetes. We have no architecture diagrams and need to document this for new hires joining next month.",
-    "assertions": [
-      "Recommends starting with a system context diagram showing the system, users, and external dependencies",
-      "Identifies that all three abstraction levels are appropriate for this complexity",
-      "Prioritizes canonical and conceptual levels for the onboarding use case",
-      "Proposes a repository structure for organizing diagrams",
-      "Establishes or recommends a glossary with consistent naming for all 8 services",
-      "Suggests specific tooling appropriate for each level",
-      "Does not try to create all diagrams at once — suggests a phased approach",
-      "Produces at least a system context diagram as the first deliverable"
-    ]
-  },
-  {
-    "name": "Over-documentation for simple system",
-    "description": "Edge case: pushing back on unnecessary documentation complexity",
-    "prompt": "I have a simple CRUD application — Express.js backend, PostgreSQL database, deployed on a single EC2 instance behind nginx. My manager read about C4 and wants me to create all four C4 levels plus deployment diagrams plus data flow diagrams. Can you help me set this up?",
-    "assertions": [
-      "Pushes back on creating all four C4 levels for a simple monolith",
-      "Applies YAGNI — recommends only the diagrams that add value for this system's complexity",
-      "Recommends at most a system context and a single container diagram",
-      "Explains that Code-level C4 diagrams are almost never worth maintaining manually",
-      "Does not refuse the request entirely — provides a pragmatic subset",
-      "Suggests conceptual-level documentation may be sufficient for this system",
-      "Explains what would trigger the need for more detailed documentation in the future"
-    ]
-  },
-  {
-    "name": "Architecture decision vs architecture documentation",
-    "description": "Scope boundary: separating decision-making from documentation",
-    "prompt": "We're debating whether to extract our authentication logic into a separate service or keep it in the monolith. Can you also create architecture diagrams showing what the system would look like after the split?",
-    "assertions": [
-      "Recognizes the decision-making aspect and defers to the architecture-design skill for the ADR",
-      "Does not make the architectural decision within this skill's workflow",
-      "Offers to diagram the proposed architecture as a conceptual or draft diagram to support the decision",
-      "Clearly separates the decision process from the documentation process",
-      "If diagramming the proposed state, labels it as non-canonical since the decision hasn't been made yet",
-      "Suggests that canonical diagrams should only be updated after the decision is accepted"
-    ]
-  }
-]
+{
+  "skill_name": "architecture-documentation",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "We have a system with 8 microservices: a React frontend, a BFF API gateway, a user service, an order service, a payment service, a notification service, a search service, and a recommendation engine. We use PostgreSQL, Redis, RabbitMQ, and Elasticsearch. Everything runs on Kubernetes. We have no architecture diagrams and need to document this for new hires joining next month.",
+      "expected_output": "Happy path — documenting a moderately complex distributed system for new hires. Should start from a system context diagram, judge which abstraction levels this complexity warrants, propose a repository structure and a glossary covering all 8 services, and phase the work rather than producing every diagram at once.",
+      "assertions": [
+        "Recommends starting with a system context diagram showing the system, users, and external dependencies",
+        "Identifies that all three abstraction levels are appropriate for this complexity",
+        "Prioritizes canonical and conceptual levels for the onboarding use case",
+        "Proposes a repository structure for organizing diagrams",
+        "Establishes or recommends a glossary with consistent naming for all 8 services",
+        "Suggests specific tooling appropriate for each level",
+        "Does not try to create all diagrams at once — suggests a phased approach",
+        "Produces at least a system context diagram as the first deliverable"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "I have a simple CRUD application — Express.js backend, PostgreSQL database, deployed on a single EC2 instance behind nginx. My manager read about C4 and wants me to create all four C4 levels plus deployment diagrams plus data flow diagrams. Can you help me set this up?",
+      "expected_output": "Edge case — pushing back on unnecessary documentation complexity. Should apply YAGNI and recommend only the diagrams that earn their maintenance for a simple monolith, without refusing the request outright, and name what would later justify more detail.",
+      "assertions": [
+        "Pushes back on creating all four C4 levels for a simple monolith",
+        "Applies YAGNI — recommends only the diagrams that add value for this system's complexity",
+        "Recommends at most a system context and a single container diagram",
+        "Explains that Code-level C4 diagrams are almost never worth maintaining manually",
+        "Does not refuse the request entirely — provides a pragmatic subset",
+        "Suggests conceptual-level documentation may be sufficient for this system",
+        "Explains what would trigger the need for more detailed documentation in the future"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "We're debating whether to extract our authentication logic into a separate service or keep it in the monolith. Can you also create architecture diagrams showing what the system would look like after the split?",
+      "expected_output": "Scope boundary — separating decision-making from documentation. Should defer the extract-or-keep decision to architecture-design for an ADR, while offering to diagram the proposed state as explicitly non-canonical until the decision is accepted.",
+      "assertions": [
+        "Recognizes the decision-making aspect and defers to the architecture-design skill for the ADR",
+        "Does not make the architectural decision within this skill's workflow",
+        "Offers to diagram the proposed architecture as a conceptual or draft diagram to support the decision",
+        "Clearly separates the decision process from the documentation process",
+        "If diagramming the proposed state, labels it as non-canonical since the decision hasn't been made yet",
+        "Suggests that canonical diagrams should only be updated after the decision is accepted"
+      ]
+    }
+  ]
+}

package/skills/cicd-pipeline/evals/evals.json

@@ -29,6 +29,19 @@
         "Recommends concurrency control to cancel outdated runs",
         "Provides a restructured pipeline diagram"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "The pipeline itself is fine. What I actually need is to work out what version number each release gets, what goes in the changelog, and when we're allowed to publish to npm. Can you just build all that into the workflow file?",
+      "expected_output": "Scope boundary — versioning policy, changelog content, and publish gates are release-management decisions; this skill automates them once decided. Should route the decision to release-management and offer to wire the resulting policy in as pipeline stages, without inventing a semver scheme itself.",
+      "assertions": [
+        "Recognizes that semver policy, changelog content, and publish gating are release-management concerns",
+        "Hands off to or invokes the release-management skill for those decisions",
+        "Distinguishes deciding the release policy from automating it in the pipeline",
+        "Offers to implement the policy as pipeline stages once it is decided",
+        "Does not invent a versioning scheme or changelog format inside the CI workflow",
+        "Does not respond by producing another full CI/CD pipeline from scratch"
+      ]
     }
   ]
 }

package/skills/code-reviewing/SKILL.md

@@ -20,8 +20,14 @@ a PR):
 !`git diff --stat 2>/dev/null || true`
 
 If the summary above is empty or irrelevant to what you were asked to review,
-proceed with the code as provided. Before reviewing line-by-line, understand the
-big picture:
+proceed with the code as provided.
+
+**Called from the pre-commit gate?** When `verification-before-completion` routes
+here before a commit, the review target is `git diff --staged` — the change about
+to be published, not the whole branch. Scope to it, keep the pass proportionate to
+what tripped the gate, and report findings in time to fix them before the commit.
+
+Before reviewing line-by-line, understand the big picture:
 
 - **What is this code supposed to do?** Read the PR description, linked issue, or ask the user.
 - **What changed?** If reviewing a diff, understand the scope of changes — run the
@@ -47,6 +53,17 @@ the diff, run three greps before accepting it:
   exported just to silence the unused-symbol checker).
 - **Any consumer?** A new schema field, prop, or param that nothing reads or sets is
   speculative plumbing — the API looks finished but is wired to nothing.
+- **A comment in the wrong place?** For any new comment block of four or more lines,
+  ask what it explains. System-level rationale — why the component exists, what was
+  rejected, how it relates to other services — belongs in an ADR or architecture doc
+  with the comment reduced to a pointer; a local invariant the types can't express
+  stays inline; a restatement of the code goes nowhere. The essay-in-a-header rots
+  unread and is unfindable by anyone grepping the docs for it.
+- **A reference nobody can resolve?** `Phase 2`, `CP1.3`, `T02`, `finding #17`,
+  `checkpoint 3/5` in comments, names, or strings — check whether a **git-tracked**
+  document defines it (`git ls-files '*.md' | xargs grep -l`). Anchored, it's a real
+  cross-reference. Unanchored, it names a conversation the reader wasn't in: the
+  finding is "write the meaning instead of the label".
 - **Any doc still describing the old shape?** If the diff renames, removes, or adds
   something a user types or configures — a flag, env var, endpoint, config key,
   command, script, install step — grep the README and `docs/` for it, old name
@@ -67,8 +84,8 @@ Review the code against these categories, in order of importance. See [reference
 
 **Severity levels for findings:**
 
-- 🔴 **Blocker**: Must fix before merge (bugs, security issues, data loss risks) — including a doc the diff makes *wrong* where the doc is an instruction someone follows: install steps, setup commands, the API contract, config keys. A stale instruction fails for the next person, silently, and is worse than no instruction.
-- 🟡 **Suggestion**: Should fix, significantly improves quality (principle violations, missing tests, docs the diff leaves incomplete rather than wrong)
+- 🔴 **Blocker**: Must fix before merge (bugs, security issues, data loss risks) — including a doc the diff makes *wrong* where the doc is an instruction someone follows: install steps, setup commands, the API contract, config keys. A stale instruction fails for the next person, silently, and is worse than no instruction. **Secrets or real personal data in the diff are always blockers** — they survive removal in git history, so flag them before the commit, not after.
+- 🟡 **Suggestion**: Should fix, significantly improves quality (principle violations, missing tests, docs the diff leaves incomplete rather than wrong, misplaced comment essays, unanchored internal terminology)
 - 🔵 **Nit**: Optional improvement (naming, style, minor simplification)
 
 ### Step 4: Present Findings
@@ -105,7 +122,7 @@ These are the lenses through which code is examined:
 
 **Testability**: Is the code testable? Are there tests? Do tests test behavior or implementation?
 
-**Readability**: Can someone unfamiliar with this code understand it in one reading? Good naming, appropriate comments (why, not what), manageable function length.
+**Readability**: Can someone unfamiliar with this code understand it in one reading? Good naming, manageable function length, and comments that are both *why, not what* and **in the right place** — the why of a system belongs in an ADR the comment points at; the why of a line belongs on the line. Judge "unfamiliar" literally: a reader with the repo and nothing else, who was in none of the conversations that produced it.
 
 **Error Handling**: Are errors caught, logged, and handled appropriately? Are error messages helpful? See [references/error-handling.md](references/error-handling.md) for detailed patterns (null safety, exception context, caller-oriented exceptions).
 
@@ -123,4 +140,5 @@ These are the lenses through which code is examined:
 - `refactoring` — when the review surfaces code smells worth a structured cleanup
 - `code-slop-cleanup` — strip AI-slop patterns from the diff before (or instead of) debating them in review
 - `security-audit` — for a dedicated, deep security pass beyond the review checklist
-- `verification-before-completion` — verify the change actually runs before approving, not from the diff alone
+- `verification-before-completion` — verify the change actually runs before approving, not from the diff alone; also the pre-commit gate that routes staged diffs here
+- `architecture-design` — where a system-level rationale found in a comment belongs instead (an ADR)

package/skills/code-reviewing/evals/evals.json

@@ -21,9 +21,9 @@
       "prompt": "Can you review the changes in my last git commit?",
       "expected_output": "Should read the git diff, understand context from surrounding files, and perform a structured review.",
       "assertions": [
-        "Runs git diff or git show to read the changes",
-        "Reads surrounding code for context",
-        "For any new helper, export, or block, greps the repo beyond the diff for an existing twin, for importers, and for consumers (duplication / dead export / unconsumed surface are invisible in a single hunk)",
+        "Names the command that reads the change (git show / git diff HEAD~1) and scopes the review to that commit",
+        "Establishes the surrounding code as context for the review rather than judging the hunk in isolation",
+        "For any new helper, export, or block, states it will search the repo beyond the diff for an existing twin, for importers, and for consumers (duplication / dead export / unconsumed surface are invisible in a single hunk)",
         "Follows the structured review format (summary, blockers, suggestions, nits)",
         "Provides specific file:line references for findings"
       ]
@@ -89,6 +89,22 @@
         "Justifies the severity: a wrong instruction is followed and then fails, which is worse than a missing one",
         "Does NOT approve on 'code is consistent and CI is green' alone"
       ]
+    },
+    {
+      "id": 4,
+      "prompt": "The pre-commit gate routed you here. Review the staged diff: it adds `src/server/dashboard.ts` with a 14-line file-header block explaining why dashboard state lives in a second SQLite file, which tables get wiped on rebuild, and why the read handle stays read-only — the header cites ADR-018. Below that are two query functions, one commented `// Aggregates (Phase 4). Read-only.` The repo has docs/decisions/ADR-018 tracked, but no tracked doc mentions any phase.",
+      "pressure": [
+        "looks_good"
+      ],
+      "expected_behavior": "Scopes to the staged diff rather than the whole branch. Flags the 14-line header as misplaced rationale — it duplicates the ADR it cites, so it should shrink to a pointer — and flags 'Phase 4' as unanchored after checking the tracked docs, while correctly leaving the ADR-018 reference alone since that document is tracked. Assigns both as suggestions, not blockers.",
+      "assertions": [
+        "Scopes the review to the staged diff",
+        "Flags the 14-line header as rationale that belongs in the ADR, reduced to a pointer",
+        "Checks the tracked docs before judging the terminology rather than assuming",
+        "Flags 'Phase 4' as unanchored and recommends writing the meaning",
+        "Does NOT flag the ADR-018 reference, which resolves to a tracked document",
+        "Rates these as suggestions rather than blockers"
+      ]
     }
   ]
 }

package/skills/code-slop-cleanup/SKILL.md

@@ -33,8 +33,13 @@ Slop is relative — the same construct is correct in one place and slop in anot
 
 - **High**: debug leftovers, silent-failure handlers, stray working files
 - **Medium**: defensive theater on trusted paths, generic TODOs, unrequested backwards-compat shims, duplication of existing helpers
+- **Medium**: also unanchored internal terminology (`Phase 2`, `CP1.3`, `finding #17`) that no git-tracked doc defines — rewrite to the meaning
 - **Low**: comment slop, dead weight (unused imports, `=== true`, redundant returns), style inconsistent with the file
 
+Misplaced rationale — the multi-line header essay explaining a *system* instead of
+the code beneath it — is detected here but **routed, not deleted**: the reasoning
+belongs in an ADR, and deleting it loses something nothing else records.
+
 ### Step 4: Strip — Deletions Only, Production Code Only
 
 Apply removals highest severity first. Every edit must be a behavior-preserving deletion (or a trivial inlining a deletion forces). If removing something requires *rewriting* logic around it, that's refactoring — leave it and flag it.
@@ -71,6 +76,8 @@ The biggest slop category — duplication — is created at generation time and
 |--------|---------|
 | "This check might catch something someday" | If nothing upstream can produce that state, it's theater — and it hides the real bugs by normalizing noise. |
 | "The comments help junior developers" | Restating comments rot and mislead; code clarity helps juniors. Keep why-comments, cut what-comments. |
+| "This 12-line header explains the architecture — I'll delete it as comment slop" | It's misplaced, not worthless. Deleting it destroys reasoning nothing else records. Flag it for the ADR and leave the code alone. |
+| "Phase 3 is obviously meaningful, leave the reference" | Check before assuming: if no tracked doc defines it, it points at a conversation the reader wasn't in. Rewrite it to what it means. |
 | "I'll just fix this bug while I'm here" | Cleanup and fixes in one pass make both unreviewable. Flag it, route it, finish the cleanup. |
 | "Strip it all — defensive code is slop" | A boundary check is not slop. Over-removal breaks behavior, the one thing this pass must never do. |
 | "This test is obviously redundant, I'll just delete it" | "Obviously" is a guess; proof requires the behavior map. Route to `test-suite-design`. |

package/skills/code-slop-cleanup/evals/evals.json

@@ -52,6 +52,19 @@
         "Distinguishes provable slop from legitimate defensive code even under the lead's instruction",
         "Proposes a reduced-scope alternative (e.g. high-severity-only removals) rather than either full compliance or flat refusal"
       ]
+    },
+    {
+      "id": 2,
+      "prompt": "Deslop this branch before I open the PR. Biggest offender: src/sync/manifest.ts opens with a 16-line comment block laying out why sync uses content hashes instead of timestamps, which alternatives were rejected and why the version ledger is never synced. Classic AI comment bloat — nuke it. Also there's a `// retry budget (Phase 3)` on line 80.",
+      "pressure": ["authority", "looks_good"],
+      "expected_behavior": "Does not delete the 16-line block. Recognizes it as misplaced rationale rather than comment slop: the reasoning is real and recorded nowhere else, so deleting it destroys information. Flags it and routes the move to architecture-design / project-documentation, leaving the code untouched. Handles the 'Phase 3' reference separately — checks for a tracked anchor and rewrites to the meaning if there is none.",
+      "assertions": [
+        "Does NOT delete the 16-line rationale block despite being told to",
+        "Distinguishes misplaced rationale from comment slop",
+        "Routes the move to an ADR / architecture doc rather than fixing it in place",
+        "Checks whether a tracked doc defines 'Phase 3' before acting on it",
+        "Rewrites the unanchored reference to its meaning rather than deleting the comment wholesale"
+      ]
     }
   ]
 }

package/skills/code-slop-cleanup/references/slop-patterns.md

@@ -21,13 +21,15 @@ The pattern catalog for `code-slop-cleanup` (diff scope) and `technical-debt-rev
 | **Generic TODOs** | `// TODO: handle errors`, `// TODO: improve this` — no context, no ticket, no owner | TODO with a ticket reference or a concrete, actionable description |
 | **Unrequested backwards-compat shims** | `_legacy` wrappers, kept old code paths "just in case", re-exports nobody consumes, dual old/new implementations | A migration the plan actually calls for, with a removal date or ticket |
 | **Duplication of existing helpers** | A new utility that re-implements something already in the codebase or stdlib. Named shapes that recur in AI code: the same N-arg **wiring/setup ritual** pasted at every entry point; **parallel functions/commands differing only by a constant or a noun** (`block`/`workflow` branch commands, per-model queries); **row-mappers or adapters duplicated byte-for-byte** across sibling modules. Detecting these needs a **repo-wide** look — a single diff hunk hides copies #1–N elsewhere, so grep for the twin before accepting a new block. | Genuinely different concepts that happen to look similar (false DRY is worse than repetition) |
+| **Misplaced rationale (the header essay)** | A comment block of roughly four-plus lines explaining a *system* rather than a line: why the component exists, which alternatives were rejected, how it relates to other services, the whole trade-off behind a design. Often opens a file and often cites an ADR it is busy duplicating. The tell is that the comment would read correctly as a section of an architecture doc, with nothing tying it to the code beneath it. The fix is a *move* — rationale to the ADR/architecture doc, a one-line pointer left behind (`// Triage state lives in a separate writable DB; see ADR-018.`) — so the reasoning survives where it can be found and reviewed. **A move is not a deletion: during slop cleanup, flag it and route (see the routed table below); never delete the block outright,** which would destroy reasoning nothing else records. | A genuinely local invariant that happens to need several lines to state — why *this* comparison must be `date(col) >= date(?)`, a consequence the caller cannot see, a boundary condition the types don't express. Length is not the test; whether it explains the code below it is. Also: file headers the project's own convention mandates, and generated-doc blocks. |
+| **Unanchored internal terminology** | `Phase 2`, `Fase 4`, `CP1.3`, `checkpoint 3/5`, `T02`, `P2-E08`, `finding #17`, `Sprint 4` in comments, identifiers, or strings, where **no git-tracked document defines the term**. It names a plan, chat, or scratchpad the reader has no access to. Check, don't assume: `git ls-files '*.md' \| xargs grep -l "P2-E08"`. Fix by rewriting to the meaning — `// Dashboard aggregates (Phase 4). Read-only.` → `// Dashboard aggregates. Read-only.` — which usually *improves* the sentence, since the label carried no information. | The term **is** defined in a tracked doc (an execution plan under `docs/`, an ADR, a roadmap): then it's a real cross-reference and removing it loses information. Prefer making the pointer explicit. Also: issue/PR numbers on a public tracker, and version tags. |
 | **Unconsumed surface / speculative plumbing** | Exports with no importers anywhere (the `_`-prefix + `export` tell — exported only to silence the unused-symbol checker); a fully-built, unit-tested module never wired into any path; a schema field validated but never read, a prop threaded through layers but never set, a param carried "for symmetry" but unused, a value computed but never surfaced. The API *looks* finished; nothing calls it. | A library/package's genuine public API; params required by an interface/callback signature; a documented extension point with a near-term, named consumer landing in the same change |
 
 ## Low severity — cosmetic residue
 
 | Pattern | Tells | When it's NOT slop |
 |---------|-------|--------------------|
-| **Comment slop** | Comments restating the code (`// increment counter`), section dividers, agent narration ("Here we now handle…"), docstrings repeating the signature, journal comments, a **changelog / version-history block embedded in source** (belongs in CHANGELOG/ADR), a **stale comment describing a scheme the code no longer implements** (actively misleading, worse than none) | Why-comments on non-obvious decisions; consequence warnings; docstrings on public API the project mandates |
+| **Comment slop** | Comments restating the code (`// increment counter`), section dividers, agent narration ("Here we now handle…"), docstrings repeating the signature, journal comments, a **changelog / version-history block embedded in source** (belongs in CHANGELOG/ADR), a **stale comment describing a scheme the code no longer implements** (actively misleading, worse than none) | Why-comments on non-obvious decisions *about the code they sit on*; consequence warnings; docstrings on public API the project mandates. Where the why concerns the system rather than the line, see **Misplaced rationale** above — that one is moved, not deleted |
 | **Dead weight** | Unused imports/params/variables, `return undefined`, `=== true`, `Promise.all` around a single promise, unreachable branches | Parameters required by an interface/callback signature; re-exports that are the module's public API |
 | **Style inconsistency** | Naming, formatting, or idiom that doesn't match the surrounding file (e.g. `fetchData` in a file that uses `getX` everywhere) | The file itself is inconsistent and the new code follows the project-level convention |
 
@@ -37,6 +39,7 @@ These are slop, but this catalog's consumers don't fix them in place:
 
 | Pattern | Tells | Route to |
 |---------|-------|----------|
+| **Misplaced rationale (the header essay)** | See the medium table above. Detection is in scope here; the fix isn't. | `architecture-design` (the ADR the rationale belongs in) or `project-documentation` (an existing architecture doc). Deleting the block during cleanup would lose reasoning nothing else records — flag it, name the destination, leave the code alone. |
 | **Test-file slop** | Redundant/duplicate tests, trivial asserts (`expect(true)`), asserting on the mock, implementation-mirroring tests | `test-suite-design` (Pruning Test Slop) — deleting a test is never behavior-neutral for the safety net; it needs behavior-map judgment. **Never delete a test during diff cleanup.** |
 | **Type slop** | `any`/`unknown` to silence the checker, casts papering over mismatches, single-implementer interfaces "for testing", overly broad types (`env: object` then `type: ignore` to reach a field), **stale `type: ignore` matching no current error**, a **declared strict gate (`mypy strict`, `--max-warnings 0`) sitting on N errors** because nobody runs it | `technical-debt-review` (type debt) — usually cross-file; fixing it changes signatures. Running the gate green is the `verification-before-completion` "done" step |
 | **Config/scaffolding slop** | Config keys and feature flags nothing reads, orphan `.env` entries, checked-in generated files, half-finished TODO scaffolds | `technical-debt-review` (config/scaffolding debt) — proving "unread" needs repo-wide tracing |

package/skills/containerization/evals/evals.json

@@ -31,6 +31,18 @@
         "Recommends adding HEALTHCHECK",
         "Provides a corrected Dockerfile"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "The image builds fine now. Next step: provision the EKS cluster, the VPC with private subnets, the RDS instance and the ECR repository on AWS. Write me the Terraform for all of it.",
+      "expected_output": "Scope boundary — provisioning cloud infrastructure is infrastructure-as-code territory, not containerization. Should hand off rather than writing Terraform modules here, while staying useful about the container-side artifacts that will run on that cluster.",
+      "assertions": [
+        "Recognizes that provisioning EKS/VPC/RDS/ECR is infrastructure-as-code work, not containerization",
+        "Hands off to or invokes the infrastructure-as-code skill",
+        "Does not write the Terraform modules as part of the containerization workflow",
+        "Distinguishes authoring images and Kubernetes manifests from provisioning the platform they run on",
+        "Remains useful by offering the container-side artifacts (manifests, image build) that pair with the provisioned cluster"
+      ]
     }
   ]
 }

package/skills/effort-estimation/evals/evals.json

@@ -27,6 +27,19 @@
         "Recommends a technical spike before committing to precise numbers",
         "Does NOT produce a precise single-number estimate for 6 months of unknown work"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Give me story points for the new billing feature. I don't have it broken down yet — there's just the one-line ask from the customer. Just point it so I can put it in the sprint.",
+      "expected_output": "Scope boundary — there is nothing to estimate yet. Should route the breakdown to feature-planning, explain that a point estimate on an undecomposed one-line ask is a guess dressed as data, and offer to estimate once tasks and acceptance criteria exist.",
+      "assertions": [
+        "Identifies that no breakdown exists yet, so there is nothing meaningful to estimate",
+        "Hands off to or invokes the feature-planning skill to decompose the work first",
+        "Does not emit a story-point number for the undecomposed one-line request",
+        "Explains that estimating requires tasks and acceptance criteria to size against",
+        "Offers to run the estimation workflow once the breakdown exists",
+        "Does not simply refuse — gives the user a concrete next step"
+      ]
     }
   ]
 }

package/skills/feature-planning/evals/evals.json

@@ -4,15 +4,14 @@
     {
       "id": 1,
       "prompt": "I need to add a notification system to our app. Users should get email and in-app notifications when someone comments on their posts. We're using a Node.js/Express backend with PostgreSQL and a React frontend. Can you help me plan this out?",
-      "expected_output": "A structured plan with clarifying questions asked first, acceptance criteria in Given/When/Then format, 5-10 granular tasks with dependencies, risk identification, and a saved markdown plan file.",
+      "expected_output": "A structured plan with clarifying questions asked first, acceptance criteria in Given/When/Then format, 5-10 granular tasks with dependencies, risk identification, and a saved markdown plan file. NOTE: an assertion requiring the plan to be saved as a markdown file was removed — both eval arms generate a reply and cannot write files, so it could never pass and contributed no gate signal. Verify file creation in a real session instead.",
       "assertions": [
         "Asks clarifying questions before producing the plan",
         "Includes acceptance criteria in Given/When/Then format",
         "Tasks are granular (1-4 hours each) with clear deliverables",
         "Dependency map is present showing task ordering",
         "Risks section identifies at least 2 technical risks",
-        "Out of scope section explicitly excludes something",
-        "Saves the plan as a markdown file"
+        "Out of scope section explicitly excludes something"
       ]
     },
     {

package/skills/frontend-architecture/evals/evals.json

@@ -29,6 +29,18 @@
         "Does NOT recommend adding yet another state management library on top",
         "Addresses the performance issue specifically (memoization, selector optimization)"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Component structure is settled. Now go through our modal, dropdown and data-table components and tell me everything failing WCAG — keyboard traps, missing ARIA, focus order, contrast ratios. We have an accessibility audit next month.",
+      "expected_output": "Scope boundary — a WCAG conformance review of existing components is accessibility-design work, not frontend architecture. Should hand off rather than answering with component hierarchy or state-management advice.",
+      "assertions": [
+        "Recognizes that WCAG conformance, ARIA, focus order and contrast are accessibility-design concerns",
+        "Hands off to or invokes the accessibility-design skill",
+        "Does not answer with component hierarchy, state management, or folder structure guidance",
+        "Does not attempt a full WCAG audit inside the frontend-architecture workflow",
+        "May note where architectural choices (focus management, component composition) support the accessibility work"
+      ]
     }
   ]
 }