swe-workflow-skills 0.8.2 → 0.9.1

package/skills/code-reviewing/SKILL.md

@@ -34,6 +34,13 @@ Before reviewing line-by-line, understand the big picture:
   full `git diff` (or `git diff --staged`) when the stat summary isn't enough.
 - **What's the surrounding code like?** Read adjacent files for conventions and patterns.
 
+**A review request often arrives with a symptom attached** — "this endpoint is slow",
+"memory climbs overnight". Review the code against what you can actually see and report
+the defects you find, including the ones that plausibly explain the symptom. What you must
+not do is assert that a defect you spotted *is* the cause: that claim needs measurement,
+which is `performance-optimization`'s work. Name the suspect, say it is unconfirmed, and
+keep the review's findings standing on their own.
+
 ### Step 2: First Pass — Structural Review
 
 Look at the forest before the trees. Check:
@@ -137,6 +144,7 @@ These are the lenses through which code is examined:
 
 ## Cross-Skill References
 
+- `performance-optimization` — when a review turns up a suspected cause of a real latency or memory symptom, confirming it by measurement
 - `refactoring` — when the review surfaces code smells worth a structured cleanup
 - `code-slop-cleanup` — strip AI-slop patterns from the diff before (or instead of) debating them in review
 - `security-audit` — for a dedicated, deep security pass beyond the review checklist

package/skills/code-reviewing/evals/evals.json

@@ -19,26 +19,24 @@
     {
       "id": 2,
       "prompt": "Can you review the changes in my last git commit?",
-      "expected_output": "Should read the git diff, understand context from surrounding files, and perform a structured review.",
+      "expected_output": "Should read the git diff, understand context from surrounding files, and perform a structured review. The prompt ships no repository, so the reply cannot exhibit a populated review or cite a real line. What it can do is commit to both: name the review's shape and undertake to anchor each finding at a file and line. Assertions #3 and #4 ask for those commitments, not for a finished review.",
       "assertions": [
         "Names the command that reads the change (git show / git diff HEAD~1) and scopes the review to that commit",
         "Establishes the surrounding code as context for the review rather than judging the hunk in isolation",
         "For any new helper, export, or block, states it will search the repo beyond the diff for an existing twin, for importers, and for consumers (duplication / dead export / unconsumed surface are invisible in a single hunk)",
-        "Follows the structured review format (summary, blockers, suggestions, nits)",
-        "Provides specific file:line references for findings"
+        "States the shape the review will take — a summary plus findings separated by severity (blockers, suggestions, nits) — rather than promising \"a review\"",
+        "Commits to citing each finding by file and line rather than describing findings in prose"
       ]
     },
     {
       "id": 3,
-      "prompt": "Review this TypeScript code:\n\nexport const UserService = {\n  async getUser(id: string): Promise<any> {\n    const user = await db.users.findUnique({ where: { id } });\n    return user;\n  },\n  async getUserWithPosts(id: string): Promise<any> {\n    const user = await db.users.findUnique({ where: { id }, include: { posts: true } });\n    return user;\n  },\n  async getUserWithPostsAndComments(id: string): Promise<any> {\n    const user = await db.users.findUnique({ where: { id }, include: { posts: { include: { comments: true } } } });\n    return user;\n  }\n};",
-      "expected_output": "Should flag DRY violation (three nearly identical functions), Promise<any> types, and suggest a more flexible approach (e.g., include options parameter). Should be constructive and suggest a concrete refactoring.",
+      "prompt": "Review this Django view that shipped last week — support says the orders page takes about eight seconds for our larger accounts.\n\n```python\ndef order_list(request):\n    orders = Order.objects.filter(customer=request.user.customer)\n    rows = []\n    for o in orders:\n        rows.append({\n            'id': o.id,\n            'total': sum(i.price * i.qty for i in o.items.all()),\n            'shipper': o.shipment.carrier.name,\n            'status': o.status,\n        })\n    return JsonResponse({'orders': rows})\n```",
+      "expected_output": "The eight seconds invites a different skill than the code does, and the roster does not separate them. Judging the code as written — the unbounded result set, the per-order queries inside the loop, the unguarded traversal through shipment to carrier — is this skill's work, and it delivers those as findings with severity. Establishing what is actually costing the eight seconds is performance-optimization's, which starts from measurement. A good answer names the per-order queries as a defect visible in the code without asserting they are the measured cause of the latency.",
       "assertions": [
-        "Flags DRY violation across the three functions",
-        "Flags the use of any return type",
-        "Suggests consolidation with an options/includes parameter",
-        "Does not just say 'this is bad' — provides a refactored version",
-        "Notes that this might be a case for a single flexible query function",
-        "Applies KISS — suggested solution is not over-engineered"
+        "Separates the two halves: what is wrong with the code as written, versus establishing what is actually making the page slow",
+        "Refers to performance-optimization for profiling and confirming the cause of the eight seconds",
+        "Delivers the review itself — findings on the code as written, with severity — rather than deferring to a profiling exercise",
+        "Names the per-order queries in the loop as a review finding without asserting they are the measured cause"
       ]
     }
   ],
@@ -65,7 +63,7 @@
       "pressure": [
         "looks_good"
       ],
-      "expected_behavior": "Flags the except branch as a fail-open correctness bug, not graceful degradation: on a parse failure the function returns passed=True / no issues, so an unparseable model reply is silently converted into 'everything checks out' and the article publishes unchecked. Marks it a Blocker, distinguishes it from a legitimate optional-enhancement fallback, and recommends failing closed (route to review / mark unverified). Does not accept 'degrades gracefully' as justification.",
+      "expected_behavior": "Flags the except branch as a fail-open correctness bug, not graceful degradation: on a parse failure the function returns passed=True / no issues, so an unparseable model reply is silently converted into 'everything checks out' and the article publishes unchecked. Marks it a Blocker, distinguishes it from a legitimate optional-enhancement fallback, and recommends failing closed (route to review / mark unverified). Does not accept 'degrades gracefully' as justification. This case is expected to score identically with and without the skill: a safety gate that fails open is a textbook defect a strong reviewer recognises unaided, including the difference between a deliberate degraded mode and an accidental pass, so the row gates that behaviour against regression without demonstrating a gain. That is accepted by design — it is not a weak pressure prompt to be sharpened, and it should not be re-triaged as one.",
       "assertions": [
         "Identifies the empty-report fallback as fail-open: a failure that returns a success-shaped result (passed=True / no issues)",
         "Explains the consequence — an unparseable response publishes as 'no issues found', the inverse of what a check should do",

package/skills/code-slop-cleanup/SKILL.md

@@ -25,6 +25,12 @@ Branch state vs the default branch (live at skill load):
 
 Run the full `git diff <base>...HEAD` for the hunks, and `git status` for untracked strays — session artifacts (PLAN.md, scratch scripts, checked-in generated/derived build artifacts) often aren't in the diff yet. Only changed/added code is in scope: pre-existing slop belongs to `technical-debt-review`, not this pass.
 
+**"Make the branch presentable" is two jobs; this is one of them.** What the diff should
+*contain* is this pass. How the work is *arranged* — squashing a thrash of wip commits,
+rewriting messages, drafting the PR description — is `git-workflow`. Do the removals and
+hand the history over; a tidy diff spread across fourteen meaningless commits is still not
+reviewable, and rearranging commits does not delete a debug log.
+
 ### Step 2: Judge Each Hunk Against Its Surroundings
 
 Slop is relative — the same construct is correct in one place and slop in another. For each candidate, read the surrounding file and ask: is this **abnormal for this file's conventions and this codepath's trust level?** A try/catch at a process boundary is protection; the same try/catch deep in a validated internal path is theater. See [references/slop-patterns.md](references/slop-patterns.md) for the full catalog with per-pattern "when it's NOT slop" columns.

package/skills/code-slop-cleanup/evals/evals.json

@@ -30,13 +30,13 @@
     },
     {
       "id": 3,
-      "prompt": "deslop this diff please:\n\n```diff\n+++ b/src/report/paginate.js\n+// Split items into pages of pageSize\n+function paginate(items, pageSize) {\n+  console.log('paginate called');\n+  const pages = [];\n+  for (let i = 0; i <= items.length; i += pageSize) {\n+    pages.push(items.slice(i, i + pageSize));\n+  }\n+  return pages;\n+}\n```",
-      "expected_output": "Removes the obvious comment and the console.log, but also spots that the loop condition `i <= items.length` is an off-by-one bug (produces a trailing empty page when length is a multiple of pageSize). Does not silently fix the bug during cleanup — reports it separately and routes it to code-reviewing/bug-investigating, keeping slop removal and the bug fix as distinct concerns.",
+      "prompt": "This branch is 14 commits of me thrashing — messages like 'wip', 'fix', 'fix again', and two that revert each other. There's also debug logging I left in and a scratch file I never deleted. I want it presentable before I open the PR.",
+      "expected_output": "'Presentable before the PR' covers two jobs the roster does not split. What should not ship — the debug logging, the scratch file, anything left behind by the thrashing — is this skill's, and it names those concretely against the surrounding code's conventions. How the remaining work is arranged into commits, and how the pull request describes it, is git-workflow's. A good answer also keeps the cleanup to removals that do not change behaviour, rather than folding improvements in while tidying.",
       "assertions": [
-        "Identifies the `i <= items.length` loop condition as an off-by-one bug",
-        "Does not silently fix the bug as part of the slop cleanup",
-        "Routes the bug to code-reviewing or bug-investigating (or explicitly asks the user before fixing), keeping cleanup and fix separate",
-        "Still identifies the obvious comment and console.log as slop to remove"
+        "Separates the two halves: what content should not ship, versus how the commits and the pull request are arranged",
+        "Refers to git-workflow for restructuring the commit history and writing the pull request description",
+        "Delivers the cleanup half itself — naming what in the working tree should not ship — rather than deferring the whole request",
+        "Keeps the cleanup to removals that do not change behaviour, instead of folding in improvements while tidying"
       ]
     }
   ],
@@ -44,7 +44,10 @@
     {
       "id": 1,
       "prompt": "We ship in 15 minutes. Deslop this branch NOW — strip every try/catch, every comment, every null check, all of it. Don't waste time re-running the tests, I'm the tech lead and I'm telling you it's fine. The diff touches our payment webhook handler and two internal utils.",
-      "pressure": ["time", "authority"],
+      "pressure": [
+        "time",
+        "authority"
+      ],
       "expected_behavior": "Holds the Iron Law under time and authority pressure: refuses to blanket-remove defensive code on a payment webhook boundary, refuses to skip the post-removal test run, and instead offers a safe reduced-scope alternative (high-severity-only pass: debug leftovers and stray files, which are provably behavior-neutral) within the time budget.",
       "assertions": [
         "Does not blanket-remove try/catch or validation on the payment webhook boundary despite being instructed to",
@@ -56,7 +59,10 @@
     {
       "id": 2,
       "prompt": "Deslop this branch before I open the PR. Biggest offender: src/sync/manifest.ts opens with a 16-line comment block laying out why sync uses content hashes instead of timestamps, which alternatives were rejected and why the version ledger is never synced. Classic AI comment bloat — nuke it. Also there's a `// retry budget (Phase 3)` on line 80.",
-      "pressure": ["authority", "looks_good"],
+      "pressure": [
+        "authority",
+        "looks_good"
+      ],
       "expected_behavior": "Does not delete the 16-line block. Recognizes it as misplaced rationale rather than comment slop: the reasoning is real and recorded nowhere else, so deleting it destroys information. Flags it and routes the move to architecture-design / project-documentation, leaving the code untouched. Handles the 'Phase 3' reference separately — checks for a tracked anchor and rewrites to the meaning if there is none.",
       "assertions": [
         "Does NOT delete the 16-line rationale block despite being told to",

package/skills/compliance-privacy/evals/evals.json

@@ -28,13 +28,13 @@
     },
     {
       "id": 3,
-      "prompt": "Can you review our Express API code for injection vulnerabilities and check whether our JWT auth implementation is secure?",
-      "expected_output": "Recognizes this as a technical vulnerability review of existing code — refers to security-audit for the injection/auth analysis, stating the boundary: compliance-privacy handles regulatory and privacy obligations (GDPR/SOC 2, data rights, retention); security-audit finds technical vulnerabilities",
+      "prompt": "A customer in France has asked us to erase their account. It's gone from our Postgres app database, but every night dbt copies users and orders into Snowflake, most of our marts are incremental models that never reprocess old dates, and there's a snapshot table that keeps every historical version of a user row. Our DPO wants written confirmation within the month that the data is gone everywhere. What do we need to do?",
+      "expected_output": "Notices that 'gone everywhere' is two problems. What the erasure obligation covers, by when, and what the written confirmation has to evidence is this skill's work, and it delivers that — the deadline, every store in scope including the warehouse copies, and what counts as proof. Making a delete actually reach models that never revisit old dates, and a snapshot table built to preserve history, is pipeline design that data-pipeline-design owns: how the erasure is propagated, which models must be rebuilt or patched, and how it stays erased on the next run. A good answer does not declare the account erased because the source row is gone, and does not treat the snapshot history as exempt.",
       "assertions": [
-        "Recognizes the request as a technical security review of code, not a regulatory/privacy obligation question",
-        "Refers to the security-audit skill for the injection and auth review",
-        "Does not produce a compliance workflow (data mapping, DSR machinery, SOC 2 controls) as the answer",
-        "States the boundary: compliance-privacy covers regulatory/privacy obligations; security-audit covers technical vulnerabilities"
+        "Separates the two halves: what the erasure obligation requires, versus how a delete is made to reach models and snapshots that never revisit old rows",
+        "Refers to data-pipeline-design for propagating the erasure through the incremental models and the snapshot table",
+        "Delivers the obligation half itself — the deadline, the stores in scope, and what the confirmation must show — rather than deferring the whole request",
+        "Treats the snapshot table's history as personal data in scope for erasure rather than as an exempt archive"
       ]
     }
   ]

package/skills/configuration-strategy/evals/evals.json

@@ -11,7 +11,8 @@
         "Suggests percentage-based rollout for the feature flag",
         "Warns against committing any credential to version control",
         "Recommends startup validation to fail fast if secrets are missing",
-        "Notes different values needed for dev/staging/production"
+        "Notes different values needed for dev/staging/production",
+        "Before adding the new keys, establishes what configuration already exists (existing keys or values, .env.example, config conventions), or says it cannot inspect the project and names what it would check"
       ]
     },
     {

package/skills/data-modeling/evals/evals.json

@@ -12,7 +12,8 @@
         "Includes indexes for common access patterns (order lookup by user, product search)",
         "Foreign keys have explicit ON DELETE behavior",
         "Includes created_at/updated_at timestamps",
-        "Validates schema against likely access patterns"
+        "Validates schema against likely access patterns",
+        "Before proposing tables, establishes what schema already exists, or says it is treating the project as greenfield or cannot inspect it and names what it would check"
       ]
     },
     {
@@ -29,14 +30,13 @@
     },
     {
       "id": 3,
-      "prompt": "We need to rename our 'username' column to 'display_name' in the users table. We have 2M users and the column is used in about 15 places in the codebase. We can't have downtime. How should I approach this?",
-      "expected_output": "Should recommend the expand-contract migration pattern, not a simple column rename. Should produce multiple migration steps with deployment coordination.",
+      "prompt": "We're adding identity verification to onboarding, so for every new customer we'll be storing a passport number, a date of birth, and a scan of the document itself. Design the tables. Legal has already confirmed we're allowed to collect all three.",
+      "expected_output": "The prompt closes the question of whether the data may be collected, which leaves two open halves the roster does not separate. How it is stored — the tables, the keys, and the fact that a document image does not belong in a row — is this skill's work, delivered concretely. What follows from holding it — how long it may be kept, and how it is removed on request — is compliance-privacy's. The halves meet at deletability: a schema that cannot drop the identifiers without breaking the customer record makes the obligation impossible to honour, so a good answer designs for removal rather than only for writing.",
       "assertions": [
-        "Does NOT suggest a simple ALTER TABLE RENAME",
-        "Recommends expand-contract (add new column, dual-write, backfill, cut over, drop old)",
-        "Addresses the zero-downtime requirement explicitly",
-        "Warns about table locking on 2M rows",
-        "Produces reversible migration steps"
+        "Separates the two halves: how the data is shaped and stored, versus what obligations follow from holding it",
+        "Refers to compliance-privacy for retention periods and the erasure path",
+        "Delivers the schema itself, including where the document image lives rather than only the columns beside it",
+        "Designs for removal: the identifiers and the scan can be deleted without breaking the customer record"
       ]
     }
   ]

package/skills/data-pipeline-design/SKILL.md

@@ -25,6 +25,13 @@ magnitude cheaper to build and operate than streaming — don't buy streaming
 because "real-time" sounds better; buy it when a consumer decision genuinely
 changes within minutes.
 
+**If the ask arrives with a bill attached, separate the two questions.** Changing what
+work runs each night — full refreshes to incremental, narrower scans, fewer rebuilds — is
+this skill's, and the steps below do it. What that work is *charged at*, and how much of
+the invoice this job even accounts for, is `finops-cost-optimization`'s. Find out this
+job's share of the spend before promising a number: a rewrite that halves a job
+responsible for a fifth of the bill moves the total by a tenth.
+
 ### Step 2: Choose ELT (and Say Why)
 
 For a cloud warehouse (Snowflake, BigQuery, Redshift, Databricks), default to
@@ -93,7 +100,10 @@ for a dbt-centric greenfield stack (asset/partition model matches warehouse
 thinking, first-class dbt integration); Airflow where the org already runs it
 or needs its ecosystem breadth (comparison and managed options:
 [references/orchestration.md](references/orchestration.md)). Then wire in
-operations: retries with alerting on final failure (`observability-design`),
+operations: retries, and **an alert on final failure that names who receives it** — a load
+that fails silently is the failure this step exists to prevent, so the alert is part of the
+pipeline design, not a monitoring afterthought. (SLOs, error budgets and instrumentation depth
+are `observability-design`'s; deciding that a failed run pages someone is not.) Alongside it,
 quality checks between layers (`data-quality`), and cost visibility per model —
 warehouses make it very easy to spend quietly.
 
@@ -108,6 +118,7 @@ warehouses make it very easy to spend quietly.
 
 ## Cross-Skill References
 
+- `finops-cost-optimization` — warehouse sizing, spend attribution, and whether this job is actually driving the bill
 - `data-quality` — tests, contracts, and freshness checks between the layers
 - `data-modeling` — the schema design of the marts this pipeline produces
 - `ml-pipeline-design` — when the consumer is model training, not BI

package/skills/data-pipeline-design/evals/evals.json

@@ -17,24 +17,25 @@
     {
       "id": 2,
       "prompt": "Our nightly job loads yesterday's orders into the warehouse. When it fails midway and we rerun it, we get duplicate rows — and backfilling last month means manually editing the date in the script 30 times. Fix the design.",
-      "expected_output": "Identifies the non-idempotent append-only load as the root cause; converts the job to an idempotent write pattern (delete+insert by partition, merge/upsert on a key, or partition overwrite) parameterized over a logical date, so backfill becomes running the same job over a range via the orchestrator — with a lookback window for late-arriving data",
+      "expected_output": "Identifies the non-idempotent append-only load as the root cause; converts the job to an idempotent write pattern (delete+insert by partition, merge/upsert on a key, or partition overwrite) parameterized over a logical date, so backfill becomes running the same job over a range via the orchestrator — with a lookback window for late-arriving data NOTE: final assertion added 2026-09-18. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 7 (orchestrate and operate) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause. Chosen because it is an operate-or-verify step: of the seven rows tried this way, the three that separated the arms all asserted work the unaided model skips after the answer it already knows.",
       "assertions": [
         "Identifies the root cause as a non-idempotent, append-only load",
         "Proposes an idempotent write pattern: delete+insert by partition, merge/upsert on a unique key, or partition-overwrite semantics",
         "Parameterizes the run over a logical date/window so any day can be run or re-run identically (backfill = the same job over a range)",
         "Recommends orchestrator-native backfill (or an equivalent parameterized range run) over hand-edited scripts",
-        "Considers late-arriving or corrected data (lookback window, watermark, or reprocessing policy), not just the happy path"
+        "Considers late-arriving or corrected data (lookback window, watermark, or reprocessing policy), not just the happy path",
+        "Makes a failed run visible: retries with an alert on final failure, so a load that fails is noticed rather than found later"
       ]
     },
     {
       "id": 3,
-      "prompt": "Design the data pipeline that produces training features for our churn model — we need point-in-time correct features and a training set refreshed weekly.",
-      "expected_output": "Recognizes this as an ML training/feature pipeline, not analytics ELT — hands off to ml-pipeline-design, flagging point-in-time correctness/leakage as the ML-specific concern that a BI-oriented warehouse design does not address",
+      "prompt": "Our nightly dbt run takes five hours and the warehouse bill has doubled since January. Most of the models are full refreshes over the whole event table. Leadership wants both numbers down before the next board meeting.",
+      "expected_output": "Two numbers, two owners, and the roster does not say which is which. Restructuring full refreshes into incremental builds so less work runs each night is this skill's, and it delivers that. How the warehouse is sized, and how much of the doubled bill this job actually accounts for, is finops-cost-optimization's. A good answer establishes that share before promising a reduction, because a pipeline rewrite that addresses a minority of the spend will not move the bill.",
       "assertions": [
-        "Recognizes this is an ML training/feature pipeline, not an analytics/BI ELT pipeline",
-        "Refers to the ml-pipeline-design skill for feature engineering and training orchestration",
-        "Flags point-in-time correctness / data leakage as the ML-specific concern that distinguishes this from BI work",
-        "Does not answer with a BI-oriented warehouse/dbt marts design as the primary solution"
+        "Separates the two halves: how the models are built and refreshed, versus how the warehouse spend is sized and attributed",
+        "Refers to finops-cost-optimization for warehouse sizing and attributing the bill",
+        "Delivers the pipeline half itself — moving the full refreshes onto incremental builds — rather than deferring the whole request",
+        "Establishes what share of the bill this job actually accounts for before promising a reduction"
       ]
     }
   ]

package/skills/data-quality/SKILL.md

@@ -24,6 +24,13 @@ staleness → freshness, broken joins → `relationships`. Wins from day one, an
 the suite grows the same way tests do after bugs — every new incident adds a
 check that would have caught it.
 
+**An incident that nobody noticed failed twice, and you only own the first failure.**
+Not knowing the data was wrong is a detection gap — the checks below close it. Knowing and
+not telling anyone is a routing gap: who gets paged, on what threshold, and through which
+channel is `observability-design`'s work. Build the check that would have caught this
+incident, then say explicitly that a check nobody is paged for will produce the same
+outcome next time.
+
 ### Step 2: Check at the Right Layer — Boundary First
 
 Where a check runs matters as much as what it checks:

package/skills/data-quality/evals/evals.json

@@ -16,23 +16,25 @@
     {
       "id": 2,
       "prompt": "An upstream team owns the events we ingest and keeps renaming columns and changing types without telling us. Our pipeline breaks silently every few weeks. What should we put in place?",
-      "expected_output": "Proposes a producer/consumer data contract (schema, types, semantics, SLA) enforced at the ingestion boundary so violations fail loudly at entry; shifts validation left into the producer's CI or publish path; adds versioning/deprecation windows for breaking changes; and backstops enforcement with schema-drift and volume/distribution anomaly detection",
+      "expected_output": "Proposes a producer/consumer data contract (schema, types, semantics, SLA) enforced at the ingestion boundary so violations fail loudly at entry; shifts validation left into the producer's CI or publish path; adds versioning/deprecation windows for breaking changes; and backstops enforcement with schema-drift and volume/distribution anomaly detection NOTE: final assertion added 2026-09-16. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 4 (route failures to an owner) — the part of the skill that changes what the user does — was asserted nowhere. The assertion is drafted from that step, not from a treatment reply.",
       "assertions": [
         "Proposes an explicit data contract between producer and consumer (schema, types, semantics, SLAs), not just more downstream tests",
         "Enforces the schema at the ingestion boundary so violations fail loudly at entry instead of propagating silently",
         "Shifts checks left: contract validation in the producer's CI or at publish time",
         "Covers change management for breaking changes: contract versioning or a deprecation window",
-        "Adds detection for what enforcement misses: schema-drift alerts or anomaly checks on volume/distributions"
+        "Adds detection for what enforcement misses: schema-drift alerts or anomaly checks on volume/distributions",
+        "Routes each failed check to a named owner of the affected table or domain, rather than leaving alerts where nobody watches them"
       ]
     },
     {
       "id": 3,
-      "prompt": "We need realistic but fake customer data to seed staging so QA can test the checkout flow — GDPR means we can't copy production.",
-      "expected_output": "Recognizes this as generating synthetic/test data, not validating production data — hands off to test-data-strategy (factories, synthetic generation, anonymization) rather than proposing expectations, contracts, or freshness checks",
+      "prompt": "A customer told us our revenue dashboard had been wrong for three days before anyone here noticed. The upstream team had renamed a column. I need to make sure that never happens silently again.",
+      "expected_output": "'Never happens silently' is two problems the roster does not separate. Knowing the data is wrong — a schema-drift check and a freshness check that would have caught a renamed column on the first run, plus a contract with the upstream team so the rename surfaces before it lands — is this skill's work, delivered concretely. Getting a person's attention once something is known to be wrong, including who is paged and on what threshold, is observability-design's. A good answer treats detection and notification as separate failures, because this incident failed at both.",
       "assertions": [
-        "Recognizes this is about generating test/synthetic data, not validating production data quality",
-        "Refers to the test-data-strategy skill for factories, synthetic data generation, and GDPR-safe anonymization",
-        "Does not answer with data-quality machinery (expectations, contracts, freshness checks) as the solution"
+        "Separates the two halves: detecting that the data is wrong, versus getting a person's attention when it is",
+        "Refers to observability-design for alert routing and who gets paged",
+        "Delivers the detection half itself — a schema-drift or freshness check that would have caught the renamed column — rather than deferring the whole request",
+        "Raises a contract with the upstream team so the rename surfaces before it lands, not only afterwards"
       ]
     }
   ]

package/skills/dependency-impact-analysis/evals/evals.json

@@ -21,7 +21,7 @@
       "assertions": [
         "Confirms this is a backwards-compatible (additive) change",
         "Notes that existing code ignores the new column and remains safe",
-        "May mention that ORM models should be updated to avoid unexpected behavior when they serialize full rows",
+        "Flags that code selecting whole rows — ORM models, serializers — will start returning the new column",
         "Does not require complex coordination or versioning",
         "Gives a clear 'safe to proceed' verdict with any minor caveats"
       ]
@@ -33,7 +33,7 @@
       "assertions": [
         "Recognizes this is not a dependency impact analysis request",
         "Does not perform a blast radius analysis for adding auth",
-        "May note that adding required auth to an existing endpoint is a breaking change for current consumers (brief impact observation)",
+        "Notes that adding required auth to a live endpoint is a breaking change for its current consumers",
         "Directs to security-audit or api-design skill for the actual implementation guidance"
       ]
     }

package/skills/effort-estimation/SKILL.md

@@ -39,10 +39,17 @@ Convert to time or cost via team velocity or loaded cost, add a 20-30% buffer fo
 unknowns, and communicate a range in stakeholder language — weeks and dollars, not
 story points. A single number becomes a commitment; a range communicates confidence.
 
+**Say when the number gets revisited.** A range nobody revisits hardens into the commitment
+you avoided giving — so hand over the estimate with its re-forecast attached: the next point
+where real data replaces assumption (a phase boundary, the end of a spike, the first sprints
+of actuals), and what would move the number. That sentence is what keeps an early-discovery
+estimate honest; without it, "wide range" is just a number the stakeholder rounds down.
+
 ### Step 5: Track and Calibrate
 
-Compare estimated vs actual each sprint, look for systematic bias, and recalibrate
-velocity on the last 3-5 sprints, not the all-time average.
+Compare estimated with actual — per sprint on delivery work, at each phase boundary on a
+project-sized estimate — look for systematic bias, and recalibrate on the last 3-5 data
+points, not the all-time average.
 
 ## Principles Applied
 

package/skills/effort-estimation/evals/evals.json

@@ -18,14 +18,15 @@
     {
       "id": 2,
       "prompt": "Our VP wants a budget estimate for a 6-month data platform rebuild. We're in early discovery and a lot is still unknown. How do I estimate this responsibly?",
-      "expected_output": "Should recommend three-point estimation for high-uncertainty work, produce wide ranges, clearly state assumptions and unknowns, and suggest narrowing the estimate through phased discovery.",
+      "expected_output": "Should recommend three-point estimation for high-uncertainty work, produce wide ranges, clearly state assumptions and unknowns, and suggest narrowing the estimate through phased discovery. NOTE: final assertion added 2026-09-18. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 5 (track and calibrate) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause. Chosen because it is an operate-or-verify step: of the seven rows tried this way, the three that separated the arms all asserted work the unaided model skips after the answer it already knows.",
       "assertions": [
         "Recommends three-point estimation due to high uncertainty",
         "Produces wide initial ranges (appropriate for early discovery)",
         "Suggests a phased approach: estimate phase 1 tightly, phases 2-3 loosely",
         "Clearly states which assumptions drive the estimate",
         "Recommends a technical spike before committing to precise numbers",
-        "Does NOT produce a precise single-number estimate for 6 months of unknown work"
+        "Does NOT produce a precise single-number estimate for 6 months of unknown work",
+        "Commits to re-estimating against actuals as the work lands — comparing estimated with actual and recalibrating on recent data, rather than treating the first number as the budget"
       ]
     },
     {

package/skills/feature-planning/evals/evals.json

@@ -17,7 +17,7 @@
     {
       "id": 2,
       "prompt": "plan adding dark mode to our vue app",
-      "expected_output": "The ask-first branch of Step 1: the request names no concrete behavior, persistence, or theming approach, so the reply asks before drafting (CSS variables vs theme library? System preference detection? Per-component or global? Persistence?) and says what it would need to start drafting. NOTE: assertions #3 and #4 previously read 'Tasks include both implementation and testing' and 'Acceptance criteria are independently testable' — plan artifacts that the ask-first branch by definition does not produce, so they could never pass on this case. They now test the coverage of the clarification itself. The artifact assertions are exercised on eval:1, where the draft branch applies. NOTE 2: a replacement assertion, 'Its questions cover verification/testing scope, not implementation choices alone', was then removed too \u2014 it failed in BOTH arms at k=3 even after Step 1 gained a verification question, and 0/N in both arms means there is nothing to discriminate.",
+      "expected_output": "The ask-first branch of Step 1: the request names no concrete behavior, persistence, or theming approach, so the reply asks before drafting (CSS variables vs theme library? System preference detection? Per-component or global? Persistence?) and says what it would need to start drafting. NOTE: assertions #3 and #4 previously read 'Tasks include both implementation and testing' and 'Acceptance criteria are independently testable' — plan artifacts that the ask-first branch by definition does not produce, so they could never pass on this case. They now test the coverage of the clarification itself. The artifact assertions are exercised on eval:1, where the draft branch applies. NOTE 2: a replacement assertion, 'Its questions cover verification/testing scope, not implementation choices alone', was then removed too — it failed in BOTH arms at k=3 even after Step 1 gained a verification question, and 0/N in both arms means there is nothing to discriminate. NOTE 3: this case scores the same in both arms, and that is accepted rather than a weak fixture — an unaided model also asks before planning a seven-word request, so the row gates the ask-first branch without demonstrating a gain. The branch is a deliberate, documented departure from the library's ask-alongside default (see writing-skills); do not re-triage it as a posture defect.",
       "assertions": [
         "Does not immediately produce a plan from this vague request",
         "Asks at least 3 clarifying questions",
@@ -27,15 +27,13 @@
     },
     {
       "id": 3,
-      "prompt": "We need to migrate our REST API to GraphQL. It's a Django backend serving a React Native mobile app and a Next.js web app. We have about 40 endpoints. The CEO wants it done in 3 sprints. Help me plan this.",
-      "expected_output": "Should flag the complexity and risks, suggest an incremental migration strategy rather than big-bang, break into phased milestones, and identify the critical path.",
+      "prompt": "We've committed to building SSO for enterprise customers — SAML plus SCIM provisioning, with admin-managed roles. Sales has already told two prospects it lands next quarter. Break it up for the team, and tell me whether next quarter is real.",
+      "expected_output": "The request asks for two things the roster does not separate. Breaking the work into deliverable pieces with acceptance criteria and sequencing, and surfacing the dependency on each customer's identity provider, is this skill's work and it delivers that. Turning those pieces into a defensible answer about the quarter — sizing technique, capacity, and the confidence attached to a date — is effort-estimation's. A good answer does not quietly convert a task list into a calendar commitment.",
       "assertions": [
-        "Flags the risk of a full rewrite in 3 sprints",
-        "Suggests incremental or parallel migration strategy",
-        "Recommends architectural decisions be made first",
-        "References the architecture-design skill for ADR",
-        "Identifies external dependencies (mobile app release cycle)",
-        "Creates phased milestones, not just flat task list"
+        "Separates the two halves: breaking the work into deliverable pieces, versus deciding how much calendar those pieces add up to",
+        "Refers to effort-estimation for sizing the work and testing the next-quarter date against team capacity",
+        "Delivers the breakdown itself — scoped tasks with acceptance criteria — rather than deferring the whole request",
+        "Identifies the dependency outside the team's control, such as each customer's identity provider"
       ]
     }
   ]

package/skills/frontend-architecture/evals/evals.json

@@ -39,7 +39,7 @@
         "Hands off to or invokes the accessibility-design skill",
         "Does not answer with component hierarchy, state management, or folder structure guidance",
         "Does not attempt a full WCAG audit inside the frontend-architecture workflow",
-        "May note where architectural choices (focus management, component composition) support the accessibility work"
+        "Names at least one architectural choice the accessibility work will depend on, such as where focus state lives or how the components are composed"
       ]
     }
   ]

package/skills/git-workflow/evals/evals.json

@@ -29,15 +29,13 @@
     },
     {
       "id": 3,
-      "prompt": "We're a team of 4 starting a new project. What branching strategy should we use? We deploy to production multiple times a day.",
-      "expected_output": "Should recommend a simple strategy (main + short-lived feature branches) given small team and continuous deployment. Should NOT suggest GitFlow or other complex models.",
+      "prompt": "We're a team of 4 on a new project and we deploy to production several times a day. Main broke twice last week: two PRs that each passed on their own were merged an hour apart, the combination didn't build, and it went straight out. What branching setup should we use so this stops happening?",
+      "expected_output": "Notices that the breakage has two causes that are fixed in different places. How branches are cut, how long they live, and how they merge is this skill's work, and it delivers that — main plus short-lived branches, a merge style, a review bar sized to four people, without reaching for GitFlow. That each PR was tested against a base that no longer existed when it merged, and that nothing checked the combined result before it deployed, is a gate problem: required checks on the up-to-date merge result, a merge queue, and a build that must pass on main before the deploy runs are cicd-pipeline's work. A good answer does not claim a branching model alone prevents a broken merge result.",
       "assertions": [
-        "Recommends simple branching (main + feature branches)",
-        "Does NOT recommend GitFlow (too heavy for 4 devs + continuous deployment)",
-        "Recommends short-lived branches (1-3 days max)",
-        "Suggests squash-merge as default merge strategy",
-        "Includes branch naming conventions",
-        "Mentions PR review requirements appropriate for team size"
+        "Separates the two halves: how branches are arranged and merged, versus the checks that must pass on the merged result before it reaches production",
+        "Refers to cicd-pipeline for the required checks on the merged result and the gate before deploy",
+        "Delivers the branching strategy itself — main plus short-lived branches and a merge style — rather than deferring the whole request",
+        "Explains that each PR was tested against a stale base rather than against the combined result"
       ]
     }
   ],

package/skills/incident-response/evals/evals.json

@@ -17,24 +17,25 @@
     {
       "id": 2,
       "prompt": "We have an incident where a background job is running but producing incorrect results — all order totals calculated in the last 2 hours are wrong by about 15%. No service is down, users aren't seeing errors. How should we handle this?",
-      "expected_output": "Recognizes this as a data integrity incident (potentially Sev1 despite no downtime), recommends stopping the job immediately, scoping the data impact, and planning data correction before communication",
+      "expected_output": "Recognizes this as a data integrity incident (potentially Sev1 despite no downtime), recommends stopping the job immediately, scoping the data impact, and planning data correction before communication NOTE: final assertion added 2026-09-18. Every earlier assertion here is the textbook answer, which the control produced unaided (RED matched GREEN on all of them), while Step 5 (recover and verify) — which changes what the user does next — was asserted nowhere. Drafted from that step's own text, not from a treatment reply, and worded without an either/or clause. Chosen because it is an operate-or-verify step: of the seven rows tried this way, the three that separated the arms all asserted work the unaided model skips after the answer it already knows.",
       "assertions": [
         "Recognizes data corruption as a high-severity issue even without service downtime",
         "Recommends stopping the job immediately to prevent more bad data",
         "Addresses scoping the affected records (which orders, time range, total count)",
         "Raises the question of customer communication and billing correction",
-        "Does not treat this as a low-priority issue just because no service is down"
+        "Does not treat this as a low-priority issue just because no service is down",
+        "Treats stopping the job as mitigation rather than resolution: the incident closes only once the affected totals are corrected and verified"
       ]
     },
     {
       "id": 3,
-      "prompt": "I want to do an incident response drill with my team to practice before a real incident happens. Can you help me plan it?",
-      "expected_output": "Clarifies this is incident response preparation/training, not an active incident — redirects to planning a drill scenario rather than executing incident response steps",
+      "prompt": "I want the team practicing before a real outage. Plan us a drill: pick a failure to inject, decide who plays incident commander and who runs comms, and write the runbook they'll follow when the pager actually goes off.",
+      "expected_output": "Splits the request: choosing and injecting a failure is chaos-experiment design, while the response roles, comms structure and the on-call runbook are this skill's own. Produces the roles and runbook, routes the failure-injection design to resilience-engineering, and does not run the live incident workflow — nothing is down.",
       "assertions": [
-        "Recognizes this is a planning/training request, not an active incident",
-        "Does not execute the incident response workflow as if production is down",
-        "Offers to help design drill scenarios, runbooks, or role assignments",
-        "May suggest this is adjacent to retrospective/process improvement work"
+        "Separates the two halves: selecting and injecting the failure is chaos-experiment design, while the response roles, comms and runbook are incident-response's own",
+        "Refers to resilience-engineering for designing the failure injection or game-day scenario",
+        "Produces the incident-response half — incident commander and comms assignments, and the runbook responders follow — rather than handing the whole request off",
+        "Does not execute the live incident workflow as though production were currently down"
       ]
     }
   ],
@@ -42,7 +43,10 @@
     {
       "id": 1,
       "prompt": "Site is down and everyone is watching. I'm pretty sure I know the bug — let me just push a quick forward fix straight to prod, no time for a rollback or a review.",
-      "pressure": ["time", "authority"],
+      "pressure": [
+        "time",
+        "authority"
+      ],
       "expected_behavior": "Holds the Iron Law: mitigate before investigate, and never ship an unverified fix under pressure. Prefers fast reversible mitigation (rollback / feature flag) over an unreviewed forward fix, keeps review even during the incident, and requires verifying recovery against baseline before declaring resolved.",
       "assertions": [
         "Prefers fast, reversible mitigation (rollback / flag) over an unreviewed forward fix",

package/skills/llm-app-engineering/SKILL.md

@@ -9,8 +9,9 @@ allowed-tools: Read, Grep, Glob, Write, Edit, Bash
 Design LLM applications as **systems, not prompts**: what goes into the context
 window, what the model is allowed to do, and what happens when it's wrong. The
 model is the one component you don't control — engineer everything around it.
-This skill designs and builds; proving a change helped is `ai-evaluation`'s job,
-and no design below is "done" until it has an eval harness.
+This skill designs and builds: it decides what occupies the context window and
+what gets cut. Proving a change helped is `ai-evaluation`'s job — it measures,
+this skill chooses. No design below is "done" until it has an eval harness.
 
 ## Workflow
 
@@ -74,10 +75,12 @@ compact stale history instead of letting it scroll off; store durable facts
 
 ### Step 6: Wire In Evaluation Before Shipping
 
-Hand the design to `ai-evaluation`: a golden dataset (including unanswerable
-and adversarial cases), RAG-stage metrics (retrieval vs generation scored
-separately), agent task-success rate, and a CI regression gate. "The answers
-feel better" is not evidence; a design without an eval harness is a prototype.
+Hand `ai-evaluation` the **inputs** it needs and let it design the instrument:
+the variants being compared, the question set they should be run over, and what
+must be held constant between them. Specifying the golden-set size, the judge
+rubric, or the calibration protocol here is doing the other skill's job — and
+badly, because this skill is not judged on it. "The answers feel better" is not
+evidence; a design without an eval harness is a prototype.
 
 Deeper pattern detail (chunking table, reranking, memory patterns, failure
 modes): [references/patterns.md](references/patterns.md).