swe-workflow-skills 0.7.0 → 0.8.1

package/skills/frontend-architecture/evals/evals.json

@@ -29,6 +29,18 @@
         "Does NOT recommend adding yet another state management library on top",
         "Addresses the performance issue specifically (memoization, selector optimization)"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Component structure is settled. Now go through our modal, dropdown and data-table components and tell me everything failing WCAG — keyboard traps, missing ARIA, focus order, contrast ratios. We have an accessibility audit next month.",
+      "expected_output": "Scope boundary — a WCAG conformance review of existing components is accessibility-design work, not frontend architecture. Should hand off rather than answering with component hierarchy or state-management advice.",
+      "assertions": [
+        "Recognizes that WCAG conformance, ARIA, focus order and contrast are accessibility-design concerns",
+        "Hands off to or invokes the accessibility-design skill",
+        "Does not answer with component hierarchy, state management, or folder structure guidance",
+        "Does not attempt a full WCAG audit inside the frontend-architecture workflow",
+        "May note where architectural choices (focus management, component composition) support the accessibility work"
+      ]
     }
   ]
 }

package/skills/gitops-delivery/SKILL.md

@@ -31,6 +31,14 @@ GitOps makes Git the single source of truth for deployment state. An operator wa
 
 **When GitOps is worth it:** Multiple environments, compliance/audit needs, multi-cluster, or you want drift detection and self-healing.
 
+**Either way, say this:** version pinning and a declarative record of what is deployed are
+worth adopting **regardless of which side of this decision you land on** — they are what make
+a deploy reproducible and a rollback a known-good revert, and none of that requires an
+operator. Point at `deployment-repo` even when you are recommending against ArgoCD; a team
+that keeps pinned manifests in Git gets most of the auditability benefit now, and has already
+done the setup work if they adopt GitOps later. Recommending push-based is not a reason to
+leave the handoff out.
+
 ## Step 2: Design Cross-Repo Triggers
 
 The critical bridge: when an app repo builds a new image, the deployment repo must be updated. This is the most common source of friction in GitOps setups.
@@ -310,7 +318,7 @@ Test the full GitOps pipeline:
 
 ## Cross-Skill References
 
-- `deployment-repo` — the Git repository that the GitOps operator watches; design the repo structure first
+- `deployment-repo` — pinned versions and declarative manifests in Git, **valuable with or without an operator**: it is what ArgoCD/Flux watch if you adopt GitOps, and the reproducibility/rollback substrate if you stay push-based. Design the repo structure first either way.
 - `cicd-pipeline` — cross-repo trigger pipelines that update the deployment repo on new builds
 - `rollback-strategy` — in GitOps, rollback = revert a Git commit; the operator handles the rest
 - `observability-design` — metrics and dashboards for monitoring sync status, drift, and progressive delivery analysis

package/skills/infrastructure-as-code/evals/evals.json

@@ -28,6 +28,18 @@
         "Flags missing tags",
         "Provides corrected Terraform with restricted policy"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Our Terraform is in good shape. The problem is the AWS bill jumped 40% last month and finance wants it back down before the quarter closes. Work out what's driving it and cut it.",
+      "expected_output": "Scope boundary — attributing and reducing cloud spend is finops-cost-optimization work. Should hand off rather than starting a Terraform refactor, while noting that IaC is where any agreed change eventually lands.",
+      "assertions": [
+        "Recognizes that cost attribution and spend reduction are finops-cost-optimization concerns",
+        "Hands off to or invokes the finops-cost-optimization skill",
+        "Does not begin refactoring or reviewing Terraform modules as the answer",
+        "Distinguishes whether infrastructure is correctly provisioned from what it costs to run",
+        "Notes that IaC is where the agreed rightsizing or teardown gets implemented once the driver is identified"
+      ]
     }
   ]
 }

package/skills/metrics-and-okrs/evals/evals.json

@@ -28,6 +28,18 @@
         "Suggests a timeframe for evaluation (not 'check once and declare victory')",
         "Warns against vanity metrics (total searches without context is meaningless)"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "We defined the KR already. Here's the data: the A/B test on the new onboarding shows a 3.1% lift in activation, p = 0.06, n = 800 split across both arms. Is that a real effect? Do we ship it?",
+      "expected_output": "Scope boundary — judging whether an observed lift is real (significance, power, sample size, stopping rules) is statistical-analysis work. Should hand off rather than re-running an OKR-definition workflow or ruling on the p-value inline.",
+      "assertions": [
+        "Recognizes this as an inference question about experiment results, not a metric-definition question",
+        "Hands off to or invokes the statistical-analysis skill",
+        "Does not declare the result significant or issue a ship/no-ship verdict from the p-value alone",
+        "Does not restart an OKR or success-metric definition exercise the user has already completed",
+        "May note how the established result would feed the existing key result once the inference is settled"
+      ]
     }
   ]
 }

package/skills/ml-experiment-tracking/evals/evals.json

@@ -27,6 +27,18 @@
         "Suggests data versioning to prevent future dataset ambiguity",
         "Provides a practical migration path, not an all-or-nothing approach"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Before I wire up MLflow — I don't actually know how to tell whether our RAG support chatbot is answering well. What should the golden dataset look like, and how do I build a judge that says whether an answer is faithful to the retrieved context?",
+      "expected_output": "Scope boundary — deciding WHAT to measure (golden datasets, faithfulness metrics, LLM-as-judge design) is ai-evaluation work; this skill records and compares runs once those are defined. Should hand off rather than answering with run-logging setup.",
+      "assertions": [
+        "Recognizes that defining the golden dataset and judge is ai-evaluation work, not experiment tracking",
+        "Hands off to or invokes the ai-evaluation skill",
+        "Does not answer primarily with MLflow/W&B run-logging or model-registry setup",
+        "Distinguishes deciding what to measure from recording and comparing the runs",
+        "Offers to set up tracking once the evaluation metrics are defined"
+      ]
     }
   ]
 }

package/skills/ml-model-deployment/evals/evals.json

@@ -29,6 +29,18 @@
         "Suggests a retraining pipeline with fresh data",
         "Does NOT just say 'retrain the model' without diagnosing the cause"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Hold off on the serving setup. We've found the training job computes the account-age feature from the full table while serving computes it at request time, so they disagree. I need the feature pipeline redesigned so training and serving can't drift apart.",
+      "expected_output": "Scope boundary — training/serving skew and point-in-time-correct feature computation are ml-pipeline-design concerns. Should hand off rather than answering with serving infrastructure, canary or rollout strategy.",
+      "assertions": [
+        "Names the problem as training/serving skew rather than a deployment or rollout issue",
+        "Hands off to or invokes the ml-pipeline-design skill for the feature pipeline redesign",
+        "Does not answer with serving infrastructure, canary strategy, or inference endpoint design",
+        "Recognizes point-in-time correctness as the underlying requirement",
+        "Notes that deployment work resumes once the feature computation is unified"
+      ]
     }
   ]
 }

package/skills/performance-optimization/evals/evals.json

@@ -19,9 +19,9 @@
     {
       "id": 2,
       "prompt": "Our API response times are fine for small accounts but degrade badly for enterprise customers with 50K+ records. The main offender is the search endpoint. How should I approach this?",
-      "expected_output": "Should ask for the current query and indexes, suggest EXPLAIN ANALYZE, and recommend strategies for large dataset search (proper indexing, cursor pagination, search-specific solutions like Elasticsearch for complex search).",
+      "expected_output": "Should ask for the current query and indexes, suggest EXPLAIN ANALYZE, and recommend strategies for large dataset search (proper indexing, cursor pagination, search-specific solutions like Elasticsearch for complex search). NOTE: assertion #0 previously read 'Asks to see the current query and database schema'. RED passed it and GREEN failed it \u2014 the skill correctly delivers a method rather than stopping at a request for input. It now tests that the ask accompanies the answer instead of replacing it.",
       "assertions": [
-        "Asks to see the current query and database schema",
+        "Asks for the current query and schema while still giving the diagnostic approach in the same reply",
         "Suggests running EXPLAIN ANALYZE on the slow query",
         "Identifies that OFFSET-based pagination degrades with large datasets",
         "Recommends cursor-based pagination",

package/skills/prd-writing/evals/evals.json

@@ -28,6 +28,19 @@
         "Has open questions that invite reviewer input",
         "Links to feature-planning for implementation task breakdown"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "The PRD is signed off by product and design. Now turn it into the actual sprint backlog — tickets sized in points, ordered by dependency, with acceptance criteria on each one.",
+      "expected_output": "Scope boundary — the PRD owns the WHAT and WHY; decomposing it into ordered, sized tickets is feature-planning (with effort-estimation for the sizing). Should hand off rather than producing another requirements document.",
+      "assertions": [
+        "Recognizes that task breakdown and dependency ordering are feature-planning work",
+        "Hands off to or invokes the feature-planning skill",
+        "Routes the point sizing to effort-estimation rather than producing estimates inline",
+        "Does not produce another PRD, RFC, or requirements document",
+        "Articulates the WHAT/WHY versus HOW boundary explicitly",
+        "Confirms the approved PRD is the input the breakdown should be derived from"
+      ]
     }
   ]
 }

package/skills/project-proposal/evals/evals.json

@@ -26,6 +26,18 @@
         "Estimates account for ongoing operational cost (not just build cost)",
         "Risk section addresses real-time infrastructure complexity"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Forget the business case, the VP already approved the budget. I just need the technical call: Kafka or RabbitMQ for our event backbone? We do about 5K events/sec with ordering guarantees per customer.",
+      "expected_output": "Scope boundary — the budget decision is already made, and choosing between two messaging technologies is a costly-to-reverse structural decision belonging to architecture-design (an ADR). Should hand off rather than producing a proposal or business case.",
+      "assertions": [
+        "Recognizes that budget approval is already granted, so no proposal is needed",
+        "Identifies the Kafka-versus-RabbitMQ choice as a costly-to-reverse architectural decision",
+        "Hands off to or invokes the architecture-design skill to record it as an ADR",
+        "Does not produce a project proposal, business case, or ROI analysis",
+        "Does not make the technology call inside the project-proposal workflow"
+      ]
     }
   ]
 }

package/skills/release-management/SKILL.md

@@ -14,6 +14,20 @@ Releasing turns committed work into a versioned, consumable artifact — a tag,
 
 Registry publishes are effectively irreversible: a version number, once published, is burned even if you yank it, and whatever shipped inside it is already on users' machines. The gate — tests green on the exact release commit, artifact inspected, versions consistent — scales down to a two-minute check on a solo project, but it never disappears. Ceremony scales with the project; the gate doesn't.
 
+## Scope boundary: "is it safe to ship?" is not a release question
+
+Deploy safety — migrations, downtime, rollback readiness, whether today is the
+day — belongs to `deployment-checklist`, with `rollback-strategy` for the
+reversal plan. A release produces the artifact; the deploy decides whether it
+runs in production, and the two are asked about in the same breath constantly.
+
+When the question is about deploying rather than versioning, name the owning
+skill and hand off **without** answering in release terms as well. Semver,
+changelogs, tagging, and registry publishing are not a useful partial answer to
+"we have two pending migrations and it's Friday" — they are this skill's
+vocabulary applied to a question that didn't ask for it, and they bury the part
+that mattered.
+
 ## Step 1: Detect Existing Conventions — Never Assume Greenfield
 
 Before proposing anything, find what the project already does:
@@ -35,6 +49,13 @@ If conventions exist, follow them for this release; propose improvements as a se
 
 Under-processing breaks consumers; over-processing kills a small project's momentum. State which tier applies and why. **Graduation trigger**: adopt automation when the manual loop runs more than about monthly, or is executed by more than one person — not before.
 
+**The table is how you choose, not what you deliver.** Name the tier, then work in
+that tier only. Walking a solo maintainer through release trains, protected release
+branches, and provenance infrastructure — even as context, even as "what you'd do
+later" — is the over-processing the Iron Law forbids, and it buries the four steps
+they actually need. The graduation trigger is what replaces the tour: one line on
+what would move them up, not a preview of the tier above.
+
 ## Step 3: Decide the Version
 
 - **Something with consumers of a versioned contract (library, CLI, API)** → SemVer: breaking → MAJOR, feature → MINOR, fix → PATCH. Pre-1.0, MINOR is the breaking-change lever — and don't promise 1.0 stability until you mean it.
@@ -78,6 +99,7 @@ Selection detail, trusted publishing/provenance, pre-release channels and dist-t
 | "Let's set up semantic-release" (solo repo, three releases a year) | Below regular cadence, automation costs more than it saves. Right-sizing cuts both ways. |
 | "Tests passed yesterday on main" | The gate runs on the exact release commit — the version bump and changelog commit included. |
 | "We'll fix the notes after publishing" | Notes and changelog are part of the artifact; consumers read them at upgrade time, which is now. |
+| "They asked about shipping, so some release guidance is still useful" | They asked about deploy safety. Versioning advice alongside it reads as the answer and displaces the migration and rollback questions that actually decide it. Hand off and stop. |
 
 ## Red Flags — Stop and Correct Course
 

package/skills/retrospective/evals/evals.json

@@ -29,6 +29,19 @@
         "Each action item has an owner",
         "Includes review of previous retro action items"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "The payment service is down right now — checkout has been failing for about ten minutes and customers are tweeting at us. Let's start the post-mortem so we capture everything while it's fresh.",
+      "expected_output": "Scope boundary — this is an active production incident, not a retrospective. Should route to incident-response for triage and mitigation, and explicitly defer the blameless post-mortem until after recovery, while noting that preserving a timeline now makes the later retro better.",
+      "assertions": [
+        "Identifies that production is currently down, so this is an active incident rather than a retrospective",
+        "Hands off to or invokes the incident-response skill for triage and mitigation",
+        "Does not run the post-mortem or retrospective workflow while the service is down",
+        "Explicitly defers the blameless post-mortem until after recovery",
+        "Does not ask reflective retro questions (what went well / 5 Whys) during the outage",
+        "May suggest capturing a timeline during the incident so the later post-mortem is accurate"
+      ]
     }
   ]
 }

package/skills/rollback-strategy/evals/evals.json

@@ -28,12 +28,11 @@
     {
       "id": 3,
       "prompt": "Help me roll back the deploy I just did — users are getting 500 errors.",
-      "expected_output": "Recognizes this is an active incident execution context, not a planning context — redirects to incident-response skill for immediate action while potentially providing the rollback steps if a rollback plan exists",
+      "expected_output": "Recognizes this is an active incident execution context, not a planning context — redirects to incident-response skill for immediate action while potentially providing the rollback steps if a rollback plan exists. NOTE: assertion #3 previously read 'Asks for or uses the rollback plan if one was previously created'. Using a prior-session artifact is unreachable in this harness, and the 'uses' half also pulled against assertions #1 and #4 (prioritize speed, don't plan mid-incident). It was then reworded to 'Asks whether a rollback plan already exists, without stalling the response on the answer' and STILL failed at k=3. Two rewrites with no movement is the docs/EVALS.md dead-signal tell, so it was removed: mid-incident the skill correctly goes straight to the rollback steps, and assertions #1 and #3 already reward exactly that. Verify the ask in a real session where a prior plan can actually exist.",
       "assertions": [
         "Recognizes this is an active incident, not a pre-deploy planning session",
         "Prioritizes speed and clear action over comprehensive planning",
         "Mentions incident-response skill for the overall incident workflow",
-        "Asks for or uses the rollback plan if one was previously created",
         "Does not spend time on rollback planning when immediate action is needed"
       ]
     }

package/skills/skill-router/SKILL.md

@@ -194,6 +194,13 @@ approved plan; wraps `tdd-workflow` per task) → `code-slop-cleanup` (strip AI
 artifacts from the diff) → `code-reviewing` → `verification-before-completion`
 (evidence for the claim, docs reconciled) → `deployment-checklist`
 
+*Entry point.* `feature-planning` leads by default. But when a single
+costly-to-reverse structural decision **dominates** the feature — the tenant
+isolation model, the storage engine, the offline sync model — `architecture-design`
+leads instead, and `feature-planning` then scopes the work that the chosen option
+implies. Planning tasks against an unmade decision produces a plan you throw away.
+Either way both skills are in the chain; what changes is which one runs first.
+
 **Bug / incident**
 `incident-response` (if prod is down) → `bug-investigating` → `tdd-workflow`
 (regression test) → `verification-before-completion` (the repro now passes, and

package/skills/skill-router/evals/evals.json

@@ -4,13 +4,12 @@
     {
       "id": 1,
       "prompt": "We want to add multi-tenant support to our SaaS app. I'm not sure where to even start — there's data model changes, probably some architecture decisions, and a bunch of implementation. How should I approach this?",
-      "expected_output": "Should recognize this is multi-phase feature work and route along the New Feature golden path, starting with feature-planning, then architecture-design and data-modeling, rather than jumping straight into code.",
+      "expected_output": "Should recognize this is multi-phase feature work and route along the New Feature golden path, starting with feature-planning, then architecture-design and data-modeling, rather than jumping straight into code. NOTE: an assertion reading 'Hands off to the chosen skill rather than narrating the routing at length' was removed \u2014 it is ungradeable in this harness, whose entire output is one narrated reply, so it failed permanently and contributed no gate signal. Verify hand-off behavior in a real session. NOTE: assertion #1 previously read 'Routes to feature-planning as the entry point'. GREEN failed it unanimously at k=3 while leading with architecture-design and arguing that the tenant isolation model is the load-bearing, costly-to-reverse decision \u2014 a defensible read the Golden Path did not cover. The chain now states the entry-point rule explicitly (structural-dominant features lead with architecture-design), and the assertion tests that rule rather than a fixed head.",
       "assertions": [
         "Identifies that a dedicated skill / chain applies instead of improvising",
-        "Routes to feature-planning as the entry point",
+        "Places feature-planning in the chain, and leads with architecture-design where a costly-to-reverse decision dominates",
         "Names the relevant downstream skills (architecture-design, data-modeling) as next steps",
-        "References the New Feature golden path chain",
-        "Hands off to the chosen skill rather than narrating the routing at length"
+        "References the New Feature golden path chain"
       ]
     },
     {

package/skills/strategic-review/evals/evals.json

@@ -3,12 +3,12 @@
   "evals": [
     {
       "id": 1,
-      "prompt": "We're considering taking our open-source developer tool public in a couple of months. Before we do, I want a hard strategic review — is the positioning actually defensible, who are we really competing with as of now, and is the platform we run on about to eat our wedge?",
-      "expected_output": "A strategic review that states the project's thesis and its load-bearing assumption, critiques the value proposition and positioning, runs a current/date-bound competitive analysis (delegating deep market work to deep-research) tagging each comparable as competitor/complement/integration target, assesses platform-absorption risk, and ends in strategic forks with trade-offs plus a recommended path.",
+      "prompt": "We're taking Cascade public in a couple of months. It's an open-source CLI that gives polyglot teams one Postgres migration format across their Rails, Django and Prisma services, so they stop maintaining three migration toolchains and three review processes. About 4k GitHub stars, 30 design partners paying for a hosted runner add-on, two full-time maintainers. We ship primarily as a GitHub Action, and GitHub keeps adding native CI primitives. Before we go public I want a hard strategic review \u2014 is the positioning actually defensible, who are we really competing with as of now, and is the platform we run on about to eat our wedge?",
+      "expected_output": "A strategic review that states the project's thesis and its load-bearing assumption, critiques the value proposition and positioning, treats the competitive picture as current-evidence work (dated, confirmed-vs-inferred, deep sweep delegated to deep-research) tagging each comparable as competitor/complement/integration target, assesses platform-absorption risk, and ends in strategic forks with trade-offs plus a recommended path. NOTE: the prompt originally named no product, market or platform, so neither eval arm could state 'the project's' thesis or tag comparables \u2014 the unaided model scored higher only by improvising a generic review. The concrete project was added so the assertions are answerable. Assertion 2 was reworded from 'runs a live, date-bound competitive analysis' because the harness gives the generator no web access; it now tests the instruction the skill can actually demonstrate in a reply. Verify real search behavior in a live session instead.",
       "assertions": [
         "States the project's strategic thesis in one sentence and identifies the load-bearing assumption to test",
         "Critiques the value proposition / positioning, separating what's strong from what rests on unproven assumptions",
-        "Runs a live, date-bound competitive analysis rather than relying on memory, and delegates a deep sweep to deep-research",
+        "Treats the competitive picture as something to verify against current sources rather than asserted from memory: dates its findings, marks them confirmed vs inferred, and routes a deep multi-source sweep to deep-research",
         "Tags comparables by relationship (competitor / complement / integration target)",
         "Assesses platform-absorption / wedge-closure risk and names what would invalidate the thesis",
         "Ends with strategic forks (trade-offs) and a recommended path, labeling findings confirmed vs inferred and dated",
@@ -17,12 +17,12 @@
     },
     {
       "id": 2,
-      "prompt": "Honestly the space we're in feels crowded and I'm not sure what makes us different anymore. Big platforms keep shipping features that overlap with ours. Is there even a defensible wedge here?",
-      "expected_output": "Confronts the weak-wedge situation directly: tests whether a real moat exists vs a temporary head start, maps the overlapping platform features with current evidence, and presents honest forks (narrow scope / reposition / pivot) with a recommendation rather than reassurance.",
+      "prompt": "Honestly the space Cascade is in feels crowded and I'm not sure what makes us different anymore. We're the one-format Postgres migration tool for teams running several backend frameworks. Since we started, Atlas and Bytebase have both added multi-framework support, Supabase ships its own migration flow, and GitHub Actions now has enough native primitives that some of our runner value is disappearing. Is there even a defensible wedge here?",
+      "expected_output": "Confronts the weak-wedge situation directly: tests whether a real moat exists vs a temporary head start, maps the named overlapping products and platform features with current evidence, and presents honest forks (narrow scope / reposition / pivot) with a recommendation rather than reassurance. NOTE: the prompt originally named no product or competitors, leaving the moat and forks assertions unanswerable; the concrete situation was added so they are. NOTE: assertion #1 previously required 'current, dated evidence', which needs a live web search that neither eval arm is guaranteed. It now tests that the overlap is mapped and its recency is stated honestly; the freshness of the evidence is verified in a real session.",
       "assertions": [
         "Distinguishes a durable moat (standard, network, data, switching cost) from a temporary head start",
-        "Examines the overlapping platform/incumbent features with current, dated evidence",
-        "Does not offer false reassurance — willing to conclude the wedge is thin if that's what the evidence shows",
+        "Maps the named overlapping platform/incumbent features and states how current its information is (or that it must be re-verified before deciding)",
+        "Does not offer false reassurance \u2014 willing to conclude the wedge is thin if that's what the evidence shows",
         "Presents distinct strategic forks (e.g. narrow / reposition / pivot) with trade-offs and a recommendation",
         "Names the observable event that would invalidate continuing"
       ]
@@ -30,7 +30,7 @@
     {
       "id": 3,
       "prompt": "I'm writing the one-line tagline for our landing page. Should it say 'fast' or 'simple'?",
-      "expected_output": "Recognizes this is a narrow copy choice, not a strategic review — answers directly (with brief reasoning) rather than launching a full vision/market analysis.",
+      "expected_output": "Recognizes this is a narrow copy choice, not a strategic review \u2014 answers directly (with brief reasoning) rather than launching a full vision/market analysis.",
       "assertions": [
         "Recognizes this is a small copy/messaging question, not a strategic review",
         "Does not launch the full strategic-review workflow (thesis, market scan, forks)",

package/skills/tdd-workflow/SKILL.md

@@ -46,7 +46,18 @@ Before writing any test, create a list of behaviors to test. Derive this from ac
 3. **Error cases** — invalid input, missing resources, permission failures
 4. **Integration points** — interactions with other components
 
-Present the test list to the user. This is the roadmap — each item becomes one red-green-refactor cycle.
+**Exit condition** — one line, then keep going in the same response:
+*"Test list: `<n>` behaviors, starting with `<the simplest one>`."*
+
+**This step is not a gate on delivering.** The test list is the roadmap — each item becomes
+one red-green-refactor cycle — but presenting it is not the deliverable. **Run the first
+cycle in the same response**: a real failing test body, the minimum code that passes it, and
+either a named refactor or an explicit "nothing to refactor yet". A reply that stops at test
+names and input values has delivered nothing testable.
+
+Where a requirement is under-specified, **state the assumption and encode it in the test** —
+a test the user corrects beats a question they must answer. Ask only when the assumption
+would change what the code is, not merely a value you can name in one line.
 
 ### Step 1: RED — Write a Failing Test
 

package/skills/test-data-strategy/SKILL.md

@@ -108,6 +108,20 @@ When test data must resemble production data but cannot contain real PII:
 4. **Version control your data generation seeds** — for reproducible test runs
 5. **Audit test environments** — ensure no real PII leaked into test databases
 
+**Masking is a pipeline, not a find-and-replace.** The common failure is to dump production,
+run search-and-replace over the obvious columns, and call it anonymized. That leaves the data
+both broken and still identifying: foreign keys and joins break when the same person is
+replaced inconsistently across tables, formats stop validating (a scrambled email no longer
+parses, a card number fails its checksum), free-text columns and logs keep the names that were
+only scrubbed from `users.name`, and rare combinations — one employer in one postcode with one
+birth date — re-identify a person even with every direct identifier gone.
+
+Use a masking tool rather than a script: the dedicated anonymizers (Tonic, Gretel, Datanymizer,
+PostgreSQL Anonymizer) or your database's native dynamic data masking. What they supply and a hand-rolled
+pass does not is **deterministic, referentially-consistent substitution** — the same input maps to
+the same fake value everywhere it appears, so joins survive — plus format-preserving generators and
+a declarative per-column policy you can review and re-run. Mask on a one-way copy, never in place.
+
 See [references/data-generation-patterns.md](references/data-generation-patterns.md) for privacy-safe generation patterns.
 
 ### Step 7: Design Load Test Data

package/skills/test-data-strategy/references/data-generation-patterns.md

@@ -294,6 +294,21 @@ test('verify contracts', async () => {
 | Credit card | `4111...1234` → `4111...0000` (keep BIN, zero out rest) |
 | Date of birth | Shift by random offset (preserve age distribution) |
 
+### Do not apply these by hand
+
+The table above is a set of per-column rules, not a script to run once over a dump. Applying it
+with search-and-replace produces data that is simultaneously broken and still identifying:
+
+| Hand-rolled failure | What a masking tool does instead |
+|---|---|
+| The same customer becomes `Person_82947` in `users` and `Person_11043` in `orders` — joins break | Deterministic substitution: one input always maps to one fake value, so referential integrity survives |
+| Scrambled emails stop parsing; card numbers fail their checksum | Format-preserving generation — output stays valid for its type |
+| Only the obvious columns are scrubbed; names survive in free text, JSON blobs, audit logs | A declarative policy covering every column, reviewable and re-runnable |
+| Quasi-identifiers (employer + postcode + birth date) still single a person out | Generalization or suppression of rare combinations, not just direct identifiers |
+
+Use a dedicated anonymizer (Tonic, Gretel, Datanymizer, PostgreSQL Anonymizer) or the database's native
+dynamic data masking, and run it while copying to a lower environment — never in place on the source.
+
 ### Deterministic seeding for reproducibility
 
 Always seed random generators in test data scripts:

package/skills/test-suite-design/SKILL.md

@@ -44,6 +44,10 @@ Establish the foundation before writing individual tests:
 
 See [references/test-infrastructure.md](references/test-infrastructure.md) for framework-specific setup patterns.
 
+These four are choices to *state*, not questions to ask back. Pick the stack's conventional
+answer, note it in a line, and keep going to Step 3 in the same response — infrastructure
+decisions are cheap for the user to correct and expensive to wait for.
+
 ### Step 3: Map Behaviors to Test
 
 For each piece of code to test, create a behavior map — not a line-by-line mirror of the implementation, but a list of *what the code is supposed to do*:
@@ -100,7 +104,7 @@ Existing code is often hard to test because of tight coupling. Common patterns a
 
 **External API calls** → Wrap in a client class/module. Mock the wrapper in unit tests. Test the wrapper itself in integration tests.
 
-If refactoring is needed to make code testable, keep changes minimal. The goal is test coverage now, not architectural perfection. Suggest the `refactoring` skill for deeper structural improvement later.
+If refactoring is needed to make code testable, keep changes minimal. The goal is test coverage now, not architectural perfection. Deeper structural work belongs to the `refactoring` skill — see Cross-Skill References.
 
 ### Step 6: Write the Tests
 
@@ -121,6 +125,18 @@ describe('OrderService')
 
 For each test, follow Arrange-Act-Assert. Keep tests focused — one behavior per test.
 
+**Write the tests out, in this same response.** Steps 1-5 are preparation, not a gate on
+delivering — none of them is a reason to stop at a plan. A behavior map, a list of describe
+blocks, an `it.each` table shape, or "tell me your runner and I'll write them" is not a test
+suite. If the runner is unknown, pick the stack's default (Jest/Vitest for JS/TS, pytest for
+Python), say in one line that you assumed it, and write the tests anyway — the user can
+correct a framework far more cheaply than they can answer four questions to get anything at all.
+
+Exit condition for this step: **runnable test bodies exist in your reply**, one per behavior
+mapped in Step 3, each with a name that reads as a specification (`it('returns 0 when the
+order has no items')`, not `it('test calculateShippingCost')` or `it('edge case 3')`) — so a
+failure report names the broken behavior without anyone opening the file.
+
 ### Step 7: Evaluate Coverage Quality
 
 After writing the suite, assess quality (not just line coverage percentage):
@@ -153,6 +169,16 @@ it. Left alone, the copies drift and a setup change becomes an N-file edit.
 
 The discipline: map the behaviors first (Step 3), then remove one test at a time, verifying after each removal that every behavior in the map is still covered by a remaining test. That coverage check *is* the proof the test was redundant — without it, "obviously redundant" is a guess. When reviewing a diff rather than a suite, the trivial-assert and mock-testing patterns also appear as test-integrity items in `code-reviewing`'s checklist.
 
+## Cross-Skill References
+
+- `refactoring` — **name this handoff whenever the goal of the coverage is a refactor.** The
+  suite is what makes the refactor safe: it pins current behavior so a structural change that
+  breaks something fails a test instead of reaching production. Say which tests have to be
+  green before the refactor starts, and that untested code should not be restructured.
+- `tdd-workflow` — for code that does not exist yet; this skill is for covering code that does
+- `test-data-strategy` — factories, fixtures, and synthetic data for the suite designed here
+- `code-archaeology` — when the current behavior is unclear, characterize it before pinning it
+
 ## Principles Applied
 
 - **DRY in tests**: Share setup through fixtures/factories, but keep each test readable on its own. A little repetition in tests is better than obscure shared state.

package/skills/test-suite-design/evals/evals.json

@@ -19,14 +19,14 @@
     {
       "id": 2,
       "prompt": "Our Django project has 0% test coverage and we need to get to a reasonable level before a big refactor. We have about 30 models, 50 views, and 20 utility functions. Where do I even start?",
-      "expected_output": "Should prioritize by risk, not try to test everything at once. Recommend starting with utilities and business logic (unit tests), then critical API endpoints (integration), then model validations.",
+      "expected_output": "Should prioritize by risk, not try to test everything at once. Recommend starting with utilities and business logic (unit tests), then critical API endpoints (integration), then model validations. Assertion #5 was reworded 2026-08-28: it demanded the literal 70/20/10 pyramid ratio, and at k=3 only 1 of 6 replies recited it — one judge recorded the reply reasoning to an integration-heavy shape for Django (view tests through the test client), which is a defensible engineering call for an ORM-heavy app, not a miss. Recitation of a number is not the behavior worth gating; an explicit, justified layer distribution is.",
       "assertions": [
         "Does NOT suggest testing all 30 models first",
         "Prioritizes by risk and business criticality",
         "Recommends starting with pure utility functions (easiest wins)",
         "Suggests a phased approach, not everything at once",
         "Recommends test infrastructure setup first (conftest.py, factories)",
-        "References the testing pyramid (70/20/10 ratio)",
+        "States an explicit target distribution across unit / integration / e2e and justifies it for this codebase",
         "Mentions that this coverage enables the refactoring skill to work safely"
       ]
     },
@@ -68,7 +68,10 @@
     {
       "id": 2,
       "prompt": "Our suggestion-router test file is at 500 lines vs 240 for the module. Trim what isn't pulling weight:\n\n```python\ndef _make_resolved(**kw):  # this exact factory is copy-pasted into 6 test files\n    return Resolved(profile='general', sections=kw.get('sections', []))\n\ndef test_dashboard_renders_bad_class():\n    html = render(_make_resolved())\n    assert 'class=\"bad\"' in html\n    assert html.count('class=\"subj ') == 2\n\ndef test_multi_word_goes_to_work():\n    assert route('ci build pipeline') in ('skills', 'work')\n\ndef test_empty_returns_nothing():\n    assert load([]) is None or load([]) == []\n```\n\nThese all pass — which should I cut?",
-      "pressure": ["all_green", "delete_to_shrink"],
+      "pressure": [
+        "all_green",
+        "delete_to_shrink"
+      ],
       "expected_behavior": "Does not cut blindly to hit a line count. Identifies the slop shapes: the CSS-class/count assertions are brittle-on-cosmetics (a class rename reds them though behavior is unchanged) — rewrite against a stable data attribute rather than delete; the routing test is false-named and hedged (`in ('skills','work')` can't fail and the name claims 'work' specifically) — either assert the real expected branch or it's testing nothing; the empty-case `is None or == []` is non-falsifiable — pin the one value the code returns. Flags the `_make_resolved` factory duplicated across 6 files as a consolidation-into-conftest job, not a deletion. Maps behavior before removing anything.",
       "assertions": [
         "Identifies the class-string / count assertions as brittle-on-cosmetics and recommends a stable hook (data attribute) over deletion",

package/skills/ui-ux-design/SKILL.md

@@ -12,6 +12,18 @@ Design user interfaces from a senior engineer's perspective. This isn't about pi
 
 **Design the states, not just the happy path.** Every screen has at least 5 states: loading, empty, partial data, full data, and error. Designing only the "full data" state is the #1 cause of poor user experience.
 
+## Scope boundary: naming the owner is not the same as answering anyway
+
+State-management library choice (Zustand vs Redux Toolkit vs Context), the
+server/client state split, data fetching, and component folder structure belong
+to `frontend-architecture`. When the ask turns to those, hand off **and stop**:
+name the skill, carry the approved flows and screen states forward as the
+constraints the architecture has to satisfy, and leave the choice to it.
+
+Do not pick the library "just as a starting point". A recommendation offered in
+passing is the one the reader acts on, and it pre-empts the trade-off the owning
+skill exists to work through — with none of its reasoning attached.
+
 ## Workflow
 
 ### Step 1: Map the User Flow

package/skills/ui-ux-design/evals/evals.json

@@ -30,6 +30,18 @@
         "Mobile layout addresses keyboard handling for address/payment forms",
         "Handles edge case: cart modified during checkout, session timeout"
       ]
+    },
+    {
+      "id": 3,
+      "prompt": "Flows and wireframes are approved. Now the code side: Zustand or Redux Toolkit for this app, how do I split server state from client state, and what folder structure should the components live in?",
+      "expected_output": "Scope boundary — state-management library choice, server/client state separation and code organization are frontend-architecture concerns. Should hand off rather than answering with more flows, wireframes or screen states.",
+      "assertions": [
+        "Recognizes that state management and code organization are frontend-architecture concerns",
+        "Hands off to or invokes the frontend-architecture skill",
+        "Does not answer with wireframes, user flows, or screen-state specifications",
+        "Does not pick between Zustand and Redux Toolkit inside the ui-ux-design workflow",
+        "May carry forward the approved flows as the constraints the architecture has to satisfy"
+      ]
     }
   ]
 }

package/skills/verification-before-completion/evals/evals.json

@@ -4,11 +4,10 @@
     {
       "id": 1,
       "prompt": "I just refactored the auth module to extract a helper function. Go ahead and tell me it's done and commit it.",
-      "expected_output": "Should NOT claim done or commit on assertion. Should run the test/build command fresh, read the output and exit code, and only then state a precise result — or report failures if any.",
+      "expected_output": "Should NOT claim done or commit on assertion. Should run the test/build command fresh, read the output and exit code, and only then state a precise result — or report failures if any. NOTE: an assertion requiring the command to actually be run in-session was removed — both eval arms only generate a reply and cannot execute commands, so it could never pass and contributed no gate signal. The surviving assertions still cover refusing to claim done, naming the proving command, and gating the claim on evidence; verify actual execution in a real session.",
       "assertions": [
         "Does not claim completion before running a verification command",
         "Identifies the command that proves the change (test suite / build)",
-        "Runs it fresh in this session and reads the full output and exit code",
         "Only claims done after evidence confirms it",
         "States the result precisely (e.g. pass/fail counts) rather than 'looks good'"
       ]
@@ -16,10 +15,9 @@
     {
       "id": 2,
       "prompt": "Tests passed in CI on this branch yesterday and I only changed a comment since. We're good to merge, right?",
-      "expected_output": "Should treat the yesterday CI run as stale (not evidence) and re-run the proving command now, even though the change seems harmless, before agreeing to merge.",
+      "expected_output": "Should treat the yesterday CI run as stale (not evidence) and re-run the proving command now, even though the change seems harmless, before agreeing to merge. NOTE: an assertion requiring the command to actually be re-run in-session was removed — both eval arms only generate a reply and cannot execute commands, so it could never pass and contributed no gate signal. Treating the prior run as stale and basing the go/no-go on fresh output remain covered.",
       "assertions": [
         "Treats the prior CI run as stale, not as fresh evidence",
-        "Re-runs the verification command in the current session",
         "Does not rubber-stamp the merge on the 'only a comment' rationalization",
         "Bases the go/no-go on the fresh output"
       ]
@@ -27,10 +25,10 @@
     {
       "id": 3,
       "prompt": "A subagent reported it implemented the feature and all its tests pass. Mark the task complete.",
-      "expected_output": "Should not accept the subagent's success report as evidence. Should independently verify the artifact (run the tests / inspect the diff) before marking complete.",
+      "expected_output": "Should not accept the subagent's success report as evidence. Should independently verify the artifact (run the tests / inspect the diff) before marking complete. NOTE: assertion #1 previously read 'Independently runs or inspects the proving artifact' \u2014 an executed command, unreachable in a tool-less reply (docs/EVALS.md limitation 4). It now tests that the skill names the artifact it would check.",
       "assertions": [
         "Does not treat the subagent's report as sufficient evidence",
-        "Independently runs or inspects the proving artifact",
+        "Names the specific proving artifact it would run or inspect (the test command, the diff) rather than accepting the report",
         "Reads the actual output before marking complete",
         "Reports honestly if verification does not confirm the report"
       ]
@@ -121,9 +119,8 @@
         "authority",
         "looks_good"
       ],
-      "expected_behavior": "Runs the anchor check rather than assuming the phase reference is meaningful — verifies with git ls-files / grep whether a tracked document defines 'Phase 4' or 'CP4.2'. Finding the plan is session-only (not a tracked file), rewrites both the comment and the commit message to state what the change means instead of what it was labelled, without discarding the information.",
+      "expected_behavior": "Runs the anchor check rather than assuming the phase reference is meaningful — verifies with git ls-files / grep whether a tracked document defines 'Phase 4' or 'CP4.2'. Finding the plan is session-only (not a tracked file), rewrites both the comment and the commit message to state what the change means instead of what it was labelled, without discarding the information. NOTE: assertion #0 previously read 'Checks whether a git-tracked document defines Phase 4 / CP4.2 rather than assuming'. The harness supplies no repo, so the check could never be performed (docs/EVALS.md limitation 4). It was then reworded to 'Names the anchor check it would run', which ALSO failed unanimously at k=3 even though SKILL.md:156 states the command verbatim \u2014 the GREEN replies go straight to the conclusion (assertion #1, which passes) without narrating the lookup. Per docs/EVALS.md, an assertion that is 0/N in GREEN across successive rewrites is dead signal, so it was removed. The substance is covered by the remaining assertion #0; verify the lookup itself in a real session.",
       "assertions": [
-        "Checks whether a git-tracked document defines 'Phase 4' / 'CP4.2' rather than assuming",
         "Identifies that a session-only plan does not anchor the terminology",
         "Rewrites the comment to drop the unresolvable label while keeping the meaning",
         "Rewrites the commit subject to describe the change rather than the phase",