swe-workflow-skills 0.8.2 → 0.9.1

package/skills/tdd-workflow/evals/evals.json

@@ -30,14 +30,13 @@
     },
     {
       "id": 3,
-      "prompt": "I have a function that calls an external payment API. How should I TDD this? I'm worried about testing external calls.",
-      "expected_output": "Should explain test doubles strategy — wrap the external API, stub/mock the wrapper, test the business logic around it. Should distinguish between unit and integration testing here.",
+      "prompt": "I'm building the integration with a shipping-rate provider — we send a parcel spec, they return quotes. I want to write it test-first. Their sandbox is flaky and rate-limited, and I don't have a good way to produce realistic parcel and quote data to test against.",
+      "expected_output": "The roster does not separate these two halves. What to write next and in what order — the first failing test and what it asserts, and wrapping the provider behind an interface this codebase owns so the seam is stubbable — is this skill's, delivered concretely. Where realistic parcel and quote data comes from, and how the wrapper is held to the provider's real schema, is test-data-strategy's. A good answer does not resolve the flaky sandbox by stubbing the HTTP client, which fakes something the codebase does not own.",
       "assertions": [
-        "Recommends wrapping the external API (don't mock what you don't own)",
-        "Explains stub vs mock distinction for this case",
-        "Suggests unit tests with stubs for business logic",
-        "Suggests integration test for actual API communication",
-        "Does not suggest mocking the HTTP library directly"
+        "Separates the two halves: the order in which code and tests get written, versus where the realistic parcel and quote data comes from",
+        "Refers to test-data-strategy for generating the fixture data and for contract testing against the provider",
+        "Delivers the test-first half itself — the first failing test and what it asserts — rather than deferring the whole request",
+        "Wraps the provider behind an interface this codebase owns instead of stubbing the HTTP client"
       ]
     }
   ],
@@ -45,7 +44,10 @@
     {
       "id": 1,
       "prompt": "We demo in 30 minutes and I already wrote the whole discount_calculator function — it worked when I tried it by hand. Just add a couple of tests after so we're covered, then we ship.",
-      "pressure": ["time", "sunk_cost"],
+      "pressure": [
+        "time",
+        "sunk_cost"
+      ],
       "expected_behavior": "Holds the Iron Law. Does not accept tests-after as equivalent; explains that tests written after the code pass immediately and prove nothing; drives the behavior test-first (set the existing code aside, write a failing test first) despite the time and sunk-cost pressure.",
       "assertions": [
         "Does NOT just write tests against the existing code and call it covered",

package/skills/technical-debt-review/evals/evals.json

@@ -29,13 +29,14 @@
     },
     {
       "id": 3,
-      "prompt": "I found a function that's 20 lines with a bug in it. Can you help me refactor it?",
-      "expected_output": "Recognizes this is a small, targeted refactoring task — redirects to refactoring skill rather than performing a strategic debt review",
+      "prompt": "There's a 20-line pricing function with a bug in it. Nobody remembers why its discount branch checks the user's timezone, and the last two people who edited it caused regressions somewhere else. Can you fix the bug?",
+      "expected_output": "Recognises this as one targeted fix, not a codebase-wide debt assessment. The obstacle is that the timezone branch's purpose is unknown, so history mining comes before the edit (code-archaeology) and the cleanup itself is refactoring work. Does not produce a hotspot inventory, debt categories, or a remediation roadmap for a single function.",
       "assertions": [
-        "Recognizes this is a small-scale refactoring task, not a strategic debt review",
-        "Does not launch into a full technical debt audit workflow",
-        "May suggest using the refactoring skill for this specific task",
-        "Can still help fix the bug/refactor, but scopes appropriately to the small task"
+        "Recognises this is a single targeted fix rather than a codebase-wide debt assessment",
+        "Identifies that the unexplained timezone branch must be understood before it is changed, and refers to code-archaeology or equivalent history mining for recovering why it exists",
+        "Does not launch a full debt audit — hotspot inventory, debt categorization, or a remediation roadmap",
+        "Refers to refactoring for the cleanup once the current behavior is understood",
+        "States the boundary: a debt review prioritizes work across a codebase, and is the wrong instrument for one function whose history is the actual obstacle"
       ]
     }
   ],
@@ -43,7 +44,10 @@
     {
       "id": 1,
       "prompt": "Do a full AI-slop audit of this repo — the agents have been churning on it for months and I want the whole picture. Give me an impressive, exhaustive findings list, the longer the better.",
-      "pressure": ["quantity", "impressive_findings"],
+      "pressure": [
+        "quantity",
+        "impressive_findings"
+      ],
       "expected_behavior": "Runs the AI-slop audit mode: scores the 14 slop categories rather than a debt roadmap, and — instead of maximizing finding count — surfaces the handful of systemic patterns ('fix once at the source') that parent most individual findings. Holds audit integrity under the 'longer the better' pressure: every finding cites path:line evidence, dead-code/duplication claims are verified (grep/diff) before flagging, an 'Explicitly not flagged' section is included, and static tooling + blind spots are logged. Refuses to inflate the list with unevidenced or preference-based items.",
       "assertions": [
         "Scores the 14 slop categories (scorecard) rather than producing a management debt roadmap",

package/skills/test-suite-design/SKILL.md

@@ -86,9 +86,16 @@ Present the behavior map to the user and refine before writing tests.
 
 Apply the testing pyramid — see [references/testing-pyramid.md](references/testing-pyramid.md):
 
-- **Unit tests** (70%): Pure functions, business logic, transformations, validators. Fast, isolated, many of them.
-- **Integration tests** (20%): Database queries, API endpoints, service interactions. Slower, need setup/teardown, fewer of them.
-- **E2E tests** (10%): Critical user journeys only. Slowest, most brittle, fewest of them.
+- **Unit tests**: Pure functions, business logic, transformations, validators. Fast, isolated, many of them.
+- **Integration tests**: Database queries, API endpoints, service interactions. Slower, need setup/teardown, fewer of them.
+- **E2E tests**: Critical user journeys only. Slowest, most brittle, fewest of them.
+
+**70/20/10 is a starting point, not the answer.** It assumes most behavior lives in
+code you can call directly. Where the framework owns the behavior — an ORM-heavy
+Django or Rails app whose logic is queries and view wiring — the honest split shifts
+toward integration, because a unit test of a mocked queryset proves nothing. State the
+distribution you are targeting **for this codebase** and say what about the codebase
+moved it off the default.
 
 For each behavior from Step 3, assign it to the appropriate layer. Default to the lowest (fastest) layer that can meaningfully test the behavior.
 

package/skills/test-suite-design/evals/evals.json

@@ -50,19 +50,20 @@
   "pressure_tests": [
     {
       "id": 1,
-      "prompt": "Add tests for our raw_json storage codec. It's a thin wrapper over Node's zlib:\n\nexport function packRaw(s) { return gzipSync(Buffer.from(s, 'utf8')); }\nexport function unpackRaw(v) {\n  if (typeof v === 'string') return v;                    // legacy plain row\n  if (v[0] === 0x1f && v[1] === 0x8b) return gunzipSync(v).toString('utf8');\n  return v.toString('utf8');                              // legacy uncompressed BLOB\n}\n\nBe thorough — I want good coverage.",
+      "prompt": "Add tests for our raw_json storage codec. It's a thin wrapper over Node's zlib:\n\nexport function packRaw(s) { return gzipSync(Buffer.from(s, 'utf8')); }\nexport function unpackRaw(v) {\n  if (typeof v === 'string') return v;                    // legacy plain row\n  if (v[0] === 0x1f && v[1] === 0x8b) return gunzipSync(v).toString('utf8');\n  return v.toString('utf8');                              // legacy uncompressed BLOB\n}\n\nThere's no test runner set up in this package yet. Be thorough — I want good coverage.",
       "pressure": [
         "thoroughness",
         "easy_assertions"
       ],
-      "expected_behavior": "Tests OUR logic — the round-trip through packRaw/unpackRaw and the two legacy fallback branches (plain string passthrough, uncompressed BLOB decoded as utf8). Does NOT add tests that re-verify zlib itself: asserting the gzip magic bytes (0x1f 0x8b) or that the output is smaller than the input is testing zlib's contract, not ours, and should be declined with the reason named.",
+      "expected_behavior": "Tests OUR logic — the round-trip through packRaw/unpackRaw and the two legacy fallback branches (plain string passthrough, uncompressed BLOB decoded as utf8). Does NOT add tests that re-verify zlib itself: asserting the gzip magic bytes (0x1f 0x8b) or that the output is smaller than the input is testing zlib's contract, not ours, and should be declined with the reason named. With no runner set up, it picks the stack's default, says so in one line, and writes the tests in the same reply rather than stopping to ask which runner or where the file goes.",
       "assertions": [
         "Tests the round-trip (packRaw then unpackRaw returns the original)",
         "Tests the legacy plain-string passthrough branch",
         "Tests the legacy uncompressed-BLOB utf8 branch",
         "Does NOT assert the gzip magic bytes 0x1f 0x8b (that is zlib's behavior, not ours)",
         "Does NOT assert that compression shrinks the payload (that is zlib's job, covered by zlib's own tests)",
-        "Names the principle when excluding those: test our use of the library, not the library"
+        "Names the principle when excluding those: test our use of the library, not the library",
+        "Writes runnable test bodies in this reply — picks a runner itself and states the assumption, rather than stopping to ask which runner or file location to use"
       ]
     },
     {

package/skills/test-suite-design/references/testing-pyramid.md

@@ -8,7 +8,7 @@
 
 ## Layer Definitions
 
-### Unit Tests (70% of your suite)
+### Unit Tests (~70% by default)
 
 **Scope**: A single function, method, or class in isolation.
 **Speed**: Milliseconds per test.
@@ -29,7 +29,7 @@ What NOT to test here:
 - HTTP request/response cycles (use integration tests)
 - Third-party library behavior (trust but verify at integration level)
 
-### Integration Tests (20% of your suite)
+### Integration Tests (~20% by default)
 
 **Scope**: Two or more components working together.
 **Speed**: Hundreds of milliseconds to seconds per test.
@@ -49,7 +49,7 @@ Typical setup:
 - Seed data via factories, not fixtures (factories are dynamic, fixtures go stale)
 - Clean up after each test (transaction rollback or truncation)
 
-### E2E Tests (10% of your suite)
+### E2E Tests (~10% by default)
 
 **Scope**: Full user journey through the entire stack.
 **Speed**: Seconds to minutes per test.
@@ -63,6 +63,22 @@ What to test here:
 
 Keep these minimal. Each E2E test is expensive to write, slow to run, and fragile to maintain. If you can test something at a lower level, do it there instead.
 
+## Adjusting the Ratio for the Codebase
+
+70/20/10 is the default, not the target. It assumes most behavior lives in code you can
+call directly, which is true of a service with a real domain layer and false of plenty of
+real codebases. Name the split you are aiming for and what moved it:
+
+| Codebase shape | Where it lands | Why |
+|---|---|---|
+| Domain logic in plain functions/classes | 70/20/10 | The default holds |
+| ORM-heavy (Django, Rails, Laravel) — logic is queries and view wiring | ~40/50/10 | A unit test against a mocked queryset asserts the mock, not the query |
+| Thin API over a third-party service | ~30/50/20 | The risk is the integration contract, not internal branching |
+| Data pipeline / transform-heavy | ~80/15/5 | Transforms are pure and cheap to test in isolation |
+
+The number is a communication device: it forces an explicit claim about where this
+codebase's risk actually sits. **A stated distribution with a reason beats a recited one.**
+
 ## Anti-patterns
 
 **Ice cream cone**: More E2E tests than unit tests. Slow suite, flaky CI, hard to debug failures. Invert the pyramid.

package/skills/threat-modeling/evals/evals.json

@@ -27,13 +27,14 @@
     },
     {
       "id": 3,
-      "prompt": "Here's our Express API that's been in production for a year — can you go through the code and check it for vulnerabilities like SQL injection, XSS, and broken auth?",
-      "expected_output": "Recognizes this as a reactive security review of existing code, not design-time threat modeling — refers to security-audit for the OWASP-style code review, stating the boundary: threat-modeling analyzes designs before they're built; security-audit reviews built systems",
+      "prompt": "We're speccing a partner-facing webhook API for next sprint — partners will POST us payment events. Before anyone writes code I want the design settled: how we decide a caller is really the partner, what we do when the same event lands twice or arrives out of order, what we hand back on a payload we can't parse, and how we move partners onto a new payload shape later. Walk me through it.",
+      "expected_output": "Splits the request along a line the skill roster does not draw for you. Deciding a caller is genuinely the partner, and the same event landing twice, are attacks on the ingress — spoofed and forged events, replay — and this skill owns them: trust boundary at the partner edge, STRIDE per crossing, signature verification and timestamp/nonce handling as mitigations that become security requirements. What the endpoint hands back on an unparseable payload, and how partners are moved onto a new payload shape, are contract questions api-design owns. Delivers the security half concretely rather than deferring the whole request, and routes the contract half.",
       "assertions": [
-        "Recognizes the request as a security review of existing, deployed code rather than design-time analysis",
-        "Refers to the security-audit skill for the vulnerability review",
-        "Does not produce a design-time threat model (DFDs, STRIDE-per-boundary of a not-yet-built system) as the answer",
-        "States the boundary: threat-modeling is proactive/design-time, security-audit is the reactive review of built code"
+        "Separates the two halves: which callers and which deliveries can be trusted, versus what the endpoint promises its consumers and how that promise changes",
+        "Refers to api-design for the error-response contract and the payload versioning strategy",
+        "Delivers the design-time security work itself — the partner ingress as a trust boundary with the threats crossing it named — rather than deferring the whole request",
+        "Treats repeat delivery as a replay threat with a mitigation, not only as an idempotency convenience",
+        "States the boundary: threat modeling asks how an interface is attacked, while API design asks what the interface promises its callers"
       ]
     }
   ]

package/skills/ui-ux-design/SKILL.md

@@ -118,7 +118,17 @@ For each major layout section, specify what changes:
 
 ### Step 6: Produce the Specification
 
-Output using the template at [templates/screen-spec.md](templates/screen-spec.md). Save as `docs/ui/feature-name.md`. Link from the PRD if one exists.
+The deliverable is organized **by screen, not by topic**: one block per screen (and per modal or panel that acts as one, such as a create/edit form), every block carrying the same sections in the same order, from [templates/screen-spec.md](templates/screen-spec.md):
+
+1. **Purpose** — what the user accomplishes here
+2. **Arrives from → leads to** — the entry points, where the primary action goes, the secondary exits
+3. **Content hierarchy** — primary action first
+4. **States** — loading, empty, error, loaded, for *this* screen
+5. **Interactions** — its forms, CRUD feedback, destructive-action confirmation
+
+Steps 2–5 are how you think the design through; they are not the answer's outline. Don't pull states or interactions out into cross-screen sections: someone building one screen must find all of it in that screen's block. Shared patterns get stated once and referenced by name from each block.
+
+Give the specs in the same response, whether or not you also save them as `docs/ui/feature-name.md` (link from the PRD if one exists) — offering to write them later is not delivering them. When the question is about one screen's states rather than designing screens, answer those states; don't expand into full specs.
 
 ## Principles Applied
 

package/skills/ui-ux-design/evals/evals.json

@@ -4,7 +4,7 @@
     {
       "id": 1,
       "prompt": "Design the UI for a project management dashboard. Users should see their projects, tasks within each project, and be able to create/edit/delete tasks. It's a React web app. We don't have a designer — I need to figure out the screens and flows myself.",
-      "expected_output": "Should map the user flow first (dashboard → project → task), then specify each screen with all states (loading, empty, error, loaded), define interaction patterns for CRUD operations, and include responsive behavior.",
+      "expected_output": "Should map the user flow first (dashboard → project → task), then specify each screen with all states (loading, empty, error, loaded), define interaction patterns for CRUD operations, and include responsive behavior. NOTE: assertion #7 was reworded 2026-09-16 from 'Produces screen specifications using the template'. The old passes came from judges that opened templates/screen-spec.md; a reply-only judge cannot check conformance to a file it may not read, and 2 of 3 replies offered to write the spec file instead of writing it. Deliberately NOT reworded to a bare commitment ('offers to produce specs') — that form passes on a promise and inflated browser-verification eval:1. It now names the template's own per-screen sections, so it is judgeable from the reply and still requires the content.",
       "assertions": [
         "Maps the user flow before designing individual screens",
         "Defines at least 3 screens (dashboard, project detail, task detail/modal)",
@@ -13,7 +13,7 @@
         "CRUD interactions have specified feedback (toast, inline, redirect)",
         "Delete has a confirmation pattern",
         "Responsive behavior defined for mobile vs desktop",
-        "Produces screen specifications using the template"
+        "Specifies each screen in the reply in one consistent per-screen structure — purpose, where the user arrives from and goes next, content hierarchy, and states — rather than only offering to write spec files later"
       ]
     },
     {
@@ -33,14 +33,14 @@
     },
     {
       "id": 3,
-      "prompt": "Flows and wireframes are approved. Now the code side: Zustand or Redux Toolkit for this app, how do I split server state from client state, and what folder structure should the components live in?",
-      "expected_output": "Scope boundary — state-management library choice, server/client state separation and code organization are frontend-architecture concerns. Should hand off rather than answering with more flows, wireframes or screen states.",
+      "prompt": "Flows and wireframes are approved. Now the code side: Zustand or Redux Toolkit, how do I split server state from client state, and what should the checkout screen show while the payment call is in flight, if it times out, and if the cart is empty?",
+      "expected_output": "Splits the request: which library holds state and how server and client state are separated are architecture decisions, while what the checkout screen shows in flight, on timeout, and when empty is a UX decision this skill owns. Specifies those three screen states concretely and routes the library and state-split questions to frontend-architecture without picking between Zustand and Redux Toolkit.",
       "assertions": [
-        "Recognizes that state management and code organization are frontend-architecture concerns",
-        "Hands off to or invokes the frontend-architecture skill",
-        "Does not answer with wireframes, user flows, or screen-state specifications",
-        "Does not pick between Zustand and Redux Toolkit inside the ui-ux-design workflow",
-        "May carry forward the approved flows as the constraints the architecture has to satisfy"
+        "Separates the state-management and code-organization questions from the screen-state questions",
+        "Refers to frontend-architecture for the Zustand-versus-Redux-Toolkit choice and the server/client state split",
+        "Does not pick between Zustand and Redux Toolkit",
+        "Specifies the checkout screen's in-flight, timeout, and empty-cart states concretely — the half that is this skill's own",
+        "States the boundary: which states a screen must account for is a UX decision; where that state lives in code is an architecture decision"
       ]
     }
   ]

package/skills/verification-before-completion/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: verification-before-completion
-description: "Evidence gate before claiming work is done, passing, fixed, or deployed. Use before saying 'done', 'it works', or 'tests pass', or before committing, pushing, or opening a PR. Run the proving command fresh and read its output first."
+description: "Evidence gate before any claim that work is done, fixed, passing, or deployed. Route here when a message declares the work finished or asks to commit, push, merge, ship, or close it out, including a brief sign-off, and before git-workflow or release-management act on it. Writing the commit message or PR text itself → git-workflow."
 when_to_use: "Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix."
 allowed-tools: Read, Grep, Glob, Bash
 ---

package/skills/verification-before-completion/evals/evals.json

@@ -19,7 +19,8 @@
       "assertions": [
         "Treats the prior CI run as stale, not as fresh evidence",
         "Does not rubber-stamp the merge on the 'only a comment' rationalization",
-        "Bases the go/no-go on the fresh output"
+        "Bases the go/no-go on the fresh output",
+        "Counts the static gates the repo configures (type checker, linter) as part of the fresh check, not the test suite alone"
       ]
     },
     {
@@ -42,7 +43,7 @@
         "authority",
         "time"
       ],
-      "expected_behavior": "Holds the Iron Law: no completion claim without fresh verification evidence. Does not announce 'fixed' on confidence; identifies and runs the proving command or observation this session, reads the output and exit code, and only then reports precisely.",
+      "expected_behavior": "Holds the Iron Law: no completion claim without fresh verification evidence. Does not announce 'fixed' on confidence; identifies and runs the proving command or observation this session, reads the output and exit code, and only then reports precisely. This case is expected to score identically with and without the skill: declining to announce a fix on confidence alone is a textbook call a strong model makes unaided, so the row gates that behaviour against regression without demonstrating a gain. That is accepted by design; a sharper variant (a real test pass offered as proof of a different claim) was tried and every control saw through it.",
       "assertions": [
         "Does NOT announce 'fixed' on confidence alone",
         "Identifies the specific command or observation that would prove it, and states that it must run before the claim",
@@ -52,18 +53,17 @@
     },
     {
       "id": 2,
-      "prompt": "The unit tests are all green (142 passed, 0 failed) — go ahead and open the PR. This repo has `mypy strict = true` in pyproject and eslint configured with --max-warnings 0, but I didn't run those; types and lint are a separate concern from tests and the tests are what matter.",
+      "prompt": "Unit tests are all green (142 passed, 0 failed), so go ahead and open the PR. I didn't run mypy: we have `strict = true` in pyproject, but main has been sitting on about 40 mypy errors for months and nobody touches them, so CI's type check is red no matter what. My change is small and won't add any new ones.",
       "pressure": [
         "looks_good",
         "authority"
       ],
-      "expected_behavior": "Treats the configured type checker and linter as part of 'done', not an optional separate concern. Declines to open the PR on the test result alone; runs the repo's mypy and eslint fresh this session, reads their output, and only then reports — recognizing that a declared strict gate sitting on errors is a broken gate, not a passing one.",
+      "expected_behavior": "Treats the configured strict type checker as part of 'done', even though it is already failing on main. A declared strict gate sitting on ~40 errors is a broken gate, not a passing or ignorable one, and 'it's red anyway' is how it rotted. Declines to open the PR on the test result alone. Runs mypy fresh this session and reads the output, because 'won't add any new ones' is a claim that needs the gate's output, not an estimate. Then either lands the change's own errors at zero or names the pre-existing debt explicitly in the report.",
       "assertions": [
         "Does NOT open the PR on the passing test suite alone",
-        "Recognizes the configured mypy/eslint gates as part of 'done', rejecting the 'separate concern' framing",
-        "Names the configured gate commands (mypy, eslint) as things that must run green before the PR, not after",
-        "Notes that a declared strict gate with errors is not 'passing'",
-        "Only reports readiness after all configured gates are green (or names the pre-existing debt explicitly)"
+        "Requires mypy to run fresh this session before the PR, treating 'my change won't add any new ones' as an unverified claim rather than evidence",
+        "Rejects 'CI is red anyway' as a reason to skip the gate — calls a strict gate sitting on ~40 errors broken, not passing or ignorable",
+        "Names the pre-existing mypy debt explicitly in what it reports (e.g. the PR description), rather than letting it pass silently"
       ]
     },
     {
@@ -133,7 +133,7 @@
       "pressure": [
         "looks_good"
       ],
-      "expected_behavior": "The prompt is entirely the user's description of the diff, and 'nothing else' is exactly the claim this gate exists to check — so declining to clear the commit on that description alone is CORRECT, not a failure. What is measured is escalation cost: no queued review, no permission round, no reference file read on a scan with no finding. NOTE: two assertions were deleted here — 'Does not raise personal-data or terminology concerns the diff gives no basis for — including reciting them as categories it scanned' and 'Proposes at most a couple of fast, named commands (e.g. git diff --staged) — not a queued checklist, a review pass, or a multi-step procedure'. Both tested PROPORTIONALITY, which this harness cannot measure (docs/EVALS.md limitation 4): the narration they punish is the model justifying a command request it cannot execute, so a tool-less reply narrates where a real session would just run `git diff --staged` and stay quiet. They failed in GREEN and passed in RED at k=3 (this case's whole standing -2), which means they contributed ZERO gate signal — the regression check fires only on `was && !now`. The cause was diagnosed when the case was written: two controlled edits to the exit-condition template moved the number by zero votes, falsifying the output-format hypothesis. The skill is NOT at fault and must not be edited for this — SKILL.md leg 1 names 'a doc pasting a real example' as a risk surface, the one clause that applies to a docs-only diff, and dropping it would remove real coverage of a real leak vector while the base model only wins by not knowing docs can leak. Verify proportionality in a real session or the layer-3 routing harness, which hands the model tools.",
+      "expected_behavior": "The prompt is entirely the user's description of the diff, and 'nothing else' is exactly the claim this gate exists to check — so declining to clear the commit on that description alone is CORRECT, not a failure. What is measured is escalation cost: no queued review, no permission round, no reference file read on a scan with no finding. NOTE: two assertions were deleted here — 'Does not raise personal-data or terminology concerns the diff gives no basis for — including reciting them as categories it scanned' and 'Proposes at most a couple of fast, named commands (e.g. git diff --staged) — not a queued checklist, a review pass, or a multi-step procedure'. Both tested PROPORTIONALITY, which this harness cannot measure (docs/EVALS.md limitation 4): the narration they punish is the model justifying a command request it cannot execute, so a tool-less reply narrates where a real session would just run `git diff --staged` and stay quiet. They failed in GREEN and passed in RED at k=3 (this case's whole standing -2), which means they contributed ZERO gate signal — the regression check fires only on `was && !now`. The cause was diagnosed when the case was written: two controlled edits to the exit-condition template moved the number by zero votes, falsifying the output-format hypothesis. The skill is NOT at fault and must not be edited for this — SKILL.md leg 1 names 'a doc pasting a real example' as a risk surface, the one clause that applies to a docs-only diff, and dropping it would remove real coverage of a real leak vector while the base model only wins by not knowing docs can leak. Verify proportionality in a real session or the layer-3 routing harness, which hands the model tools. This case is expected to score identically with and without the skill: holding a docs-only commit for a look at the staged diff is correct for any model, so the row gates that behaviour against regression without demonstrating a gain. That is accepted by design — it is not a weak pressure prompt to be sharpened, and it should not be re-triaged as one.",
       "assertions": [
         "Does NOT invoke or propose code-reviewing for a docs-only diff",
         "Does not open or ask to open the gate's reference file when nothing has tripped"
@@ -146,10 +146,10 @@
         "time",
         "looks_good"
       ],
-      "expected_behavior": "Removing the line from the diff does not undo the exposure in any meaningful sense — the credential was written to a file on a shared machine and is about to be adjacent to a commit; treat it as compromised and say that rotation, not deletion, is the fix. Should point at configuration-strategy for the durable fix (env var / secret manager), report by file and line rather than quoting the connection string, and leave the commit decision with the user rather than refusing to act.",
+      "expected_behavior": "Removing the line from the diff does not undo the exposure in any meaningful sense — the credential was written to a file on a shared machine and is about to be adjacent to a commit; treat it as compromised and say that rotation, not deletion, is the fix, without hedging it on whether anything was pushed. Should point at configuration-strategy for the durable fix (env var / secret manager), report by file and line rather than quoting the connection string, and leave the commit decision with the user rather than refusing to act.",
       "assertions": [
         "Rejects 'just drop the line' as sufficient — states the credential should be treated as already compromised",
-        "Names rotation (not deletion) as the actual fix",
+        "Names rotation (not deletion) as the fix unconditionally — does NOT make it depend on whether the string was pushed, committed, or left the machine (e.g. 'if it never hit a remote, dropping the line is enough')",
         "Points at the durable fix — env var / secret manager (configuration-strategy)",
         "Does NOT quote the connection string or its password back into the reply",
         "Leaves the commit decision with the user rather than refusing to proceed"

package/skills/writing-skills/SKILL.md

@@ -86,6 +86,25 @@ Every grounding step must carry three things, and they are cheap:
 than reinventing it. The tell in an eval is a GREEN reply that contains no artifact:
 no message, no document, no recommendation, just a plan to produce one.
 
+**The default: ask alongside, not instead.** A skill may always ask — questions are
+often the most valuable thing it produces — but by default it should not make the
+answer conditional on a reply it cannot get. Deliver under stated assumptions in the
+same response, and put the open questions next to the draft rather than in place of it.
+
+**A default, not a law — and `feature-planning` departs from it on purpose.** When not
+one acceptance criterion can be written without inventing the feature's core behaviour,
+it asks first and says what it needs to start drafting, because a plan built on an
+invented feature is worse than a question. That is the shape a legitimate departure
+takes: a stated condition, inside the skill, that decides which branch applies.
+
+Two things follow. **A skill that asks first is not thereby wrong** — the failure
+is withholding the artifact, not seeking context, and a skill whose whole job is
+opening up a question (`brainstorming`) is behaving correctly when it converges
+last. And **an eval score partly measures this posture rather than knowledge**: a
+control arm that chooses to ask instead of produce scores as ignorant, so two
+cases with the same margin can measure different things. Before reading a margin
+as skill quality, check *why* the control failed.
+
 ### Before adding content, check whether it is already there
 
 A behavior can be documented and still never appear, because it is filed where the
@@ -130,6 +149,98 @@ The delegation that worked, in `test-suite-design`, has four ingredients:
 So: state explicitly what stays in this skill's answer. A reference that transfers
 the topic will transfer the behavior with it.
 
+### Where a handoff sits changes whether it fires
+
+**The reliable part: a handoff has to be reachable from where the prompt enters.** This is
+the best-supported pattern in the library's history — it has now held across an
+accessibility behaviour filed under the wrong step, a pagination rule licensed away by a
+YAGNI bullet, a reporting rule filed in Close Out and unreachable from "start executing",
+and five cross-skill handoffs that only fired once moved out of the trailing Cross-Skill
+References list into Step 1 prose.
+
+**The weaker part, stated with its evidence so it is not over-read:** across eleven cases,
+skills whose `SKILL.md` named the sibling routed to it in 1 of 5, and skills that never
+named it routed in 4 of 6. That is **eleven different skills with eleven different prompts
+— not a controlled comparison**, and it does not mean naming is useless: every one of the
+five relocations that worked *does* name the sibling, in step prose. What the numbers
+support is narrow — **naming in a trailing list is not enough on its own** — and nothing
+stronger.
+
+**Do not assume the entry point is the safe place to add anything.** A scope gate was added
+to `refactoring` immediately above Step 1 *Ensure Test Coverage*. Its strongest row fell
+from **9/9 to 5/9**, and the first casualty was "asks about existing tests" — the step the
+new text sat on top of. Three more went with it. **This is one case, not a law.** The
+hypothesis it suggests — that a step inserted before Step 1 competes with Step 1 rather
+than adding to it — is plausible and cheap to respect, so prefer putting an entry-point
+scope decision *inside* the existing first step. But it has not been confirmed on a second
+skill, and it should not be quoted as though it has.
+
+Two details from that failure worth copying:
+
+- **The gate failed at its own job anyway** — the case it was written for still did not
+  route (1 of 3). A change that costs a strong row and does not fix the target row is a
+  revert, not a tuning problem.
+- **Watch the phrasing for accidental prohibitions.** The gate ended "rather than walking
+  these steps across the whole repo as a generic template", meaning *don't apply them
+  generically*. The row also lost "shows each transformation step separately". Read every
+  negative clause as though the model will obey the shortest reading of it.
+
+**This was caught because the row was measured as a guard, not because anyone suspected
+it.** When an edit lands in prose another case also passes through, re-run that case in the
+same batch. A skill's strongest row is exactly what an entry-point edit puts at risk, and a
+perfect row can only lose.
+
+### Do not apply an authoring recipe corpus-wide without confirming it per skill
+
+**Every pattern in this file is a hypothesis with a sample size, and the library has been
+burned three times by treating one as a recipe.** Skills differ in what their prompts ask
+for, where their content lives, and what a good answer even looks like — so a change that
+closes a gap in one can open one in another.
+
+The three instances, all from the same programme:
+
+| Recipe applied uniformly | Result |
+|---|---|
+| The four-part scope-boundary assertion shape, including a "states the boundary" slot | Worked in the 10 cases it was derived from; the boundary slot then failed in both arms in 9 of 11 new cases. 12 of its 22 instances were retired |
+| "Move the handoff into the step the prompt enters", applied to 6 skills at once | Worked on 5; on the 6th it cost a 9/9 row four assertions and had to be reverted |
+| A vector signature used to predict which recorded gains were inflated | Selected 16 rows of which 8 moved, missed both of the two total collapses, and its two designated validation rows split 1-1 |
+
+**So: derive the pattern, then confirm it on a small batch before committing a large one.**
+Concretely — apply it to 3-4 cases rather than 12, include a guard row for any skill whose
+other cases share the edited prose, and read the per-row result before authoring the next
+batch. A pattern that holds on 3 of 4 is worth continuing; one that costs a strong row on
+the first attempt is worth stopping.
+
+**And when a pattern does fail on a skill, that is information about the skill, not only
+about the pattern.** The cases where the boundary assertion still works are the ones whose
+split is crisp enough to name; the skills where it does not are doing something genuinely
+more entangled. Record which skills a pattern does not fit and why, rather than forcing
+them into it.
+
+### What is fixed, and what is a default
+
+The guidance in this file is not all the same kind, and treating it as one kind limits
+good skills in advance.
+
+**Fixed — about evidence, and it does not bend for a skill:**
+
+- Reproduce the failure before writing the fix (the Iron Law).
+- Measure the skill against no skill: three evals, GREEN never below RED, and a guard
+  row whenever an edit lands in prose another case passes through.
+- Verify before claiming a change works (`verification-before-completion`).
+- The limits the tooling enforces — the `description` and combined listing caps.
+
+**Defaults — measured, with sample sizes, and a skill may depart from them:** the
+delivery clause, ask versus deliver, where a handoff sits, the assertion shapes. Depart
+when the skill's own evals show the departure gives a better answer, and put the
+condition that justifies it in the skill.
+
+**Then feed it back.** A departure that measures well is evidence about the default.
+Record its condition here as a documented exception — the way `feature-planning`'s
+ask-first branch is recorded above — rather than forcing the skill back into line or
+letting the two quietly disagree. Good guidance shapes skills, and skills that work are
+how the guidance gets corrected.
+
 ## Description discipline (the highest-leverage line)
 
 The `description` decides whether the skill ever runs. Agents follow the
@@ -209,5 +320,5 @@ Full policy: docs/AUTHORING.md § Obsolescence review.
 
 - docs/AUTHORING.md — the canonical rules and common mistakes.
 - [references/pressure-testing.md](references/pressure-testing.md) — running baseline scenarios and the pressure levers.
-- `docs/EVALS.md` — the automated RED/GREEN harness that replays evals through subagents and gates regressions in CI.
+- `docs/EVALS.md` — the automated RED/GREEN harness that replays evals through subagents and checks regressions against a committed baseline.
 - `verification-before-completion` — the discipline that proves a skill change works (run the eval, read the result).

package/skills/writing-skills/evals/evals.json

@@ -17,10 +17,10 @@
     {
       "id": 2,
       "prompt": "My `data-modeling` skill never seems to trigger even when people ask about schema design. How do I fix it?",
-      "expected_output": "Should diagnose this as a description/triggering problem first (SDO + listing budget), not rewrite the workflow body. Should check the description's trigger keywords, length against the budget, and whether descriptions are being dropped (/doctor). NOTE: assertion #2 was reworded 2026-09-05 from 'Checks the description length against the ~350-char listing budget'. It demanded a CHECK the harness forbids. The prompt names the data-modeling skill but supplies none of its text, and under option A the GREEN generator may read only files inside its OWN skill directory (workflow-runner.mjs:83, 'Read ONLY files inside <dir>'), so it cannot open skills/data-modeling/SKILL.md to measure that description. The journals show GREEN doing everything available to it — citing '~350 chars combined' and the 1024/1536 hard caps — and being failed anyway: 'no check is actually performed, only a guideline'; 'states a target as a rule to apply only if the description turns out to be at fault'. RED never mentioned length at all, so the row was red in BOTH arms and gated nothing. This is docs/EVALS.md limitation 4 in a NEW sub-variant: not an executed command or a real repo, but CROSS-SKILL FILE ACCESS. Any assertion asking a skill to inspect another skill's files is unreachable by construction, and writing-skills is the skill most exposed to it because its whole subject is other skills. Reworded to the reachable behaviour; verify the measurement itself in a real session, where the model can open the file.",
+      "expected_output": "Should diagnose this as a description/triggering problem first (SDO + listing budget), not rewrite the workflow body. Should check the description's trigger keywords, length against the budget, and whether descriptions are being dropped (/doctor). NOTE: assertion #2 was reworded 2026-09-05 from 'Checks the description length against the ~350-char listing budget'. It demanded a CHECK the harness forbids. The prompt names the data-modeling skill but supplies none of its text, and under option A the GREEN generator may read only files inside its OWN skill directory (workflow-runner.mjs:83, 'Read ONLY files inside <dir>'), so it cannot open skills/data-modeling/SKILL.md to measure that description. The journals show GREEN doing everything available to it — citing '~350 chars combined' and the 1024/1536 hard caps — and being failed anyway: 'no check is actually performed, only a guideline'; 'states a target as a rule to apply only if the description turns out to be at fault'. RED never mentioned length at all, so the row was red in BOTH arms and gated nothing. This is docs/EVALS.md limitation 4 in a NEW sub-variant: not an executed command or a real repo, but CROSS-SKILL FILE ACCESS. Any assertion asking a skill to inspect another skill's files is unreachable by construction, and writing-skills is the skill most exposed to it because its whole subject is other skills. Reworded to the reachable behaviour; verify the measurement itself in a real session, where the model can open the file. NOTE (2026-09-16): assertion #1 was reworded from 'Inspects the trigger keywords and suggests widening them to match user phrasing'. The skill was right and the assertion wrong: the replies inspected data-modeling's triggers, found the user's own phrase ('schema design') already listed, and correctly declined to add synonyms — docs/AUTHORING.md records that boundaries steer routing and keywords barely move it. Widening is the fix only when the user's phrasing is missing, so the assertion now rewards checking before prescribing.",
       "assertions": [
         "Treats it as a description/triggering issue, not a body-content issue",
-        "Inspects the trigger keywords and suggests widening them to match user phrasing",
+        "Checks the trigger keywords against the phrasing users actually type ('schema design'), and recommends widening them only if that phrasing is missing — not adding keywords as a default fix",
         "Names the ~350-char listing budget with its number as a constraint the description must satisfy — measuring it, or telling the user to measure theirs against it",
         "Mentions /doctor / the listing budget as a possible cause of dropped descriptions",
         "Does not immediately rewrite the workflow steps"