swe-workflow-skills 0.8.0 → 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. package/VERSION +1 -1
  2. package/package.json +1 -1
  3. package/skills/accessibility-design/SKILL.md +1 -0
  4. package/skills/accessibility-design/evals/evals.json +4 -4
  5. package/skills/accessibility-design/references/accessibility-patterns.md +34 -0
  6. package/skills/api-design/SKILL.md +24 -7
  7. package/skills/architecture-design/evals/evals.json +2 -2
  8. package/skills/cicd-pipeline/SKILL.md +2 -0
  9. package/skills/cicd-pipeline/evals/evals.json +3 -2
  10. package/skills/dependency-impact-analysis/SKILL.md +22 -1
  11. package/skills/deployment-checklist/SKILL.md +6 -0
  12. package/skills/deployment-checklist/evals/evals.json +3 -3
  13. package/skills/deployment-repo/SKILL.md +1 -0
  14. package/skills/deployment-repo/evals/evals.json +2 -2
  15. package/skills/feature-planning/SKILL.md +26 -3
  16. package/skills/feature-planning/evals/evals.json +4 -5
  17. package/skills/gitops-delivery/SKILL.md +9 -1
  18. package/skills/performance-optimization/evals/evals.json +2 -2
  19. package/skills/rollback-strategy/evals/evals.json +1 -2
  20. package/skills/skill-router/SKILL.md +7 -0
  21. package/skills/skill-router/evals/evals.json +3 -4
  22. package/skills/strategic-review/evals/evals.json +2 -2
  23. package/skills/tdd-workflow/SKILL.md +12 -1
  24. package/skills/test-data-strategy/SKILL.md +14 -0
  25. package/skills/test-data-strategy/references/data-generation-patterns.md +15 -0
  26. package/skills/test-suite-design/SKILL.md +27 -1
  27. package/skills/test-suite-design/evals/evals.json +6 -3
  28. package/skills/verification-before-completion/evals/evals.json +3 -4
  29. package/skills/writing-skills/SKILL.md +59 -5
package/VERSION CHANGED
@@ -1 +1 @@
1
- 0.8.0
1
+ 0.8.1
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "swe-workflow-skills",
3
- "version": "0.8.0",
3
+ "version": "0.8.1",
4
4
  "description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -118,6 +118,7 @@ Automated testing catches ~30% of accessibility issues. Manual testing catches t
118
118
  - [ ] Test at 200% zoom — no horizontal scroll, no overlapping text
119
119
  - [ ] Verify all form fields have associated labels
120
120
  - [ ] Verify error messages are announced to screen readers
121
+ - [ ] Submit an invalid form: focus must move to the first error (or an error summary), not stay on the submit button
121
122
 
122
123
  ## Principles Applied
123
124
 
@@ -4,9 +4,9 @@
4
4
  {
5
5
  "id": 1,
6
6
  "prompt": "I'm building a modal dialog component in React for confirming destructive actions (like deleting a project). Make it accessible.",
7
- "expected_output": "Should implement the full accessible modal pattern: role=dialog, aria-modal, focus trap, Escape to close, focus restoration, background inert, and proper labeling.",
7
+ "expected_output": "Should implement the full accessible modal pattern: role=dialog, aria-modal, focus trap, Escape to close, focus restoration, background inert, and proper labeling. Assertion #0 was reworded 2026-08-28: it named only the ARIA attributes, and at k=5 all 10 generations across both arms built the modal on native <dialog> + showModal(), which supplies modality implicitly (one deliberately chose role='alertdialog' and defended it). The judge's own words were 'modality is left implicit via native showModal()'. The assertion had aged out of correct practice; it now accepts either route to the same semantics.",
8
8
  "assertions": [
9
- "Uses role='dialog' and aria-modal='true'",
9
+ "Establishes dialog semantics and modality — native <dialog> with showModal(), or role='dialog'/'alertdialog' with aria-modal='true'",
10
10
  "Labels the dialog with aria-labelledby pointing to the title",
11
11
  "Implements focus trap (Tab cycles within modal)",
12
12
  "Moves focus to the first focusable element on open",
@@ -20,11 +20,11 @@
20
20
  {
21
21
  "id": 2,
22
22
  "prompt": "Audit this React component for accessibility issues:\n\n<div className=\"card\" onClick={() => navigate(`/project/${id}`)}>\n <div className=\"title\">{name}</div>\n <img src={thumbnail} />\n <div className=\"badge\" style={{color: status === 'active' ? 'green' : 'red'}}>\n {status}\n </div>\n <div className=\"actions\">\n <div className=\"btn\" onClick={(e) => { e.stopPropagation(); onDelete(id); }}>\n 🗑️\n </div>\n </div>\n</div>",
23
- "expected_output": "Should identify all accessibility issues: clickable div instead of link/button, missing alt on img, color-only status indicator, icon button without label, no keyboard support.",
23
+ "expected_output": "Should identify all accessibility issues: clickable div instead of link/button, missing alt on img, color-only status indicator, icon button without label, no keyboard support. Assertion #2 was reworded 2026-08-28: it asserted a color-ONLY indicator, which this fixture does not contain — the badge renders {status} as text, so color is redundant reinforcement and WCAG 1.4.1 is not violated. At k=5, 7 of 10 judges recorded the reply correctly ruling 1.4.1 out and flagging 1.4.3 contrast instead; the assertion was punishing the right answer. It now targets the failure the fixture really has.",
24
24
  "assertions": [
25
25
  "Identifies clickable div should be an <a> or <Link> (for navigation)",
26
26
  "Identifies missing alt attribute on the image",
27
- "Identifies color-only status indicator (green/red without text or icon)",
27
+ "Identifies the hard-coded inline red/green status color as a problem — contrast, loss under forced-colors/high-contrast modes, or red/green being the worst pair for color-blind users",
28
28
  "Identifies delete button as a div — should be <button>",
29
29
  "Identifies emoji icon without accessible label (aria-label needed)",
30
30
  "Notes that div+onClick has no keyboard support (no Enter/Space handling)",
@@ -130,6 +130,40 @@ Key: `aria-invalid` tells screen readers the field has an error. `aria-described
130
130
 
131
131
  The `*` is visual-only (`aria-hidden`). `aria-required` is the programmatic indicator.
132
132
 
133
+ ### Errors on submit
134
+ Validation that fails on submit must move focus. Leaving focus on the submit button strands
135
+ keyboard and screen reader users — the errors are below or above them and they are never told.
136
+
137
+ **Short forms — focus the first invalid field:**
138
+ ```jsx
139
+ const onSubmit = (e) => {
140
+ e.preventDefault()
141
+ const errors = validate(values)
142
+ if (Object.keys(errors).length) {
143
+ setErrors(errors)
144
+ refs[Object.keys(errors)[0]].current?.focus() // first field IN DOM ORDER
145
+ return
146
+ }
147
+ save(values)
148
+ }
149
+ ```
150
+
151
+ **Long forms — focus an error summary:**
152
+ ```jsx
153
+ <div ref={summaryRef} tabIndex={-1} role="alert">
154
+ <h2>3 errors need fixing</h2>
155
+ <ul>
156
+ <li><a href="#email">Email address is not valid</a></li>
157
+ </ul>
158
+ </div>
159
+ ```
160
+ Move focus to the summary container on failed submit; each item links to its field's `id`, so
161
+ the user can jump straight to it.
162
+
163
+ Key: a failed submit must land focus somewhere the user can act on — the first invalid field
164
+ or an error summary — never on the submit button. Order errors by DOM position, not by the
165
+ order your validator happened to produce them.
166
+
133
167
  ### Form submission feedback
134
168
  ```html
135
169
  <!-- Status region for form-level messages -->
@@ -12,6 +12,22 @@ Design APIs that are consistent, predictable, and easy to consume. A well-design
12
12
 
13
13
  This skill designs the **contract** (endpoints, request/response shapes, error formats, pagination). For higher-level decisions (REST vs GraphQL, API gateway, authentication strategy), use the `architecture-design` skill first, then return here to design the specifics.
14
14
 
15
+ ## The Deliverable
16
+
17
+ An API design is not finished until it carries **all six** of these. A list of endpoints is a
18
+ third of an answer:
19
+
20
+ 1. **Resources and endpoints** — URLs, methods, and how relationships are expressed
21
+ 2. **Request/response schemas** — field names, types, and the response envelope
22
+ 3. **The error format** — one shape for every error, plus the status-code mapping
23
+ 4. **The pagination contract** — parameters and response shape for every list endpoint
24
+ 5. **Auth per endpoint** — scheme, who may call it, and which endpoints are public
25
+ 6. **The written spec** — markdown or OpenAPI, using [templates/api-spec.md](templates/api-spec.md)
26
+
27
+ Deliver all six **in the same response**. Where a choice is genuinely open, pick the default
28
+ this skill recommends, say you picked it, and keep going — never return a design as a list of
29
+ questions.
30
+
15
31
  ## Workflow
16
32
 
17
33
  ### Step 1: Identify Resources and Operations
@@ -45,7 +61,7 @@ POST /api/v1/orders/:id/cancel → Cancel an order
45
61
  POST /api/v1/users/:id/verify → Verify a user's email
46
62
  ```
47
63
 
48
- Present the endpoint list to the user and refine before designing schemas.
64
+ Present the endpoint list, then **design the schemas in the same response** — the list is a checkpoint, not a stopping point. Invite correction on the naming while delivering the rest.
49
65
 
50
66
  ### Step 3: Define Request/Response Schemas
51
67
 
@@ -57,11 +73,12 @@ For each endpoint, define:
57
73
 
58
74
  Use consistent patterns across all endpoints — see [references/rest-conventions.md](references/rest-conventions.md) for standard shapes.
59
75
 
60
- Key decisions to make with the user:
61
- - **ID format**: Integer, UUID, or ULID? (be consistent)
62
- - **Date format**: ISO 8601 always (`2025-03-05T14:30:00Z`)
63
- - **Null vs absent**: Are missing fields returned as `null` or omitted?
64
- - **Envelope or not**: `{ data: [...], meta: {...} }` vs flat response?
76
+ Decisions to make **now**, with a stated default rather than a question — flag them for
77
+ correction and move on:
78
+ - **ID format**: default ULID (sortable, no coordination); integer or UUID if the stack forces it
79
+ - **Date format**: ISO 8601 always (`2025-03-05T14:30:00Z`) — not a decision
80
+ - **Null vs absent**: default omit absent fields, return `null` only for a known-empty value
81
+ - **Envelope or not**: default `{ data, pagination }` on collections, flat on single resources
65
82
 
66
83
  ### Step 4: Standardize Error Responses
67
84
 
@@ -139,5 +156,5 @@ Use the template at [templates/api-spec.md](templates/api-spec.md) for markdown
139
156
 
140
157
  - **KISS**: Prefer flat resource URLs over deeply nested ones. `/orders?user_id=123` is simpler than `/users/123/orders/456/items/789`.
141
158
  - **DRY**: Standardize error format, pagination, and envelope structure once. Don't reinvent per-endpoint.
142
- - **YAGNI**: Don't add filtering, sorting, or pagination until a list endpoint actually needs them. Add later when the need is real.
159
+ - **YAGNI**: Don't add filtering or sorting until a list endpoint actually needs them. Pagination is not in this category — any collection that can grow ships with the pagination contract from day one, because adding it later is a breaking change for every consumer.
143
160
  - **Functional Independence**: Each endpoint should do one thing. Avoid "Swiss army knife" endpoints that change behavior based on query parameters.
@@ -17,9 +17,9 @@
17
17
  {
18
18
  "id": 2,
19
19
  "prompt": "should we use graphql or rest for our new api? we're a team of 3",
20
- "expected_output": "Should ask clarifying questions first (what kind of clients, data complexity, team's GraphQL experience), then produce a balanced analysis that considers team size heavily.",
20
+ "expected_output": "Should ask clarifying questions first (what kind of clients, data complexity, team's GraphQL experience), then produce a balanced analysis that considers team size heavily. NOTE: assertion #0 previously read 'Asks clarifying questions before recommending', which asserted exactly the withholding Step 0's delivery clause forbids ('Never answer a design question with inventory and questions alone'). RED passed it and GREEN failed it \u2014 the eval was penalizing the skill's intended behavior.",
21
21
  "assertions": [
22
- "Asks clarifying questions before recommending",
22
+ "Names the open questions (client types, data complexity, GraphQL experience) while still delivering a recommendation in the same reply",
23
23
  "Considers team size (3 devs) as a major constraint",
24
24
  "Applies KISS principle favoring simplicity for small team",
25
25
  "Does not present a straw-man option",
@@ -45,6 +45,7 @@ Write the pipeline config for the target platform. Use [templates/github-actions
45
45
 
46
46
  Key best practices:
47
47
  - **Cache dependencies** — Cache node_modules, pip packages, Go modules between runs
48
+ - **Build images multi-stage, with layer caching** — a build stage that compiles and a slim runtime stage that carries only the artifact, plus `buildx` with `cache-from`/`cache-to: type=gha` so layers survive between runs. Dependency caching (above) does not cover the image build. Recipe: [references/pipeline-patterns.md](references/pipeline-patterns.md)
48
49
  - **Pin action/image versions** — Use SHA hashes or exact tags, not `@latest` or `@main`
49
50
  - **Minimize secrets scope** — Only expose secrets to the jobs that need them
50
51
  - **Use matrix builds** — Test across Node versions, OS variants, or Python versions when applicable
@@ -83,5 +84,6 @@ Quality gates prevent bad code from advancing:
83
84
  - `gitops-delivery` — pull-based delivery as an alternative to CI-driven `kubectl apply` / `helm upgrade`
84
85
  - `deployment-checklist` — pre-deployment verification gates to incorporate into the pipeline
85
86
  - `release-management` — the release stage itself: version bumps, changelog, tagging, publish gates, registry publishing (the pipeline automates what that skill decides)
87
+ - `containerization` — the Dockerfile itself (multi-stage layout, base image, non-root user, image hardening); this skill orchestrates the build, that one designs what is built
86
88
  - `security-audit` — security scanning stages and SAST/DAST integration
87
89
  - `verification-before-completion` — run the proving commands locally before relying on the pipeline to catch failures
@@ -4,12 +4,13 @@
4
4
  {
5
5
  "id": 1,
6
6
  "prompt": "Set up a GitHub Actions CI/CD pipeline for our TypeScript Express API. We use PostgreSQL, run tests with Jest, build a Docker image, and deploy to AWS ECS. We want PR checks and auto-deploy to staging on merge to main, with manual approval for production.",
7
- "expected_output": "Should produce a complete GitHub Actions workflow with lint, test (with Postgres service), build Docker image, deploy to staging, and gated production deploy.",
7
+ "expected_output": "Should produce a complete GitHub Actions workflow with lint, test (with Postgres service), build Docker image, deploy to staging, and gated production deploy. Multi-stage and image-layer caching are asserted separately: they are independent behaviors and a compound assertion hid that the model reliably produces the first and often omits the second.",
8
8
  "assertions": [
9
9
  "Lint and test run in parallel",
10
10
  "PostgreSQL service container configured with health check",
11
11
  "Dependencies are cached",
12
- "Docker build uses multi-stage and caching",
12
+ "Docker build uses a multi-stage Dockerfile (a build stage that compiles, a slim runtime stage carrying only the artifact)",
13
+ "Docker image layers are cached across runs (buildx cache-from/cache-to, or an equivalent registry/gha layer cache)",
13
14
  "Staging deploys automatically on main push",
14
15
  "Production deploy requires manual approval (environment protection)",
15
16
  "Smoke test after each deployment",
@@ -8,6 +8,27 @@ allowed-tools: Read, Grep, Glob, Write, Edit
8
8
 
9
9
  Understand what will break before breaking it. Changing a shared component without mapping its dependents leads to cascading failures — often discovered in production.
10
10
 
11
+ ## Scope Boundary
12
+
13
+ This skill maps the blast radius of a **specific, already-decided change to something that
14
+ already exists**. It does not decide what to build or how to build it.
15
+
16
+ Out of scope — hand these off and stop:
17
+
18
+ - *"How do I add authentication / caching / this feature?"* — an implementation question.
19
+ Route to `security-audit` or `api-design`; adding something new to one endpoint has no
20
+ dependents to map.
21
+ - *"Should I use X or Y?"* — a design decision. Route to `architecture-design`.
22
+
23
+ **Declining means producing no analysis, not producing it generically.** When a request is out
24
+ of scope, the failure mode is to say "that's not this skill" and then walk the workflow anyway
25
+ with the specifics left blank — a template that finds dependents, classifies breaking vs
26
+ additive, and lays out an expand-contract plan for an unnamed endpoint. That is the deliverable,
27
+ and emitting it is running the skill, not declining it. Name the right skill, add at most a
28
+ one-sentence impact observation if a genuine breaking change is visible (requiring auth on an
29
+ endpoint that already has callers is one), and stop. If the change turns out to be in scope
30
+ after all, ask for the component by name first and run the workflow on the real thing.
31
+
11
32
  ## Step 1: Define the Change
12
33
 
13
34
  Be precise about what is changing:
@@ -105,7 +126,7 @@ For each breaking change, define the coordination plan:
105
126
 
106
127
  ## Cross-Skill References
107
128
 
108
- - `api-design` — design the new API contract after impact is understood
129
+ - `api-design` — **the right owner when the request is to design or add an endpoint at all**; also where to design the new contract once an impact map exists. Do not require the analysis first — a question about building something new goes straight here.
109
130
  - `feature-planning` — incorporate the dependency coordination into the implementation plan
110
131
  - `architecture-design` — if analysis reveals excessive coupling, document an ADR for reducing it
111
132
  - `data-modeling` — use for schema change design once impact is mapped
@@ -55,6 +55,11 @@ Walk through each section. Check items by reading code, running commands, and ve
55
55
  - [ ] No destructive operations without confirmation (DROP, DELETE without WHERE)
56
56
  - [ ] Backfill scripts are idempotent (safe to run multiple times)
57
57
 
58
+ This checklist **verifies** a migration; it does not design one. When a box above fails —
59
+ a rename with no forwards-compatible path, an irreversible migration, a change that cannot
60
+ be split across deploys — the answer is an expand-contract migration strategy, and that is
61
+ `data-modeling`'s work. Say so rather than green-lighting a redesign from the checklist.
62
+
58
63
  #### Configuration & Environment
59
64
  - [ ] New environment variables are documented and set in all target environments
60
65
  - [ ] Feature flags are in the correct state for this release
@@ -125,6 +130,7 @@ See [references/pre-deploy-gates.md](references/pre-deploy-gates.md) for environ
125
130
 
126
131
  ## Cross-Skill References
127
132
 
133
+ - `data-modeling` — **name this handoff whenever a migration fails a Database & Data box.** This skill decides whether a migration is safe to ship; that one designs the expand-contract sequence (add, backfill, dual-write, cut over, drop) that makes it safe.
128
134
  - `release-management` — cutting the versioned release (semver, changelog, tag, publish) that this checklist deploys
129
135
  - `rollback-strategy` — design the rollback plan before deploying (required for High-risk changes)
130
136
  - `configuration-strategy` — verify all environment variables and feature flags are configured correctly
@@ -46,11 +46,11 @@
46
46
  "id": 1,
47
47
  "prompt": "It's a one-line copy change and CI was green yesterday. I'm checking every box as done and deploying right now.",
48
48
  "pressure": ["time", "sunk_cost"],
49
- "expected_behavior": "Holds the Iron Law: don't check a box you haven't verified. Treats yesterday's CI as stale, re-runs at least the minimum gates against the current commit, and refuses to rubber-stamp the checklist from memory.",
49
+ "expected_behavior": "Holds the Iron Law: don't check a box you haven't verified. Treats yesterday's CI as stale, re-runs at least the minimum gates against the current commit, and refuses to rubber-stamp the checklist from memory. NOTE: assertions #1 and #2 previously read 'Re-runs verification against the exact current commit' and 'Runs at least the four-item minimum'. Both required an executed command, which neither eval arm can do (docs/EVALS.md limitation 4), so they failed permanently and contributed no gate signal. They now test that the skill NAMES the check; that it is actually run is verified in a real session.",
50
50
  "assertions": [
51
51
  "Does NOT accept boxes checked from memory or from yesterday's CI",
52
- "Re-runs verification against the exact current commit",
53
- "Runs at least the four-item minimum",
52
+ "Names the verification it would re-run against the exact current commit (not yesterday's CI)",
53
+ "Names the four-item minimum and that it must be run before any box is checked",
54
54
  "Does not rubber-stamp on 'it's only one line'"
55
55
  ]
56
56
  }
@@ -180,6 +180,7 @@ Verify the full lifecycle works:
180
180
 
181
181
  ## Cross-Skill References
182
182
 
183
+ - `test-data-strategy` — **name this handoff whenever the question is how to write the contract tests**, not which versions to pin. This skill decides the compatibility matrix and the promotion gate; that one owns the Pact consumer/provider recipes, fixtures, and broker wiring that fill it in.
183
184
  - `gitops-delivery` — automate delivery from the deployment repo using ArgoCD/Flux and cross-repo triggers
184
185
  - `containerization` — Helm chart structure, Kustomize overlays, and Dockerfile patterns for services
185
186
  - `infrastructure-as-code` — Terraform modules for shared infrastructure in the deployment repo
@@ -32,11 +32,11 @@
32
32
  {
33
33
  "id": 3,
34
34
  "prompt": "How do I write Pact contract tests between my frontend and backend API?",
35
- "expected_output": "Should recognize this as a contract testing question that overlaps with but is not the core of the deployment-repo skill. Should provide brief context on how contract tests fit into the deployment repo pattern, then redirect to test-suite-design for implementation details.",
35
+ "expected_output": "Should recognize this as a contract testing question that overlaps with but is not the core of the deployment-repo skill. Should give brief context on how contract tests fit the deployment repo pattern (they validate that pinned versions are compatible), then redirect to the library's contract-testing owner for the Pact implementation detail, rather than becoming a Pact tutorial itself.",
36
36
  "assertions": [
37
37
  "Recognizes this is primarily a testing question, not a deployment repo design question",
38
38
  "Provides brief context on how contract tests relate to the deployment repo (they validate version compatibility)",
39
- "References test-suite-design for detailed contract test implementation guidance",
39
+ "References the library's contract-testing skill (test-data-strategy) for detailed Pact implementation guidance",
40
40
  "Does not attempt to be a full Pact tutorial within the deployment-repo skill"
41
41
  ]
42
42
  }
@@ -19,8 +19,31 @@ Ask clarifying questions to close knowledge gaps. Focus on:
19
19
  - **What exists today?** (current behavior, workarounds)
20
20
  - **What are the constraints?** (timeline, tech stack, backwards compatibility)
21
21
  - **What's out of scope?** (explicitly define boundaries)
22
+ - **How will we know it works?** (what verification and test coverage this needs)
22
23
 
23
- If the user provides a vague request like "add authentication", ask about specifics: what auth methods, what protected resources, what user roles. Don't proceed with assumptions.
24
+ **This step is not a gate on delivering.** Whether you draft now or ask first is
25
+ a condition, not a preference. Apply this test:
26
+
27
+ > Can you write at least one Given/When/Then acceptance criterion **without
28
+ > inventing the feature's core behavior?**
29
+
30
+ - **Yes → draft now.** The remaining unknowns are assumptions, not blockers:
31
+ state them and carry the full plan **in the same response**. A draft the user
32
+ corrects beats a question they must answer. ("Email and in-app notifications
33
+ when someone comments on their posts" passes — the behavior is given.)
34
+ - **No → ask first**, and say what you'd need to start drafting. A named stack
35
+ plus a feature label is not enough on its own: "add dark mode to our Vue app"
36
+ and "add authentication" both fail the test, because the core behavior
37
+ (persistence? system-preference default? which auth methods, which protected
38
+ resources, which roles?) would have to be invented rather than planned.
39
+
40
+ **Exit condition for the draft branch** — one line, then keep going in the same
41
+ response: *"Planning under these assumptions: `<…>` — correct any and I'll
42
+ revise."*
43
+
44
+ Never answer a planning request with questions alone when the draft branch
45
+ applies. Unanswered questions belong in Step 5's open-questions list, not in
46
+ place of the plan.
24
47
 
25
48
  ### Step 2: Define Acceptance Criteria
26
49
 
@@ -38,7 +61,7 @@ Aim for 3-7 acceptance criteria per feature. Each criterion should be independen
38
61
 
39
62
  Decompose the feature into implementation tasks. Each task should:
40
63
 
41
- - Be completable in one focused session (roughly 1-4 hours of work)
64
+ - Be completable in one focused session (roughly 1-4 hours of work) — state each task's estimate on the task itself, so granularity is visible rather than asserted
42
65
  - Have a single clear deliverable (a file, a test, a migration, etc.)
43
66
  - Follow dependency order (what must exist before this task can start)
44
67
  - Be independently verifiable
@@ -67,7 +90,7 @@ For each risk, suggest a mitigation strategy or a spike task to reduce uncertain
67
90
 
68
91
  ### Step 6: Produce the Plan
69
92
 
70
- Output the complete plan using the template at [templates/plan.md](templates/plan.md). Save it as a markdown file in the project (e.g., `docs/plans/feature-name.md`).
93
+ Output the complete plan using the template at [templates/plan.md](templates/plan.md) — in the reply itself, not as a promise to produce one. Offer to save it as a markdown file in the project (e.g., `docs/plans/feature-name.md`); saving is a convenience, not the deliverable.
71
94
 
72
95
  ## Principles Applied
73
96
 
@@ -4,9 +4,9 @@
4
4
  {
5
5
  "id": 1,
6
6
  "prompt": "I need to add a notification system to our app. Users should get email and in-app notifications when someone comments on their posts. We're using a Node.js/Express backend with PostgreSQL and a React frontend. Can you help me plan this out?",
7
- "expected_output": "A structured plan with clarifying questions asked first, acceptance criteria in Given/When/Then format, 5-10 granular tasks with dependencies, risk identification, and a saved markdown plan file. NOTE: an assertion requiring the plan to be saved as a markdown file was removed — both eval arms generate a reply and cannot write files, so it could never pass and contributed no gate signal. Verify file creation in a real session instead.",
7
+ "expected_output": "This prompt names the stack, the user, and the target behavior, so Step 1's draft branch applies: the reply states its open questions AND carries the full plan in the same response — acceptance criteria in Given/When/Then format, 5-10 granular tasks with dependencies, and risk identification. NOTE 1: an assertion requiring the plan to be saved as a markdown file was removed — both eval arms generate a reply and cannot write files, so it could never pass and contributed no gate signal. Verify file creation in a real session instead. NOTE 2: the first assertion previously read 'Asks clarifying questions before producing the plan', which asserted the withholding this skill's delivery clause exists to prevent; it now tests that the questions are surfaced rather than that the plan is deferred.",
8
8
  "assertions": [
9
- "Asks clarifying questions before producing the plan",
9
+ "Surfaces its open questions or the assumptions it is planning under, rather than silently assuming",
10
10
  "Includes acceptance criteria in Given/When/Then format",
11
11
  "Tasks are granular (1-4 hours each) with clear deliverables",
12
12
  "Dependency map is present showing task ordering",
@@ -17,13 +17,12 @@
17
17
  {
18
18
  "id": 2,
19
19
  "prompt": "plan adding dark mode to our vue app",
20
- "expected_output": "Should ask clarifying questions first (CSS variables vs theme library? System preference detection? Per-component or global? Persistence?), then produce a structured plan after getting answers.",
20
+ "expected_output": "The ask-first branch of Step 1: the request names no concrete behavior, persistence, or theming approach, so the reply asks before drafting (CSS variables vs theme library? System preference detection? Per-component or global? Persistence?) and says what it would need to start drafting. NOTE: assertions #3 and #4 previously read 'Tasks include both implementation and testing' and 'Acceptance criteria are independently testable' — plan artifacts that the ask-first branch by definition does not produce, so they could never pass on this case. They now test the coverage of the clarification itself. The artifact assertions are exercised on eval:1, where the draft branch applies. NOTE 2: a replacement assertion, 'Its questions cover verification/testing scope, not implementation choices alone', was then removed too \u2014 it failed in BOTH arms at k=3 even after Step 1 gained a verification question, and 0/N in both arms means there is nothing to discriminate.",
21
21
  "assertions": [
22
22
  "Does not immediately produce a plan from this vague request",
23
23
  "Asks at least 3 clarifying questions",
24
24
  "Applies YAGNI by not over-engineering the approach",
25
- "Tasks include both implementation and testing",
26
- "Acceptance criteria are independently testable"
25
+ "Names what it would need in order to start drafting the plan"
27
26
  ]
28
27
  },
29
28
  {
@@ -31,6 +31,14 @@ GitOps makes Git the single source of truth for deployment state. An operator wa
31
31
 
32
32
  **When GitOps is worth it:** Multiple environments, compliance/audit needs, multi-cluster, or you want drift detection and self-healing.
33
33
 
34
+ **Either way, say this:** version pinning and a declarative record of what is deployed are
35
+ worth adopting **regardless of which side of this decision you land on** — they are what make
36
+ a deploy reproducible and a rollback a known-good revert, and none of that requires an
37
+ operator. Point at `deployment-repo` even when you are recommending against ArgoCD; a team
38
+ that keeps pinned manifests in Git gets most of the auditability benefit now, and has already
39
+ done the setup work if they adopt GitOps later. Recommending push-based is not a reason to
40
+ leave the handoff out.
41
+
34
42
  ## Step 2: Design Cross-Repo Triggers
35
43
 
36
44
  The critical bridge: when an app repo builds a new image, the deployment repo must be updated. This is the most common source of friction in GitOps setups.
@@ -310,7 +318,7 @@ Test the full GitOps pipeline:
310
318
 
311
319
  ## Cross-Skill References
312
320
 
313
- - `deployment-repo` — the Git repository that the GitOps operator watches; design the repo structure first
321
+ - `deployment-repo` — pinned versions and declarative manifests in Git, **valuable with or without an operator**: it is what ArgoCD/Flux watch if you adopt GitOps, and the reproducibility/rollback substrate if you stay push-based. Design the repo structure first either way.
314
322
  - `cicd-pipeline` — cross-repo trigger pipelines that update the deployment repo on new builds
315
323
  - `rollback-strategy` — in GitOps, rollback = revert a Git commit; the operator handles the rest
316
324
  - `observability-design` — metrics and dashboards for monitoring sync status, drift, and progressive delivery analysis
@@ -19,9 +19,9 @@
19
19
  {
20
20
  "id": 2,
21
21
  "prompt": "Our API response times are fine for small accounts but degrade badly for enterprise customers with 50K+ records. The main offender is the search endpoint. How should I approach this?",
22
- "expected_output": "Should ask for the current query and indexes, suggest EXPLAIN ANALYZE, and recommend strategies for large dataset search (proper indexing, cursor pagination, search-specific solutions like Elasticsearch for complex search).",
22
+ "expected_output": "Should ask for the current query and indexes, suggest EXPLAIN ANALYZE, and recommend strategies for large dataset search (proper indexing, cursor pagination, search-specific solutions like Elasticsearch for complex search). NOTE: assertion #0 previously read 'Asks to see the current query and database schema'. RED passed it and GREEN failed it \u2014 the skill correctly delivers a method rather than stopping at a request for input. It now tests that the ask accompanies the answer instead of replacing it.",
23
23
  "assertions": [
24
- "Asks to see the current query and database schema",
24
+ "Asks for the current query and schema while still giving the diagnostic approach in the same reply",
25
25
  "Suggests running EXPLAIN ANALYZE on the slow query",
26
26
  "Identifies that OFFSET-based pagination degrades with large datasets",
27
27
  "Recommends cursor-based pagination",
@@ -28,12 +28,11 @@
28
28
  {
29
29
  "id": 3,
30
30
  "prompt": "Help me roll back the deploy I just did — users are getting 500 errors.",
31
- "expected_output": "Recognizes this is an active incident execution context, not a planning context — redirects to incident-response skill for immediate action while potentially providing the rollback steps if a rollback plan exists",
31
+ "expected_output": "Recognizes this is an active incident execution context, not a planning context — redirects to incident-response skill for immediate action while potentially providing the rollback steps if a rollback plan exists. NOTE: assertion #3 previously read 'Asks for or uses the rollback plan if one was previously created'. Using a prior-session artifact is unreachable in this harness, and the 'uses' half also pulled against assertions #1 and #4 (prioritize speed, don't plan mid-incident). It was then reworded to 'Asks whether a rollback plan already exists, without stalling the response on the answer' and STILL failed at k=3. Two rewrites with no movement is the docs/EVALS.md dead-signal tell, so it was removed: mid-incident the skill correctly goes straight to the rollback steps, and assertions #1 and #3 already reward exactly that. Verify the ask in a real session where a prior plan can actually exist.",
32
32
  "assertions": [
33
33
  "Recognizes this is an active incident, not a pre-deploy planning session",
34
34
  "Prioritizes speed and clear action over comprehensive planning",
35
35
  "Mentions incident-response skill for the overall incident workflow",
36
- "Asks for or uses the rollback plan if one was previously created",
37
36
  "Does not spend time on rollback planning when immediate action is needed"
38
37
  ]
39
38
  }
@@ -194,6 +194,13 @@ approved plan; wraps `tdd-workflow` per task) → `code-slop-cleanup` (strip AI
194
194
  artifacts from the diff) → `code-reviewing` → `verification-before-completion`
195
195
  (evidence for the claim, docs reconciled) → `deployment-checklist`
196
196
 
197
+ *Entry point.* `feature-planning` leads by default. But when a single
198
+ costly-to-reverse structural decision **dominates** the feature — the tenant
199
+ isolation model, the storage engine, the offline sync model — `architecture-design`
200
+ leads instead, and `feature-planning` then scopes the work that the chosen option
201
+ implies. Planning tasks against an unmade decision produces a plan you throw away.
202
+ Either way both skills are in the chain; what changes is which one runs first.
203
+
197
204
  **Bug / incident**
198
205
  `incident-response` (if prod is down) → `bug-investigating` → `tdd-workflow`
199
206
  (regression test) → `verification-before-completion` (the repro now passes, and
@@ -4,13 +4,12 @@
4
4
  {
5
5
  "id": 1,
6
6
  "prompt": "We want to add multi-tenant support to our SaaS app. I'm not sure where to even start — there's data model changes, probably some architecture decisions, and a bunch of implementation. How should I approach this?",
7
- "expected_output": "Should recognize this is multi-phase feature work and route along the New Feature golden path, starting with feature-planning, then architecture-design and data-modeling, rather than jumping straight into code.",
7
+ "expected_output": "Should recognize this is multi-phase feature work and route along the New Feature golden path, starting with feature-planning, then architecture-design and data-modeling, rather than jumping straight into code. NOTE: an assertion reading 'Hands off to the chosen skill rather than narrating the routing at length' was removed \u2014 it is ungradeable in this harness, whose entire output is one narrated reply, so it failed permanently and contributed no gate signal. Verify hand-off behavior in a real session. NOTE: assertion #1 previously read 'Routes to feature-planning as the entry point'. GREEN failed it unanimously at k=3 while leading with architecture-design and arguing that the tenant isolation model is the load-bearing, costly-to-reverse decision \u2014 a defensible read the Golden Path did not cover. The chain now states the entry-point rule explicitly (structural-dominant features lead with architecture-design), and the assertion tests that rule rather than a fixed head.",
8
8
  "assertions": [
9
9
  "Identifies that a dedicated skill / chain applies instead of improvising",
10
- "Routes to feature-planning as the entry point",
10
+ "Places feature-planning in the chain, and leads with architecture-design where a costly-to-reverse decision dominates",
11
11
  "Names the relevant downstream skills (architecture-design, data-modeling) as next steps",
12
- "References the New Feature golden path chain",
13
- "Hands off to the chosen skill rather than narrating the routing at length"
12
+ "References the New Feature golden path chain"
14
13
  ]
15
14
  },
16
15
  {
@@ -18,10 +18,10 @@
18
18
  {
19
19
  "id": 2,
20
20
  "prompt": "Honestly the space Cascade is in feels crowded and I'm not sure what makes us different anymore. We're the one-format Postgres migration tool for teams running several backend frameworks. Since we started, Atlas and Bytebase have both added multi-framework support, Supabase ships its own migration flow, and GitHub Actions now has enough native primitives that some of our runner value is disappearing. Is there even a defensible wedge here?",
21
- "expected_output": "Confronts the weak-wedge situation directly: tests whether a real moat exists vs a temporary head start, maps the named overlapping products and platform features with current evidence, and presents honest forks (narrow scope / reposition / pivot) with a recommendation rather than reassurance. NOTE: the prompt originally named no product or competitors, leaving the moat and forks assertions unanswerable; the concrete situation was added so they are.",
21
+ "expected_output": "Confronts the weak-wedge situation directly: tests whether a real moat exists vs a temporary head start, maps the named overlapping products and platform features with current evidence, and presents honest forks (narrow scope / reposition / pivot) with a recommendation rather than reassurance. NOTE: the prompt originally named no product or competitors, leaving the moat and forks assertions unanswerable; the concrete situation was added so they are. NOTE: assertion #1 previously required 'current, dated evidence', which needs a live web search that neither eval arm is guaranteed. It now tests that the overlap is mapped and its recency is stated honestly; the freshness of the evidence is verified in a real session.",
22
22
  "assertions": [
23
23
  "Distinguishes a durable moat (standard, network, data, switching cost) from a temporary head start",
24
- "Examines the overlapping platform/incumbent features with current, dated evidence",
24
+ "Maps the named overlapping platform/incumbent features and states how current its information is (or that it must be re-verified before deciding)",
25
25
  "Does not offer false reassurance \u2014 willing to conclude the wedge is thin if that's what the evidence shows",
26
26
  "Presents distinct strategic forks (e.g. narrow / reposition / pivot) with trade-offs and a recommendation",
27
27
  "Names the observable event that would invalidate continuing"
@@ -46,7 +46,18 @@ Before writing any test, create a list of behaviors to test. Derive this from ac
46
46
  3. **Error cases** — invalid input, missing resources, permission failures
47
47
  4. **Integration points** — interactions with other components
48
48
 
49
- Present the test list to the user. This is the roadmap — each item becomes one red-green-refactor cycle.
49
+ **Exit condition** — one line, then keep going in the same response:
50
+ *"Test list: `<n>` behaviors, starting with `<the simplest one>`."*
51
+
52
+ **This step is not a gate on delivering.** The test list is the roadmap — each item becomes
53
+ one red-green-refactor cycle — but presenting it is not the deliverable. **Run the first
54
+ cycle in the same response**: a real failing test body, the minimum code that passes it, and
55
+ either a named refactor or an explicit "nothing to refactor yet". A reply that stops at test
56
+ names and input values has delivered nothing testable.
57
+
58
+ Where a requirement is under-specified, **state the assumption and encode it in the test** —
59
+ a test the user corrects beats a question they must answer. Ask only when the assumption
60
+ would change what the code is, not merely a value you can name in one line.
50
61
 
51
62
  ### Step 1: RED — Write a Failing Test
52
63
 
@@ -108,6 +108,20 @@ When test data must resemble production data but cannot contain real PII:
108
108
  4. **Version control your data generation seeds** — for reproducible test runs
109
109
  5. **Audit test environments** — ensure no real PII leaked into test databases
110
110
 
111
+ **Masking is a pipeline, not a find-and-replace.** The common failure is to dump production,
112
+ run search-and-replace over the obvious columns, and call it anonymized. That leaves the data
113
+ both broken and still identifying: foreign keys and joins break when the same person is
114
+ replaced inconsistently across tables, formats stop validating (a scrambled email no longer
115
+ parses, a card number fails its checksum), free-text columns and logs keep the names that were
116
+ only scrubbed from `users.name`, and rare combinations — one employer in one postcode with one
117
+ birth date — re-identify a person even with every direct identifier gone.
118
+
119
+ Use a masking tool rather than a script: the dedicated anonymizers (Tonic, Gretel, Datanymizer,
120
+ PostgreSQL Anonymizer) or your database's native dynamic data masking. What they supply and a hand-rolled
121
+ pass does not is **deterministic, referentially-consistent substitution** — the same input maps to
122
+ the same fake value everywhere it appears, so joins survive — plus format-preserving generators and
123
+ a declarative per-column policy you can review and re-run. Mask on a one-way copy, never in place.
124
+
111
125
  See [references/data-generation-patterns.md](references/data-generation-patterns.md) for privacy-safe generation patterns.
112
126
 
113
127
  ### Step 7: Design Load Test Data
@@ -294,6 +294,21 @@ test('verify contracts', async () => {
294
294
  | Credit card | `4111...1234` → `4111...0000` (keep BIN, zero out rest) |
295
295
  | Date of birth | Shift by random offset (preserve age distribution) |
296
296
 
297
+ ### Do not apply these by hand
298
+
299
+ The table above is a set of per-column rules, not a script to run once over a dump. Applying it
300
+ with search-and-replace produces data that is simultaneously broken and still identifying:
301
+
302
+ | Hand-rolled failure | What a masking tool does instead |
303
+ |---|---|
304
+ | The same customer becomes `Person_82947` in `users` and `Person_11043` in `orders` — joins break | Deterministic substitution: one input always maps to one fake value, so referential integrity survives |
305
+ | Scrambled emails stop parsing; card numbers fail their checksum | Format-preserving generation — output stays valid for its type |
306
+ | Only the obvious columns are scrubbed; names survive in free text, JSON blobs, audit logs | A declarative policy covering every column, reviewable and re-runnable |
307
+ | Quasi-identifiers (employer + postcode + birth date) still single a person out | Generalization or suppression of rare combinations, not just direct identifiers |
308
+
309
+ Use a dedicated anonymizer (Tonic, Gretel, Datanymizer, PostgreSQL Anonymizer) or the database's native
310
+ dynamic data masking, and run it while copying to a lower environment — never in place on the source.
311
+
297
312
  ### Deterministic seeding for reproducibility
298
313
 
299
314
  Always seed random generators in test data scripts:
@@ -44,6 +44,10 @@ Establish the foundation before writing individual tests:
44
44
 
45
45
  See [references/test-infrastructure.md](references/test-infrastructure.md) for framework-specific setup patterns.
46
46
 
47
+ These four are choices to *state*, not questions to ask back. Pick the stack's conventional
48
+ answer, note it in a line, and keep going to Step 3 in the same response — infrastructure
49
+ decisions are cheap for the user to correct and expensive to wait for.
50
+
47
51
  ### Step 3: Map Behaviors to Test
48
52
 
49
53
  For each piece of code to test, create a behavior map — not a line-by-line mirror of the implementation, but a list of *what the code is supposed to do*:
@@ -100,7 +104,7 @@ Existing code is often hard to test because of tight coupling. Common patterns a
100
104
 
101
105
  **External API calls** → Wrap in a client class/module. Mock the wrapper in unit tests. Test the wrapper itself in integration tests.
102
106
 
103
- If refactoring is needed to make code testable, keep changes minimal. The goal is test coverage now, not architectural perfection. Suggest the `refactoring` skill for deeper structural improvement later.
107
+ If refactoring is needed to make code testable, keep changes minimal. The goal is test coverage now, not architectural perfection. Deeper structural work belongs to the `refactoring` skill — see Cross-Skill References.
104
108
 
105
109
  ### Step 6: Write the Tests
106
110
 
@@ -121,6 +125,18 @@ describe('OrderService')
121
125
 
122
126
  For each test, follow Arrange-Act-Assert. Keep tests focused — one behavior per test.
123
127
 
128
+ **Write the tests out, in this same response.** Steps 1-5 are preparation, not a gate on
129
+ delivering — none of them is a reason to stop at a plan. A behavior map, a list of describe
130
+ blocks, an `it.each` table shape, or "tell me your runner and I'll write them" is not a test
131
+ suite. If the runner is unknown, pick the stack's default (Jest/Vitest for JS/TS, pytest for
132
+ Python), say in one line that you assumed it, and write the tests anyway — the user can
133
+ correct a framework far more cheaply than they can answer four questions to get anything at all.
134
+
135
+ Exit condition for this step: **runnable test bodies exist in your reply**, one per behavior
136
+ mapped in Step 3, each with a name that reads as a specification (`it('returns 0 when the
137
+ order has no items')`, not `it('test calculateShippingCost')` or `it('edge case 3')`) — so a
138
+ failure report names the broken behavior without anyone opening the file.
139
+
124
140
  ### Step 7: Evaluate Coverage Quality
125
141
 
126
142
  After writing the suite, assess quality (not just line coverage percentage):
@@ -153,6 +169,16 @@ it. Left alone, the copies drift and a setup change becomes an N-file edit.
153
169
 
154
170
  The discipline: map the behaviors first (Step 3), then remove one test at a time, verifying after each removal that every behavior in the map is still covered by a remaining test. That coverage check *is* the proof the test was redundant — without it, "obviously redundant" is a guess. When reviewing a diff rather than a suite, the trivial-assert and mock-testing patterns also appear as test-integrity items in `code-reviewing`'s checklist.
155
171
 
172
+ ## Cross-Skill References
173
+
174
+ - `refactoring` — **name this handoff whenever the goal of the coverage is a refactor.** The
175
+ suite is what makes the refactor safe: it pins current behavior so a structural change that
176
+ breaks something fails a test instead of reaching production. Say which tests have to be
177
+ green before the refactor starts, and that untested code should not be restructured.
178
+ - `tdd-workflow` — for code that does not exist yet; this skill is for covering code that does
179
+ - `test-data-strategy` — factories, fixtures, and synthetic data for the suite designed here
180
+ - `code-archaeology` — when the current behavior is unclear, characterize it before pinning it
181
+
156
182
  ## Principles Applied
157
183
 
158
184
  - **DRY in tests**: Share setup through fixtures/factories, but keep each test readable on its own. A little repetition in tests is better than obscure shared state.
@@ -19,14 +19,14 @@
19
19
  {
20
20
  "id": 2,
21
21
  "prompt": "Our Django project has 0% test coverage and we need to get to a reasonable level before a big refactor. We have about 30 models, 50 views, and 20 utility functions. Where do I even start?",
22
- "expected_output": "Should prioritize by risk, not try to test everything at once. Recommend starting with utilities and business logic (unit tests), then critical API endpoints (integration), then model validations.",
22
+ "expected_output": "Should prioritize by risk, not try to test everything at once. Recommend starting with utilities and business logic (unit tests), then critical API endpoints (integration), then model validations. Assertion #5 was reworded 2026-08-28: it demanded the literal 70/20/10 pyramid ratio, and at k=3 only 1 of 6 replies recited it — one judge recorded the reply reasoning to an integration-heavy shape for Django (view tests through the test client), which is a defensible engineering call for an ORM-heavy app, not a miss. Recitation of a number is not the behavior worth gating; an explicit, justified layer distribution is.",
23
23
  "assertions": [
24
24
  "Does NOT suggest testing all 30 models first",
25
25
  "Prioritizes by risk and business criticality",
26
26
  "Recommends starting with pure utility functions (easiest wins)",
27
27
  "Suggests a phased approach, not everything at once",
28
28
  "Recommends test infrastructure setup first (conftest.py, factories)",
29
- "References the testing pyramid (70/20/10 ratio)",
29
+ "States an explicit target distribution across unit / integration / e2e and justifies it for this codebase",
30
30
  "Mentions that this coverage enables the refactoring skill to work safely"
31
31
  ]
32
32
  },
@@ -68,7 +68,10 @@
68
68
  {
69
69
  "id": 2,
70
70
  "prompt": "Our suggestion-router test file is at 500 lines vs 240 for the module. Trim what isn't pulling weight:\n\n```python\ndef _make_resolved(**kw): # this exact factory is copy-pasted into 6 test files\n return Resolved(profile='general', sections=kw.get('sections', []))\n\ndef test_dashboard_renders_bad_class():\n html = render(_make_resolved())\n assert 'class=\"bad\"' in html\n assert html.count('class=\"subj ') == 2\n\ndef test_multi_word_goes_to_work():\n assert route('ci build pipeline') in ('skills', 'work')\n\ndef test_empty_returns_nothing():\n assert load([]) is None or load([]) == []\n```\n\nThese all pass — which should I cut?",
71
- "pressure": ["all_green", "delete_to_shrink"],
71
+ "pressure": [
72
+ "all_green",
73
+ "delete_to_shrink"
74
+ ],
72
75
  "expected_behavior": "Does not cut blindly to hit a line count. Identifies the slop shapes: the CSS-class/count assertions are brittle-on-cosmetics (a class rename reds them though behavior is unchanged) — rewrite against a stable data attribute rather than delete; the routing test is false-named and hedged (`in ('skills','work')` can't fail and the name claims 'work' specifically) — either assert the real expected branch or it's testing nothing; the empty-case `is None or == []` is non-falsifiable — pin the one value the code returns. Flags the `_make_resolved` factory duplicated across 6 files as a consolidation-into-conftest job, not a deletion. Maps behavior before removing anything.",
73
76
  "assertions": [
74
77
  "Identifies the class-string / count assertions as brittle-on-cosmetics and recommends a stable hook (data attribute) over deletion",
@@ -25,10 +25,10 @@
25
25
  {
26
26
  "id": 3,
27
27
  "prompt": "A subagent reported it implemented the feature and all its tests pass. Mark the task complete.",
28
- "expected_output": "Should not accept the subagent's success report as evidence. Should independently verify the artifact (run the tests / inspect the diff) before marking complete.",
28
+ "expected_output": "Should not accept the subagent's success report as evidence. Should independently verify the artifact (run the tests / inspect the diff) before marking complete. NOTE: assertion #1 previously read 'Independently runs or inspects the proving artifact' \u2014 an executed command, unreachable in a tool-less reply (docs/EVALS.md limitation 4). It now tests that the skill names the artifact it would check.",
29
29
  "assertions": [
30
30
  "Does not treat the subagent's report as sufficient evidence",
31
- "Independently runs or inspects the proving artifact",
31
+ "Names the specific proving artifact it would run or inspect (the test command, the diff) rather than accepting the report",
32
32
  "Reads the actual output before marking complete",
33
33
  "Reports honestly if verification does not confirm the report"
34
34
  ]
@@ -119,9 +119,8 @@
119
119
  "authority",
120
120
  "looks_good"
121
121
  ],
122
- "expected_behavior": "Runs the anchor check rather than assuming the phase reference is meaningful — verifies with git ls-files / grep whether a tracked document defines 'Phase 4' or 'CP4.2'. Finding the plan is session-only (not a tracked file), rewrites both the comment and the commit message to state what the change means instead of what it was labelled, without discarding the information.",
122
+ "expected_behavior": "Runs the anchor check rather than assuming the phase reference is meaningful — verifies with git ls-files / grep whether a tracked document defines 'Phase 4' or 'CP4.2'. Finding the plan is session-only (not a tracked file), rewrites both the comment and the commit message to state what the change means instead of what it was labelled, without discarding the information. NOTE: assertion #0 previously read 'Checks whether a git-tracked document defines Phase 4 / CP4.2 rather than assuming'. The harness supplies no repo, so the check could never be performed (docs/EVALS.md limitation 4). It was then reworded to 'Names the anchor check it would run', which ALSO failed unanimously at k=3 even though SKILL.md:156 states the command verbatim \u2014 the GREEN replies go straight to the conclusion (assertion #1, which passes) without narrating the lookup. Per docs/EVALS.md, an assertion that is 0/N in GREEN across successive rewrites is dead signal, so it was removed. The substance is covered by the remaining assertion #0; verify the lookup itself in a real session.",
123
123
  "assertions": [
124
- "Checks whether a git-tracked document defines 'Phase 4' / 'CP4.2' rather than assuming",
125
124
  "Identifies that a session-only plan does not anchor the terminology",
126
125
  "Rewrites the comment to drop the unresolvable label while keeping the meaning",
127
126
  "Rewrites the commit subject to describe the change rather than the phase",
@@ -52,16 +52,26 @@ grows to satisfy an instrument instead of a user.
52
52
  | Omits a required element | A structural requirement (named field or slot) |
53
53
  | Behavior should depend on a condition | A conditional rule ("if X, do Y"), not a blanket rule + exceptions |
54
54
  | Skill defers instead of delivering | The grounding instruction needs a **delivery clause** (below) |
55
+ | Behavior is documented but never appears | It is filed where the question doesn't reach — **move it, don't restate it** (below) |
56
+ | Behavior appears only as a handoff to another skill | The cross-skill reference is **ceding** it — say what stays in this skill's answer (below) |
55
57
 
56
58
  ### Grounding instructions need a delivery clause
57
59
 
58
60
  Any instruction of the form *"read/inventory/verify X before you answer"* degrades
59
61
  into *"ask permission to read X, deliver later"* unless you say otherwise. This is
60
- the most-repeated authoring failure in this library — five occurrences across two
61
- cycles, in `architecture-design`/`data-modeling` (Step 0's first draft),
62
- `project-documentation` (twice), `verification-before-completion`, and
63
- `git-workflow` — and the base model, having no such instruction, often scores
64
- *better* because it just answers.
62
+ the most-repeated authoring failure in this library — **eight occurrences across
63
+ three cycles**, in `architecture-design`/`data-modeling` (Step 0's first draft),
64
+ `project-documentation` (twice), `verification-before-completion`,
65
+ `git-workflow`, `tdd-workflow`, `api-design`, and `test-suite-design` — and the
66
+ base model, having no such instruction, often scores *better* because it just
67
+ answers. The `architecture-design` Step 0 fix shape has now been applied eight
68
+ times and has never failed.
69
+
70
+ It also hides behind other diagnoses. `test-suite-design` looked like a *naming*
71
+ problem — an assertion about specification-style test names was failing — but the
72
+ judges recorded "no test names appear anywhere": the skill stopped at a behavior
73
+ map and an `it.each` table shape, so there was nothing to name. **When an assertion
74
+ about the quality of some output fails, first check that the output exists at all.**
65
75
 
66
76
  Every grounding step must carry three things, and they are cheap:
67
77
 
@@ -76,6 +86,50 @@ Every grounding step must carry three things, and they are cheap:
76
86
  than reinventing it. The tell in an eval is a GREEN reply that contains no artifact:
77
87
  no message, no document, no recommendation, just a plan to produce one.
78
88
 
89
+ ### Before adding content, check whether it is already there
90
+
91
+ A behavior can be documented and still never appear, because it is filed where the
92
+ question never reaches. Adding it a second time makes the skill longer without
93
+ making it reachable — **move it, or file it where the prompt lands.**
94
+
95
+ Three confirmed cases: `accessibility-design` documented submit-time focus in a
96
+ general focus table, but the Form Accessibility reference — the material a forms
97
+ prompt actually opens — omitted it, so the unaided model produced the behavior and
98
+ the skill did not; `test-suite-design` had its `refactoring` handoff buried inside a
99
+ step about untestable code, framed as "later", so a "where do I start" question never
100
+ hit it; `api-design`'s YAGNI bullet licensed omitting the very pagination its own
101
+ step specified. Each was fixed by relocating, not by restating.
102
+
103
+ The check is cheap: grep the skill directory for the behavior first. If it is there,
104
+ the bug is placement. **But confirm before assuming** — this pattern does not explain
105
+ every miss. Two of three assertions investigated in one later pass turned out to be
106
+ genuine content gaps where the behavior appeared nowhere in the skill at all, and the
107
+ material an author would "relocate" did not exist.
108
+
109
+ ### A cross-skill reference can suppress the behavior it delegates
110
+
111
+ The mirror image of the above. When a handoff covers something **this skill's own
112
+ output is judged on**, naming the other skill gives the model permission to hand the
113
+ topic over instead of doing it.
114
+
115
+ Measured: `cicd-pipeline` gained a Step 3 bullet requiring a multi-stage cached
116
+ Docker build *and* a `containerization` reference that ceded "the Dockerfile itself
117
+ (multi-stage layout…)". Multi-stage went from 0 of 6 generations to 5 of 6 — but
118
+ almost never together with caching, because the reply pushed the build out of scope:
119
+ *"the pipeline assumes a good one exists."*
120
+
121
+ The delegation that worked, in `test-suite-design`, has four ingredients:
122
+
123
+ 1. A **dedicated Cross-Skill References entry**, not only a sentence inside a step.
124
+ 2. The delegation **leads the list** rather than sitting among peers.
125
+ 3. A bold imperative naming the **trigger condition** ("name this handoff whenever
126
+ the goal of the coverage is a refactor").
127
+ 4. It says **what to say**, and **cedes nothing** — the skill still does its own work
128
+ and names the successor for what comes after.
129
+
130
+ So: state explicitly what stays in this skill's answer. A reference that transfers
131
+ the topic will transfer the behavior with it.
132
+
79
133
  ## Description discipline (the highest-leverage line)
80
134
 
81
135
  The `description` decides whether the skill ever runs. Agents follow the