@rune-kit/rune 2.10.0 → 2.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (205) hide show
  1. package/LICENSE +21 -21
  2. package/README.md +8 -6
  3. package/commands/rune.md +168 -168
  4. package/contexts/dev.md +34 -34
  5. package/contexts/research.md +43 -43
  6. package/contexts/review.md +55 -55
  7. package/extensions/ai-ml/PACK.md +88 -88
  8. package/extensions/ai-ml/skills/ai-agents.md +172 -172
  9. package/extensions/ai-ml/skills/code-sandbox.md +187 -187
  10. package/extensions/ai-ml/skills/deep-research.md +146 -146
  11. package/extensions/ai-ml/skills/embedding-search.md +66 -66
  12. package/extensions/ai-ml/skills/fine-tuning-guide.md +74 -74
  13. package/extensions/ai-ml/skills/llm-architect.md +125 -125
  14. package/extensions/ai-ml/skills/llm-integration.md +64 -64
  15. package/extensions/ai-ml/skills/prompt-patterns.md +72 -72
  16. package/extensions/ai-ml/skills/rag-patterns.md +66 -66
  17. package/extensions/ai-ml/skills/web-extraction.md +114 -114
  18. package/extensions/analytics/PACK.md +92 -92
  19. package/extensions/analytics/skills/ab-testing.md +72 -72
  20. package/extensions/analytics/skills/dashboard-patterns.md +83 -83
  21. package/extensions/analytics/skills/data-validation.md +68 -68
  22. package/extensions/analytics/skills/funnel-analysis.md +81 -81
  23. package/extensions/analytics/skills/sql-patterns.md +57 -57
  24. package/extensions/analytics/skills/statistical-analysis.md +79 -79
  25. package/extensions/analytics/skills/tracking-setup.md +71 -71
  26. package/extensions/backend/PACK.md +104 -104
  27. package/extensions/backend/skills/api-patterns.md +84 -84
  28. package/extensions/backend/skills/async-pipeline.md +193 -193
  29. package/extensions/backend/skills/auth-patterns.md +97 -97
  30. package/extensions/backend/skills/background-jobs.md +133 -133
  31. package/extensions/backend/skills/caching-patterns.md +108 -108
  32. package/extensions/backend/skills/cli-generation.md +133 -133
  33. package/extensions/backend/skills/database-patterns.md +87 -87
  34. package/extensions/backend/skills/middleware-patterns.md +104 -104
  35. package/extensions/chrome-ext/PACK.md +93 -93
  36. package/extensions/chrome-ext/skills/cws-preflight.md +143 -143
  37. package/extensions/chrome-ext/skills/cws-publish.md +104 -104
  38. package/extensions/chrome-ext/skills/ext-ai-integration.md +251 -251
  39. package/extensions/chrome-ext/skills/ext-messaging.md +139 -139
  40. package/extensions/chrome-ext/skills/ext-storage.md +133 -133
  41. package/extensions/chrome-ext/skills/mv3-scaffold.md +164 -164
  42. package/extensions/content/PACK.md +96 -96
  43. package/extensions/content/skills/blog-patterns.md +88 -88
  44. package/extensions/content/skills/cms-integration.md +131 -131
  45. package/extensions/content/skills/content-scoring.md +107 -107
  46. package/extensions/content/skills/i18n.md +83 -83
  47. package/extensions/content/skills/mdx-authoring.md +137 -137
  48. package/extensions/content/skills/reference.md +1014 -1014
  49. package/extensions/content/skills/seo-patterns.md +67 -67
  50. package/extensions/content/skills/video-repurpose.md +153 -153
  51. package/extensions/devops/PACK.md +101 -101
  52. package/extensions/devops/skills/chaos-testing.md +67 -67
  53. package/extensions/devops/skills/ci-cd.md +75 -75
  54. package/extensions/devops/skills/docker.md +58 -58
  55. package/extensions/devops/skills/edge-serverless.md +163 -163
  56. package/extensions/devops/skills/infra-as-code.md +158 -158
  57. package/extensions/devops/skills/kubernetes.md +110 -110
  58. package/extensions/devops/skills/monitoring.md +57 -57
  59. package/extensions/devops/skills/server-setup.md +64 -64
  60. package/extensions/devops/skills/ssl-domain.md +42 -42
  61. package/extensions/ecommerce/PACK.md +116 -116
  62. package/extensions/ecommerce/skills/cart-system.md +79 -79
  63. package/extensions/ecommerce/skills/inventory-mgmt.md +102 -102
  64. package/extensions/ecommerce/skills/order-management.md +126 -126
  65. package/extensions/ecommerce/skills/payment-integration.md +472 -472
  66. package/extensions/ecommerce/skills/shopify-dev.md +69 -69
  67. package/extensions/ecommerce/skills/subscription-billing.md +93 -93
  68. package/extensions/ecommerce/skills/tax-compliance.md +117 -117
  69. package/extensions/gamedev/PACK.md +142 -142
  70. package/extensions/gamedev/skills/asset-pipeline.md +74 -74
  71. package/extensions/gamedev/skills/audio-system.md +129 -129
  72. package/extensions/gamedev/skills/camera-system.md +87 -87
  73. package/extensions/gamedev/skills/ecs.md +98 -98
  74. package/extensions/gamedev/skills/game-loops.md +72 -72
  75. package/extensions/gamedev/skills/input-system.md +199 -199
  76. package/extensions/gamedev/skills/multiplayer.md +180 -180
  77. package/extensions/gamedev/skills/particles.md +105 -105
  78. package/extensions/gamedev/skills/physics-engine.md +89 -89
  79. package/extensions/gamedev/skills/scene-management.md +146 -146
  80. package/extensions/gamedev/skills/threejs-patterns.md +90 -90
  81. package/extensions/gamedev/skills/webgl.md +71 -71
  82. package/extensions/mobile/PACK.md +106 -106
  83. package/extensions/mobile/skills/app-store-connect.md +152 -152
  84. package/extensions/mobile/skills/app-store-prep.md +66 -66
  85. package/extensions/mobile/skills/deep-linking.md +109 -109
  86. package/extensions/mobile/skills/flutter.md +60 -60
  87. package/extensions/mobile/skills/ios-build-pipeline.md +142 -142
  88. package/extensions/mobile/skills/native-bridge.md +66 -66
  89. package/extensions/mobile/skills/ota-updates.md +97 -97
  90. package/extensions/mobile/skills/push-notifications.md +111 -111
  91. package/extensions/mobile/skills/react-native.md +82 -82
  92. package/extensions/saas/PACK.md +116 -116
  93. package/extensions/saas/skills/billing-integration.md +200 -200
  94. package/extensions/saas/skills/feature-flags.md +130 -130
  95. package/extensions/saas/skills/multi-tenant.md +103 -103
  96. package/extensions/saas/skills/onboarding-flow.md +139 -139
  97. package/extensions/saas/skills/subscription-flow.md +95 -95
  98. package/extensions/saas/skills/team-management.md +144 -144
  99. package/extensions/security/PACK.md +99 -99
  100. package/extensions/security/skills/api-security.md +140 -140
  101. package/extensions/security/skills/compliance.md +68 -68
  102. package/extensions/security/skills/owasp-audit.md +64 -64
  103. package/extensions/security/skills/pentest-patterns.md +77 -77
  104. package/extensions/security/skills/secret-mgmt.md +65 -65
  105. package/extensions/security/skills/supply-chain.md +65 -65
  106. package/extensions/trading/PACK.md +80 -80
  107. package/extensions/trading/skills/chart-components.md +55 -55
  108. package/extensions/trading/skills/experiment-loop.md +125 -125
  109. package/extensions/trading/skills/fintech-patterns.md +47 -47
  110. package/extensions/trading/skills/indicator-library.md +58 -58
  111. package/extensions/trading/skills/quant-analysis.md +111 -111
  112. package/extensions/trading/skills/realtime-data.md +58 -58
  113. package/extensions/trading/skills/trade-logic.md +104 -104
  114. package/extensions/ui/PACK.md +130 -130
  115. package/extensions/ui/skills/a11y-audit.md +91 -91
  116. package/extensions/ui/skills/animation-patterns.md +127 -127
  117. package/extensions/ui/skills/component-patterns.md +100 -100
  118. package/extensions/ui/skills/design-decision.md +108 -108
  119. package/extensions/ui/skills/design-system.md +68 -68
  120. package/extensions/ui/skills/landing-patterns.md +155 -155
  121. package/extensions/ui/skills/palette-picker.md +173 -173
  122. package/extensions/ui/skills/react-health.md +90 -90
  123. package/extensions/ui/skills/type-system.md +125 -125
  124. package/extensions/ui/skills/web-vitals.md +153 -153
  125. package/extensions/zalo/PACK.md +145 -145
  126. package/extensions/zalo/skills/zalo-oa-mcp.md +317 -317
  127. package/extensions/zalo/skills/zalo-oa-messaging.md +429 -429
  128. package/extensions/zalo/skills/zalo-oa-setup.md +236 -236
  129. package/extensions/zalo/skills/zalo-oa-webhook.md +189 -189
  130. package/extensions/zalo/skills/zalo-personal-messaging.md +194 -194
  131. package/extensions/zalo/skills/zalo-personal-setup.md +153 -153
  132. package/extensions/zalo/skills/zalo-rate-guard.md +219 -219
  133. package/hooks/auto-format/index.cjs +48 -48
  134. package/hooks/hooks.json +111 -111
  135. package/hooks/post-session-reflect/index.cjs +189 -189
  136. package/hooks/pre-compact/index.cjs +95 -95
  137. package/hooks/run-hook.cmd +1 -1
  138. package/hooks/secrets-scan/index.cjs +100 -100
  139. package/hooks/session-start/index.cjs +71 -71
  140. package/hooks/typecheck/index.cjs +65 -65
  141. package/package.json +63 -63
  142. package/references/ui-pro-max-data/LICENSE-UI-PRO-MAX +21 -21
  143. package/references/ui-pro-max-data/charts.csv +26 -26
  144. package/references/ui-pro-max-data/colors.csv +161 -161
  145. package/references/ui-pro-max-data/styles.csv +68 -68
  146. package/references/ui-pro-max-data/typography.csv +74 -74
  147. package/references/ui-pro-max-data/ui-reasoning.csv +162 -162
  148. package/references/ui-pro-max-data/ux-guidelines.csv +99 -99
  149. package/skills/adversary/SKILL.md +283 -283
  150. package/skills/asset-creator/SKILL.md +157 -157
  151. package/skills/audit/SKILL.md +147 -2
  152. package/skills/autopsy/SKILL.md +335 -335
  153. package/skills/brainstorm/SKILL.md +342 -342
  154. package/skills/browser-pilot/SKILL.md +168 -168
  155. package/skills/constraint-check/SKILL.md +165 -165
  156. package/skills/context-engine/SKILL.md +404 -404
  157. package/skills/cook/SKILL.md +917 -863
  158. package/skills/db/SKILL.md +273 -273
  159. package/skills/debug/SKILL.md +465 -465
  160. package/skills/dependency-doctor/SKILL.md +265 -235
  161. package/skills/deploy/SKILL.md +274 -231
  162. package/skills/design/DESIGN-REFERENCE.md +365 -365
  163. package/skills/design/SKILL.md +589 -589
  164. package/skills/doc-processor/SKILL.md +254 -254
  165. package/skills/docs/SKILL.md +374 -374
  166. package/skills/docs-seeker/SKILL.md +177 -177
  167. package/skills/fix/SKILL.md +330 -330
  168. package/skills/git/SKILL.md +339 -339
  169. package/skills/hallucination-guard/SKILL.md +219 -219
  170. package/skills/incident/SKILL.md +254 -253
  171. package/skills/integrity-check/SKILL.md +169 -169
  172. package/skills/journal/SKILL.md +240 -240
  173. package/skills/launch/SKILL.md +344 -344
  174. package/skills/logic-guardian/SKILL.md +251 -251
  175. package/skills/marketing/SKILL.md +290 -289
  176. package/skills/mcp-builder/SKILL.md +425 -425
  177. package/skills/neural-memory/SKILL.md +362 -362
  178. package/skills/onboard/SKILL.md +404 -403
  179. package/skills/perf/SKILL.md +346 -346
  180. package/skills/plan/SKILL.md +433 -428
  181. package/skills/preflight/SKILL.md +415 -415
  182. package/skills/problem-solver/SKILL.md +380 -284
  183. package/skills/rescue/SKILL.md +474 -474
  184. package/skills/retro/SKILL.md +3 -1
  185. package/skills/review/SKILL.md +612 -588
  186. package/skills/review-intake/SKILL.md +249 -249
  187. package/skills/safeguard/SKILL.md +200 -200
  188. package/skills/sast/SKILL.md +190 -190
  189. package/skills/scaffold/SKILL.md +328 -287
  190. package/skills/scope-guard/SKILL.md +180 -180
  191. package/skills/scout/SKILL.md +263 -263
  192. package/skills/sentinel/SKILL.md +382 -381
  193. package/skills/sentinel-env/SKILL.md +254 -254
  194. package/skills/sequential-thinking/SKILL.md +234 -234
  195. package/skills/session-bridge/SKILL.md +543 -543
  196. package/skills/skill-forge/SKILL.md +581 -581
  197. package/skills/skill-router/SKILL.md +3 -0
  198. package/skills/surgeon/SKILL.md +215 -215
  199. package/skills/team/SKILL.md +556 -537
  200. package/skills/test/SKILL.md +614 -614
  201. package/skills/trend-scout/SKILL.md +145 -145
  202. package/skills/verification/SKILL.md +326 -326
  203. package/skills/video-creator/SKILL.md +201 -201
  204. package/skills/watchdog/SKILL.md +168 -168
  205. package/skills/worktree/SKILL.md +140 -140
@@ -1,614 +1,614 @@
1
- ---
2
- name: test
3
- description: "TDD test writer. Writes failing tests FIRST (red), then verifies they pass after implementation (green). Covers unit, integration, and e2e tests."
4
- metadata:
5
- author: runedev
6
- version: "1.2.0"
7
- layer: L2
8
- model: sonnet
9
- group: development
10
- tools: "Read, Write, Edit, Bash, Glob, Grep"
11
- emit: tests.passed, tests.failed
12
- listen: code.changed, db.migrated
13
- ---
14
-
15
- # test
16
-
17
- <HARD-GATE>
18
- Tests define the EXPECTED BEHAVIOR. They MUST be written BEFORE implementation code.
19
- If tests pass without implementation → the tests are wrong. Rewrite them.
20
- The only exception: when retrofitting tests for existing untested code.
21
-
22
- THE IRON LAW: Write code before test? DELETE IT. Start over.
23
- - Do NOT keep it as "reference"
24
- - Do NOT "adapt" it while writing tests
25
- - Do NOT look at it to "inform" test design
26
- - Delete means delete. `git checkout -- <file>` or remove the changes entirely.
27
- This is not negotiable. This is not optional. "But I already wrote it" is a sunk cost fallacy.
28
-
29
- ROLE BOUNDARY: Test writes TEST FILES only. NEVER modify source/implementation files.
30
- - Do NOT "quickly fix" a broken import in source to make tests run
31
- - Do NOT refactor source code to be "more testable"
32
- - Do NOT add missing exports to source files
33
- - If source needs changes → hand off to `rune:fix`. Test's job ends at the test file.
34
- This separation ensures test never writes code biased toward passing its own tests.
35
- </HARD-GATE>
36
-
37
- ## Instructions
38
-
39
- ### Phase 1: Understand What to Test
40
-
41
- 1. Read the implementation plan or task description carefully
42
- 2. Use `Glob` to find existing test files: `**/*.test.*`, `**/*.spec.*`, `**/test_*`
43
- 3. Use `Read` on 2-3 existing test files to understand:
44
- - Test framework in use
45
- - File naming convention (e.g., `foo.test.ts` mirrors `foo.ts`)
46
- - Test directory structure (co-located vs `__tests__/` vs `tests/`)
47
- - Assertion style and patterns
48
- 4. Use `Glob` to find the source file(s) being tested
49
-
50
- ```
51
- TodoWrite: [
52
- { content: "Understand scope and find existing test patterns", status: "in_progress" },
53
- { content: "Detect test framework and conventions", status: "pending" },
54
- { content: "Write failing tests (RED phase)", status: "pending" },
55
- { content: "Run tests — verify they FAIL", status: "pending" },
56
- { content: "After implementation: verify tests PASS (GREEN phase)", status: "pending" }
57
- ]
58
- ```
59
-
60
- ### Phase 2: Detect Test Framework
61
-
62
- Use `Glob` to find config files and identify the framework:
63
-
64
- - `jest.config.*` or `"jest"` key in `package.json` → Jest
65
- - `vitest.config.*` or `"vitest"` key in `package.json` → Vitest
66
- - `pytest.ini`, `[tool.pytest.ini_options]` in `pyproject.toml` → pytest
67
- - **Async check**: If pytest detected AND source files contain `async def`:
68
- - Check if `pytest-asyncio` is in dependencies (`pyproject.toml [project.dependencies]` or `[project.optional-dependencies]`)
69
- - Check if `asyncio_mode` is set in `[tool.pytest.ini_options]` (values: `auto`, `strict`, or absent)
70
- - If async code exists but no `asyncio_mode` configured → **WARN**: "pytest-asyncio not configured. Async tests may silently pass without executing async code. Recommend adding `asyncio_mode = \"auto\"` to `[tool.pytest.ini_options]` in pyproject.toml."
71
- - `Cargo.toml` with `#[cfg(test)]` pattern → built-in `cargo test`
72
- - `*_test.go` files present → built-in `go test`
73
- - `cypress.config.*` → Cypress (E2E)
74
- - `playwright.config.*` → Playwright (E2E)
75
-
76
- **Verification gate**: Framework identified before writing any test code.
77
-
78
- ### Phase 3: Write Failing Tests
79
-
80
- Use `Write` to create test files following the detected conventions:
81
-
82
- 1. Mirror source file location: if source is `src/auth/login.ts`, test is `src/auth/login.test.ts`
83
- 2. Structure tests with clear `describe` / `it` blocks (or language equivalent):
84
- - `describe('Feature name')`
85
- - `it('should [expected behavior] when [condition]')`
86
- 3. Cover all three categories:
87
- - **Happy path**: valid inputs, expected success output
88
- - **Edge cases**: empty input, boundary values, large input
89
- - **Error cases**: invalid input, missing data, network failure simulation
90
-
91
- 4. Use proper assertions. Do NOT use implementation details — test behavior:
92
- - Jest/Vitest: `expect(result).toBe(expected)`
93
- - pytest: `assert result == expected`
94
- - Rust: `assert_eq!(result, expected)`
95
- - Go: `if result != expected { t.Errorf(...) }`
96
-
97
- 5. For async code: use `async/await` or pytest `@pytest.mark.asyncio`
98
-
99
- #### Python Async Tests (pytest-asyncio)
100
-
101
- When writing tests for async Python code:
102
-
103
- 1. **Verify setup before writing tests**:
104
- - Confirm `pytest-asyncio` is in project dependencies
105
- - Confirm `asyncio_mode` is set in `pyproject.toml` `[tool.pytest.ini_options]` (recommend `"auto"`)
106
- - If neither is configured, warn the caller and suggest setup before proceeding
107
-
108
- 2. **Writing async test functions**:
109
- - With `asyncio_mode = "auto"`: just write `async def test_something():` — no decorator needed
110
- - With `asyncio_mode = "strict"`: every async test needs `@pytest.mark.asyncio`
111
- - Without asyncio_mode set: always use `@pytest.mark.asyncio` decorator explicitly
112
-
113
- 3. **Async fixtures**:
114
- - Use `@pytest_asyncio.fixture` (NOT `@pytest.fixture`) for async setup/teardown
115
- - Scope rules: async fixtures default to `function` scope — use `scope="session"` carefully with async
116
-
117
- 4. **Common pitfalls**:
118
- - Tests that `pass` without `await` — they run but don't execute the async path
119
- - Missing `pytest-asyncio` makes `async def test_*` silently pass as empty coroutines
120
- - Mixing sync and async fixtures can cause event loop errors
121
-
122
- ### Phase 4: Run Tests — Verify They FAIL (RED)
123
-
124
- Use `Bash` to run ONLY the newly created test files (not full suite):
125
-
126
- - **Jest**: `npx jest path/to/test.ts --no-coverage`
127
- - **Vitest**: `npx vitest run path/to/test.ts`
128
- - **pytest**: `pytest path/to/test_file.py -v` (if async tests and no `asyncio_mode` in config: add `--asyncio-mode=auto`)
129
- - **Rust**: `cargo test test_module_name`
130
- - **Go**: `go test ./path/to/package/... -run TestFunctionName`
131
-
132
- **Hard gate**: ALL new tests MUST fail at this point.
133
-
134
- - If ANY test passes before implementation exists → that test is not testing real behavior. Rewrite it to be stricter.
135
- - If tests fail with import/syntax errors (not assertion errors) → fix the test code, re-run
136
-
137
- ### Phase 5: After Implementation — Verify Tests PASS (GREEN)
138
-
139
- After `rune:fix` writes implementation code, run the same test command again:
140
-
141
- 1. ALL tests in the new test files MUST pass
142
- 2. Run the full test suite with `Bash` to check for regressions:
143
- - `npm test`, `pytest`, `cargo test`, `go test ./...`
144
- 3. If any test fails: report clearly which test, what was expected, what was received
145
- 4. If an existing test now fails (regression): escalate to `rune:debug`
146
-
147
- **Verification gate**: 100% of new tests pass AND 0 regressions in existing tests.
148
-
149
- ### Phase 6: Coverage Check
150
-
151
- After GREEN phase, call `verification` to check coverage threshold (80% minimum):
152
-
153
- - If coverage drops below 80%: identify uncovered lines, write additional tests
154
- - Report coverage gaps with file:line references
155
-
156
- ### Phase 6.5: Diff-Aware Mode (optional)
157
-
158
- When invoked with `mode: "diff-aware"` or by `cook` after implementation:
159
-
160
- 1. Run `git diff main --name-only` to get changed files
161
- 2. For each changed file, trace its **blast radius**: what imports it? what routes does it serve? what components render it?
162
- 3. Map changed files → affected routes/endpoints/pages
163
- 4. Prioritize tests: files with most downstream dependents get tested first
164
- 5. Generate targeted test commands that cover ONLY affected paths — skip unchanged modules
165
-
166
- This mode is valuable for large codebases where running the full suite is slow. It answers: "what could this diff have broken?"
167
-
168
- ```
169
- Input: git diff main --name-only
170
- Output: Prioritized test plan targeting only affected paths
171
- ```
172
-
173
- ## Test Types — 4-Layer Methodology
174
-
175
- Tests are organized in 4 layers. Each layer catches a different failure class. Higher layers are slower but catch integration issues lower layers miss.
176
-
177
- | Layer | Type | What It Catches | Framework | Speed |
178
- |-------|------|-----------------|-----------|-------|
179
- | L1 | **Unit** | Logic bugs, boundary violations, pure function errors | jest/vitest/pytest/cargo test | Fast |
180
- | L2 | **Integration** | API contract breaks, DB query errors, service interaction failures | supertest/httpx/reqwest | Medium |
181
- | L3 | **True Backend** | Real tool/service output correctness (not just exit 0) | Same + real software invocation | Medium-Slow |
182
- | L4 | **E2E / Subprocess** | Full workflow from user/agent perspective, installed app works | Playwright/Cypress/subprocess | Slow |
183
-
184
- **Layer rules:**
185
- - **L1 (Unit)**: Synthetic data, no external deps. Every function tested in isolation. Fast, deterministic, CI-friendly
186
- - **L2 (Integration)**: Tests service boundaries — API endpoints, DB operations, message queues. May need test DB or mock server
187
- - **L3 (True Backend)**: **Invokes the REAL tool/service** and verifies output programmatically. No graceful degradation — if the dependency isn't installed, tests FAIL (not skip). Verify: magic bytes, file size > 0, content structure. Print artifact paths for manual inspection
188
- - **L4 (E2E/Subprocess)**: Tests the installed command/app via subprocess or browser automation. Full user workflow: input → process → output → verify
189
-
190
- **"No graceful degradation" rule** (L3/L4): Hard dependencies MUST be installed. Tests MUST NOT skip or produce fake results when the dependency is missing. A silently skipping test is worse than a loudly failing test.
191
-
192
- Additional modes:
193
-
194
- | Type | When | Speed |
195
- |------|------|-------|
196
- | Regression | After bug fixes | Fast |
197
- | Diff-aware | After implementation, large codebases (Phase 6.5) | Fast (targeted) |
198
-
199
- ## TEST.md — Test Plan + Results Document
200
-
201
- For non-trivial features (3+ test files or 20+ test cases), create a `TEST.md` in the test directory. This is BOTH a planning doc (written BEFORE tests) and results doc (appended AFTER tests pass).
202
-
203
- ### Before writing tests — write the plan:
204
- ```markdown
205
- # Test Plan: [Feature Name]
206
-
207
- ## Test Inventory
208
- - `test_core.py`: ~XX unit tests planned (L1)
209
- - `test_integration.py`: ~XX integration tests planned (L2)
210
- - `test_e2e.py`: ~XX E2E tests planned (L3/L4)
211
-
212
- ## Unit Test Plan (L1)
213
- | Module | Functions | Edge Cases | Est. Tests | Req IDs |
214
- |--------|-----------|------------|------------|---------|
215
- | `core/auth.py` | login, register, refresh | expired token, invalid creds, rate limit | 12 | REQ-001, REQ-003 |
216
-
217
- ## E2E Scenarios (L3/L4)
218
- | Workflow | Simulates | Operations | Verified | Req IDs |
219
- |----------|-----------|------------|----------|---------|
220
- | User signup | New user onboarding | register → verify → login | Token valid, profile created | REQ-005 |
221
-
222
- ## Realistic Workflow Scenarios
223
- - **[Name]**: [Step 1] → [Step 2] → verify [output properties]
224
- ```
225
-
226
- ### After tests pass — append results:
227
- ```markdown
228
- ## Test Results
229
- [Paste full `pytest -v --tb=no` or `npm test` output]
230
-
231
- ## Summary
232
- - Total: XX | Passed: XX | Failed: 0
233
- - Execution time: X.Xs | Coverage: XX%
234
-
235
- ## Requirement Coverage
236
- | Req ID | Test File(s) | Status |
237
- |--------|-------------|--------|
238
- | REQ-001 | `test_auth.py::test_login` | ✅ Covered |
239
- | REQ-002 | — | ❌ Not covered |
240
-
241
- ## Gaps
242
- - [Areas not covered and why]
243
- ```
244
-
245
- **Why TEST.md**: Planning tests before code catches missing edge cases early. Appending results creates permanent evidence. One document = complete testing story.
246
-
247
- ## Skill Behavior Tests (Eval Scenarios)
248
-
249
- For testing SKILL.md behavior (not code), use **Eval Scenarios** — unit tests for skill files, not code files.
250
-
251
- ### Eval Scenario Format
252
-
253
- ```markdown
254
- ## Eval: E[NN] — [scenario name]
255
-
256
- ### Prompt
257
- [The exact situation/message an agent receives]
258
-
259
- ### Expected Reasoning
260
- [Step-by-step reasoning the agent SHOULD follow]
261
-
262
- ### Must Include
263
- - [Assertion 1: what the output MUST contain or do]
264
- - [Assertion 2]
265
-
266
- ### Must NOT
267
- - [Anti-pattern 1: what the output MUST NOT do]
268
- - [Anti-pattern 2]
269
-
270
- ### Category
271
- happy-path | adversarial | edge-case | jailbreak | credential-leak
272
- ```
273
-
274
- ### Eval Coverage Requirements
275
-
276
- A skill is **behavior-tested** when it has evals covering:
277
-
278
- | Category | Min Evals | Purpose |
279
- |----------|-----------|---------|
280
- | Happy path | 1 | Core workflow executes correctly |
281
- | Edge case | 1 | Empty input, missing context, unusual state |
282
- | Adversarial | 1 | Time pressure, sunk cost, authority pressure |
283
- | Jailbreak / injection | 1 | Prompt injection attempt, "ignore instructions" |
284
-
285
- **Minimum**: 4 evals per skill (1 per category). Security-critical skills (sentinel, safeguard): 8+ evals.
286
-
287
- ### Eval Storage
288
-
289
- Save eval files as `skills/<name>/evals.md`. Each eval is a numbered scenario (E01–E24 range). skill-forge Phase 7 checks for evals presence before ship.
290
-
291
-
292
- ## Error Recovery
293
-
294
- - If test framework not found: ask calling skill to specify, or check `package.json` `devDependencies`
295
- - If `Write` to test file fails: check if directory exists, create it first with `Bash mkdir -p`
296
- - If tests error on import (module not found): check that source file path is correct, adjust imports
297
- - If `Bash` test runner hangs beyond 120 seconds: kill and report as TIMEOUT
298
-
299
- ## Called By (inbound)
300
-
301
- - `cook` (L1): Phase 3 TEST — write tests first
302
- - `fix` (L2): verify fix passes tests
303
- - `review` (L2): untested edge case found → write test for it
304
- - `deploy` (L2): pre-deployment full test suite
305
- - `preflight` (L2): run targeted regression tests on affected code
306
- - `surgeon` (L2): verify refactored code
307
- - `launch` (L1): pre-deployment test suite
308
- - `safeguard` (L2): writing characterization tests for legacy code
309
- - `review-intake` (L2): write tests for issues identified during review intake
310
-
311
- ## Calls (outbound)
312
-
313
- - `verification` (L3): Phase 6 — coverage check (80% minimum threshold)
314
- - `browser-pilot` (L3): Phase 4 — e2e and visual testing for UI flows
315
- - `debug` (L2): Phase 5 — when existing test regresses unexpectedly
316
-
317
- ## Data Flow
318
-
319
- ### Feeds Into →
320
-
321
- - `cook` (L1): test results (pass/fail/coverage) → cook's Phase 5 quality gate evidence
322
- - `completion-gate` (L3): test runner stdout → evidence for "tests pass" claims
323
- - `fix` (L2): failing test output → fix's target (what to make green)
324
-
325
- ### Fed By ←
326
-
327
- - `plan` (L2): phase file test tasks → test's RED phase targets (what to test)
328
- - `review` (L2): untested edge cases found during review → new test targets
329
- - `fix` (L2): implemented code → test's GREEN phase verification target
330
-
331
- ### Feedback Loops ↻
332
-
333
- - `test` ↔ `fix`: test writes failing tests (RED) → fix implements to pass → test verifies (GREEN) → if new failures emerge, loop continues
334
- - `test` ↔ `debug`: test discovers regression → debug diagnoses root cause → test writes regression test to prevent recurrence
335
-
336
- ## Anti-Rationalization Table
337
-
338
- | Excuse | Reality |
339
- |---|---|
340
- | "Too simple to need tests first" | Simple code breaks. Test takes 30 seconds. Write it first. |
341
- | "I'll write tests after — same result" | Tests-after = "what does this do?" Tests-first = "what SHOULD this do?" Completely different. |
342
- | "I already wrote the code, let me just add tests" | Iron Law: delete the code. Start over with tests. Sunk cost is not an argument. |
343
- | "Tests after achieve the same goals" | They don't. Tests-after are biased by the implementation you just wrote. |
344
- | "It's about spirit not ritual" | Violating the letter IS violating the spirit. Write the test first. |
345
- | "I mentally tested it" | Mental testing is not testing. Run the command, show the output. |
346
- | "This is different because..." | It's not. Write the test first. |
347
-
348
- ## Advanced: Oracle-Injection E2E Testing
349
-
350
- For **data pipelines, AI workflows, and multi-stage processing** where comparing full output structures is impractical, use oracle injection:
351
-
352
- 1. **Generate a UUID oracle token**: `const oracle = crypto.randomUUID()`
353
- 2. **Inject into synthetic input**: embed the oracle in realistic test data that flows through the pipeline
354
- 3. **Run the full pipeline**: input → all stages → output
355
- 4. **Search for oracle in output**: if found → data flowed end-to-end correctly
356
-
357
- ```
358
- // Example: testing a document processing pipeline
359
- const oracle = "ORACLE-" + crypto.randomUUID();
360
- const testDoc = `Meeting notes: discussed ${oracle} integration timeline`;
361
- const result = await pipeline.process(testDoc);
362
- assert(result.output.includes(oracle), "Oracle not found — pipeline lost data");
363
- ```
364
-
365
- **When to use**: E2E tests for pipelines with 3+ stages, LLM-based processing, ETL workflows, or any system where output structure is complex/non-deterministic but data preservation is critical.
366
-
367
- **When NOT to use**: Unit tests, simple CRUD, or when exact output comparison is feasible.
368
-
369
-
370
- ## Spec→Test Traceability
371
-
372
- When a plan with acceptance criteria exists (`.rune/features/<name>/plan.md` or phase file), every criterion MUST map to at least one test case.
373
-
374
- ```
375
- Plan Acceptance Criteria → Test Case → Implementation
376
-
377
- AC-1: "User can reset password via email" → test_password_reset_sends_email()
378
- AC-2: "Rate limit: max 3 reset attempts/hour" → test_password_reset_rate_limit()
379
- AC-3: "Expired tokens rejected" → test_expired_reset_token_rejected()
380
- ```
381
-
382
- **Validation step** (after writing tests): Cross-check plan's acceptance criteria against test names. For each criterion:
383
- - Has test → OK
384
- - No test → flag as UNTESTED REQUIREMENT (more serious than uncovered lines)
385
-
386
- **Why this is stronger than coverage**: Coverage checks that lines were EXECUTED. Traceability checks that INTENT was VERIFIED. You can have 100% coverage but miss a requirement if the test doesn't assert the right behavior.
387
-
388
- **Skip if**: No plan exists (ad-hoc fix), or plan has no acceptance criteria section.
389
-
390
- ## Eval-Driven Development
391
-
392
- Define **capability evals** and **regression evals** BEFORE writing implementation code. Evals go beyond unit tests — they verify that the agent/system can handle the feature's intent, not just its mechanics.
393
-
394
- ### Two Eval Types
395
-
396
- | Type | Purpose | Pass Criteria | When |
397
- |------|---------|---------------|------|
398
- | **Capability eval** | Can the system do this new thing? | pass@k: ≥1 success in k attempts (k=3-5) | Before implementation |
399
- | **Regression eval** | Did we break existing behavior? | pass^k: ALL k attempts must pass | After implementation |
400
-
401
- **pass@k** (capability): At least 1 of k runs succeeds. Used for new features where some variance is acceptable. Threshold: ≥90% pass@3 for standard features, ≥95% pass@5 for critical paths.
402
-
403
- **pass^k** (regression): ALL k runs must pass. Used for existing behavior that must never break. If ANY run fails, it's a regression. Threshold: 100% pass^3.
404
-
405
- ### Eval File Format
406
-
407
- Store evals in `.rune/evals/<feature>.md`:
408
-
409
- ```markdown
410
- # Eval: <feature name>
411
-
412
- ## Capability Evals (pass@k)
413
- | ID | Description | k | Threshold | Status |
414
- |----|-------------|---|-----------|--------|
415
- | CAP-1 | [what the system should be able to do] | 3 | 90% | pending |
416
-
417
- ## Regression Evals (pass^k)
418
- | ID | Description | k | Status |
419
- |----|-------------|---|--------|
420
- | REG-1 | [existing behavior that must not break] | 3 | pending |
421
- ```
422
-
423
- ### Anti-Pattern: Eval Overfitting
424
-
425
- Do NOT overfit evals to specific prompts or known examples. Evals should test the **capability**, not the **exact input**.
426
-
427
- - BAD: `"When user says 'hello', respond with 'Hi there!'"` — tests exact string match
428
- - GOOD: `"When user greets, respond with a greeting"` — tests capability
429
-
430
- ### Integration with TDD
431
-
432
- 1. Write eval definitions (capability + regression) → `.rune/evals/<feature>.md`
433
- 2. Write unit/integration tests (RED phase) → test files
434
- 3. Implement feature (GREEN phase) → source files
435
- 4. Run evals to verify capability achieved + no regressions
436
- 5. Preflight checks eval results as part of quality gate
437
-
438
- ## Red Flags — STOP and Start Over
439
-
440
- If you catch yourself with ANY of these, delete implementation code and restart with tests:
441
-
442
- - Code exists before test file
443
- - "I already manually tested it"
444
- - "Tests after achieve the same purpose"
445
- - "It's about spirit not ritual"
446
- - "This is different because..."
447
- - "Let me just finish this, then add tests"
448
-
449
- **All of these mean: Delete code. Start over with TDD.**
450
-
451
- ## Constraints
452
-
453
- 1. MUST write tests BEFORE implementation code — if tests pass without implementation, they are wrong
454
- 2. MUST cover happy path + edge cases + error cases — not just happy path
455
- 3. MUST run tests to verify they FAIL before implementation exists (RED phase is mandatory)
456
- 4. MUST NOT write tests that test mock behavior instead of real code behavior
457
- 5. MUST achieve 80% coverage minimum — identify and fill gaps
458
- 6. MUST use the project's existing test framework and conventions — don't introduce a new one
459
- 7. MUST NOT say "tests pass" without showing actual test runner output
460
- 8. MUST delete implementation code written before tests — Iron Law, no exceptions
461
- 9. MUST show RED phase output (actual failure) — "I confirmed they fail" without output is REJECTED
462
- 10. MUST NOT modify source/implementation files — test writes test files ONLY, hand off source changes to rune:fix
463
-
464
- ## Mesh Gates
465
-
466
- | Gate | Requires | If Missing |
467
- |------|----------|------------|
468
- | RED Gate | All new tests FAIL before implementation | If any pass, rewrite stricter tests |
469
- | GREEN Gate | All tests PASS after implementation | Fix code, not tests |
470
- | Coverage Gate | 80%+ coverage verified via verification | Write additional tests for gaps |
471
-
472
- ## Output Format
473
-
474
- ```
475
- ## Test Report
476
- - **Framework**: [detected]
477
- - **Files Created**: [list of new test file paths]
478
- - **Tests Written**: [count]
479
- - **Status**: RED (failing as expected) | GREEN (all passing)
480
-
481
- ### Test Cases
482
- | Test | Status | Description |
483
- |------|--------|-------------|
484
- | `test_name` | FAIL/PASS | [what it tests] |
485
-
486
- ### Coverage
487
- - Lines: [X]% | Branches: [Y]%
488
- - Gaps: `path/to/file.ts:42-58` — uncovered branch (error handling)
489
-
490
- ### Regressions (if any)
491
- - [existing test that broke, with error details]
492
- ```
493
-
494
- ## Testing Anti-Patterns (Gate Functions)
495
-
496
- Before writing tests, check yourself against these 5 anti-patterns. Each has a **gate function** — a question you MUST answer before proceeding.
497
-
498
- ### Anti-Pattern 1: Testing Mock Behavior
499
- Asserting that a mock exists (e.g., `testId="sidebar-mock"`) instead of testing real component behavior. You're proving the mock works, not the code.
500
- **Gate**: "Am I testing real component behavior or just mock existence?" → If mock existence: STOP. Rewrite to test real behavior.
501
-
502
- ### Anti-Pattern 2: Test-Only Methods in Production
503
- Adding `destroy()`, `reset()`, or `__testSetup()` methods to production classes that are ONLY called from test files. Production code should not know tests exist.
504
- **Gate**: "Is this method only called by tests?" → If yes: STOP. Move to test utilities or test helper file, not production class.
505
-
506
- ### Anti-Pattern 3: Mocking Without Understanding Side Effects
507
- Mocking a function without first understanding ALL its side effects. The real function may write config files, update caches, or emit events that downstream code depends on.
508
- **Gate**: Before mocking, STOP and answer: "What side effects does the REAL function have? Does this test depend on any of those?" → Run with real implementation first, observe what happens, THEN add minimal mocking.
509
-
510
- ### Anti-Pattern 4: Incomplete Mocks
511
- Partial mock missing fields that downstream code consumes. Your test passes because it only checks the fields you mocked, but production code reads fields your mock doesn't have → runtime crash.
512
- **Iron Rule**: Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses. Examine actual API response / real data shape before writing mock.
513
-
514
- ### Anti-Pattern 5: Mock Setup Longer Than Test Logic
515
- If mock setup is 30 lines and the actual test assertion is 3 lines, the test is testing infrastructure, not behavior. This is a code smell that indicates wrong abstraction level.
516
- **Gate**: "Is my mock setup longer than my test logic?" → If yes: test at a higher level (integration) or extract mock factories.
517
-
518
- ### Anti-Pattern 6: Test Slop (Framework-Behavior Tests)
519
- Tests that verify the framework works rather than YOUR code works. If the test would still pass with an empty component/function, it's testing infrastructure.
520
- **Gate**: "Would this test pass if I deleted my business logic?" → If yes: STOP. Rewrite to test behavior that YOUR code introduces.
521
-
522
- Examples of test slop:
523
- - "renders without crashing" (tests that React works, not your component)
524
- - "route responds with 200" without checking response body (tests Express, not your handler)
525
- - Asserting a mock was called N times without checking the RESULT of those calls
526
- - Type existence tests (`typeof result === 'object'`) when you should test the actual value
527
-
528
- **Red flags — any of these means STOP and rethink:**
529
- - Mock setup longer than test logic
530
- - `*-mock` test IDs in assertions
531
- - Methods only called in test files
532
- - Can't explain in one sentence why a mock is needed
533
- - Test would pass with empty implementation (test slop)
534
-
535
- ## Returns
536
-
537
- | Artifact | Format | Location |
538
- |----------|--------|----------|
539
- | Test files | Source files | Co-located or `__tests__/` per project convention |
540
- | Test plan + results | Markdown | `TEST.md` in test directory (non-trivial features only) |
541
- | Eval scenarios | Markdown | `skills/<name>/evals.md` (for skill behavior testing) |
542
- | Coverage report | Inline stdout | Shown in Test Report |
543
- | Test Report | Markdown (inline) | Emitted to calling skill (cook, fix, review) |
544
-
545
- ## Chain Metadata
546
-
547
- Append to Test Report when invoked standalone. Suppress when called as sub-skill inside an L1 orchestrator (cook, team, etc.) — the orchestrator emits a consolidated block. See `docs/references/chain-metadata.md`.
548
-
549
- ```yaml
550
- chain_metadata:
551
- skill: "rune:test"
552
- version: "1.2.0"
553
- status: "[DONE]"
554
- domain: "[area tested]"
555
- files_changed:
556
- - "[test files created/modified]"
557
- exports:
558
- test_results: { passed: [N], failed: [N], coverage: [N] }
559
- test_files: ["[paths to test files]"]
560
- status: "[RED | GREEN]" # RED = TDD failing (expected), GREEN = all pass
561
- suggested_next: # status-aware — pick based on RED or GREEN
562
- # When GREEN:
563
- - skill: "rune:preflight"
564
- reason: "[grounded in results — e.g., 'All 15 tests GREEN, check edge case completeness']"
565
- consumes: ["test_results", "test_files"]
566
- # When RED (TDD expected):
567
- - skill: "rune:fix"
568
- reason: "[grounded in failures — e.g., '3 tests RED as expected, implement to make them pass']"
569
- consumes: ["test_results", "test_files"]
570
- ```
571
-
572
- ## Sharp Edges
573
-
574
- Known failure modes for this skill. Check these before declaring done.
575
-
576
- | Failure Mode | Severity | Mitigation |
577
- |---|---|---|
578
- | Tests passing before implementation exists | CRITICAL | RED Gate: rewrite stricter tests — passing without code = not testing real behavior |
579
- | Skipping the RED phase (not confirming FAIL) | HIGH | Run tests, confirm FAIL output before calling cook/fix to implement |
580
- | Testing mock behavior instead of real code | HIGH | Anti-Pattern 1 gate: "Am I testing real behavior or mock existence?" |
581
- | Mocking without understanding side effects | HIGH | Anti-Pattern 3 gate: run with real impl first, observe side effects, THEN mock minimally |
582
- | Incomplete mocks missing downstream fields | HIGH | Anti-Pattern 4 iron rule: mock COMPLETE data structure, not just fields your test checks |
583
- | Coverage below 80% without filling gaps | MEDIUM | Coverage Gate: identify uncovered lines and write additional tests |
584
- | Introducing a new test framework instead of using existing one | MEDIUM | Constraint 6: detect framework first, use project's existing one always |
585
- | Modifying source files to make tests work | HIGH | Role boundary: test writes test files ONLY — source changes go to rune:fix |
586
- | Test-only methods leaking into production code | MEDIUM | Anti-Pattern 2 gate: if method only called by tests → move to test utilities |
587
-
588
- ## Self-Validation
589
-
590
- ```
591
- SELF-VALIDATION (run before emitting Test Report):
592
- - [ ] Every test file has at least one assertion — no empty test bodies
593
- - [ ] RED phase output shows actual failures (not "0 tests") — tests were real, not stubs
594
- - [ ] No test modifies source code — test files only, source changes belong to fix
595
- - [ ] Test names describe behavior, not implementation ("should reject expired token" not "test function X")
596
- - [ ] No mocks of the thing being tested — only mock external dependencies
597
- - [ ] If BA requirements exist (REQ-xxx), every requirement has at least one test — check plan's Traceability Matrix
598
- ```
599
-
600
- ## Done When
601
-
602
- - Test framework detected from project config files
603
- - Tests cover happy path + at least 2 edge cases + error case
604
- - All new tests FAIL (RED phase — actual failure output shown)
605
- - After implementation: all tests PASS (GREEN phase — actual pass output shown)
606
- - Coverage ≥80% verified via verification
607
- - Test Report emitted with framework, test count, RED/GREEN status, and coverage
608
- - Self-Validation: all checks passed
609
-
610
- ## Cost Profile
611
-
612
- ~$0.03-0.08 per invocation. Sonnet for writing tests, Bash for running them. Frequent invocation in TDD workflow.
613
-
614
- **Scope guardrail**: Do not modify source or implementation files to make tests pass unless explicitly delegated by the parent agent.
1
+ ---
2
+ name: test
3
+ description: "TDD test writer. Writes failing tests FIRST (red), then verifies they pass after implementation (green). Covers unit, integration, and e2e tests."
4
+ metadata:
5
+ author: runedev
6
+ version: "1.2.0"
7
+ layer: L2
8
+ model: sonnet
9
+ group: development
10
+ tools: "Read, Write, Edit, Bash, Glob, Grep"
11
+ emit: tests.passed, tests.failed
12
+ listen: code.changed, db.migrated
13
+ ---
14
+
15
+ # test
16
+
17
+ <HARD-GATE>
18
+ Tests define the EXPECTED BEHAVIOR. They MUST be written BEFORE implementation code.
19
+ If tests pass without implementation → the tests are wrong. Rewrite them.
20
+ The only exception: when retrofitting tests for existing untested code.
21
+
22
+ THE IRON LAW: Write code before test? DELETE IT. Start over.
23
+ - Do NOT keep it as "reference"
24
+ - Do NOT "adapt" it while writing tests
25
+ - Do NOT look at it to "inform" test design
26
+ - Delete means delete. `git checkout -- <file>` or remove the changes entirely.
27
+ This is not negotiable. This is not optional. "But I already wrote it" is a sunk cost fallacy.
28
+
29
+ ROLE BOUNDARY: Test writes TEST FILES only. NEVER modify source/implementation files.
30
+ - Do NOT "quickly fix" a broken import in source to make tests run
31
+ - Do NOT refactor source code to be "more testable"
32
+ - Do NOT add missing exports to source files
33
+ - If source needs changes → hand off to `rune:fix`. Test's job ends at the test file.
34
+ This separation ensures test never writes code biased toward passing its own tests.
35
+ </HARD-GATE>
36
+
37
+ ## Instructions
38
+
39
+ ### Phase 1: Understand What to Test
40
+
41
+ 1. Read the implementation plan or task description carefully
42
+ 2. Use `Glob` to find existing test files: `**/*.test.*`, `**/*.spec.*`, `**/test_*`
43
+ 3. Use `Read` on 2-3 existing test files to understand:
44
+ - Test framework in use
45
+ - File naming convention (e.g., `foo.test.ts` mirrors `foo.ts`)
46
+ - Test directory structure (co-located vs `__tests__/` vs `tests/`)
47
+ - Assertion style and patterns
48
+ 4. Use `Glob` to find the source file(s) being tested
49
+
50
+ ```
51
+ TodoWrite: [
52
+ { content: "Understand scope and find existing test patterns", status: "in_progress" },
53
+ { content: "Detect test framework and conventions", status: "pending" },
54
+ { content: "Write failing tests (RED phase)", status: "pending" },
55
+ { content: "Run tests — verify they FAIL", status: "pending" },
56
+ { content: "After implementation: verify tests PASS (GREEN phase)", status: "pending" }
57
+ ]
58
+ ```
59
+
60
+ ### Phase 2: Detect Test Framework
61
+
62
+ Use `Glob` to find config files and identify the framework:
63
+
64
+ - `jest.config.*` or `"jest"` key in `package.json` → Jest
65
+ - `vitest.config.*` or `"vitest"` key in `package.json` → Vitest
66
+ - `pytest.ini`, `[tool.pytest.ini_options]` in `pyproject.toml` → pytest
67
+ - **Async check**: If pytest detected AND source files contain `async def`:
68
+ - Check if `pytest-asyncio` is in dependencies (`pyproject.toml [project.dependencies]` or `[project.optional-dependencies]`)
69
+ - Check if `asyncio_mode` is set in `[tool.pytest.ini_options]` (values: `auto`, `strict`, or absent)
70
+ - If async code exists but no `asyncio_mode` configured → **WARN**: "pytest-asyncio not configured. Async tests may silently pass without executing async code. Recommend adding `asyncio_mode = \"auto\"` to `[tool.pytest.ini_options]` in pyproject.toml."
71
+ - `Cargo.toml` with `#[cfg(test)]` pattern → built-in `cargo test`
72
+ - `*_test.go` files present → built-in `go test`
73
+ - `cypress.config.*` → Cypress (E2E)
74
+ - `playwright.config.*` → Playwright (E2E)
75
+
76
+ **Verification gate**: Framework identified before writing any test code.
77
+
78
+ ### Phase 3: Write Failing Tests
79
+
80
+ Use `Write` to create test files following the detected conventions:
81
+
82
+ 1. Mirror source file location: if source is `src/auth/login.ts`, test is `src/auth/login.test.ts`
83
+ 2. Structure tests with clear `describe` / `it` blocks (or language equivalent):
84
+ - `describe('Feature name')`
85
+ - `it('should [expected behavior] when [condition]')`
86
+ 3. Cover all three categories:
87
+ - **Happy path**: valid inputs, expected success output
88
+ - **Edge cases**: empty input, boundary values, large input
89
+ - **Error cases**: invalid input, missing data, network failure simulation
90
+
91
+ 4. Use proper assertions. Do NOT use implementation details — test behavior:
92
+ - Jest/Vitest: `expect(result).toBe(expected)`
93
+ - pytest: `assert result == expected`
94
+ - Rust: `assert_eq!(result, expected)`
95
+ - Go: `if result != expected { t.Errorf(...) }`
96
+
97
+ 5. For async code: use `async/await` or pytest `@pytest.mark.asyncio`
98
+
99
+ #### Python Async Tests (pytest-asyncio)
100
+
101
+ When writing tests for async Python code:
102
+
103
+ 1. **Verify setup before writing tests**:
104
+ - Confirm `pytest-asyncio` is in project dependencies
105
+ - Confirm `asyncio_mode` is set in `pyproject.toml` `[tool.pytest.ini_options]` (recommend `"auto"`)
106
+ - If neither is configured, warn the caller and suggest setup before proceeding
107
+
108
+ 2. **Writing async test functions**:
109
+ - With `asyncio_mode = "auto"`: just write `async def test_something():` — no decorator needed
110
+ - With `asyncio_mode = "strict"`: every async test needs `@pytest.mark.asyncio`
111
+ - Without asyncio_mode set: always use `@pytest.mark.asyncio` decorator explicitly
112
+
113
+ 3. **Async fixtures**:
114
+ - Use `@pytest_asyncio.fixture` (NOT `@pytest.fixture`) for async setup/teardown
115
+ - Scope rules: async fixtures default to `function` scope — use `scope="session"` carefully with async
116
+
117
+ 4. **Common pitfalls**:
118
+ - Tests that `pass` without `await` — they run but don't execute the async path
119
+ - Missing `pytest-asyncio` makes `async def test_*` silently pass as empty coroutines
120
+ - Mixing sync and async fixtures can cause event loop errors
121
+
122
+ ### Phase 4: Run Tests — Verify They FAIL (RED)
123
+
124
+ Use `Bash` to run ONLY the newly created test files (not full suite):
125
+
126
+ - **Jest**: `npx jest path/to/test.ts --no-coverage`
127
+ - **Vitest**: `npx vitest run path/to/test.ts`
128
+ - **pytest**: `pytest path/to/test_file.py -v` (if async tests and no `asyncio_mode` in config: add `--asyncio-mode=auto`)
129
+ - **Rust**: `cargo test test_module_name`
130
+ - **Go**: `go test ./path/to/package/... -run TestFunctionName`
131
+
132
+ **Hard gate**: ALL new tests MUST fail at this point.
133
+
134
+ - If ANY test passes before implementation exists → that test is not testing real behavior. Rewrite it to be stricter.
135
+ - If tests fail with import/syntax errors (not assertion errors) → fix the test code, re-run
136
+
137
+ ### Phase 5: After Implementation — Verify Tests PASS (GREEN)
138
+
139
+ After `rune:fix` writes implementation code, run the same test command again:
140
+
141
+ 1. ALL tests in the new test files MUST pass
142
+ 2. Run the full test suite with `Bash` to check for regressions:
143
+ - `npm test`, `pytest`, `cargo test`, `go test ./...`
144
+ 3. If any test fails: report clearly which test, what was expected, what was received
145
+ 4. If an existing test now fails (regression): escalate to `rune:debug`
146
+
147
+ **Verification gate**: 100% of new tests pass AND 0 regressions in existing tests.
148
+
149
+ ### Phase 6: Coverage Check
150
+
151
+ After GREEN phase, call `verification` to check coverage threshold (80% minimum):
152
+
153
+ - If coverage drops below 80%: identify uncovered lines, write additional tests
154
+ - Report coverage gaps with file:line references
155
+
156
+ ### Phase 6.5: Diff-Aware Mode (optional)
157
+
158
+ When invoked with `mode: "diff-aware"` or by `cook` after implementation:
159
+
160
+ 1. Run `git diff main --name-only` to get changed files
161
+ 2. For each changed file, trace its **blast radius**: what imports it? what routes does it serve? what components render it?
162
+ 3. Map changed files → affected routes/endpoints/pages
163
+ 4. Prioritize tests: files with most downstream dependents get tested first
164
+ 5. Generate targeted test commands that cover ONLY affected paths — skip unchanged modules
165
+
166
+ This mode is valuable for large codebases where running the full suite is slow. It answers: "what could this diff have broken?"
167
+
168
+ ```
169
+ Input: git diff main --name-only
170
+ Output: Prioritized test plan targeting only affected paths
171
+ ```
172
+
173
+ ## Test Types — 4-Layer Methodology
174
+
175
+ Tests are organized in 4 layers. Each layer catches a different failure class. Higher layers are slower but catch integration issues lower layers miss.
176
+
177
+ | Layer | Type | What It Catches | Framework | Speed |
178
+ |-------|------|-----------------|-----------|-------|
179
+ | L1 | **Unit** | Logic bugs, boundary violations, pure function errors | jest/vitest/pytest/cargo test | Fast |
180
+ | L2 | **Integration** | API contract breaks, DB query errors, service interaction failures | supertest/httpx/reqwest | Medium |
181
+ | L3 | **True Backend** | Real tool/service output correctness (not just exit 0) | Same + real software invocation | Medium-Slow |
182
+ | L4 | **E2E / Subprocess** | Full workflow from user/agent perspective, installed app works | Playwright/Cypress/subprocess | Slow |
183
+
184
+ **Layer rules:**
185
+ - **L1 (Unit)**: Synthetic data, no external deps. Every function tested in isolation. Fast, deterministic, CI-friendly
186
+ - **L2 (Integration)**: Tests service boundaries — API endpoints, DB operations, message queues. May need test DB or mock server
187
+ - **L3 (True Backend)**: **Invokes the REAL tool/service** and verifies output programmatically. No graceful degradation — if the dependency isn't installed, tests FAIL (not skip). Verify: magic bytes, file size > 0, content structure. Print artifact paths for manual inspection
188
+ - **L4 (E2E/Subprocess)**: Tests the installed command/app via subprocess or browser automation. Full user workflow: input → process → output → verify
189
+
190
+ **"No graceful degradation" rule** (L3/L4): Hard dependencies MUST be installed. Tests MUST NOT skip or produce fake results when the dependency is missing. A silently skipping test is worse than a loudly failing test.
191
+
192
+ Additional modes:
193
+
194
+ | Type | When | Speed |
195
+ |------|------|-------|
196
+ | Regression | After bug fixes | Fast |
197
+ | Diff-aware | After implementation, large codebases (Phase 6.5) | Fast (targeted) |
198
+
199
+ ## TEST.md — Test Plan + Results Document
200
+
201
+ For non-trivial features (3+ test files or 20+ test cases), create a `TEST.md` in the test directory. This is BOTH a planning doc (written BEFORE tests) and results doc (appended AFTER tests pass).
202
+
203
+ ### Before writing tests — write the plan:
204
+ ```markdown
205
+ # Test Plan: [Feature Name]
206
+
207
+ ## Test Inventory
208
+ - `test_core.py`: ~XX unit tests planned (L1)
209
+ - `test_integration.py`: ~XX integration tests planned (L2)
210
+ - `test_e2e.py`: ~XX E2E tests planned (L3/L4)
211
+
212
+ ## Unit Test Plan (L1)
213
+ | Module | Functions | Edge Cases | Est. Tests | Req IDs |
214
+ |--------|-----------|------------|------------|---------|
215
+ | `core/auth.py` | login, register, refresh | expired token, invalid creds, rate limit | 12 | REQ-001, REQ-003 |
216
+
217
+ ## E2E Scenarios (L3/L4)
218
+ | Workflow | Simulates | Operations | Verified | Req IDs |
219
+ |----------|-----------|------------|----------|---------|
220
+ | User signup | New user onboarding | register → verify → login | Token valid, profile created | REQ-005 |
221
+
222
+ ## Realistic Workflow Scenarios
223
+ - **[Name]**: [Step 1] → [Step 2] → verify [output properties]
224
+ ```
225
+
226
+ ### After tests pass — append results:
227
+ ```markdown
228
+ ## Test Results
229
+ [Paste full `pytest -v --tb=no` or `npm test` output]
230
+
231
+ ## Summary
232
+ - Total: XX | Passed: XX | Failed: 0
233
+ - Execution time: X.Xs | Coverage: XX%
234
+
235
+ ## Requirement Coverage
236
+ | Req ID | Test File(s) | Status |
237
+ |--------|-------------|--------|
238
+ | REQ-001 | `test_auth.py::test_login` | ✅ Covered |
239
+ | REQ-002 | — | ❌ Not covered |
240
+
241
+ ## Gaps
242
+ - [Areas not covered and why]
243
+ ```
244
+
245
+ **Why TEST.md**: Planning tests before code catches missing edge cases early. Appending results creates permanent evidence. One document = complete testing story.
246
+
247
+ ## Skill Behavior Tests (Eval Scenarios)
248
+
249
+ For testing SKILL.md behavior (not code), use **Eval Scenarios** — unit tests for skill files, not code files.
250
+
251
+ ### Eval Scenario Format
252
+
253
+ ```markdown
254
+ ## Eval: E[NN] — [scenario name]
255
+
256
+ ### Prompt
257
+ [The exact situation/message an agent receives]
258
+
259
+ ### Expected Reasoning
260
+ [Step-by-step reasoning the agent SHOULD follow]
261
+
262
+ ### Must Include
263
+ - [Assertion 1: what the output MUST contain or do]
264
+ - [Assertion 2]
265
+
266
+ ### Must NOT
267
+ - [Anti-pattern 1: what the output MUST NOT do]
268
+ - [Anti-pattern 2]
269
+
270
+ ### Category
271
+ happy-path | adversarial | edge-case | jailbreak | credential-leak
272
+ ```
273
+
274
+ ### Eval Coverage Requirements
275
+
276
+ A skill is **behavior-tested** when it has evals covering:
277
+
278
+ | Category | Min Evals | Purpose |
279
+ |----------|-----------|---------|
280
+ | Happy path | 1 | Core workflow executes correctly |
281
+ | Edge case | 1 | Empty input, missing context, unusual state |
282
+ | Adversarial | 1 | Time pressure, sunk cost, authority pressure |
283
+ | Jailbreak / injection | 1 | Prompt injection attempt, "ignore instructions" |
284
+
285
+ **Minimum**: 4 evals per skill (1 per category). Security-critical skills (sentinel, safeguard): 8+ evals.
286
+
287
+ ### Eval Storage
288
+
289
+ Save eval files as `skills/<name>/evals.md`. Each eval is a numbered scenario (E01–E24 range). skill-forge Phase 7 checks for evals presence before ship.
290
+
291
+
292
+ ## Error Recovery
293
+
294
+ - If test framework not found: ask calling skill to specify, or check `package.json` `devDependencies`
295
+ - If `Write` to test file fails: check if directory exists, create it first with `Bash mkdir -p`
296
+ - If tests error on import (module not found): check that source file path is correct, adjust imports
297
+ - If `Bash` test runner hangs beyond 120 seconds: kill and report as TIMEOUT
298
+
299
+ ## Called By (inbound)
300
+
301
+ - `cook` (L1): Phase 3 TEST — write tests first
302
+ - `fix` (L2): verify fix passes tests
303
+ - `review` (L2): untested edge case found → write test for it
304
+ - `deploy` (L2): pre-deployment full test suite
305
+ - `preflight` (L2): run targeted regression tests on affected code
306
+ - `surgeon` (L2): verify refactored code
307
+ - `launch` (L1): pre-deployment test suite
308
+ - `safeguard` (L2): writing characterization tests for legacy code
309
+ - `review-intake` (L2): write tests for issues identified during review intake
310
+
311
+ ## Calls (outbound)
312
+
313
+ - `verification` (L3): Phase 6 — coverage check (80% minimum threshold)
314
+ - `browser-pilot` (L3): Phase 4 — e2e and visual testing for UI flows
315
+ - `debug` (L2): Phase 5 — when existing test regresses unexpectedly
316
+
317
+ ## Data Flow
318
+
319
+ ### Feeds Into →
320
+
321
+ - `cook` (L1): test results (pass/fail/coverage) → cook's Phase 5 quality gate evidence
322
+ - `completion-gate` (L3): test runner stdout → evidence for "tests pass" claims
323
+ - `fix` (L2): failing test output → fix's target (what to make green)
324
+
325
+ ### Fed By ←
326
+
327
+ - `plan` (L2): phase file test tasks → test's RED phase targets (what to test)
328
+ - `review` (L2): untested edge cases found during review → new test targets
329
+ - `fix` (L2): implemented code → test's GREEN phase verification target
330
+
331
+ ### Feedback Loops ↻
332
+
333
+ - `test` ↔ `fix`: test writes failing tests (RED) → fix implements to pass → test verifies (GREEN) → if new failures emerge, loop continues
334
+ - `test` ↔ `debug`: test discovers regression → debug diagnoses root cause → test writes regression test to prevent recurrence
335
+
336
+ ## Anti-Rationalization Table
337
+
338
+ | Excuse | Reality |
339
+ |---|---|
340
+ | "Too simple to need tests first" | Simple code breaks. Test takes 30 seconds. Write it first. |
341
+ | "I'll write tests after — same result" | Tests-after = "what does this do?" Tests-first = "what SHOULD this do?" Completely different. |
342
+ | "I already wrote the code, let me just add tests" | Iron Law: delete the code. Start over with tests. Sunk cost is not an argument. |
343
+ | "Tests after achieve the same goals" | They don't. Tests-after are biased by the implementation you just wrote. |
344
+ | "It's about spirit not ritual" | Violating the letter IS violating the spirit. Write the test first. |
345
+ | "I mentally tested it" | Mental testing is not testing. Run the command, show the output. |
346
+ | "This is different because..." | It's not. Write the test first. |
347
+
348
+ ## Advanced: Oracle-Injection E2E Testing
349
+
350
+ For **data pipelines, AI workflows, and multi-stage processing** where comparing full output structures is impractical, use oracle injection:
351
+
352
+ 1. **Generate a UUID oracle token**: `const oracle = crypto.randomUUID()`
353
+ 2. **Inject into synthetic input**: embed the oracle in realistic test data that flows through the pipeline
354
+ 3. **Run the full pipeline**: input → all stages → output
355
+ 4. **Search for oracle in output**: if found → data flowed end-to-end correctly
356
+
357
+ ```
358
+ // Example: testing a document processing pipeline
359
+ const oracle = "ORACLE-" + crypto.randomUUID();
360
+ const testDoc = `Meeting notes: discussed ${oracle} integration timeline`;
361
+ const result = await pipeline.process(testDoc);
362
+ assert(result.output.includes(oracle), "Oracle not found — pipeline lost data");
363
+ ```
364
+
365
+ **When to use**: E2E tests for pipelines with 3+ stages, LLM-based processing, ETL workflows, or any system where output structure is complex/non-deterministic but data preservation is critical.
366
+
367
+ **When NOT to use**: Unit tests, simple CRUD, or when exact output comparison is feasible.
368
+
369
+
370
+ ## Spec→Test Traceability
371
+
372
+ When a plan with acceptance criteria exists (`.rune/features/<name>/plan.md` or phase file), every criterion MUST map to at least one test case.
373
+
374
+ ```
375
+ Plan Acceptance Criteria → Test Case → Implementation
376
+
377
+ AC-1: "User can reset password via email" → test_password_reset_sends_email()
378
+ AC-2: "Rate limit: max 3 reset attempts/hour" → test_password_reset_rate_limit()
379
+ AC-3: "Expired tokens rejected" → test_expired_reset_token_rejected()
380
+ ```
381
+
382
+ **Validation step** (after writing tests): Cross-check plan's acceptance criteria against test names. For each criterion:
383
+ - Has test → OK
384
+ - No test → flag as UNTESTED REQUIREMENT (more serious than uncovered lines)
385
+
386
+ **Why this is stronger than coverage**: Coverage checks that lines were EXECUTED. Traceability checks that INTENT was VERIFIED. You can have 100% coverage but miss a requirement if the test doesn't assert the right behavior.
387
+
388
+ **Skip if**: No plan exists (ad-hoc fix), or plan has no acceptance criteria section.
389
+
390
+ ## Eval-Driven Development
391
+
392
+ Define **capability evals** and **regression evals** BEFORE writing implementation code. Evals go beyond unit tests — they verify that the agent/system can handle the feature's intent, not just its mechanics.
393
+
394
+ ### Two Eval Types
395
+
396
+ | Type | Purpose | Pass Criteria | When |
397
+ |------|---------|---------------|------|
398
+ | **Capability eval** | Can the system do this new thing? | pass@k: ≥1 success in k attempts (k=3-5) | Before implementation |
399
+ | **Regression eval** | Did we break existing behavior? | pass^k: ALL k attempts must pass | After implementation |
400
+
401
+ **pass@k** (capability): At least 1 of k runs succeeds. Used for new features where some variance is acceptable. Threshold: ≥90% pass@3 for standard features, ≥95% pass@5 for critical paths.
402
+
403
+ **pass^k** (regression): ALL k runs must pass. Used for existing behavior that must never break. If ANY run fails, it's a regression. Threshold: 100% pass^3.
404
+
405
+ ### Eval File Format
406
+
407
+ Store evals in `.rune/evals/<feature>.md`:
408
+
409
+ ```markdown
410
+ # Eval: <feature name>
411
+
412
+ ## Capability Evals (pass@k)
413
+ | ID | Description | k | Threshold | Status |
414
+ |----|-------------|---|-----------|--------|
415
+ | CAP-1 | [what the system should be able to do] | 3 | 90% | pending |
416
+
417
+ ## Regression Evals (pass^k)
418
+ | ID | Description | k | Status |
419
+ |----|-------------|---|--------|
420
+ | REG-1 | [existing behavior that must not break] | 3 | pending |
421
+ ```
422
+
423
+ ### Anti-Pattern: Eval Overfitting
424
+
425
+ Do NOT overfit evals to specific prompts or known examples. Evals should test the **capability**, not the **exact input**.
426
+
427
+ - BAD: `"When user says 'hello', respond with 'Hi there!'"` — tests exact string match
428
+ - GOOD: `"When user greets, respond with a greeting"` — tests capability
429
+
430
+ ### Integration with TDD
431
+
432
+ 1. Write eval definitions (capability + regression) → `.rune/evals/<feature>.md`
433
+ 2. Write unit/integration tests (RED phase) → test files
434
+ 3. Implement feature (GREEN phase) → source files
435
+ 4. Run evals to verify capability achieved + no regressions
436
+ 5. Preflight checks eval results as part of quality gate
437
+
438
+ ## Red Flags — STOP and Start Over
439
+
440
+ If you catch yourself with ANY of these, delete implementation code and restart with tests:
441
+
442
+ - Code exists before test file
443
+ - "I already manually tested it"
444
+ - "Tests after achieve the same purpose"
445
+ - "It's about spirit not ritual"
446
+ - "This is different because..."
447
+ - "Let me just finish this, then add tests"
448
+
449
+ **All of these mean: Delete code. Start over with TDD.**
450
+
451
+ ## Constraints
452
+
453
+ 1. MUST write tests BEFORE implementation code — if tests pass without implementation, they are wrong
454
+ 2. MUST cover happy path + edge cases + error cases — not just happy path
455
+ 3. MUST run tests to verify they FAIL before implementation exists (RED phase is mandatory)
456
+ 4. MUST NOT write tests that test mock behavior instead of real code behavior
457
+ 5. MUST achieve 80% coverage minimum — identify and fill gaps
458
+ 6. MUST use the project's existing test framework and conventions — don't introduce a new one
459
+ 7. MUST NOT say "tests pass" without showing actual test runner output
460
+ 8. MUST delete implementation code written before tests — Iron Law, no exceptions
461
+ 9. MUST show RED phase output (actual failure) — "I confirmed they fail" without output is REJECTED
462
+ 10. MUST NOT modify source/implementation files — test writes test files ONLY, hand off source changes to rune:fix
463
+
464
+ ## Mesh Gates
465
+
466
+ | Gate | Requires | If Missing |
467
+ |------|----------|------------|
468
+ | RED Gate | All new tests FAIL before implementation | If any pass, rewrite stricter tests |
469
+ | GREEN Gate | All tests PASS after implementation | Fix code, not tests |
470
+ | Coverage Gate | 80%+ coverage verified via verification | Write additional tests for gaps |
471
+
472
+ ## Output Format
473
+
474
+ ```
475
+ ## Test Report
476
+ - **Framework**: [detected]
477
+ - **Files Created**: [list of new test file paths]
478
+ - **Tests Written**: [count]
479
+ - **Status**: RED (failing as expected) | GREEN (all passing)
480
+
481
+ ### Test Cases
482
+ | Test | Status | Description |
483
+ |------|--------|-------------|
484
+ | `test_name` | FAIL/PASS | [what it tests] |
485
+
486
+ ### Coverage
487
+ - Lines: [X]% | Branches: [Y]%
488
+ - Gaps: `path/to/file.ts:42-58` — uncovered branch (error handling)
489
+
490
+ ### Regressions (if any)
491
+ - [existing test that broke, with error details]
492
+ ```
493
+
494
+ ## Testing Anti-Patterns (Gate Functions)
495
+
496
+ Before writing tests, check yourself against these 5 anti-patterns. Each has a **gate function** — a question you MUST answer before proceeding.
497
+
498
+ ### Anti-Pattern 1: Testing Mock Behavior
499
+ Asserting that a mock exists (e.g., `testId="sidebar-mock"`) instead of testing real component behavior. You're proving the mock works, not the code.
500
+ **Gate**: "Am I testing real component behavior or just mock existence?" → If mock existence: STOP. Rewrite to test real behavior.
501
+
502
+ ### Anti-Pattern 2: Test-Only Methods in Production
503
+ Adding `destroy()`, `reset()`, or `__testSetup()` methods to production classes that are ONLY called from test files. Production code should not know tests exist.
504
+ **Gate**: "Is this method only called by tests?" → If yes: STOP. Move to test utilities or test helper file, not production class.
505
+
506
+ ### Anti-Pattern 3: Mocking Without Understanding Side Effects
507
+ Mocking a function without first understanding ALL its side effects. The real function may write config files, update caches, or emit events that downstream code depends on.
508
+ **Gate**: Before mocking, STOP and answer: "What side effects does the REAL function have? Does this test depend on any of those?" → Run with real implementation first, observe what happens, THEN add minimal mocking.
509
+
510
+ ### Anti-Pattern 4: Incomplete Mocks
511
+ Partial mock missing fields that downstream code consumes. Your test passes because it only checks the fields you mocked, but production code reads fields your mock doesn't have → runtime crash.
512
+ **Iron Rule**: Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses. Examine actual API response / real data shape before writing mock.
513
+
514
+ ### Anti-Pattern 5: Mock Setup Longer Than Test Logic
515
+ If mock setup is 30 lines and the actual test assertion is 3 lines, the test is testing infrastructure, not behavior. This is a code smell that indicates wrong abstraction level.
516
+ **Gate**: "Is my mock setup longer than my test logic?" → If yes: test at a higher level (integration) or extract mock factories.
517
+
518
+ ### Anti-Pattern 6: Test Slop (Framework-Behavior Tests)
519
+ Tests that verify the framework works rather than YOUR code works. If the test would still pass with an empty component/function, it's testing infrastructure.
520
+ **Gate**: "Would this test pass if I deleted my business logic?" → If yes: STOP. Rewrite to test behavior that YOUR code introduces.
521
+
522
+ Examples of test slop:
523
+ - "renders without crashing" (tests that React works, not your component)
524
+ - "route responds with 200" without checking response body (tests Express, not your handler)
525
+ - Asserting a mock was called N times without checking the RESULT of those calls
526
+ - Type existence tests (`typeof result === 'object'`) when you should test the actual value
527
+
528
+ **Red flags — any of these means STOP and rethink:**
529
+ - Mock setup longer than test logic
530
+ - `*-mock` test IDs in assertions
531
+ - Methods only called in test files
532
+ - Can't explain in one sentence why a mock is needed
533
+ - Test would pass with empty implementation (test slop)
534
+
535
+ ## Returns
536
+
537
+ | Artifact | Format | Location |
538
+ |----------|--------|----------|
539
+ | Test files | Source files | Co-located or `__tests__/` per project convention |
540
+ | Test plan + results | Markdown | `TEST.md` in test directory (non-trivial features only) |
541
+ | Eval scenarios | Markdown | `skills/<name>/evals.md` (for skill behavior testing) |
542
+ | Coverage report | Inline stdout | Shown in Test Report |
543
+ | Test Report | Markdown (inline) | Emitted to calling skill (cook, fix, review) |
544
+
545
+ ## Chain Metadata
546
+
547
+ Append to Test Report when invoked standalone. Suppress when called as sub-skill inside an L1 orchestrator (cook, team, etc.) — the orchestrator emits a consolidated block. See `docs/references/chain-metadata.md`.
548
+
549
+ ```yaml
550
+ chain_metadata:
551
+ skill: "rune:test"
552
+ version: "1.2.0"
553
+ status: "[DONE]"
554
+ domain: "[area tested]"
555
+ files_changed:
556
+ - "[test files created/modified]"
557
+ exports:
558
+ test_results: { passed: [N], failed: [N], coverage: [N] }
559
+ test_files: ["[paths to test files]"]
560
+ status: "[RED | GREEN]" # RED = TDD failing (expected), GREEN = all pass
561
+ suggested_next: # status-aware — pick based on RED or GREEN
562
+ # When GREEN:
563
+ - skill: "rune:preflight"
564
+ reason: "[grounded in results — e.g., 'All 15 tests GREEN, check edge case completeness']"
565
+ consumes: ["test_results", "test_files"]
566
+ # When RED (TDD expected):
567
+ - skill: "rune:fix"
568
+ reason: "[grounded in failures — e.g., '3 tests RED as expected, implement to make them pass']"
569
+ consumes: ["test_results", "test_files"]
570
+ ```
571
+
572
+ ## Sharp Edges
573
+
574
+ Known failure modes for this skill. Check these before declaring done.
575
+
576
+ | Failure Mode | Severity | Mitigation |
577
+ |---|---|---|
578
+ | Tests passing before implementation exists | CRITICAL | RED Gate: rewrite stricter tests — passing without code = not testing real behavior |
579
+ | Skipping the RED phase (not confirming FAIL) | HIGH | Run tests, confirm FAIL output before calling cook/fix to implement |
580
+ | Testing mock behavior instead of real code | HIGH | Anti-Pattern 1 gate: "Am I testing real behavior or mock existence?" |
581
+ | Mocking without understanding side effects | HIGH | Anti-Pattern 3 gate: run with real impl first, observe side effects, THEN mock minimally |
582
+ | Incomplete mocks missing downstream fields | HIGH | Anti-Pattern 4 iron rule: mock COMPLETE data structure, not just fields your test checks |
583
+ | Coverage below 80% without filling gaps | MEDIUM | Coverage Gate: identify uncovered lines and write additional tests |
584
+ | Introducing a new test framework instead of using existing one | MEDIUM | Constraint 6: detect framework first, use project's existing one always |
585
+ | Modifying source files to make tests work | HIGH | Role boundary: test writes test files ONLY — source changes go to rune:fix |
586
+ | Test-only methods leaking into production code | MEDIUM | Anti-Pattern 2 gate: if method only called by tests → move to test utilities |
587
+
588
+ ## Self-Validation
589
+
590
+ ```
591
+ SELF-VALIDATION (run before emitting Test Report):
592
+ - [ ] Every test file has at least one assertion — no empty test bodies
593
+ - [ ] RED phase output shows actual failures (not "0 tests") — tests were real, not stubs
594
+ - [ ] No test modifies source code — test files only, source changes belong to fix
595
+ - [ ] Test names describe behavior, not implementation ("should reject expired token" not "test function X")
596
+ - [ ] No mocks of the thing being tested — only mock external dependencies
597
+ - [ ] If BA requirements exist (REQ-xxx), every requirement has at least one test — check plan's Traceability Matrix
598
+ ```
599
+
600
+ ## Done When
601
+
602
+ - Test framework detected from project config files
603
+ - Tests cover happy path + at least 2 edge cases + error case
604
+ - All new tests FAIL (RED phase — actual failure output shown)
605
+ - After implementation: all tests PASS (GREEN phase — actual pass output shown)
606
+ - Coverage ≥80% verified via verification
607
+ - Test Report emitted with framework, test count, RED/GREEN status, and coverage
608
+ - Self-Validation: all checks passed
609
+
610
+ ## Cost Profile
611
+
612
+ ~$0.03-0.08 per invocation. Sonnet for writing tests, Bash for running them. Frequent invocation in TDD workflow.
613
+
614
+ **Scope guardrail**: Do not modify source or implementation files to make tests pass unless explicitly delegated by the parent agent.