universal-dev-standards 4.2.0 → 5.0.0-beta.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (542) hide show
  1. package/README.md +3 -3
  2. package/bin/uds.js +165 -5
  3. package/bundled/core/acceptance-test-driven-development.md +19 -721
  4. package/bundled/core/accessibility-standards.md +862 -0
  5. package/bundled/core/ai-agreement-standards.md +95 -0
  6. package/bundled/core/ai-friendly-architecture.md +2 -0
  7. package/bundled/core/ai-instruction-standards.md +2 -0
  8. package/bundled/core/anti-hallucination.md +90 -637
  9. package/bundled/core/behavior-driven-development.md +20 -735
  10. package/bundled/core/changelog-standards.md +4 -0
  11. package/bundled/core/checkin-standards.md +21 -4
  12. package/bundled/core/code-review-checklist.md +3 -0
  13. package/bundled/core/commit-message-guide.md +14 -3
  14. package/bundled/core/deployment-standards.md +296 -0
  15. package/bundled/core/developer-memory.md +568 -0
  16. package/bundled/core/documentation-structure.md +114 -18
  17. package/bundled/core/documentation-writing-standards.md +272 -2
  18. package/bundled/core/error-code-standards.md +3 -0
  19. package/bundled/core/forward-derivation-standards.md +134 -8
  20. package/bundled/core/git-workflow.md +72 -686
  21. package/bundled/core/guides/anti-hallucination-guide.md +517 -0
  22. package/bundled/core/guides/git-workflow-guide.md +954 -0
  23. package/bundled/core/guides/performance-guide.md +508 -0
  24. package/bundled/core/guides/refactoring-guide.md +608 -0
  25. package/bundled/core/guides/security-guide.md +485 -0
  26. package/bundled/core/logging-standards.md +337 -2
  27. package/bundled/core/performance-standards.md +231 -0
  28. package/bundled/core/project-context-memory.md +180 -0
  29. package/bundled/core/project-structure.md +315 -2
  30. package/bundled/core/refactoring-standards.md +81 -761
  31. package/bundled/core/requirement-engineering.md +39 -0
  32. package/bundled/core/reverse-engineering-standards.md +6 -4
  33. package/bundled/core/security-standards.md +244 -0
  34. package/bundled/core/spec-driven-development.md +20 -319
  35. package/bundled/core/test-completeness-dimensions.md +213 -11
  36. package/bundled/core/test-driven-development.md +22 -978
  37. package/bundled/core/testing-standards.md +222 -2903
  38. package/bundled/core/versioning.md +16 -55
  39. package/bundled/core/virtual-organization-standards.md +85 -0
  40. package/bundled/locales/README.md +4 -4
  41. package/bundled/locales/zh-CN/CHANGELOG.md +18 -18
  42. package/bundled/locales/zh-CN/CLAUDE.md +3 -3
  43. package/bundled/locales/zh-CN/MAINTENANCE.md +7 -7
  44. package/bundled/locales/zh-CN/README.md +2 -2
  45. package/bundled/locales/zh-CN/STANDARDS-MAPPING.md +1 -1
  46. package/bundled/locales/zh-CN/adoption/ADOPTION-GUIDE.md +3 -3
  47. package/bundled/locales/zh-CN/adoption/DAILY-WORKFLOW-GUIDE.md +66 -7
  48. package/bundled/locales/zh-CN/adoption/STATIC-DYNAMIC-GUIDE.md +22 -22
  49. package/bundled/locales/zh-CN/adoption/checklists/enterprise.md +5 -5
  50. package/bundled/locales/zh-CN/adoption/checklists/minimal.md +4 -4
  51. package/bundled/locales/zh-CN/adoption/checklists/recommended.md +5 -5
  52. package/bundled/locales/zh-CN/ai/MAINTENANCE.md +13 -13
  53. package/bundled/locales/zh-CN/core/ai-instruction-standards.md +1 -1
  54. package/bundled/locales/zh-CN/core/anti-hallucination.md +39 -3
  55. package/bundled/locales/zh-CN/core/checkin-standards.md +18 -5
  56. package/bundled/locales/zh-CN/core/commit-message-guide.md +1 -1
  57. package/bundled/locales/zh-CN/core/developer-memory.md +576 -0
  58. package/bundled/locales/zh-CN/core/documentation-structure.md +43 -5
  59. package/bundled/locales/zh-CN/core/documentation-writing-standards.md +146 -5
  60. package/bundled/locales/zh-CN/core/git-workflow.md +179 -4
  61. package/bundled/locales/zh-CN/core/logging-standards.md +177 -4
  62. package/bundled/locales/zh-CN/core/project-structure.md +117 -5
  63. package/bundled/locales/zh-CN/core/refactoring-standards.md +3 -3
  64. package/bundled/locales/zh-CN/core/reverse-engineering-standards.md +6 -6
  65. package/bundled/locales/zh-CN/core/spec-driven-development.md +516 -14
  66. package/bundled/locales/zh-CN/core/test-completeness-dimensions.md +155 -24
  67. package/bundled/locales/zh-CN/core/test-driven-development.md +71 -20
  68. package/bundled/locales/zh-CN/core/testing-standards.md +4 -4
  69. package/bundled/locales/zh-CN/core/versioning.md +1 -1
  70. package/bundled/locales/zh-CN/docs/AI-AGENT-ROADMAP.md +90 -51
  71. package/bundled/locales/zh-CN/docs/CHEATSHEET.md +184 -0
  72. package/bundled/locales/zh-CN/docs/CLI-FLOW.md +239 -0
  73. package/bundled/locales/zh-CN/docs/FEATURE-REFERENCE.md +320 -0
  74. package/bundled/locales/zh-CN/docs/OPERATION-WORKFLOW.md +43 -43
  75. package/bundled/locales/zh-CN/docs/STANDARDS-REFERENCE.md +440 -0
  76. package/bundled/locales/zh-CN/docs/USAGE-MODES-COMPARISON.md +1 -1
  77. package/bundled/locales/zh-CN/docs/WINDOWS-GUIDE.md +1 -1
  78. package/bundled/locales/zh-CN/integrations/codex/README.md +3 -3
  79. package/bundled/locales/zh-CN/integrations/gemini-cli/README.md +3 -3
  80. package/bundled/locales/zh-CN/integrations/github-copilot/README.md +5 -5
  81. package/bundled/locales/zh-CN/integrations/github-copilot/copilot-instructions.md +9 -9
  82. package/bundled/locales/zh-CN/integrations/github-copilot/skills-mapping.md +18 -6
  83. package/bundled/locales/zh-CN/integrations/google-antigravity/README.md +2 -2
  84. package/bundled/locales/zh-CN/integrations/opencode/README.md +3 -3
  85. package/bundled/locales/zh-CN/integrations/opencode/skills-mapping.md +22 -10
  86. package/bundled/locales/zh-CN/integrations/openspec/README.md +1 -1
  87. package/bundled/locales/zh-CN/integrations/spec-kit/AGENTS.md +3 -3
  88. package/bundled/locales/zh-CN/integrations/spec-kit/README.md +1 -1
  89. package/bundled/locales/zh-CN/options/commit-message/bilingual.md +1 -1
  90. package/bundled/locales/zh-CN/options/commit-message/english.md +1 -1
  91. package/bundled/locales/zh-CN/options/commit-message/traditional-chinese.md +1 -1
  92. package/bundled/locales/zh-CN/options/git-workflow/gitflow.md +1 -1
  93. package/bundled/locales/zh-CN/options/git-workflow/github-flow.md +1 -1
  94. package/bundled/locales/zh-CN/options/git-workflow/merge-commit.md +1 -1
  95. package/bundled/locales/zh-CN/options/git-workflow/rebase-ff.md +1 -1
  96. package/bundled/locales/zh-CN/options/git-workflow/squash-merge.md +1 -1
  97. package/bundled/locales/zh-CN/options/git-workflow/trunk-based.md +1 -1
  98. package/bundled/locales/zh-CN/options/project-structure/dotnet.md +1 -1
  99. package/bundled/locales/zh-CN/options/project-structure/go.md +1 -1
  100. package/bundled/locales/zh-CN/options/project-structure/java.md +1 -1
  101. package/bundled/locales/zh-CN/options/project-structure/nodejs.md +1 -1
  102. package/bundled/locales/zh-CN/options/project-structure/python.md +1 -1
  103. package/bundled/locales/zh-CN/options/testing/e2e-testing.md +1 -1
  104. package/bundled/locales/zh-CN/options/testing/integration-testing.md +1 -1
  105. package/bundled/locales/zh-CN/options/testing/system-testing.md +1 -1
  106. package/bundled/locales/zh-CN/options/testing/unit-testing.md +1 -1
  107. package/bundled/locales/zh-CN/skills/{claude-code/CONTRIBUTING.template.md → CONTRIBUTING.template.md} +2 -2
  108. package/bundled/locales/zh-CN/skills/README.md +131 -81
  109. package/bundled/locales/zh-CN/skills/agents/README.md +281 -0
  110. package/bundled/locales/zh-CN/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/SKILL.md +10 -10
  111. package/bundled/locales/zh-CN/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/anti-hallucination.md +8 -8
  112. package/bundled/locales/zh-CN/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/certainty-labels.md +8 -8
  113. package/bundled/locales/zh-CN/skills/ai-friendly-architecture/SKILL.md +221 -0
  114. package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +214 -0
  115. package/bundled/locales/zh-CN/skills/atdd-assistant/SKILL.md +71 -0
  116. package/bundled/locales/zh-CN/skills/atdd-assistant/acceptance-criteria-guide.md +428 -0
  117. package/bundled/locales/zh-CN/skills/atdd-assistant/atdd-workflow.md +484 -0
  118. package/bundled/locales/zh-CN/skills/atdd-assistant/guide.md +368 -0
  119. package/bundled/locales/zh-CN/skills/bdd-assistant/SKILL.md +68 -0
  120. package/bundled/locales/zh-CN/skills/bdd-assistant/bdd-workflow.md +406 -0
  121. package/bundled/locales/zh-CN/skills/bdd-assistant/gherkin-guide.md +454 -0
  122. package/bundled/locales/zh-CN/skills/bdd-assistant/guide.md +324 -0
  123. package/bundled/locales/zh-CN/skills/changelog-guide/SKILL.md +64 -0
  124. package/bundled/locales/zh-CN/skills/{claude-code/changelog-guide/SKILL.md → changelog-guide/guide.md} +6 -7
  125. package/bundled/locales/zh-CN/skills/checkin-assistant/SKILL.md +56 -0
  126. package/bundled/locales/zh-CN/skills/{claude-code/checkin-assistant/SKILL.md → checkin-assistant/guide.md} +6 -7
  127. package/bundled/locales/zh-CN/skills/code-review-assistant/SKILL.md +56 -0
  128. package/bundled/locales/zh-CN/skills/{claude-code/code-review-assistant → code-review-assistant}/checkin-checklist.md +4 -4
  129. package/bundled/locales/zh-CN/skills/{claude-code/code-review-assistant/SKILL.md → code-review-assistant/guide.md} +5 -6
  130. package/bundled/locales/zh-CN/skills/{claude-code/code-review-assistant → code-review-assistant}/review-checklist.md +4 -4
  131. package/bundled/locales/zh-CN/skills/{claude-code/commands → commands}/bdd.md +2 -2
  132. package/bundled/locales/zh-CN/skills/{claude-code/commands → commands}/methodology.md +3 -3
  133. package/bundled/locales/zh-CN/skills/{claude-code/commands → commands}/refactor.md +3 -3
  134. package/bundled/locales/zh-CN/skills/commit-standards/SKILL.md +61 -0
  135. package/bundled/locales/zh-CN/skills/{claude-code/commit-standards → commit-standards}/conventional-commits.md +4 -4
  136. package/bundled/locales/zh-CN/skills/{claude-code/commit-standards/SKILL.md → commit-standards/guide.md} +5 -6
  137. package/bundled/locales/zh-CN/skills/{claude-code/commit-standards → commit-standards}/language-options.md +3 -3
  138. package/bundled/locales/zh-CN/skills/docs-generator/SKILL.md +59 -0
  139. package/bundled/locales/zh-CN/skills/docs-generator/guide.md +262 -0
  140. package/bundled/locales/zh-CN/skills/{claude-code/documentation-guide → documentation-guide}/SKILL.md +7 -8
  141. package/bundled/locales/zh-CN/skills/{claude-code/documentation-guide → documentation-guide}/documentation-structure.md +4 -4
  142. package/bundled/locales/zh-CN/skills/{claude-code/documentation-guide → documentation-guide}/readme-template.md +3 -3
  143. package/bundled/locales/zh-CN/skills/{claude-code/error-code-guide → error-code-guide}/SKILL.md +6 -6
  144. package/bundled/locales/zh-CN/skills/forward-derivation/SKILL.md +62 -0
  145. package/bundled/locales/zh-CN/skills/forward-derivation/guide.md +130 -0
  146. package/bundled/locales/zh-CN/skills/{claude-code/git-workflow-guide → git-workflow-guide}/SKILL.md +6 -6
  147. package/bundled/locales/zh-CN/skills/{claude-code/git-workflow-guide → git-workflow-guide}/branch-naming.md +3 -3
  148. package/bundled/locales/zh-CN/skills/{claude-code/git-workflow-guide → git-workflow-guide}/git-workflow.md +4 -4
  149. package/bundled/locales/zh-CN/skills/{claude-code/logging-guide → logging-guide}/SKILL.md +6 -6
  150. package/bundled/locales/zh-CN/skills/methodology-system/SKILL.md +78 -0
  151. package/bundled/locales/zh-CN/skills/{claude-code/methodology-system → methodology-system}/create-methodology.md +2 -2
  152. package/bundled/locales/zh-CN/skills/{claude-code/methodology-system/SKILL.md → methodology-system/guide.md} +2 -2
  153. package/bundled/locales/zh-CN/skills/{claude-code/methodology-system → methodology-system}/runtime.md +3 -3
  154. package/bundled/locales/zh-CN/skills/project-discovery/SKILL.md +66 -0
  155. package/bundled/locales/zh-CN/skills/project-discovery/guide.md +371 -0
  156. package/bundled/locales/zh-CN/skills/{claude-code/project-structure-guide → project-structure-guide}/SKILL.md +4 -4
  157. package/bundled/locales/zh-CN/skills/{claude-code/project-structure-guide → project-structure-guide}/language-patterns.md +1 -1
  158. package/bundled/locales/zh-CN/skills/refactoring-assistant/SKILL.md +67 -0
  159. package/bundled/locales/zh-CN/skills/{claude-code/refactoring-assistant/SKILL.md → refactoring-assistant/guide.md} +7 -8
  160. package/bundled/locales/zh-CN/skills/release-standards/SKILL.md +64 -0
  161. package/bundled/locales/zh-CN/skills/{claude-code/release-standards → release-standards}/changelog-format.md +3 -3
  162. package/bundled/locales/zh-CN/skills/{claude-code/release-standards/SKILL.md → release-standards/guide.md} +5 -6
  163. package/bundled/locales/zh-CN/skills/{claude-code/release-standards → release-standards}/release-workflow.md +7 -7
  164. package/bundled/locales/zh-CN/skills/{claude-code/release-standards → release-standards}/semantic-versioning.md +3 -3
  165. package/bundled/locales/zh-CN/skills/requirement-assistant/SKILL.md +67 -0
  166. package/bundled/locales/zh-CN/skills/{claude-code/requirement-assistant/SKILL.md → requirement-assistant/guide.md} +4 -5
  167. package/bundled/locales/zh-CN/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-checklist.md +3 -3
  168. package/bundled/locales/zh-CN/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-writing.md +3 -3
  169. package/bundled/locales/zh-CN/skills/reverse-engineer/SKILL.md +67 -0
  170. package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer → reverse-engineer}/bdd-extraction.md +2 -2
  171. package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer/SKILL.md → reverse-engineer/guide.md} +17 -18
  172. package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer → reverse-engineer}/tdd-analysis.md +2 -2
  173. package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer → reverse-engineer}/workflow.md +3 -3
  174. package/bundled/locales/zh-CN/skills/spec-driven-dev/SKILL.md +75 -0
  175. package/bundled/locales/zh-CN/skills/{claude-code/spec-driven-dev/SKILL.md → spec-driven-dev/guide.md} +10 -11
  176. package/bundled/locales/zh-CN/skills/tdd-assistant/SKILL.md +66 -0
  177. package/bundled/locales/zh-CN/skills/{claude-code/tdd-assistant/SKILL.md → tdd-assistant/guide.md} +12 -16
  178. package/bundled/locales/zh-CN/skills/{claude-code/tdd-assistant → tdd-assistant}/language-examples.md +3 -3
  179. package/bundled/locales/zh-CN/skills/{claude-code/tdd-assistant → tdd-assistant}/tdd-workflow.md +3 -3
  180. package/bundled/locales/zh-CN/skills/test-coverage-assistant/SKILL.md +65 -0
  181. package/bundled/locales/zh-CN/skills/test-coverage-assistant/guide.md +284 -0
  182. package/bundled/locales/zh-CN/skills/{claude-code/testing-guide → testing-guide}/SKILL.md +8 -8
  183. package/bundled/locales/zh-CN/skills/{claude-code/testing-guide → testing-guide}/testing-pyramid.md +4 -4
  184. package/bundled/locales/zh-CN/skills/{cline → tools/cline}/README.md +1 -1
  185. package/bundled/locales/zh-CN/skills/{copilot → tools/copilot}/README.md +1 -1
  186. package/bundled/locales/zh-CN/skills/{copilot → tools/copilot}/copilot-instructions.md +1 -1
  187. package/bundled/locales/zh-CN/skills/{cursor → tools/cursor}/README.md +1 -1
  188. package/bundled/locales/zh-CN/skills/{windsurf → tools/windsurf}/README.md +1 -1
  189. package/bundled/locales/zh-TW/CHANGELOG.md +16 -16
  190. package/bundled/locales/zh-TW/CLAUDE.md +3 -3
  191. package/bundled/locales/zh-TW/MAINTENANCE.md +7 -7
  192. package/bundled/locales/zh-TW/README.md +13 -7
  193. package/bundled/locales/zh-TW/STANDARDS-MAPPING.md +1 -1
  194. package/bundled/locales/zh-TW/adoption/ADOPTION-GUIDE.md +4 -4
  195. package/bundled/locales/zh-TW/adoption/DAILY-WORKFLOW-GUIDE.md +66 -7
  196. package/bundled/locales/zh-TW/adoption/STATIC-DYNAMIC-GUIDE.md +22 -22
  197. package/bundled/locales/zh-TW/adoption/checklists/enterprise.md +5 -5
  198. package/bundled/locales/zh-TW/adoption/checklists/minimal.md +4 -4
  199. package/bundled/locales/zh-TW/adoption/checklists/recommended.md +5 -5
  200. package/bundled/locales/zh-TW/ai/MAINTENANCE.md +13 -13
  201. package/bundled/locales/zh-TW/ai/standards/anti-hallucination.ai.yaml +3 -2
  202. package/bundled/locales/zh-TW/ai/standards/git-workflow.ai.yaml +4 -3
  203. package/bundled/locales/zh-TW/ai/standards/spec-driven-development.ai.yaml +3 -3
  204. package/bundled/locales/zh-TW/ai/standards/testing.ai.yaml +1 -0
  205. package/bundled/locales/zh-TW/core/ai-instruction-standards.md +1 -1
  206. package/bundled/locales/zh-TW/core/anti-hallucination.md +38 -701
  207. package/bundled/locales/zh-TW/core/changelog-standards.md +1 -1
  208. package/bundled/locales/zh-TW/core/checkin-standards.md +18 -5
  209. package/bundled/locales/zh-TW/core/code-review-checklist.md +1 -1
  210. package/bundled/locales/zh-TW/core/commit-message-guide.md +1 -1
  211. package/bundled/locales/zh-TW/core/developer-memory.md +576 -0
  212. package/bundled/locales/zh-TW/core/documentation-structure.md +115 -22
  213. package/bundled/locales/zh-TW/core/documentation-writing-standards.md +126 -5
  214. package/bundled/locales/zh-TW/core/forward-derivation-standards.md +2 -2
  215. package/bundled/locales/zh-TW/core/git-workflow.md +38 -812
  216. package/bundled/locales/zh-TW/core/guides/anti-hallucination-guide.md +779 -0
  217. package/bundled/locales/zh-TW/core/guides/git-workflow-guide.md +1016 -0
  218. package/bundled/locales/zh-TW/core/guides/refactoring-guide.md +864 -0
  219. package/bundled/locales/zh-TW/core/guides/testing-guide.md +2280 -0
  220. package/bundled/locales/zh-TW/core/logging-standards.md +159 -4
  221. package/bundled/locales/zh-TW/core/project-structure.md +102 -9
  222. package/bundled/locales/zh-TW/core/refactoring-standards.md +37 -823
  223. package/bundled/locales/zh-TW/core/reverse-engineering-standards.md +6 -6
  224. package/bundled/locales/zh-TW/core/spec-driven-development.md +17 -268
  225. package/bundled/locales/zh-TW/core/test-completeness-dimensions.md +128 -14
  226. package/bundled/locales/zh-TW/core/test-driven-development.md +20 -983
  227. package/bundled/locales/zh-TW/core/testing-standards.md +120 -2105
  228. package/bundled/locales/zh-TW/core/versioning.md +13 -55
  229. package/bundled/locales/zh-TW/docs/AI-AGENT-ROADMAP.md +90 -51
  230. package/bundled/locales/zh-TW/docs/CHEATSHEET.md +184 -0
  231. package/bundled/locales/zh-TW/docs/CLI-FLOW.md +239 -0
  232. package/bundled/locales/zh-TW/docs/FEATURE-REFERENCE.md +320 -0
  233. package/bundled/locales/zh-TW/docs/MIGRATION-V5.md +71 -0
  234. package/bundled/locales/zh-TW/docs/OPERATION-WORKFLOW.md +42 -42
  235. package/bundled/locales/zh-TW/docs/STANDARDS-REFERENCE.md +440 -0
  236. package/bundled/locales/zh-TW/docs/USAGE-MODES-COMPARISON.md +1 -1
  237. package/bundled/locales/zh-TW/docs/WINDOWS-GUIDE.md +1 -1
  238. package/bundled/locales/zh-TW/integrations/claude-code/README.md +44 -0
  239. package/bundled/locales/zh-TW/integrations/github-copilot/copilot-instructions.md +3 -3
  240. package/bundled/locales/zh-TW/integrations/github-copilot/skills-mapping.md +18 -6
  241. package/bundled/locales/zh-TW/integrations/opencode/skills-mapping.md +22 -10
  242. package/bundled/locales/zh-TW/methodologies/guides/sdd-guide.md +338 -0
  243. package/bundled/locales/zh-TW/methodologies/guides/tdd-guide.md +1055 -0
  244. package/bundled/locales/zh-TW/options/commit-message/bilingual.md +1 -1
  245. package/bundled/locales/zh-TW/options/commit-message/english.md +1 -1
  246. package/bundled/locales/zh-TW/options/commit-message/traditional-chinese.md +1 -1
  247. package/bundled/locales/zh-TW/options/git-workflow/gitflow.md +1 -1
  248. package/bundled/locales/zh-TW/options/git-workflow/github-flow.md +1 -1
  249. package/bundled/locales/zh-TW/options/git-workflow/merge-commit.md +1 -1
  250. package/bundled/locales/zh-TW/options/git-workflow/rebase-ff.md +1 -1
  251. package/bundled/locales/zh-TW/options/git-workflow/squash-merge.md +1 -1
  252. package/bundled/locales/zh-TW/options/git-workflow/trunk-based.md +1 -1
  253. package/bundled/locales/zh-TW/options/project-structure/dotnet.md +1 -1
  254. package/bundled/locales/zh-TW/options/project-structure/go.md +1 -1
  255. package/bundled/locales/zh-TW/options/project-structure/java.md +1 -1
  256. package/bundled/locales/zh-TW/options/project-structure/nodejs.md +1 -1
  257. package/bundled/locales/zh-TW/options/project-structure/python.md +1 -1
  258. package/bundled/locales/zh-TW/options/testing/e2e-testing.md +1 -1
  259. package/bundled/locales/zh-TW/options/testing/integration-testing.md +1 -1
  260. package/bundled/locales/zh-TW/options/testing/system-testing.md +1 -1
  261. package/bundled/locales/zh-TW/options/testing/unit-testing.md +1 -1
  262. package/bundled/locales/zh-TW/skills/{claude-code/CONTRIBUTING.template.md → CONTRIBUTING.template.md} +2 -2
  263. package/bundled/locales/zh-TW/skills/README.md +131 -79
  264. package/bundled/locales/zh-TW/skills/agents/README.md +281 -0
  265. package/bundled/locales/zh-TW/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/SKILL.md +7 -7
  266. package/bundled/locales/zh-TW/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/anti-hallucination.md +8 -8
  267. package/bundled/locales/zh-TW/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/certainty-labels.md +8 -8
  268. package/bundled/locales/zh-TW/skills/ai-friendly-architecture/SKILL.md +221 -0
  269. package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +214 -0
  270. package/bundled/locales/zh-TW/skills/atdd-assistant/SKILL.md +71 -0
  271. package/bundled/locales/zh-TW/skills/atdd-assistant/acceptance-criteria-guide.md +428 -0
  272. package/bundled/locales/zh-TW/skills/atdd-assistant/atdd-workflow.md +484 -0
  273. package/bundled/locales/zh-TW/skills/atdd-assistant/guide.md +368 -0
  274. package/bundled/locales/zh-TW/skills/bdd-assistant/SKILL.md +68 -0
  275. package/bundled/locales/zh-TW/skills/bdd-assistant/bdd-workflow.md +406 -0
  276. package/bundled/locales/zh-TW/skills/bdd-assistant/gherkin-guide.md +454 -0
  277. package/bundled/locales/zh-TW/skills/bdd-assistant/guide.md +324 -0
  278. package/bundled/locales/zh-TW/skills/changelog-guide/SKILL.md +64 -0
  279. package/bundled/locales/zh-TW/skills/{claude-code/changelog-guide/SKILL.md → changelog-guide/guide.md} +6 -7
  280. package/bundled/locales/zh-TW/skills/checkin-assistant/SKILL.md +56 -0
  281. package/bundled/locales/zh-TW/skills/{claude-code/checkin-assistant/SKILL.md → checkin-assistant/guide.md} +6 -7
  282. package/bundled/locales/zh-TW/skills/code-review-assistant/SKILL.md +56 -0
  283. package/bundled/locales/zh-TW/skills/{claude-code/code-review-assistant → code-review-assistant}/checkin-checklist.md +4 -4
  284. package/bundled/locales/zh-TW/skills/{claude-code/code-review-assistant/SKILL.md → code-review-assistant/guide.md} +2 -3
  285. package/bundled/locales/zh-TW/skills/{claude-code/code-review-assistant → code-review-assistant}/review-checklist.md +4 -4
  286. package/bundled/locales/zh-TW/skills/{claude-code/commands → commands}/bdd.md +2 -2
  287. package/bundled/locales/zh-TW/skills/{claude-code/commands → commands}/methodology.md +3 -3
  288. package/bundled/locales/zh-TW/skills/{claude-code/commands → commands}/refactor.md +3 -3
  289. package/bundled/locales/zh-TW/skills/commit-standards/SKILL.md +61 -0
  290. package/bundled/locales/zh-TW/skills/{claude-code/commit-standards → commit-standards}/conventional-commits.md +4 -4
  291. package/bundled/locales/zh-TW/skills/{claude-code/commit-standards/SKILL.md → commit-standards/guide.md} +5 -6
  292. package/bundled/locales/zh-TW/skills/{claude-code/commit-standards → commit-standards}/language-options.md +3 -3
  293. package/bundled/locales/zh-TW/skills/docs-generator/SKILL.md +59 -0
  294. package/bundled/locales/zh-TW/skills/docs-generator/guide.md +262 -0
  295. package/bundled/locales/zh-TW/skills/{claude-code/documentation-guide → documentation-guide}/SKILL.md +7 -8
  296. package/bundled/locales/zh-TW/skills/{claude-code/documentation-guide → documentation-guide}/documentation-structure.md +4 -4
  297. package/bundled/locales/zh-TW/skills/{claude-code/documentation-guide → documentation-guide}/readme-template.md +3 -3
  298. package/bundled/locales/zh-TW/skills/{claude-code/error-code-guide → error-code-guide}/SKILL.md +6 -6
  299. package/bundled/locales/zh-TW/skills/forward-derivation/SKILL.md +62 -0
  300. package/bundled/locales/zh-TW/skills/{claude-code/forward-derivation/SKILL.md → forward-derivation/guide.md} +24 -19
  301. package/bundled/locales/zh-TW/skills/{claude-code/git-workflow-guide → git-workflow-guide}/SKILL.md +3 -3
  302. package/bundled/locales/zh-TW/skills/{claude-code/git-workflow-guide → git-workflow-guide}/branch-naming.md +3 -3
  303. package/bundled/locales/zh-TW/skills/{claude-code/git-workflow-guide → git-workflow-guide}/git-workflow.md +4 -4
  304. package/bundled/locales/zh-TW/skills/{claude-code/logging-guide → logging-guide}/SKILL.md +6 -6
  305. package/bundled/locales/zh-TW/skills/methodology-system/SKILL.md +78 -0
  306. package/bundled/locales/zh-TW/skills/{claude-code/methodology-system → methodology-system}/create-methodology.md +2 -2
  307. package/bundled/locales/zh-TW/skills/{claude-code/methodology-system/SKILL.md → methodology-system/guide.md} +2 -2
  308. package/bundled/locales/zh-TW/skills/{claude-code/methodology-system → methodology-system}/runtime.md +3 -3
  309. package/bundled/locales/zh-TW/skills/project-discovery/SKILL.md +66 -0
  310. package/bundled/locales/zh-TW/skills/project-discovery/guide.md +371 -0
  311. package/bundled/locales/zh-TW/skills/project-discovery/workflow.md +574 -0
  312. package/bundled/locales/zh-TW/skills/{claude-code/project-structure-guide → project-structure-guide}/SKILL.md +4 -4
  313. package/bundled/locales/zh-TW/skills/{claude-code/project-structure-guide → project-structure-guide}/language-patterns.md +1 -1
  314. package/bundled/locales/zh-TW/skills/refactoring-assistant/SKILL.md +67 -0
  315. package/bundled/locales/zh-TW/skills/{claude-code/refactoring-assistant/SKILL.md → refactoring-assistant/guide.md} +7 -8
  316. package/bundled/locales/zh-TW/skills/release-standards/SKILL.md +64 -0
  317. package/bundled/locales/zh-TW/skills/{claude-code/release-standards → release-standards}/changelog-format.md +3 -3
  318. package/bundled/locales/zh-TW/skills/{claude-code/release-standards/SKILL.md → release-standards/guide.md} +2 -3
  319. package/bundled/locales/zh-TW/skills/{claude-code/release-standards → release-standards}/release-workflow.md +60 -28
  320. package/bundled/locales/zh-TW/skills/{claude-code/release-standards → release-standards}/semantic-versioning.md +3 -3
  321. package/bundled/locales/zh-TW/skills/requirement-assistant/SKILL.md +67 -0
  322. package/bundled/locales/zh-TW/skills/{claude-code/requirement-assistant/SKILL.md → requirement-assistant/guide.md} +2 -3
  323. package/bundled/locales/zh-TW/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-checklist.md +3 -3
  324. package/bundled/locales/zh-TW/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-writing.md +3 -3
  325. package/bundled/locales/zh-TW/skills/reverse-engineer/SKILL.md +67 -0
  326. package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer → reverse-engineer}/bdd-extraction.md +2 -2
  327. package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer/SKILL.md → reverse-engineer/guide.md} +17 -18
  328. package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer → reverse-engineer}/tdd-analysis.md +2 -2
  329. package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer → reverse-engineer}/workflow.md +3 -3
  330. package/bundled/locales/zh-TW/skills/spec-driven-dev/SKILL.md +75 -0
  331. package/bundled/locales/zh-TW/skills/{claude-code/spec-driven-dev/SKILL.md → spec-driven-dev/guide.md} +9 -10
  332. package/bundled/locales/zh-TW/skills/tdd-assistant/SKILL.md +66 -0
  333. package/bundled/locales/zh-TW/skills/{claude-code/tdd-assistant/SKILL.md → tdd-assistant/guide.md} +9 -10
  334. package/bundled/locales/zh-TW/skills/{claude-code/tdd-assistant → tdd-assistant}/language-examples.md +3 -3
  335. package/bundled/locales/zh-TW/skills/{claude-code/tdd-assistant → tdd-assistant}/tdd-workflow.md +3 -3
  336. package/bundled/locales/zh-TW/skills/test-coverage-assistant/SKILL.md +65 -0
  337. package/bundled/locales/zh-TW/skills/{claude-code/test-coverage-assistant/SKILL.md → test-coverage-assistant/guide.md} +27 -24
  338. package/bundled/locales/zh-TW/skills/{claude-code/testing-guide → testing-guide}/SKILL.md +6 -6
  339. package/bundled/locales/zh-TW/skills/{claude-code/testing-guide → testing-guide}/testing-pyramid.md +4 -4
  340. package/bundled/locales/zh-TW/skills/{cline → tools/cline}/README.md +1 -1
  341. package/bundled/locales/zh-TW/skills/{copilot → tools/copilot}/README.md +1 -1
  342. package/bundled/locales/zh-TW/skills/{copilot → tools/copilot}/copilot-instructions.md +1 -1
  343. package/bundled/locales/zh-TW/skills/{cursor → tools/cursor}/README.md +1 -1
  344. package/bundled/locales/zh-TW/skills/{windsurf → tools/windsurf}/README.md +1 -1
  345. package/bundled/skills/INTEGRATION-GUIDE.md +210 -0
  346. package/bundled/skills/README.md +95 -0
  347. package/bundled/skills/_shared/README.md +60 -0
  348. package/bundled/skills/{claude-code/agents → agents}/README.md +2 -2
  349. package/bundled/skills/{claude-code/agents → agents}/code-architect.md +1 -1
  350. package/bundled/skills/{claude-code/agents → agents}/doc-writer.md +1 -1
  351. package/bundled/skills/{claude-code/agents → agents}/reviewer.md +1 -1
  352. package/bundled/skills/{claude-code/agents → agents}/spec-analyst.md +1 -1
  353. package/bundled/skills/{claude-code/agents → agents}/test-specialist.md +1 -1
  354. package/bundled/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/SKILL.md +28 -5
  355. package/bundled/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/anti-hallucination.md +24 -5
  356. package/bundled/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/certainty-labels.md +31 -6
  357. package/bundled/skills/ai-friendly-architecture/SKILL.md +216 -0
  358. package/bundled/skills/ai-instruction-standards/SKILL.md +209 -0
  359. package/bundled/skills/atdd-assistant/SKILL.md +74 -0
  360. package/bundled/skills/{claude-code/atdd-assistant → atdd-assistant}/acceptance-criteria-guide.md +1 -1
  361. package/bundled/skills/{claude-code/atdd-assistant → atdd-assistant}/atdd-workflow.md +1 -1
  362. package/bundled/skills/{claude-code/atdd-assistant/SKILL.md → atdd-assistant/guide.md} +11 -11
  363. package/bundled/skills/bdd-assistant/SKILL.md +70 -0
  364. package/bundled/skills/{claude-code/bdd-assistant → bdd-assistant}/bdd-workflow.md +1 -1
  365. package/bundled/skills/{claude-code/bdd-assistant → bdd-assistant}/gherkin-guide.md +1 -1
  366. package/bundled/skills/{claude-code/bdd-assistant/SKILL.md → bdd-assistant/guide.md} +11 -11
  367. package/bundled/skills/changelog-guide/SKILL.md +66 -0
  368. package/bundled/skills/{claude-code/changelog-guide/SKILL.md → changelog-guide/guide.md} +8 -6
  369. package/bundled/skills/checkin-assistant/SKILL.md +52 -0
  370. package/bundled/skills/{claude-code/checkin-assistant/SKILL.md → checkin-assistant/guide.md} +6 -6
  371. package/bundled/skills/code-review-assistant/SKILL.md +53 -0
  372. package/bundled/skills/{claude-code/code-review-assistant → code-review-assistant}/checkin-checklist.md +3 -3
  373. package/bundled/skills/{claude-code/code-review-assistant/SKILL.md → code-review-assistant/guide.md} +7 -5
  374. package/bundled/skills/{claude-code/code-review-assistant → code-review-assistant}/review-checklist.md +3 -3
  375. package/bundled/skills/commands/COMMAND-FAMILY-OVERVIEW.md +290 -0
  376. package/bundled/skills/{claude-code/commands → commands}/README.md +13 -3
  377. package/bundled/skills/{claude-code/commands → commands}/atdd.md +3 -3
  378. package/bundled/skills/{claude-code/commands → commands}/bdd.md +2 -2
  379. package/bundled/skills/commands/changelog.md +58 -0
  380. package/bundled/skills/{claude-code/commands → commands}/check.md +1 -1
  381. package/bundled/skills/commands/checkin.md +53 -0
  382. package/bundled/skills/{claude-code/commands → commands}/commit.md +3 -3
  383. package/bundled/skills/{claude-code/commands → commands}/config.md +1 -1
  384. package/bundled/skills/{claude-code/commands → commands}/coverage.md +5 -4
  385. package/bundled/skills/commands/derive-all.md +53 -0
  386. package/bundled/skills/commands/derive-atdd.md +49 -0
  387. package/bundled/skills/commands/derive-bdd.md +47 -0
  388. package/bundled/skills/commands/derive-tdd.md +48 -0
  389. package/bundled/skills/commands/derive.md +72 -0
  390. package/bundled/skills/commands/discover.md +80 -0
  391. package/bundled/skills/commands/docgen.md +61 -0
  392. package/bundled/skills/commands/docs.md +59 -0
  393. package/bundled/skills/commands/guide.md +44 -0
  394. package/bundled/skills/{claude-code/commands → commands}/init.md +2 -2
  395. package/bundled/skills/{claude-code/commands → commands}/methodology.md +32 -16
  396. package/bundled/skills/{claude-code/commands → commands}/refactor.md +2 -2
  397. package/bundled/skills/commands/release.md +56 -0
  398. package/bundled/skills/{claude-code/commands → commands}/requirement.md +3 -3
  399. package/bundled/skills/commands/reverse-bdd.md +47 -0
  400. package/bundled/skills/commands/reverse-sdd.md +51 -0
  401. package/bundled/skills/commands/reverse-tdd.md +51 -0
  402. package/bundled/skills/commands/reverse.md +63 -0
  403. package/bundled/skills/{claude-code/commands → commands}/review.md +3 -3
  404. package/bundled/skills/commands/sdd.md +135 -0
  405. package/bundled/skills/{claude-code/commands → commands}/tdd.md +3 -3
  406. package/bundled/skills/{claude-code/commands → commands}/update.md +2 -2
  407. package/bundled/skills/commit-standards/SKILL.md +60 -0
  408. package/bundled/skills/{claude-code/commit-standards → commit-standards}/conventional-commits.md +3 -3
  409. package/bundled/skills/{claude-code/commit-standards/SKILL.md → commit-standards/guide.md} +17 -5
  410. package/bundled/skills/{claude-code/commit-standards → commit-standards}/language-options.md +2 -2
  411. package/bundled/skills/docs-generator/SKILL.md +69 -0
  412. package/bundled/skills/docs-generator/guide.md +258 -0
  413. package/bundled/skills/{claude-code/documentation-guide → documentation-guide}/SKILL.md +7 -7
  414. package/bundled/skills/{claude-code/documentation-guide → documentation-guide}/documentation-structure.md +3 -3
  415. package/bundled/skills/{claude-code/documentation-guide → documentation-guide}/readme-template.md +2 -2
  416. package/bundled/skills/{claude-code/error-code-guide → error-code-guide}/SKILL.md +7 -5
  417. package/bundled/skills/forward-derivation/SKILL.md +61 -0
  418. package/bundled/skills/{claude-code/forward-derivation/SKILL.md → forward-derivation/guide.md} +44 -29
  419. package/bundled/skills/{claude-code/git-workflow-guide → git-workflow-guide}/SKILL.md +7 -5
  420. package/bundled/skills/{claude-code/git-workflow-guide → git-workflow-guide}/branch-naming.md +2 -2
  421. package/bundled/skills/{claude-code/git-workflow-guide → git-workflow-guide}/git-workflow.md +3 -3
  422. package/bundled/skills/{claude-code/logging-guide → logging-guide}/SKILL.md +7 -5
  423. package/bundled/skills/methodology-system/SKILL.md +75 -0
  424. package/bundled/skills/{claude-code/methodology-system → methodology-system}/create-methodology.md +1 -1
  425. package/bundled/skills/{claude-code/methodology-system/SKILL.md → methodology-system/guide.md} +51 -16
  426. package/bundled/skills/methodology-system/integrated-flow.md +454 -0
  427. package/bundled/skills/{claude-code/methodology-system → methodology-system}/runtime.md +2 -2
  428. package/bundled/skills/project-discovery/SKILL.md +63 -0
  429. package/bundled/skills/project-discovery/guide.md +367 -0
  430. package/bundled/skills/project-discovery/workflow.md +566 -0
  431. package/bundled/skills/{claude-code/project-structure-guide → project-structure-guide}/SKILL.md +14 -4
  432. package/bundled/skills/refactoring-assistant/SKILL.md +78 -0
  433. package/bundled/skills/{claude-code/refactoring-assistant/SKILL.md → refactoring-assistant/guide.md} +7 -7
  434. package/bundled/skills/release-standards/SKILL.md +61 -0
  435. package/bundled/skills/{claude-code/release-standards → release-standards}/changelog-format.md +2 -2
  436. package/bundled/skills/{claude-code/release-standards/SKILL.md → release-standards/guide.md} +7 -5
  437. package/bundled/skills/{claude-code/release-standards → release-standards}/release-workflow.md +59 -24
  438. package/bundled/skills/{claude-code/release-standards → release-standards}/semantic-versioning.md +2 -2
  439. package/bundled/skills/requirement-assistant/SKILL.md +66 -0
  440. package/bundled/skills/{claude-code/requirement-assistant/SKILL.md → requirement-assistant/guide.md} +14 -6
  441. package/bundled/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-checklist.md +2 -2
  442. package/bundled/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-writing.md +2 -2
  443. package/bundled/skills/reverse-engineer/SKILL.md +64 -0
  444. package/bundled/skills/{claude-code/reverse-engineer → reverse-engineer}/bdd-extraction.md +1 -1
  445. package/bundled/skills/{claude-code/reverse-engineer/SKILL.md → reverse-engineer/guide.md} +21 -18
  446. package/bundled/skills/{claude-code/reverse-engineer → reverse-engineer}/tdd-analysis.md +1 -1
  447. package/bundled/skills/{claude-code/reverse-engineer → reverse-engineer}/workflow.md +2 -2
  448. package/bundled/skills/spec-driven-dev/SKILL.md +80 -0
  449. package/bundled/skills/{claude-code/spec-driven-dev/SKILL.md → spec-driven-dev/guide.md} +50 -8
  450. package/bundled/skills/tdd-assistant/SKILL.md +65 -0
  451. package/bundled/skills/{claude-code/tdd-assistant/SKILL.md → tdd-assistant/guide.md} +10 -10
  452. package/bundled/skills/{claude-code/tdd-assistant → tdd-assistant}/language-examples.md +3 -3
  453. package/bundled/skills/{claude-code/tdd-assistant → tdd-assistant}/tdd-workflow.md +2 -2
  454. package/bundled/skills/test-coverage-assistant/SKILL.md +63 -0
  455. package/bundled/skills/{claude-code/test-coverage-assistant/SKILL.md → test-coverage-assistant/guide.md} +39 -24
  456. package/bundled/skills/{claude-code/testing-guide → testing-guide}/SKILL.md +12 -7
  457. package/bundled/skills/{claude-code/testing-guide → testing-guide}/testing-pyramid.md +3 -3
  458. package/bundled/skills/testing-guide/testing-theory.md +2291 -0
  459. package/bundled/skills/tools/cline/.clinerules +79 -0
  460. package/bundled/skills/tools/cline/README.md +50 -0
  461. package/bundled/skills/tools/copilot/README.md +99 -0
  462. package/bundled/skills/tools/copilot/copilot-instructions.md +71 -0
  463. package/bundled/skills/tools/cursor/.cursorrules +101 -0
  464. package/bundled/skills/tools/cursor/README.md +50 -0
  465. package/bundled/skills/tools/windsurf/.windsurfrules +76 -0
  466. package/bundled/skills/tools/windsurf/README.md +51 -0
  467. package/bundled/skills/{claude-code/workflows → workflows}/README.md +1 -1
  468. package/bundled/skills/workflows/release.workflow.yaml +893 -0
  469. package/package.json +4 -1
  470. package/src/commands/check.js +195 -130
  471. package/src/commands/config.js +1262 -0
  472. package/src/commands/configure.js +3 -950
  473. package/src/commands/fix.js +53 -0
  474. package/src/commands/hitl.js +19 -0
  475. package/src/commands/init.js +320 -1030
  476. package/src/commands/list.js +1 -1
  477. package/src/commands/simulate.js +61 -0
  478. package/src/commands/spec.js +288 -0
  479. package/src/commands/start.js +373 -0
  480. package/src/commands/sweep.js +142 -0
  481. package/src/commands/update.js +136 -19
  482. package/src/commands/workflow.js +265 -1
  483. package/src/config/ai-agent-paths.js +89 -29
  484. package/src/core/constants.js +7 -1
  485. package/src/core/manifest.js +1 -0
  486. package/src/flows/init-flow.js +243 -0
  487. package/src/hitl/checkpoint.js +80 -0
  488. package/src/hitl/classifier.js +54 -0
  489. package/src/hitl/manager.js +88 -0
  490. package/src/i18n/messages.js +907 -99
  491. package/src/index.js +1 -0
  492. package/src/installers/integration-installer.js +237 -0
  493. package/src/installers/manifest-installer.js +131 -0
  494. package/src/installers/skills-installer.js +286 -0
  495. package/src/installers/standards-installer.js +193 -0
  496. package/src/missions/MissionManager.js +501 -0
  497. package/src/prompts/init.js +228 -130
  498. package/src/prompts/integrations.js +13 -1
  499. package/src/schemas/standard.schema.json +117 -0
  500. package/src/utils/agents-installer.js +2 -2
  501. package/src/utils/config-loader.js +47 -0
  502. package/src/utils/config-manager.js +108 -0
  503. package/src/utils/config-merger.js +47 -0
  504. package/src/utils/github.js +22 -18
  505. package/src/utils/inquirer-patch.js +98 -0
  506. package/src/utils/integration-generator.js +392 -58
  507. package/src/utils/reference-sync.js +69 -19
  508. package/src/utils/skills-installer.js +6 -3
  509. package/src/utils/standard-fixer.js +141 -0
  510. package/src/utils/standard-validator.js +329 -0
  511. package/src/utils/workflow-executor.js +886 -0
  512. package/src/utils/workflow-state.js +404 -0
  513. package/src/utils/workflows-installer.js +2 -2
  514. package/src/vibe/auto-sweep.js +349 -0
  515. package/src/vibe/micro-spec.js +428 -0
  516. package/standards-registry.json +278 -102
  517. package/bundled/locales/zh-CN/skills/claude-code/README.md +0 -174
  518. package/bundled/locales/zh-CN/skills/claude-code/test-coverage-assistant/SKILL.md +0 -282
  519. package/bundled/locales/zh-TW/skills/claude-code/README.md +0 -174
  520. package/bundled/skills/claude-code/README.md +0 -215
  521. package/bundled/skills/claude-code/commands/changelog.md +0 -57
  522. package/bundled/skills/claude-code/commands/derive-all.md +0 -214
  523. package/bundled/skills/claude-code/commands/derive-atdd.md +0 -177
  524. package/bundled/skills/claude-code/commands/derive-bdd.md +0 -158
  525. package/bundled/skills/claude-code/commands/derive-tdd.md +0 -259
  526. package/bundled/skills/claude-code/commands/docs.md +0 -75
  527. package/bundled/skills/claude-code/commands/release.md +0 -50
  528. package/bundled/skills/claude-code/commands/reverse-bdd.md +0 -264
  529. package/bundled/skills/claude-code/commands/reverse-spec.md +0 -185
  530. package/bundled/skills/claude-code/commands/reverse-tdd.md +0 -351
  531. package/bundled/skills/claude-code/commands/spec.md +0 -69
  532. package/bundled/skills/claude-code/methodology-system/integrated-flow.md +0 -322
  533. /package/bundled/skills/{claude-code/CONTRIBUTING.template.md → CONTRIBUTING.template.md} +0 -0
  534. /package/bundled/skills/{claude-code/ai → ai}/standards/checkin.ai.yaml +0 -0
  535. /package/bundled/skills/{claude-code/ai → ai}/standards/commit.ai.yaml +0 -0
  536. /package/bundled/skills/{claude-code/ai → ai}/standards/refactoring.ai.yaml +0 -0
  537. /package/bundled/skills/{claude-code/ai → ai}/standards/testing.ai.yaml +0 -0
  538. /package/bundled/skills/{claude-code/project-structure-guide → project-structure-guide}/language-patterns.md +0 -0
  539. /package/bundled/skills/{claude-code/workflows → workflows}/code-review.workflow.yaml +0 -0
  540. /package/bundled/skills/{claude-code/workflows → workflows}/feature-dev.workflow.yaml +0 -0
  541. /package/bundled/skills/{claude-code/workflows → workflows}/integrated-flow.workflow.yaml +0 -0
  542. /package/bundled/skills/{claude-code/workflows → workflows}/large-codebase-analysis.workflow.yaml +0 -0
@@ -2,2452 +2,216 @@
2
2
 
3
3
  > **Language**: English | [繁體中文](../locales/zh-TW/core/testing-standards.md)
4
4
 
5
- **Version**: 2.2.0
6
- **Last Updated**: 2026-01-20
5
+ **Version**: 3.0.0
6
+ **Last Updated**: 2026-01-29
7
7
  **Applicability**: All software projects
8
+ **Scope**: universal
9
+ **Industry Standards**: ISTQB CTFL v4.0, ISO/IEC/IEEE 29119
10
+ **References**: [istqb.org](https://istqb.org/)
8
11
 
9
12
  ---
10
13
 
11
14
  ## Purpose
12
15
 
13
- This standard defines testing conventions and best practices to ensure software quality through systematic testing at multiple levels.
16
+ This standard defines actionable testing rules and conventions for AI agents and developers. For theoretical foundations, educational content, and detailed examples, see [Testing Theory Knowledge Base](../skills/testing-guide/testing-theory.md).
14
17
 
15
18
  **Reference Standards**:
16
19
  - [ISTQB CTFL v4.0](https://istqb.org/certifications/certified-tester-foundation-level-ctfl-v4-0/)
17
20
  - [ISO/IEC/IEEE 29119](https://www.iso.org/standard/81291.html)
18
- - [SWEBOK v4.0](https://www.computer.org/education/bodies-of-knowledge/software-engineering) - IEEE Computer Society
21
+ - [SWEBOK v4.0](https://www.computer.org/education/bodies-of-knowledge/software-engineering)
19
22
 
20
23
  ---
21
24
 
22
- ## Table of Contents
23
-
24
- 1. [Testing Fundamentals](#testing-fundamentals) ⭐ NEW
25
- 2. [Testing Framework Selection](#testing-framework-selection)
26
- 3. [Testing Pyramid](#testing-pyramid-default-industry-standard)
27
- 4. [Test Levels](#unit-testing-ut) (UT, IT, ST, E2E)
28
- 5. [Static Testing](#static-testing)
29
- 6. [Test Design Techniques](#test-design-techniques)
30
- 7. [Experience-Based Testing](#experience-based-testing)
31
- 8. [Risk-Based Testing](#risk-based-testing)
32
- 9. [Specialized Testing](#specialized-testing)
33
- 10. [Test-Related Measures](#test-related-measures) ⭐ NEW
34
- 11. [Defect Management](#defect-management)
35
- 12. [Test Process Management](#test-process-management)
36
- 13. [Test Doubles](#test-doubles)
37
- 14. [Test Data Management](#test-data-management)
38
- 15. [Test Environment Isolation](#test-environment-isolation)
39
- 16. [CI/CD Integration](#cicd-integration)
40
- 17. [Best Practices](#best-practices)
41
- 18. [Test Documentation Structure](#test-documentation-structure) ⭐ NEW
25
+ ## Glossary
42
26
 
43
- ---
44
-
45
- ## Testing Fundamentals
46
-
47
- Understanding the theoretical foundations of software testing is essential for effective test design and execution.
48
-
49
- **Reference**: SWEBOK v4.0 Chapter 5
50
-
51
- ### Faults vs Failures
52
-
53
- ```
54
- ┌─────────────────────────────────────────────────────────────┐
55
- │ Faults, Errors, and Failures │
56
- ├─────────────────────────────────────────────────────────────┤
57
- │ │
58
- │ Error (Mistake) │
59
- │ │ │
60
- │ ▼ introduced during development │
61
- │ ┌─────────┐ │
62
- │ │ Fault │ (Defect/Bug in the code) │
63
- │ └─────────┘ │
64
- │ │ │
65
- │ ▼ when executed │
66
- │ ┌─────────┐ │
67
- │ │ Failure │ (Observable incorrect behavior) │
68
- │ └─────────┘ │
69
- │ │
70
- │ Key Points: │
71
- │ • A fault may exist without causing a failure │
72
- │ • A failure requires a fault to be executed │
73
- │ • Multiple faults can cause the same failure │
74
- │ • Testing finds failures; debugging finds faults │
75
- │ │
76
- └─────────────────────────────────────────────────────────────┘
77
- ```
78
-
79
- ### The Oracle Problem
80
-
81
- An **oracle** is any mechanism that determines whether observed program behavior is correct. The oracle problem addresses the challenge of knowing expected outcomes.
82
-
83
- ```
84
- ┌─────────────────────────────────────────────────────────────┐
85
- │ Oracle Types │
86
- ├─────────────────────────────────────────────────────────────┤
87
- │ │
88
- │ Human Oracle │
89
- │ └── Manual verification by domain expert │
90
- │ │
91
- │ Specification-Based Oracle │
92
- │ └── Compare against formal requirements │
93
- │ │
94
- │ Reference Implementation │
95
- │ └── Compare against known-correct implementation │
96
- │ │
97
- │ Metamorphic Oracle │
98
- │ └── Verify relationships between outputs │
99
- │ (e.g., sort(sort(x)) == sort(x)) │
100
- │ │
101
- │ Statistical Oracle │
102
- │ └── Validate against expected distributions │
103
- │ │
104
- │ Implicit Oracle │
105
- │ └── No crash, no exception, no timeout │
106
- │ │
107
- └─────────────────────────────────────────────────────────────┘
108
- ```
109
-
110
- ### Theoretical Limitations of Testing
111
-
112
- > "Program testing can be used to show the presence of bugs, but never to show their absence." — Edsger W. Dijkstra
113
-
114
- ```
115
- ┌─────────────────────────────────────────────────────────────┐
116
- │ Testing Limitations │
117
- ├─────────────────────────────────────────────────────────────┤
118
- │ │
119
- │ Exhaustive Testing is Impossible │
120
- │ ├── Infinite input domain │
121
- │ ├── State space explosion │
122
- │ └── Timing and concurrency variations │
123
- │ │
124
- │ Infeasible Paths │
125
- │ ├── Some code paths cannot be executed by any input │
126
- │ ├── Dead code may appear reachable in CFG │
127
- │ └── 100% path coverage may be mathematically impossible │
128
- │ │
129
- │ Halting Problem │
130
- │ └── Cannot algorithmically determine if all tests halt │
131
- │ │
132
- │ Implications: │
133
- │ • Testing is sampling, not proof │
134
- │ • Risk-based prioritization is essential │
135
- │ • Combine testing with formal methods where critical │
136
- │ │
137
- └─────────────────────────────────────────────────────────────┘
138
- ```
139
-
140
- ### Testability
141
-
142
- Testability measures how easily software can be tested effectively.
143
-
144
- | Factor | Description | How to Improve |
145
- |--------|-------------|----------------|
146
- | **Controllability** | Ability to set system state for testing | Provide test hooks, dependency injection |
147
- | **Observability** | Ability to observe test results | Add logging, expose internal state |
148
- | **Isolability** | Ability to test components independently | Modular design, clear interfaces |
149
- | **Understandability** | Clarity of expected behavior | Good documentation, clear specifications |
150
- | **Stability** | Low rate of change | Freeze features before testing phase |
151
- | **Simplicity** | Low complexity | Reduce cyclomatic complexity |
152
-
153
- ### Test Adequacy Criteria
154
-
155
- Test adequacy criteria answer: "Have we tested enough?"
156
-
157
- ```
158
- ┌─────────────────────────────────────────────────────────────┐
159
- │ Test Adequacy Criteria │
160
- ├─────────────────────────────────────────────────────────────┤
161
- │ │
162
- │ Coverage-Based Criteria │
163
- │ ├── Statement Coverage: % of statements executed │
164
- │ ├── Branch Coverage: % of branches taken │
165
- │ ├── Condition Coverage: % of conditions evaluated │
166
- │ └── MC/DC: Modified condition/decision coverage │
167
- │ │
168
- │ Fault-Based Criteria │
169
- │ ├── Mutation Score: % of mutants killed │
170
- │ └── Fault Seeding: % of seeded faults found │
171
- │ │
172
- │ Requirements-Based Criteria │
173
- │ └── Requirements Coverage: % of requirements tested │
174
- │ │
175
- │ Risk-Based Criteria │
176
- │ └── Risk Coverage: % of high-risk items tested │
177
- │ │
178
- │ Stopping Rules: │
179
- │ • Achieved target coverage level │
180
- │ • Defect discovery rate drops below threshold │
181
- │ • Budget/time exhausted (with documented risk) │
182
- │ │
183
- └─────────────────────────────────────────────────────────────┘
184
- ```
185
-
186
- ---
187
-
188
- ## Testing Framework Selection
189
-
190
- Choose a testing framework based on your project needs. This standard supports two approaches:
191
-
192
- ### Option A: ISTQB Standard Framework
193
-
194
- Best for enterprise projects, certification needs, and formal QA processes.
195
-
196
- **Reference**: [ISTQB Glossary v4.0](https://glossary.istqb.org)
197
-
198
- | Level | Abbreviation | Purpose |
199
- |-------|--------------|---------|
200
- | Unit Testing | UT | Verify individual code units |
201
- | Integration Testing | IT/SIT | Verify component interactions |
202
- | System Testing | ST | Verify system meets requirements |
203
- | Acceptance Testing | AT/UAT | Verify system meets business needs |
204
-
205
- **When to choose ISTQB**:
206
- - Enterprise projects with formal QA processes
207
- - Projects requiring certification or compliance
208
- - Organizations with dedicated QA teams
209
- - Projects with strict audit requirements
210
-
211
- ### Option B: Industry Testing Pyramid
212
-
213
- Best for agile development, CI/CD optimization, and rapid iteration.
214
-
215
- **Reference**: [Martin Fowler's Testing Pyramid](https://martinfowler.com/bliki/TestPyramid.html), [Google Testing Blog](https://testing.googleblog.com)
216
-
217
- | Level | Abbreviation | Ratio | Purpose |
218
- |-------|--------------|-------|---------|
219
- | Unit Testing | UT | 70% | Isolated component tests |
220
- | Integration Testing | IT/SIT* | 20% | Component interaction tests |
221
- | E2E Testing | E2E | 10% | User workflow tests |
222
-
223
- *Note on Integration Testing abbreviation:
224
- - **IT** (Integration Testing): Common in Agile/DevOps communities
225
- - **SIT** (System Integration Testing): Common in Enterprise/ISTQB contexts
226
- - Both terms refer to the same testing level
227
-
228
- **When to choose Industry Pyramid**:
229
- - Agile/Scrum development teams
230
- - CI/CD focused environments
231
- - Small to medium projects with rapid iteration
232
- - DevOps practices
233
-
234
- ---
235
-
236
- ## Testing Pyramid (Default: Industry Standard)
237
-
238
- ```
239
- ┌─────────┐
240
- │ E2E │ ← 10% (Fewer, slower, expensive)
241
- ─┴─────────┴─
242
- ┌─────────────┐
243
- │ IT/SIT │ ← 20% (Integration Testing)
244
- ─┴─────────────┴─
245
- ┌─────────────────┐
246
- │ UT │ ← 70% (Unit Testing - Foundation)
247
- └─────────────────┘
248
- ```
249
-
250
- ### Recommended Ratio (Industry Pyramid)
251
-
252
- | Level | Abbreviation | Percentage | Execution Time |
253
- |-------|--------------|------------|----------------|
254
- | Unit Testing | UT | 70% | < 10 min total |
255
- | Integration Testing | IT/SIT | 20% | < 30 min total |
256
- | E2E Testing | E2E | 10% | < 2 hours total |
257
-
258
- ### ISTQB 4-Level Structure (Alternative)
259
-
260
- | Level | Abbreviation | Performed By | Focus |
261
- |-------|--------------|--------------|-------|
262
- | Unit Testing | UT | Developers | Code correctness |
263
- | Integration Testing | IT/SIT | Developers/QA | Interface contracts |
264
- | System Testing | ST | QA Team | Requirements verification |
265
- | Acceptance Testing | AT/UAT | End Users | Business validation |
266
-
267
- ---
268
-
269
- ## Unit Testing (UT)
270
-
271
- ### Definition
272
-
273
- Tests individual functions, methods, or classes in isolation from external dependencies.
274
-
275
- ### Characteristics
276
-
277
- - **Isolated**: No database, network, or file system access
278
- - **Fast**: Each test < 100ms
279
- - **Deterministic**: Same input always produces same output
280
-
281
- ### Scope
282
-
283
- ```
284
- ┌─────────────────────────────────────────┐
285
- │ Unit Under Test │
286
- ├─────────────────────────────────────────┤
287
- │ ✅ Single function/method │
288
- │ ✅ Single class │
289
- │ ✅ Pure business logic │
290
- │ ✅ Data transformations │
291
- │ ✅ Validation rules │
292
- ├─────────────────────────────────────────┤
293
- │ ❌ Database queries │
294
- │ ❌ External API calls │
295
- │ ❌ File I/O operations │
296
- │ ❌ Multi-class interactions │
297
- └─────────────────────────────────────────┘
298
- ```
299
-
300
- ### Naming Convention
301
-
302
- **File Naming**:
303
- ```
304
- [ClassName]Tests.[ext]
305
- [ClassName].test.[ext]
306
- [ClassName].spec.[ext]
307
-
308
- Examples:
309
- UserService.test.ts
310
- UserServiceTests.cs
311
- user_service_test.py
312
- user_service_test.go
313
- ```
314
-
315
- **Method Naming**:
316
- ```
317
- [MethodName]_[Scenario]_[ExpectedResult]
318
- should_[ExpectedBehavior]_when_[Condition]
319
- test_[method]_[scenario]_[expected]
320
-
321
- Examples:
322
- CalculateTotal_WithDiscount_ReturnsDiscountedPrice()
323
- should_return_null_when_user_not_found()
324
- test_validate_email_invalid_format_returns_false()
325
- ```
326
-
327
- ### Coverage Guidelines
328
-
329
- | Metric | Minimum | Recommended |
330
- |--------|---------|-------------|
331
- | Line Coverage | 70% | 85% |
332
- | Branch Coverage | 60% | 80% |
333
- | Function Coverage | 80% | 90% |
334
-
335
- ### Example
336
-
337
- ```csharp
338
- // C# Example
339
- [TestClass]
340
- public class UserValidatorTests
341
- {
342
- private UserValidator _validator;
343
-
344
- [TestInitialize]
345
- public void Setup()
346
- {
347
- _validator = new UserValidator();
348
- }
349
-
350
- [TestMethod]
351
- public void ValidateEmail_ValidFormat_ReturnsTrue()
352
- {
353
- // Arrange
354
- var email = "user@example.com";
355
-
356
- // Act
357
- var result = _validator.ValidateEmail(email);
358
-
359
- // Assert
360
- Assert.IsTrue(result);
361
- }
362
-
363
- [TestMethod]
364
- public void ValidateEmail_InvalidFormat_ReturnsFalse()
365
- {
366
- // Arrange
367
- var email = "invalid-email";
368
-
369
- // Act
370
- var result = _validator.ValidateEmail(email);
371
-
372
- // Assert
373
- Assert.IsFalse(result);
374
- }
375
- }
376
- ```
377
-
378
- ```typescript
379
- // TypeScript Example
380
- describe('UserValidator', () => {
381
- let validator: UserValidator;
382
-
383
- beforeEach(() => {
384
- validator = new UserValidator();
385
- });
386
-
387
- describe('validateEmail', () => {
388
- it('should return true for valid email format', () => {
389
- const result = validator.validateEmail('user@example.com');
390
- expect(result).toBe(true);
391
- });
392
-
393
- it('should return false for invalid email format', () => {
394
- const result = validator.validateEmail('invalid-email');
395
- expect(result).toBe(false);
396
- });
397
- });
398
- });
399
- ```
400
-
401
- ---
402
-
403
- ## Integration Testing (IT)
404
-
405
- ### Definition
406
-
407
- Tests interactions between multiple components, modules, or external systems.
408
-
409
- ### When Integration Tests Are Required
410
-
411
- | Scenario | Reason |
412
- |----------|--------|
413
- | Query predicates | Mocks cannot verify filter expressions |
414
- | Entity relationships | Verify foreign key correctness |
415
- | Composite keys | In-memory DB may differ from real DB |
416
- | Field mapping | DTO ↔ Entity transformations |
417
- | Pagination | Row ordering and counting |
418
- | Transactions | Rollback behavior |
419
-
420
- **Decision Rule**:
421
- If your unit test uses a wildcard matcher (`any()`, `It.IsAny<>`, `Arg.Any<>`)
422
- for a query/filter parameter, that functionality MUST have an integration test.
423
-
424
- ### Characteristics
425
-
426
- - **Component Integration**: Tests module boundaries
427
- - **Real Dependencies**: Uses actual databases, APIs (often containerized)
428
- - **Slower**: Each test typically 1-10 seconds
429
-
430
- ### Scope
431
-
432
- ```
433
- ┌─────────────────────────────────────────┐
434
- │ Integration Test Scope │
435
- ├─────────────────────────────────────────┤
436
- │ ✅ Database CRUD operations │
437
- │ ✅ Repository + Database │
438
- │ ✅ Service + Repository │
439
- │ ✅ API endpoint + Service layer │
440
- │ ✅ Message queue producers/consumers │
441
- │ ✅ Cache read/write operations │
442
- ├─────────────────────────────────────────┤
443
- │ ❌ Full user workflows │
444
- │ ❌ Cross-service communication │
445
- │ ❌ UI interactions │
446
- └─────────────────────────────────────────┘
447
- ```
448
-
449
- ### Naming Convention
450
-
451
- **File Naming**:
452
- ```
453
- [ComponentName]IntegrationTests.[ext]
454
- [ComponentName].integration.test.[ext]
455
- [ComponentName].itest.[ext]
456
-
457
- Examples:
458
- UserRepositoryIntegrationTests.cs
459
- user-service.integration.test.ts
460
- user_repository_itest.py
461
- ```
462
-
463
- **Method Naming**:
464
- ```
465
- [Operation]_[Context]_[ExpectedOutcome]
466
-
467
- Examples:
468
- CreateUser_WithValidData_PersistsToDatabase()
469
- GetUserById_ExistingUser_ReturnsUserFromDatabase()
470
- SendMessage_ToQueue_ConsumerReceivesMessage()
471
- ```
472
-
473
- ### Test Fixtures
474
-
475
- ```csharp
476
- // C# Integration Test Example with Test Database
477
- [TestClass]
478
- public class UserRepositoryIntegrationTests
479
- {
480
- private TestDbContext _dbContext;
481
- private UserRepository _repository;
482
-
483
- [TestInitialize]
484
- public async Task Setup()
485
- {
486
- // Use test database (e.g., SQLite in-memory or Testcontainers)
487
- _dbContext = TestDbContextFactory.Create();
488
- _repository = new UserRepository(_dbContext);
489
- await _dbContext.Database.EnsureCreatedAsync();
490
- }
491
-
492
- [TestCleanup]
493
- public async Task Cleanup()
494
- {
495
- await _dbContext.DisposeAsync();
496
- }
497
-
498
- [TestMethod]
499
- public async Task CreateUser_WithValidData_PersistsToDatabase()
500
- {
501
- // Arrange
502
- var user = new User { Name = "Test User", Email = "test@example.com" };
503
-
504
- // Act
505
- await _repository.CreateAsync(user);
506
- var savedUser = await _repository.GetByIdAsync(user.Id);
507
-
508
- // Assert
509
- Assert.IsNotNull(savedUser);
510
- Assert.AreEqual("Test User", savedUser.Name);
511
- }
512
- }
513
- ```
514
-
515
- ---
516
-
517
- ## System Testing (ST)
518
-
519
- ### Definition
520
-
521
- Tests the complete integrated system to verify it meets specified requirements.
522
-
523
- ### Characteristics
524
-
525
- - **Complete System**: All components deployed and integrated
526
- - **Requirement-Based**: Tests against functional specifications
527
- - **Production-Like**: Uses environment similar to production
528
-
529
- ### Scope
530
-
531
- ```
532
- ┌─────────────────────────────────────────┐
533
- │ System Test Scope │
534
- ├─────────────────────────────────────────┤
535
- │ ✅ Complete API workflows │
536
- │ ✅ Cross-service transactions │
537
- │ ✅ Data flow through entire system │
538
- │ ✅ Security requirements │
539
- │ ✅ Performance under load │
540
- │ ✅ Error handling & recovery │
541
- │ ✅ Configuration validation │
542
- ├─────────────────────────────────────────┤
543
- │ ❌ UI visual testing │
544
- │ ❌ User journey simulations │
545
- │ ❌ A/B testing scenarios │
546
- └─────────────────────────────────────────┘
547
- ```
548
-
549
- ### Types of System Tests
550
-
551
- | Type | Description |
552
- |------|-------------|
553
- | Functional | Verify features work as specified |
554
- | Performance | Load, stress, scalability testing |
555
- | Security | Penetration, vulnerability scanning |
556
- | Reliability | Failover, recovery, stability |
557
- | Compatibility | Cross-platform, browser compatibility |
558
-
559
- ### Naming Convention
560
-
561
- **File Naming**:
562
- ```
563
- [Feature]SystemTests.[ext]
564
- [Feature].system.test.[ext]
565
- [Feature]_st.[ext]
566
-
567
- Examples:
568
- OrderProcessingSystemTests.cs
569
- authentication.system.test.ts
570
- payment_processing_st.py
571
- ```
572
-
573
- ### Example
574
-
575
- ```csharp
576
- // System Test Example: Complete Resource Processing Flow
577
- // Note: Replace {Resource}, {Item}, {Action} with your domain concepts
578
- [TestClass]
579
- public class ResourceProcessingSystemTests
580
- {
581
- private HttpClient _client;
582
- private TestEnvironment _env;
583
-
584
- [TestInitialize]
585
- public async Task Setup()
586
- {
587
- _env = await TestEnvironment.CreateAsync();
588
- _client = _env.CreateAuthenticatedClient();
589
- }
590
-
591
- [TestMethod]
592
- public async Task ProcessResource_CompleteFlow_CompletedSuccessfully()
593
- {
594
- // Arrange: Create test data
595
- var item = await _env.CreateTestItem(value: 100);
596
- var user = await _env.CreateTestUser();
597
-
598
- // Act: Execute complete processing flow
599
- // Step 1: Create request
600
- var requestResponse = await _client.PostAsync("/api/requests",
601
- new { itemId = item.Id, quantity = 2 });
602
- Assert.AreEqual(HttpStatusCode.OK, requestResponse.StatusCode);
603
-
604
- // Step 2: Submit processing
605
- var processResponse = await _client.PostAsync("/api/processes",
606
- new { requestId = requestResponse.RequestId, userId = user.Id });
607
- var process = await processResponse.Content.ReadAsAsync<Process>();
608
- Assert.AreEqual(HttpStatusCode.Created, processResponse.StatusCode);
609
-
610
- // Step 3: Confirm completion
611
- var confirmResponse = await _client.PostAsync($"/api/processes/{process.Id}/confirm",
612
- new { confirmationType = "standard", amount = 200 });
613
- Assert.AreEqual(HttpStatusCode.OK, confirmResponse.StatusCode);
614
-
615
- // Assert: Verify final state
616
- var finalProcess = await _client.GetAsync($"/api/processes/{process.Id}");
617
- var result = await finalProcess.Content.ReadAsAsync<Process>();
618
-
619
- Assert.AreEqual(ProcessStatus.Completed, result.Status);
620
- Assert.AreEqual(200, result.TotalAmount);
621
- Assert.IsNotNull(result.Confirmation);
622
- }
623
- }
624
- ```
625
-
626
- ---
627
-
628
- ## End-to-End Testing (E2E)
629
-
630
- ### Definition
631
-
632
- Tests complete user workflows from the user interface through all system layers.
633
-
634
- ### Characteristics
635
-
636
- - **User Perspective**: Simulates real user interactions
637
- - **Full Stack**: UI → API → Database → External Services
638
- - **Slowest**: Each test typically 30 seconds to several minutes
639
-
640
- ### Scope
641
-
642
- ```
643
- ┌─────────────────────────────────────────┐
644
- │ E2E Test Scope │
645
- ├─────────────────────────────────────────┤
646
- │ ✅ Critical user journeys │
647
- │ ✅ Login/Authentication flows │
648
- │ ✅ Core business transactions │
649
- │ ✅ Cross-browser functionality │
650
- │ ✅ Smoke tests for deployments │
651
- ├─────────────────────────────────────────┤
652
- │ ❌ Every possible user path │
653
- │ ❌ Edge cases (use UT/IT) │
654
- │ ❌ Performance benchmarking │
655
- └─────────────────────────────────────────┘
656
- ```
657
-
658
- ### Naming Convention
659
-
660
- **File Naming**:
661
- ```
662
- [UserJourney].e2e.[ext]
663
- [Feature].e2e.spec.[ext]
664
- e2e/[feature]/[scenario].[ext]
665
-
666
- Examples:
667
- user-registration.e2e.ts
668
- checkout-flow.e2e.spec.ts
669
- e2e/authentication/login.spec.ts
670
- ```
671
-
672
- ### Example
673
-
674
- ```typescript
675
- // Playwright E2E Test Example
676
- import { test, expect } from '@playwright/test';
677
-
678
- test.describe('User Registration Journey', () => {
679
- test('should complete registration and login successfully', async ({ page }) => {
680
- // Step 1: Navigate to registration page
681
- await page.goto('/register');
682
-
683
- // Step 2: Fill registration form
684
- await page.fill('[data-testid="email"]', 'newuser@example.com');
685
- await page.fill('[data-testid="password"]', 'SecurePass123!');
686
- await page.fill('[data-testid="confirm-password"]', 'SecurePass123!');
687
- await page.click('[data-testid="register-button"]');
688
-
689
- // Step 3: Verify registration success
690
- await expect(page.locator('[data-testid="success-message"]'))
691
- .toContainText('Registration successful');
692
-
693
- // Step 4: Login with new account
694
- await page.goto('/login');
695
- await page.fill('[data-testid="email"]', 'newuser@example.com');
696
- await page.fill('[data-testid="password"]', 'SecurePass123!');
697
- await page.click('[data-testid="login-button"]');
698
-
699
- // Step 5: Verify login success and dashboard redirect
700
- await expect(page).toHaveURL('/dashboard');
701
- await expect(page.locator('[data-testid="welcome-message"]'))
702
- .toContainText('Welcome, newuser@example.com');
703
- });
704
- });
705
- ```
706
-
707
- ---
708
-
709
- ## Static Testing
710
-
711
- Static testing examines work products (code, documents, requirements) without executing the software. It complements dynamic testing by finding defects earlier in the SDLC.
712
-
713
- **Reference**: ISTQB CTFL v4.0 Chapter 3, ISO/IEC/IEEE 29119-4
714
-
715
- ### Types of Static Testing
716
-
717
- ```
718
- ┌─────────────────────────────────────────────────────────────┐
719
- │ Static Testing Types │
720
- ├─────────────────────────────────────────────────────────────┤
721
- │ Reviews (Manual) │
722
- │ ├── Informal Review: Ad-hoc, no formal process │
723
- │ ├── Walkthrough: Author-led, educational purpose │
724
- │ ├── Technical Review: Peer-led, find defects │
725
- │ └── Inspection: Formal, metrics-driven, most rigorous │
726
- ├─────────────────────────────────────────────────────────────┤
727
- │ Static Analysis (Tool-based) │
728
- │ ├── Code Analysis: Linters, complexity analyzers │
729
- │ ├── Security Analysis: SAST tools (SonarQube, Checkmarx) │
730
- │ ├── Architecture Analysis: Dependency checks │
731
- │ └── Data Flow Analysis: Variable usage patterns │
732
- └─────────────────────────────────────────────────────────────┘
733
- ```
734
-
735
- ### Review Process (ISTQB)
736
-
737
- | Phase | Activities |
738
- |-------|------------|
739
- | Planning | Define scope, entry criteria, roles |
740
- | Initiate Review | Distribute work products |
741
- | Individual Review | Each reviewer examines independently |
742
- | Issue Communication | Discuss findings in meeting |
743
- | Fixing & Reporting | Author fixes, metrics collected |
744
-
745
- ### Static Analysis Tools by Language
746
-
747
- | Language | Linting | Security | Complexity |
748
- |----------|---------|----------|------------|
749
- | JavaScript/TypeScript | ESLint, Biome | npm audit, Snyk | ESLint complexity rules |
750
- | Python | Pylint, Ruff, Flake8 | Bandit, Safety | Radon |
751
- | Java | Checkstyle, PMD | SpotBugs, OWASP DC | JaCoCo |
752
- | C# | StyleCop, Roslyn | Security Code Scan | NDepend |
753
- | Go | golangci-lint | gosec | gocyclo |
754
-
755
- ### When to Use Static Testing
756
-
757
- | Work Product | Review Type | Tools |
758
- |--------------|-------------|-------|
759
- | Requirements | Inspection, Walkthrough | - |
760
- | Design Documents | Technical Review | Architecture tools |
761
- | Code | Technical Review, Tool Analysis | Linters, SAST |
762
- | Test Plans | Walkthrough | - |
763
- | User Documentation | Walkthrough | Spell/grammar checkers |
764
-
765
- ### Code Review Checklist
766
-
767
- ```
768
- ┌─────────────────────────────────────────────────────────────┐
769
- │ Code Review Focus Areas │
770
- ├─────────────────────────────────────────────────────────────┤
771
- │ Functionality │
772
- │ ├── Does the code do what it's supposed to do? │
773
- │ ├── Are edge cases handled? │
774
- │ └── Is error handling appropriate? │
775
- ├─────────────────────────────────────────────────────────────┤
776
- │ Maintainability │
777
- │ ├── Is the code readable and well-organized? │
778
- │ ├── Are names meaningful? │
779
- │ └── Is complexity manageable? │
780
- ├─────────────────────────────────────────────────────────────┤
781
- │ Security │
782
- │ ├── Input validation present? │
783
- │ ├── No hardcoded secrets? │
784
- │ └── SQL injection / XSS prevention? │
785
- ├─────────────────────────────────────────────────────────────┤
786
- │ Performance │
787
- │ ├── No obvious inefficiencies? │
788
- │ ├── Resource cleanup (connections, files)? │
789
- │ └── Appropriate data structures? │
790
- └─────────────────────────────────────────────────────────────┘
791
- ```
792
-
793
- ### Integration with CI/CD
794
-
795
- ```yaml
796
- # Example: Static analysis in CI pipeline
797
- static-analysis:
798
- stage: test
799
- script:
800
- - npm run lint # Linting
801
- - npm run lint:security # Security analysis
802
- - npx sonarqube-scanner # Quality gates
803
- rules:
804
- - if: $CI_PIPELINE_SOURCE == "push"
805
- ```
806
-
807
- ---
808
-
809
- ## Test Design Techniques
810
-
811
- Test design techniques are methods for deriving and selecting test cases. ISTQB and ISO/IEC/IEEE 29119 categorize them into three main approaches.
812
-
813
- **Reference**: ISTQB CTFL v4.0 Chapter 4, ISO/IEC/IEEE 29119-4
814
-
815
- ### Technique Categories
816
-
817
- ```
818
- ┌─────────────────────────────────────────────────────────────┐
819
- │ Test Design Technique Categories │
820
- ├─────────────────────────────────────────────────────────────┤
821
- │ Specification-Based (Black-Box) │
822
- │ └── Derive tests from requirements/specifications │
823
- ├─────────────────────────────────────────────────────────────┤
824
- │ Structure-Based (White-Box) │
825
- │ └── Derive tests from internal code structure │
826
- ├─────────────────────────────────────────────────────────────┤
827
- │ Experience-Based │
828
- │ └── Derive tests from tester knowledge and intuition │
829
- └─────────────────────────────────────────────────────────────┘
830
- ```
831
-
832
- ### Specification-Based Techniques (Black-Box)
833
-
834
- #### 1. Equivalence Partitioning (EP)
835
-
836
- Divides input data into partitions where all values should be treated the same by the system.
837
-
838
- ```
839
- Input: Age (valid range: 18-65)
840
-
841
- ┌─────────────┬─────────────┬─────────────┐
842
- │ Invalid │ Valid │ Invalid │
843
- │ < 18 │ 18 - 65 │ > 65 │
844
- ├─────────────┼─────────────┼─────────────┤
845
- │ Partition 1│ Partition 2│ Partition 3│
846
- │ Test: 10 │ Test: 30 │ Test: 70 │
847
- └─────────────┴─────────────┴─────────────┘
848
-
849
- Coverage: At least 1 test case per partition
850
- ```
851
-
852
- ```python
853
- # Python Example: Equivalence Partitioning
854
- def test_age_validation():
855
- validator = AgeValidator(min_age=18, max_age=65)
856
-
857
- # Partition 1: Below minimum (invalid)
858
- assert validator.is_valid(10) == False
859
-
860
- # Partition 2: Valid range
861
- assert validator.is_valid(30) == True
862
-
863
- # Partition 3: Above maximum (invalid)
864
- assert validator.is_valid(70) == False
865
- ```
866
-
867
- #### 2. Boundary Value Analysis (BVA)
868
-
869
- Tests at the boundaries of equivalence partitions where defects are most likely.
870
-
871
- ```
872
- Input: Quantity (valid range: 1-100)
873
-
874
- 0 1 2 ... 99 100 101
875
- │ │ │ │ │ │
876
- ▼ ▼ ▼ ▼ ▼ ▼
877
- Invalid Valid Valid Valid Valid Invalid
878
- (boundary)(boundary) (boundary)(boundary)
879
-
880
- Test Values: 0, 1, 2, 99, 100, 101
881
- ```
882
-
883
- ```typescript
884
- // TypeScript Example: Boundary Value Analysis
885
- describe('Quantity Validation', () => {
886
- const validator = new QuantityValidator(1, 100);
887
-
888
- // Lower boundary
889
- it('rejects 0 (below minimum)', () => {
890
- expect(validator.isValid(0)).toBe(false);
891
- });
892
-
893
- it('accepts 1 (minimum boundary)', () => {
894
- expect(validator.isValid(1)).toBe(true);
895
- });
896
-
897
- it('accepts 2 (above minimum)', () => {
898
- expect(validator.isValid(2)).toBe(true);
899
- });
900
-
901
- // Upper boundary
902
- it('accepts 99 (below maximum)', () => {
903
- expect(validator.isValid(99)).toBe(true);
904
- });
905
-
906
- it('accepts 100 (maximum boundary)', () => {
907
- expect(validator.isValid(100)).toBe(true);
908
- });
909
-
910
- it('rejects 101 (above maximum)', () => {
911
- expect(validator.isValid(101)).toBe(false);
912
- });
913
- });
914
- ```
915
-
916
- #### 3. Decision Table Testing
917
-
918
- Captures complex business rules with multiple conditions.
919
-
920
- ```
921
- Discount Rules:
922
- - Premium member: 20% discount
923
- - Order > $100: 10% discount
924
- - Both conditions: 25% discount (not cumulative)
925
-
926
- ┌─────────────────┬──────┬──────┬──────┬──────┐
927
- │ Conditions │ TC1 │ TC2 │ TC3 │ TC4 │
928
- ├─────────────────┼──────┼──────┼──────┼──────┤
929
- │ Premium Member │ N │ Y │ N │ Y │
930
- │ Order > $100 │ N │ N │ Y │ Y │
931
- ├─────────────────┼──────┼──────┼──────┼──────┤
932
- │ Actions │ │ │ │ │
933
- ├─────────────────┼──────┼──────┼──────┼──────┤
934
- │ Discount % │ 0% │ 20% │ 10% │ 25% │
935
- └─────────────────┴──────┴──────┴──────┴──────┘
936
- ```
937
-
938
- ```csharp
939
- // C# Example: Decision Table Testing
940
- [TestClass]
941
- public class DiscountCalculatorTests
942
- {
943
- private DiscountCalculator _calculator;
944
-
945
- [TestInitialize]
946
- public void Setup() => _calculator = new DiscountCalculator();
947
-
948
- [TestMethod]
949
- public void Calculate_NotPremium_SmallOrder_NoDiscount()
950
- {
951
- var result = _calculator.Calculate(isPremium: false, orderAmount: 50);
952
- Assert.AreEqual(0, result.DiscountPercent);
953
- }
954
-
955
- [TestMethod]
956
- public void Calculate_Premium_SmallOrder_20PercentDiscount()
957
- {
958
- var result = _calculator.Calculate(isPremium: true, orderAmount: 50);
959
- Assert.AreEqual(20, result.DiscountPercent);
960
- }
961
-
962
- [TestMethod]
963
- public void Calculate_NotPremium_LargeOrder_10PercentDiscount()
964
- {
965
- var result = _calculator.Calculate(isPremium: false, orderAmount: 150);
966
- Assert.AreEqual(10, result.DiscountPercent);
967
- }
968
-
969
- [TestMethod]
970
- public void Calculate_Premium_LargeOrder_25PercentDiscount()
971
- {
972
- var result = _calculator.Calculate(isPremium: true, orderAmount: 150);
973
- Assert.AreEqual(25, result.DiscountPercent);
974
- }
975
- }
976
- ```
977
-
978
- #### 4. State Transition Testing
979
-
980
- Tests behavior changes based on system states and transitions.
981
-
982
- ```
983
- Order State Machine:
984
-
985
- ┌─────────┐ place() ┌───────────┐ pay() ┌────────┐
986
- │ Draft │──────────────▶│ Pending │───────────▶│ Paid │
987
- └─────────┘ └───────────┘ └────────┘
988
- │ │ │
989
- │ cancel() │ cancel() │ ship()
990
- ▼ ▼ ▼
991
- ┌───────────┐ ┌───────────┐ ┌──────────┐
992
- │ Cancelled │ │ Cancelled │ │ Shipped │
993
- └───────────┘ └───────────┘ └──────────┘
994
-
995
- State Transition Table:
996
- ┌───────────┬─────────┬───────────┬───────────┬───────────┐
997
- │ Current │ place() │ pay() │ ship() │ cancel() │
998
- ├───────────┼─────────┼───────────┼───────────┼───────────┤
999
- │ Draft │ Pending │ Invalid │ Invalid │ Cancelled │
1000
- │ Pending │ Invalid │ Paid │ Invalid │ Cancelled │
1001
- │ Paid │ Invalid │ Invalid │ Shipped │ Invalid │
1002
- │ Shipped │ Invalid │ Invalid │ Invalid │ Invalid │
1003
- │ Cancelled │ Invalid │ Invalid │ Invalid │ Invalid │
1004
- └───────────┴─────────┴───────────┴───────────┴───────────┘
1005
- ```
1006
-
1007
- ```python
1008
- # Python Example: State Transition Testing
1009
- class TestOrderStateMachine:
1010
- def test_draft_to_pending_on_place(self):
1011
- order = Order(state="draft")
1012
- order.place()
1013
- assert order.state == "pending"
1014
-
1015
- def test_pending_to_paid_on_pay(self):
1016
- order = Order(state="pending")
1017
- order.pay()
1018
- assert order.state == "paid"
1019
-
1020
- def test_paid_to_shipped_on_ship(self):
1021
- order = Order(state="paid")
1022
- order.ship()
1023
- assert order.state == "shipped"
1024
-
1025
- def test_invalid_transition_pay_from_draft(self):
1026
- order = Order(state="draft")
1027
- with pytest.raises(InvalidTransitionError):
1028
- order.pay()
1029
-
1030
- def test_invalid_transition_cancel_from_shipped(self):
1031
- order = Order(state="shipped")
1032
- with pytest.raises(InvalidTransitionError):
1033
- order.cancel()
1034
- ```
1035
-
1036
- #### 5. Use Case Testing
1037
-
1038
- Tests complete user scenarios from start to finish.
1039
-
1040
- ```
1041
- Use Case: User Login
1042
-
1043
- Primary Flow:
1044
- 1. User enters username
1045
- 2. User enters password
1046
- 3. User clicks login
1047
- 4. System validates credentials
1048
- 5. System redirects to dashboard
1049
-
1050
- Alternative Flows:
1051
- A1. Invalid credentials → Show error, remain on login
1052
- A2. Account locked → Show locked message
1053
- A3. Password expired → Redirect to password change
1054
-
1055
- Test Cases:
1056
- - TC1: Valid credentials → Dashboard (Primary)
1057
- - TC2: Invalid password → Error message (A1)
1058
- - TC3: Invalid username → Error message (A1)
1059
- - TC4: Locked account → Locked message (A2)
1060
- - TC5: Expired password → Password change page (A3)
1061
- ```
1062
-
1063
- ### Structure-Based Techniques (White-Box)
1064
-
1065
- #### 1. Statement Coverage
1066
-
1067
- Ensure every statement is executed at least once.
1068
-
1069
- ```
1070
- Code:
1071
- 1 function categorize(score) {
1072
- 2 let result;
1073
- 3 if (score >= 90) {
1074
- 4 result = 'A';
1075
- 5 } else if (score >= 70) {
1076
- 6 result = 'B';
1077
- 7 } else {
1078
- 8 result = 'C';
1079
- 9 }
1080
- 10 return result;
1081
- 11 }
1082
-
1083
- 100% Statement Coverage requires tests:
1084
- - Test 1: score = 95 → executes lines 1-4, 10
1085
- - Test 2: score = 80 → executes lines 1-3, 5-6, 10
1086
- - Test 3: score = 50 → executes lines 1-3, 5, 7-10
1087
- ```
1088
-
1089
- #### 2. Branch Coverage
1090
-
1091
- Ensure every branch (decision outcome) is executed.
1092
-
1093
- ```
1094
- Code:
1095
- if (a > 0 && b > 0) {
1096
- doSomething();
1097
- }
1098
-
1099
- Branches:
1100
- - True branch: a > 0 AND b > 0 is true
1101
- - False branch: a > 0 AND b > 0 is false
1102
-
1103
- 100% Branch Coverage:
1104
- - Test 1: a = 1, b = 1 → True branch
1105
- - Test 2: a = 0, b = 1 → False branch (or a = 1, b = 0)
1106
- ```
1107
-
1108
- #### 3. Condition Coverage
1109
-
1110
- Ensure each condition in a decision is evaluated to both true and false.
1111
-
1112
- ```
1113
- Code:
1114
- if (a > 0 && b > 0) { ... }
1115
-
1116
- Conditions: (a > 0), (b > 0)
1117
-
1118
- 100% Condition Coverage:
1119
- - Test 1: a = 1, b = 1 → (a > 0) = true, (b > 0) = true
1120
- - Test 2: a = 0, b = 0 → (a > 0) = false, (b > 0) = false
1121
-
1122
- Note: Does not guarantee branch coverage!
1123
- ```
1124
-
1125
- #### 4. MC/DC (Modified Condition/Decision Coverage)
1126
-
1127
- Each condition independently affects the decision outcome. Required for safety-critical systems.
1128
-
1129
- ```
1130
- Code:
1131
- if (a && b) { ... }
1132
-
1133
- MC/DC Requirements:
1134
- 1. Each condition evaluated to true and false
1135
- 2. Each condition independently affects outcome
1136
-
1137
- Test Cases:
1138
- - Test 1: a = true, b = true → Decision = true
1139
- - Test 2: a = false, b = true → Decision = false (a changed outcome)
1140
- - Test 3: a = true, b = false → Decision = false (b changed outcome)
1141
- ```
1142
-
1143
- ### Additional Specification-Based Techniques
1144
-
1145
- #### 6. Pairwise Testing (Combinatorial Testing)
1146
-
1147
- Tests all possible pairs of input parameter values, reducing test cases while maintaining good coverage.
1148
-
1149
- **Reference**: SWEBOK v4.0 - Input Domain-Based Techniques
1150
-
1151
- ```
1152
- Example: Login Form with 3 parameters, each with 3 values
1153
-
1154
- Parameters:
1155
- - Browser: Chrome, Firefox, Safari
1156
- - OS: Windows, macOS, Linux
1157
- - Language: English, Spanish, French
1158
-
1159
- Full Combinatorial: 3 × 3 × 3 = 27 test cases
1160
- Pairwise Coverage: 9 test cases (covers all pairs)
1161
-
1162
- ┌──────┬──────────┬─────────┬──────────┐
1163
- │ TC │ Browser │ OS │ Language │
1164
- ├──────┼──────────┼─────────┼──────────┤
1165
- │ 1 │ Chrome │ Windows │ English │
1166
- │ 2 │ Chrome │ macOS │ Spanish │
1167
- │ 3 │ Chrome │ Linux │ French │
1168
- │ 4 │ Firefox │ Windows │ Spanish │
1169
- │ 5 │ Firefox │ macOS │ French │
1170
- │ 6 │ Firefox │ Linux │ English │
1171
- │ 7 │ Safari │ Windows │ French │
1172
- │ 8 │ Safari │ macOS │ English │
1173
- │ 9 │ Safari │ Linux │ Spanish │
1174
- └──────┴──────────┴─────────┴──────────┘
1175
- ```
1176
-
1177
- **Tools**: PICT (Microsoft), AllPairs, Jenny
1178
-
1179
- ```bash
1180
- # Using PICT (Pairwise Independent Combinatorial Testing)
1181
- # Input file: params.txt
1182
- # Browser: Chrome, Firefox, Safari
1183
- # OS: Windows, macOS, Linux
1184
- # Language: English, Spanish, French
1185
-
1186
- pict params.txt > test_cases.txt
1187
- ```
1188
-
1189
- #### 7. Data Flow Testing
1190
-
1191
- Tests the flow of data through a program by tracking variable definitions and uses.
1192
-
1193
- **Reference**: SWEBOK v4.0 - Code-Based Techniques
1194
-
1195
- ```
1196
- ┌─────────────────────────────────────────────────────────────┐
1197
- │ Data Flow Terminology │
1198
- ├─────────────────────────────────────────────────────────────┤
1199
- │ │
1200
- │ Definition (def): Variable is assigned a value │
1201
- │ Use (use): Variable's value is accessed │
1202
- │ - c-use: Computational use (in calculation) │
1203
- │ - p-use: Predicate use (in condition) │
1204
- │ │
1205
- │ def-use pair: Path from definition to use │
1206
- │ def-clear path: No redefinition between def and use │
1207
- │ │
1208
- │ Example: │
1209
- │ 1: x = 5 ← def(x) │
1210
- │ 2: y = x + 1 ← c-use(x), def(y) │
1211
- │ 3: if (x > 0) ← p-use(x) │
1212
- │ 4: z = y * 2 ← c-use(y), def(z) │
1213
- │ │
1214
- └─────────────────────────────────────────────────────────────┘
1215
- ```
1216
-
1217
- **Coverage Criteria**:
1218
-
1219
- | Criterion | Requirement | Strength |
1220
- |-----------|-------------|----------|
1221
- | All-Defs | Every def reaches at least one use | Weak |
1222
- | All-Uses | Every def-use pair is exercised | Medium |
1223
- | All-DU-Paths | All def-clear paths from def to use | Strong |
1224
-
1225
- ```python
1226
- # Python Example: Data Flow Testing
1227
- def calculate_discount(price, quantity, is_member):
1228
- # def: total
1229
- total = price * quantity
1230
-
1231
- # def: discount
1232
- if is_member: # p-use: is_member
1233
- discount = 0.1 # def: discount (path 1)
1234
- else:
1235
- discount = 0.0 # def: discount (path 2)
1236
-
1237
- # c-use: total, discount
1238
- if total > 100: # p-use: total
1239
- discount += 0.05 # c-use: discount, def: discount
1240
-
1241
- # c-use: total, discount
1242
- return total * (1 - discount)
1243
-
1244
- # Test cases for All-Uses coverage:
1245
- # TC1: is_member=True, total>100 → exercises def@line4, use@line10
1246
- # TC2: is_member=False, total>100 → exercises def@line6, use@line10
1247
- # TC3: is_member=True, total<=100 → exercises def@line4, use@line12
1248
- # TC4: is_member=False, total<=100 → exercises def@line6, use@line12
1249
- ```
1250
-
1251
- ### Technique Selection Guide
1252
-
1253
- | Technique | Best For | Test Level |
1254
- |-----------|----------|------------|
1255
- | Equivalence Partitioning | Input validation, ranges | UT, IT |
1256
- | Boundary Value Analysis | Numeric limits, dates | UT, IT |
1257
- | Decision Table | Complex business rules | UT, IT, ST |
1258
- | State Transition | Workflows, status changes | IT, ST |
1259
- | Use Case Testing | User scenarios | ST, E2E |
1260
- | Pairwise Testing | Multi-parameter combinations | IT, ST |
1261
- | Data Flow Testing | Variable lifecycle verification | UT |
1262
- | Statement Coverage | Basic code coverage | UT |
1263
- | Branch Coverage | Decision logic | UT |
1264
- | MC/DC | Safety-critical systems | UT |
1265
-
1266
- ---
1267
-
1268
- ## Experience-Based Testing
1269
-
1270
- Experience-based testing leverages the tester's knowledge, skills, and intuition to design and execute tests. It complements systematic techniques.
1271
-
1272
- **Reference**: ISTQB CTFL v4.0 Section 4.4
1273
-
1274
- ### Types of Experience-Based Testing
1275
-
1276
- #### 1. Exploratory Testing
1277
-
1278
- Simultaneous test design, execution, and learning. The tester explores the system without predefined scripts.
1279
-
1280
- ```
1281
- ┌─────────────────────────────────────────────────────────────┐
1282
- │ Exploratory Testing Session │
1283
- ├─────────────────────────────────────────────────────────────┤
1284
- │ Charter: "Explore the checkout process focusing on │
1285
- │ payment failure scenarios" │
1286
- ├─────────────────────────────────────────────────────────────┤
1287
- │ Time-box: 60 minutes │
1288
- ├─────────────────────────────────────────────────────────────┤
1289
- │ Notes: │
1290
- │ - Tested expired credit card → Got generic error │
1291
- │ - Tested insufficient funds → Same generic error │
1292
- │ - Found: No specific error messages for different failures │
1293
- │ - Bug: Retry with same card doesn't clear previous error │
1294
- ├─────────────────────────────────────────────────────────────┤
1295
- │ Bugs Found: 2 │
1296
- │ Areas for Further Testing: Error message specificity │
1297
- └─────────────────────────────────────────────────────────────┘
1298
- ```
1299
-
1300
- **Session-Based Test Management (SBTM)**:
1301
-
1302
- | Element | Description |
1303
- |---------|-------------|
1304
- | Charter | Mission statement for the session |
1305
- | Time-box | Fixed duration (typically 60-90 min) |
1306
- | Session Notes | Observations, questions, bugs found |
1307
- | Debrief | Review findings with team |
1308
-
1309
- #### 2. Error Guessing
1310
-
1311
- Anticipate defects based on experience with similar systems or common mistakes.
1312
-
1313
- ```
1314
- Common Error Categories to Guess:
1315
-
1316
- ┌─────────────────────────────────────────────────────────────┐
1317
- │ Input Errors │
1318
- │ ├── Empty/null inputs │
1319
- │ ├── Very long strings │
1320
- │ ├── Special characters (', ", <, >, &, etc.) │
1321
- │ ├── Unicode/emoji characters │
1322
- │ └── Negative numbers where positive expected │
1323
- ├─────────────────────────────────────────────────────────────┤
1324
- │ Boundary Errors │
1325
- │ ├── Off-by-one errors │
1326
- │ ├── Integer overflow/underflow │
1327
- │ ├── Date boundaries (leap years, month ends) │
1328
- │ └── Array index out of bounds │
1329
- ├─────────────────────────────────────────────────────────────┤
1330
- │ State Errors │
1331
- │ ├── Operations on deleted records │
1332
- │ ├── Concurrent modifications │
1333
- │ ├── Session timeout during operation │
1334
- │ └── Browser back button after submission │
1335
- ├─────────────────────────────────────────────────────────────┤
1336
- │ Environment Errors │
1337
- │ ├── Network timeout/disconnection │
1338
- │ ├── Low disk space │
1339
- │ ├── Database connection pool exhaustion │
1340
- │ └── Time zone differences │
1341
- └─────────────────────────────────────────────────────────────┘
1342
- ```
1343
-
1344
- #### 3. Checklist-Based Testing
1345
-
1346
- Use predefined checklists based on experience and standards.
1347
-
1348
- ```markdown
1349
- # API Endpoint Testing Checklist
1350
-
1351
- ## Authentication & Authorization
1352
- - [ ] Endpoint rejects unauthenticated requests
1353
- - [ ] Endpoint rejects expired tokens
1354
- - [ ] Endpoint enforces role-based access
1355
- - [ ] Different user roles see appropriate data
1356
-
1357
- ## Input Validation
1358
- - [ ] Required fields are validated
1359
- - [ ] Data types are enforced
1360
- - [ ] String length limits enforced
1361
- - [ ] Malicious input sanitized (XSS, SQL injection)
1362
-
1363
- ## Response Handling
1364
- - [ ] Success responses have correct status codes
1365
- - [ ] Error responses have meaningful messages
1366
- - [ ] Response format matches API documentation
1367
- - [ ] Pagination works correctly
1368
-
1369
- ## Performance
1370
- - [ ] Response time under load is acceptable
1371
- - [ ] No N+1 query problems
1372
- - [ ] Large datasets handled gracefully
1373
- ```
1374
-
1375
- ### When to Use Experience-Based Testing
1376
-
1377
- | Scenario | Recommended Technique |
1378
- |----------|----------------------|
1379
- | New feature exploration | Exploratory Testing |
1380
- | Legacy system with no documentation | Exploratory Testing |
1381
- | Known problematic areas | Error Guessing |
1382
- | Regression testing | Checklist-Based |
1383
- | Time pressure / quick validation | Error Guessing |
1384
- | Compliance verification | Checklist-Based |
1385
-
1386
- ### Combining with Systematic Techniques
1387
-
1388
- ```
1389
- ┌─────────────────────────────────────────────────────────────┐
1390
- │ Optimal Testing Approach │
1391
- ├─────────────────────────────────────────────────────────────┤
1392
- │ │
1393
- │ Systematic Techniques Experience-Based │
1394
- │ (70-80% of effort) (20-30% of effort) │
1395
- │ │
1396
- │ ┌──────────────────┐ ┌──────────────────┐ │
1397
- │ │ Equivalence Part.│ │ Exploratory │ │
1398
- │ │ Boundary Value │ + │ Error Guessing │ │
1399
- │ │ Decision Tables │ │ Checklists │ │
1400
- │ └──────────────────┘ └──────────────────┘ │
1401
- │ │
1402
- │ Provides: Provides: │
1403
- │ - Coverage - Edge cases │
1404
- │ - Repeatability - Real-world scenarios │
1405
- │ - Documentation - Quick feedback │
1406
- │ │
1407
- └─────────────────────────────────────────────────────────────┘
1408
- ```
1409
-
1410
- ---
1411
-
1412
- ## Risk-Based Testing
1413
-
1414
- Risk-based testing prioritizes testing efforts based on the likelihood and impact of potential failures.
1415
-
1416
- **Reference**: ISTQB CTFL v4.0 Section 5.2
1417
-
1418
- ### Risk Assessment
1419
-
1420
- #### Risk Formula
1421
-
1422
- ```
1423
- Risk Level = Likelihood × Impact
1424
-
1425
- ┌─────────────────────────────────────────────────────────────┐
1426
- │ Risk Matrix │
1427
- ├─────────────┬───────────────────────────────────────────────┤
1428
- │ │ Impact │
1429
- │ Likelihood │ Low (1) Medium (2) High (3) │
1430
- ├─────────────┼───────────────────────────────────────────────┤
1431
- │ High (3) │ Medium(3) High(6) Critical(9) │
1432
- │ Medium (2) │ Low(2) Medium(4) High(6) │
1433
- │ Low (1) │ Low(1) Low(2) Medium(3) │
1434
- └─────────────┴───────────────────────────────────────────────┘
1435
- ```
1436
-
1437
- #### Risk Factors
1438
-
1439
- **Likelihood Factors** (Technical Risk):
1440
- - Complexity of the feature
1441
- - New/unfamiliar technology
1442
- - Code quality metrics
1443
- - Developer experience
1444
- - Frequency of changes
1445
- - Integration complexity
1446
-
1447
- **Impact Factors** (Business Risk):
1448
- - Revenue impact
1449
- - User base affected
1450
- - Regulatory compliance
1451
- - Brand reputation
1452
- - Data security/privacy
1453
- - Operational criticality
1454
-
1455
- ### Risk-Based Test Prioritization
1456
-
1457
- ```
1458
- Example: E-commerce Application
1459
-
1460
- ┌──────────────────────┬─────────┬─────────┬───────┬──────────┐
1461
- │ Feature │Likelihood│ Impact │ Risk │ Priority │
1462
- ├──────────────────────┼─────────┼─────────┼───────┼──────────┤
1463
- │ Payment Processing │ 2 │ 3 │ 6 │ 1 │
1464
- │ User Authentication │ 2 │ 3 │ 6 │ 2 │
1465
- │ Shopping Cart │ 2 │ 2 │ 4 │ 3 │
1466
- │ Product Search │ 1 │ 2 │ 2 │ 4 │
1467
- │ Product Reviews │ 1 │ 1 │ 1 │ 5 │
1468
- │ Wishlist │ 1 │ 1 │ 1 │ 6 │
1469
- └──────────────────────┴─────────┴─────────┴───────┴──────────┘
1470
-
1471
- Test Effort Allocation:
1472
- - Critical (Risk 6+): 50% of test effort, most thorough coverage
1473
- - Medium (Risk 3-5): 30% of test effort, standard coverage
1474
- - Low (Risk 1-2): 20% of test effort, basic coverage
1475
- ```
1476
-
1477
- ### Risk-Based Test Planning
1478
-
1479
- ```markdown
1480
- # Risk-Based Test Plan Template
1481
-
1482
- ## 1. Risk Identification
1483
- List all features/components and their risk factors.
1484
-
1485
- ## 2. Risk Analysis
1486
- Calculate risk scores using Likelihood × Impact.
1487
-
1488
- ## 3. Test Prioritization
1489
- | Priority | Features | Test Depth |
1490
- |----------|----------|------------|
1491
- | P1 | Payment, Auth | Full coverage, all techniques |
1492
- | P2 | Cart, Checkout | Standard coverage |
1493
- | P3 | Search, Browse | Basic happy path |
1494
- | P4 | Reviews, Wishlist | Minimal, smoke tests |
1495
-
1496
- ## 4. Risk Mitigation
1497
- - P1 features: 100% automated tests, manual exploratory
1498
- - P2 features: 80% automated, sample manual tests
1499
- - P3 features: Key scenarios automated
1500
- - P4 features: Basic smoke tests only
1501
-
1502
- ## 5. Residual Risk Acceptance
1503
- Document accepted risks for low-priority features.
1504
- ```
1505
-
1506
- ### Continuous Risk Assessment
1507
-
1508
- ```
1509
- ┌─────────────────────────────────────────────────────────────┐
1510
- │ Continuous Risk Re-evaluation │
1511
- ├─────────────────────────────────────────────────────────────┤
1512
- │ │
1513
- │ Sprint Start ──▶ During Sprint ──▶ Sprint End │
1514
- │ │ │ │ │
1515
- │ ▼ ▼ ▼ │
1516
- │ Initial Risk Update based on: Review: │
1517
- │ Assessment - Defects found - Actual vs Expected │
1518
- │ - Code changes - Adjust for next │
1519
- │ - New requirements sprint │
1520
- │ │
1521
- └─────────────────────────────────────────────────────────────┘
1522
- ```
27
+ | Abbreviation | Full Term | Description |
28
+ |--------------|-----------|-------------|
29
+ | **UT** | Unit Testing | Testing individual functions/methods in isolation |
30
+ | **IT** | Integration Testing | Testing interactions between components |
31
+ | **ST** | System Testing | Testing the complete integrated system |
32
+ | **AT** | Acceptance Testing | Testing against business acceptance criteria |
33
+ | **E2E** | End-to-End Testing | Testing complete user workflows |
34
+ | **UAT** | User Acceptance Testing | Acceptance testing performed by end users |
35
+ | **SIT** | System Integration Testing | Testing integration of multiple systems |
1523
36
 
1524
- ---
1525
-
1526
- ## Specialized Testing
1527
-
1528
- Modern software development requires specialized testing types beyond traditional categories.
1529
-
1530
- ### Contract Testing
1531
-
1532
- Verifies that services communicate correctly according to agreed contracts. Essential for microservices architectures.
1533
-
1534
- ```
1535
- ┌─────────────────────────────────────────────────────────────┐
1536
- │ Consumer-Driven Contract Testing │
1537
- ├─────────────────────────────────────────────────────────────┤
1538
- │ │
1539
- │ Consumer Provider │
1540
- │ (Frontend) (API) │
1541
- │ │ │ │
1542
- │ │ 1. Define expectations │ │
1543
- │ │─────────────────────────────▶ │ │
1544
- │ │ │ │
1545
- │ │ 2. Generate contract │ │
1546
- │ │ (Pact file) │ │
1547
- │ │ │ │
1548
- │ │ 3. Share contract │ │
1549
- │ │─────────────────────────────▶ │ │
1550
- │ │ │ │
1551
- │ │ 4. Provider verifies │
1552
- │ │ against contract │
1553
- │ │ │ │
1554
- └─────────────────────────────────────────────────────────────┘
1555
- ```
1556
-
1557
- **Tools**: Pact, Spring Cloud Contract, Postman
1558
-
1559
- ```typescript
1560
- // Consumer Test (Pact.js Example)
1561
- describe('User API Contract', () => {
1562
- it('returns user by ID', async () => {
1563
- await provider.addInteraction({
1564
- state: 'user with ID 123 exists',
1565
- uponReceiving: 'a request for user 123',
1566
- withRequest: {
1567
- method: 'GET',
1568
- path: '/users/123'
1569
- },
1570
- willRespondWith: {
1571
- status: 200,
1572
- body: {
1573
- id: '123',
1574
- name: Matchers.string('John Doe'),
1575
- email: Matchers.email()
1576
- }
1577
- }
1578
- });
1579
-
1580
- const user = await userClient.getUser('123');
1581
- expect(user.id).toBe('123');
1582
- });
1583
- });
1584
- ```
1585
-
1586
- ### Mutation Testing
1587
-
1588
- Evaluates test suite quality by introducing small changes (mutants) to the code and checking if tests detect them.
1589
-
1590
- ```
1591
- ┌─────────────────────────────────────────────────────────────┐
1592
- │ Mutation Testing │
1593
- ├─────────────────────────────────────────────────────────────┤
1594
- │ │
1595
- │ Original Code: Mutant (change operator): │
1596
- │ if (a > b) if (a >= b) ← Boundary mutant │
1597
- │ if (a > b) if (a < b) ← Negation mutant │
1598
- │ return a + b; return a - b; ← Arithmetic mutant │
1599
- │ return true; return false; ← Return value mutant │
1600
- │ │
1601
- ├─────────────────────────────────────────────────────────────┤
1602
- │ Mutation Score = (Killed Mutants / Total Mutants) × 100 │
1603
- │ │
1604
- │ - Killed: Test failed → Good, test detected the change │
1605
- │ - Survived: Test passed → Bad, test missed the defect │
1606
- │ │
1607
- │ Target: > 80% mutation score for critical code │
1608
- └─────────────────────────────────────────────────────────────┘
1609
- ```
1610
-
1611
- **Tools**: Stryker (JS/TS), PITest (Java), mutmut (Python)
1612
-
1613
- ```bash
1614
- # Run mutation testing with Stryker
1615
- npx stryker run
1616
-
1617
- # Example output:
1618
- # Mutation score: 85.7%
1619
- # Killed: 180 Survived: 30 No coverage: 10
1620
- ```
1621
-
1622
- ### Chaos Engineering
1623
-
1624
- Proactively tests system resilience by injecting failures in production or staging environments.
1625
-
1626
- ```
1627
- ┌─────────────────────────────────────────────────────────────┐
1628
- │ Chaos Engineering Principles │
1629
- ├─────────────────────────────────────────────────────────────┤
1630
- │ │
1631
- │ 1. Define "steady state" (normal behavior metrics) │
1632
- │ 2. Hypothesize steady state continues during chaos │
1633
- │ 3. Introduce real-world events: │
1634
- │ - Server failures │
1635
- │ - Network latency/partition │
1636
- │ - Resource exhaustion │
1637
- │ - Clock skew │
1638
- │ 4. Try to disprove the hypothesis │
1639
- │ 5. Minimize blast radius (start small) │
1640
- │ │
1641
- └─────────────────────────────────────────────────────────────┘
1642
- ```
1643
-
1644
- **Tools**: Chaos Monkey, Gremlin, Litmus, Chaos Toolkit
1645
-
1646
- ```yaml
1647
- # Chaos Toolkit Experiment Example
1648
- title: "Service resilience when database is slow"
1649
- description: "Verify the service degrades gracefully"
1650
-
1651
- steady-state-hypothesis:
1652
- title: "Service responds within SLA"
1653
- probes:
1654
- - name: "service-responds"
1655
- type: probe
1656
- provider:
1657
- type: http
1658
- url: "http://service/health"
1659
- timeout: 3
1660
-
1661
- method:
1662
- - name: "inject-db-latency"
1663
- type: action
1664
- provider:
1665
- type: process
1666
- path: "tc"
1667
- arguments: "qdisc add dev eth0 root netem delay 500ms"
1668
- pauses:
1669
- after: 30
1670
-
1671
- rollbacks:
1672
- - name: "remove-latency"
1673
- type: action
1674
- provider:
1675
- type: process
1676
- path: "tc"
1677
- arguments: "qdisc del dev eth0 root"
1678
- ```
1679
-
1680
- ### Property-Based Testing
1681
-
1682
- Generates random test inputs based on properties the code should satisfy.
1683
-
1684
- ```python
1685
- # Python Example with Hypothesis
1686
- from hypothesis import given, strategies as st
1687
-
1688
- # Property: Sorting should be idempotent
1689
- @given(st.lists(st.integers()))
1690
- def test_sort_idempotent(xs):
1691
- sorted_once = sorted(xs)
1692
- sorted_twice = sorted(sorted_once)
1693
- assert sorted_once == sorted_twice
1694
-
1695
- # Property: Sorted list should have same elements
1696
- @given(st.lists(st.integers()))
1697
- def test_sort_preserves_elements(xs):
1698
- sorted_xs = sorted(xs)
1699
- assert sorted(xs) == sorted(sorted_xs)
1700
- assert len(xs) == len(sorted_xs)
1701
-
1702
- # Property: JSON encode/decode roundtrip
1703
- @given(st.dictionaries(st.text(), st.integers()))
1704
- def test_json_roundtrip(d):
1705
- assert json.loads(json.dumps(d)) == d
1706
- ```
1707
-
1708
- **Tools**: Hypothesis (Python), fast-check (JS/TS), QuickCheck (Haskell), FsCheck (C#)
1709
-
1710
- ### Visual Regression Testing
1711
-
1712
- Detects unintended visual changes in UI components.
1713
-
1714
- ```
1715
- ┌─────────────────────────────────────────────────────────────┐
1716
- │ Visual Regression Testing │
1717
- ├─────────────────────────────────────────────────────────────┤
1718
- │ │
1719
- │ 1. Capture baseline screenshots │
1720
- │ 2. Run tests, capture new screenshots │
1721
- │ 3. Compare pixel-by-pixel or perceptually │
1722
- │ 4. Flag differences for review │
1723
- │ │
1724
- │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
1725
- │ │ Baseline │ → │ Current │ → │ Diff │ │
1726
- │ │ Screenshot │ │ Screenshot │ │ Report │ │
1727
- │ └─────────────┘ └─────────────┘ └─────────────┘ │
1728
- │ │
1729
- └─────────────────────────────────────────────────────────────┘
1730
- ```
1731
-
1732
- **Tools**: Percy, Chromatic, BackstopJS, Playwright visual comparisons
1733
-
1734
- ```typescript
1735
- // Playwright Visual Regression Example
1736
- test('homepage visual regression', async ({ page }) => {
1737
- await page.goto('/');
1738
- await expect(page).toHaveScreenshot('homepage.png', {
1739
- maxDiffPixels: 100 // Allow minor differences
1740
- });
1741
- });
1742
-
1743
- test('button component visual regression', async ({ page }) => {
1744
- await page.goto('/components/button');
1745
- const button = page.locator('[data-testid="primary-button"]');
1746
- await expect(button).toHaveScreenshot('primary-button.png');
1747
- });
1748
- ```
1749
-
1750
- ### Accessibility Testing (a11y)
1751
-
1752
- Ensures applications are usable by people with disabilities.
1753
-
1754
- ```
1755
- ┌─────────────────────────────────────────────────────────────┐
1756
- │ Accessibility Testing Checklist │
1757
- ├─────────────────────────────────────────────────────────────┤
1758
- │ WCAG 2.1 Guidelines │
1759
- │ ├── Perceivable │
1760
- │ │ ├── Alt text for images │
1761
- │ │ ├── Captions for video │
1762
- │ │ └── Sufficient color contrast │
1763
- │ ├── Operable │
1764
- │ │ ├── Keyboard navigation │
1765
- │ │ ├── Focus indicators visible │
1766
- │ │ └── No keyboard traps │
1767
- │ ├── Understandable │
1768
- │ │ ├── Clear labels and instructions │
1769
- │ │ ├── Predictable navigation │
1770
- │ │ └── Error identification │
1771
- │ └── Robust │
1772
- │ ├── Valid HTML │
1773
- │ └── ARIA attributes correct │
1774
- └─────────────────────────────────────────────────────────────┘
1775
- ```
1776
-
1777
- **Tools**: axe-core, Pa11y, WAVE, Lighthouse
1778
-
1779
- ```typescript
1780
- // Playwright + axe-core Example
1781
- import AxeBuilder from '@axe-core/playwright';
1782
-
1783
- test('homepage accessibility', async ({ page }) => {
1784
- await page.goto('/');
1785
-
1786
- const results = await new AxeBuilder({ page })
1787
- .withTags(['wcag2a', 'wcag2aa'])
1788
- .analyze();
1789
-
1790
- expect(results.violations).toEqual([]);
1791
- });
1792
- ```
1793
-
1794
- ### Additional Specialized Testing Types (SWEBOK)
1795
-
1796
- The following test types are defined in SWEBOK v4.0 as test objectives:
1797
-
1798
- #### Installation Testing
1799
-
1800
- Verifies that the software installs correctly in the target environment.
1801
-
1802
- ```
1803
- ┌─────────────────────────────────────────────────────────────┐
1804
- │ Installation Testing Checklist │
1805
- ├─────────────────────────────────────────────────────────────┤
1806
- │ │
1807
- │ Pre-Installation │
1808
- │ ├── Prerequisites verified (OS, runtime, dependencies) │
1809
- │ ├── Disk space requirements met │
1810
- │ └── Permissions and access rights confirmed │
1811
- │ │
1812
- │ Installation Process │
1813
- │ ├── Fresh installation succeeds │
1814
- │ ├── Upgrade from previous version succeeds │
1815
- │ ├── Custom installation options work │
1816
- │ └── Silent/unattended installation works │
1817
- │ │
1818
- │ Post-Installation │
1819
- │ ├── Application launches correctly │
1820
- │ ├── Configuration files created properly │
1821
- │ ├── Registry/system settings correct │
1822
- │ └── Uninstallation removes all components │
1823
- │ │
1824
- └─────────────────────────────────────────────────────────────┘
1825
- ```
1826
-
1827
- #### Alpha and Beta Testing
1828
-
1829
- | Phase | Participants | Environment | Purpose |
1830
- |-------|--------------|-------------|---------|
1831
- | **Alpha** | Internal users, developers | Development site | Early validation, major issues |
1832
- | **Beta** | External users, customers | Customer site | Real-world validation, feedback |
1833
- | **Open Beta** | General public | Various | Wide exposure, stress testing |
1834
-
1835
- #### Recovery Testing
1836
-
1837
- Tests the system's ability to recover from failures.
1838
-
1839
- ```
1840
- ┌─────────────────────────────────────────────────────────────┐
1841
- │ Recovery Testing Scenarios │
1842
- ├─────────────────────────────────────────────────────────────┤
1843
- │ │
1844
- │ Hardware Failures │
1845
- │ ├── Power outage during transaction │
1846
- │ ├── Disk failure and RAID recovery │
1847
- │ └── Network disconnection and reconnection │
1848
- │ │
1849
- │ Software Failures │
1850
- │ ├── Application crash recovery │
1851
- │ ├── Database connection loss and reconnect │
1852
- │ └── Service restart behavior │
1853
- │ │
1854
- │ Data Recovery │
1855
- │ ├── Backup and restore procedures │
1856
- │ ├── Transaction rollback │
1857
- │ └── Point-in-time recovery │
1858
- │ │
1859
- │ Metrics: │
1860
- │ • Recovery Time Objective (RTO): Max acceptable downtime │
1861
- │ • Recovery Point Objective (RPO): Max acceptable data loss │
1862
- │ │
1863
- └─────────────────────────────────────────────────────────────┘
1864
- ```
37
+ > **Note**: "IT" in this document always refers to "Integration Testing", not "Information Technology".
1865
38
 
1866
- #### Configuration Testing
1867
-
1868
- Tests the system under different configurations.
1869
-
1870
- ```
1871
- Configuration Matrix Example:
1872
-
1873
- ┌──────────────────┬─────────┬─────────┬─────────┐
1874
- │ Configuration │ Config A│ Config B│ Config C│
1875
- ├──────────────────┼─────────┼─────────┼─────────┤
1876
- │ Memory │ 4 GB │ 8 GB │ 16 GB │
1877
- │ Database │ SQLite │ MySQL │ PostgreSQL│
1878
- │ Cache │ None │ Redis │ Memcached│
1879
- │ Load Balancer │ None │ Nginx │ HAProxy │
1880
- └──────────────────┴─────────┴─────────┴─────────┘
1881
-
1882
- Test each supported configuration combination.
1883
- ```
1884
-
1885
- #### Back-to-Back Testing
39
+ ---
1886
40
 
1887
- Compares outputs of multiple implementations with identical inputs.
41
+ ## Testing Framework Selection
1888
42
 
1889
- ```
1890
- ┌─────────────────────────────────────────────────────────────┐
1891
- │ Back-to-Back Testing │
1892
- ├─────────────────────────────────────────────────────────────┤
1893
- │ │
1894
- │ Use Cases: │
1895
- │ ├── Comparing new version against old version │
1896
- │ ├── Comparing optimized code against reference impl │
1897
- │ ├── Cross-platform consistency verification │
1898
- │ └── Validating refactored code behavior │
1899
- │ │
1900
- │ Process: │
1901
- │ ┌─────────┐ ┌─────────────┐ ┌──────────┐ │
1902
- │ │ Input │────▶│ Version A │────▶│ Output A │──┐ │
1903
- │ │ Data │ └─────────────┘ └──────────┘ │ │
1904
- │ │ │ ┌─────────────┐ ┌──────────┐ ▼ │
1905
- │ │ │────▶│ Version B │────▶│ Output B │─▶Compare │
1906
- │ └─────────┘ └─────────────┘ └──────────┘ │
1907
- │ │
1908
- └─────────────────────────────────────────────────────────────┘
1909
- ```
43
+ | Framework | Levels | Best For |
44
+ |-----------|--------|----------|
45
+ | **ISTQB** | UT → IT/SIT → ST → AT/UAT | Enterprise, compliance, formal QA |
46
+ | **Industry Pyramid** | UT (70%) → IT (20%) → E2E (10%) | Agile, DevOps, CI/CD |
1910
47
 
1911
- #### Usability Testing
48
+ ---
1912
49
 
1913
- Evaluates how easily users can learn and use the system.
50
+ ## Testing Pyramid (Default Ratios)
1914
51
 
1915
52
  ```
1916
- ┌─────────────────────────────────────────────────────────────┐
1917
- Usability Testing Methods │
1918
- ├─────────────────────────────────────────────────────────────┤
1919
- │ │
1920
- Observational Methods │
1921
- │ ├── Think-aloud protocol │
1922
- │ ├── Eye tracking │
1923
- └── Session recording │
1924
- │ │
1925
- │ Task-Based Methods │
1926
- │ ├── Task completion rate │
1927
- │ ├── Time on task │
1928
- │ └── Error rate │
1929
- │ │
1930
- │ Survey Methods │
1931
- │ ├── System Usability Scale (SUS) │
1932
- │ ├── Net Promoter Score (NPS) │
1933
- │ └── Post-task questionnaires │
1934
- │ │
1935
- │ Key Metrics: │
1936
- │ • Learnability: Time to complete first task │
1937
- │ • Efficiency: Time to complete after learning │
1938
- │ • Memorability: Performance after period of non-use │
1939
- │ • Satisfaction: User rating scores │
1940
- │ │
1941
- └─────────────────────────────────────────────────────────────┘
53
+ ┌─────────┐
54
+ E2E │ ← 10% (Fewer, slower, expensive)
55
+ ─┴─────────┴─
56
+ ┌─────────────┐
57
+ IT/SIT │ 20% (Integration Testing)
58
+ ─┴─────────────┴─
59
+ ┌─────────────────┐
60
+ UT │ 70% (Unit Testing - Foundation)
61
+ └─────────────────┘
1942
62
  ```
1943
63
 
1944
- ### Specialized Testing Summary
1945
-
1946
- | Type | Purpose | When to Use |
1947
- |------|---------|-------------|
1948
- | Contract Testing | API compatibility | Microservices |
1949
- | Mutation Testing | Test quality assessment | Critical code paths |
1950
- | Chaos Engineering | Resilience validation | Production systems |
1951
- | Property-Based | Edge case discovery | Algorithms, parsers |
1952
- | Visual Regression | UI consistency | Frontend changes |
1953
- | Accessibility | Inclusive design | All user-facing apps |
1954
- | Installation | Deployment verification | Release candidates |
1955
- | Alpha/Beta | Early user validation | Pre-release phases |
1956
- | Recovery | Failure recovery | Business-critical systems |
1957
- | Configuration | Multi-config support | Cross-platform apps |
1958
- | Back-to-Back | Version comparison | Migrations, refactoring |
1959
- | Usability | User experience | User-facing applications |
64
+ | Level | Percentage | Execution Time Target |
65
+ |-------|------------|----------------------|
66
+ | Unit Testing (UT) | 70% | < 10 min total |
67
+ | Integration Testing (IT) | 20% | < 30 min total |
68
+ | E2E Testing | 10% | < 2 hours total |
1960
69
 
1961
70
  ---
1962
71
 
1963
- ## Test-Related Measures
1964
-
1965
- Quantitative measures for evaluating both the software under test and the quality of testing itself.
1966
-
1967
- **Reference**: SWEBOK v4.0 Section 5.4
72
+ ## Test Level Requirements
1968
73
 
1969
- ### Program Evaluation Measures
74
+ ### Unit Testing (UT)
1970
75
 
1971
- #### Fault Density
1972
-
1973
- ```
1974
- Fault Density = Total Faults / Size
1975
-
1976
- Where Size can be:
1977
- - KLOC (Thousands of Lines of Code)
1978
- - Function Points
1979
- - Story Points
1980
-
1981
- Industry Benchmarks:
1982
- ┌─────────────────────┬────────────────────────────┐
1983
- │ Quality Level │ Faults per KLOC │
1984
- ├─────────────────────┼────────────────────────────┤
1985
- │ Typical │ 1 - 25 │
1986
- │ Good │ 0.5 - 1 │
1987
- │ Excellent │ < 0.5 │
1988
- │ Safety-Critical │ < 0.1 │
1989
- └─────────────────────┴────────────────────────────┘
1990
- ```
1991
-
1992
- #### Reliability Growth Models
1993
-
1994
- Track defect discovery rate over time to predict remaining defects.
1995
-
1996
- ```
1997
- ┌─────────────────────────────────────────────────────────────┐
1998
- │ Reliability Growth Curve │
1999
- ├─────────────────────────────────────────────────────────────┤
2000
- │ │
2001
- │ Defects │
2002
- │ Found │
2003
- │ │ ╭──────────────────────────────── │
2004
- │ │ ╭─╯ │
2005
- │ │ ╭╯ │
2006
- │ │ ╭╯ │
2007
- │ │╭╯ │
2008
- │ ├╯ │
2009
- │ └──────────────────────────────────────▶ Time │
2010
- │ │
2011
- │ As testing progresses: │
2012
- │ • Defect discovery rate decreases │
2013
- │ • Curve approaches asymptote (total defects) │
2014
- │ • Remaining defects can be estimated │
2015
- │ │
2016
- │ Common Models: │
2017
- │ • Goel-Okumoto: Exponential │
2018
- │ • Musa: Logarithmic │
2019
- │ • Jelinski-Moranda: De-eutrophication │
2020
- │ │
2021
- └─────────────────────────────────────────────────────────────┘
2022
- ```
76
+ **Characteristics**: Isolated, Fast (< 100ms each), Deterministic
2023
77
 
2024
- ### Test Evaluation Measures
78
+ #### Scope
2025
79
 
2026
- #### Fault Seeding
80
+ | Include | Exclude |
81
+ |---------|---------|
82
+ | Single function/method | Database queries |
83
+ | Single class | External API calls |
84
+ | Pure business logic | File I/O operations |
85
+ | Data transformations | Multi-class interactions |
86
+ | Validation rules | Network calls |
2027
87
 
2028
- Insert known faults to measure test effectiveness.
88
+ #### Naming Convention
2029
89
 
90
+ **File Naming**:
2030
91
  ```
2031
- Fault Detection Effectiveness = Seeded Faults Found / Total Seeded Faults
2032
-
2033
- If tests find 80% of seeded faults, estimate:
2034
- - Similar detection rate for real faults
2035
- - Remaining faults ≈ Found faults / Detection rate
2036
-
2037
- Example:
2038
- - Seeded 20 faults, tests found 16 (80%)
2039
- - Found 40 real faults during testing
2040
- - Estimated total real faults ≈ 40 / 0.8 = 50
2041
- - Estimated remaining ≈ 10 faults
92
+ [ClassName]Tests.[ext] # C#
93
+ [ClassName].test.[ext] # TypeScript/JavaScript
94
+ [class_name]_test.[ext] # Python, Go
2042
95
  ```
2043
96
 
2044
- #### Coverage Metrics Summary
97
+ **Method Naming** (choose ONE per project):
2045
98
 
2046
- | Metric | Formula | Target |
2047
- |--------|---------|--------|
2048
- | Statement Coverage | Executed statements / Total statements | 70% |
2049
- | Branch Coverage | Executed branches / Total branches | 60% |
2050
- | Condition Coverage | Conditions evaluated T & F / Total conditions | 80% |
2051
- | MC/DC | Conditions independently affecting decision | 100% (safety-critical) |
2052
- | Requirements Coverage | Tested requirements / Total requirements | 100% |
2053
- | Mutation Score | Killed mutants / Total mutants | ≥ 80% |
99
+ | Style | Best For | Example |
100
+ |-------|----------|---------|
101
+ | `[Method]_[Scenario]_[Result]` | C#, Java | `CalculateTotal_NegativePrice_ThrowsException()` |
102
+ | `should_[behavior]_when_[condition]` | JavaScript/TypeScript | `should_reject_login_when_account_locked()` |
103
+ | `test_[method]_[scenario]_[expected]` | Python (pytest) | `test_validate_email_invalid_format_returns_false()` |
2054
104
 
2055
- ### Testing Process Metrics
105
+ #### Coverage Thresholds
2056
106
 
2057
- ```
2058
- ┌─────────────────────────────────────────────────────────────┐
2059
- │ Key Testing Metrics │
2060
- ├─────────────────────────────────────────────────────────────┤
2061
- │ │
2062
- │ Efficiency Metrics │
2063
- │ ├── Test Case Productivity = Tests created / Effort │
2064
- │ ├── Defect Detection Rate = Defects found / Test hours │
2065
- │ └── Automation Rate = Automated tests / Total tests │
2066
- │ │
2067
- │ Effectiveness Metrics │
2068
- │ ├── Defect Leakage = Prod defects / Total defects │
2069
- │ ├── Test Effectiveness = Defects found / Defects present │
2070
- │ └── Phase Containment = Defects found in phase / Injected │
2071
- │ │
2072
- │ Progress Metrics │
2073
- │ ├── Test Execution Rate = Tests run / Tests planned │
2074
- │ ├── Pass Rate = Tests passed / Tests executed │
2075
- │ └── Blocking Rate = Blocked tests / Total tests │
2076
- │ │
2077
- └─────────────────────────────────────────────────────────────┘
2078
- ```
107
+ | Metric | Minimum | Recommended |
108
+ |--------|---------|-------------|
109
+ | Line Coverage | 70% | 85% |
110
+ | Branch Coverage | 60% | 80% |
111
+ | Function Coverage | 80% | 90% |
2079
112
 
2080
113
  ---
2081
114
 
2082
- ## Defect Management
2083
-
2084
- Systematic tracking and management of defects throughout the testing lifecycle.
2085
-
2086
- **Reference**: ISTQB CTFL v4.0 Section 5.5, ISO/IEC/IEEE 29119-3
2087
-
2088
- ### Defect Lifecycle
2089
-
2090
- ```
2091
- ┌─────────────────────────────────────────────────────────────┐
2092
- │ Defect Lifecycle │
2093
- ├─────────────────────────────────────────────────────────────┤
2094
- │ │
2095
- │ ┌────────┐ ┌────────┐ ┌──────────┐ │
2096
- │ │ New │───▶│ Open │───▶│ In Progress│ │
2097
- │ └────────┘ └────────┘ └──────────┘ │
2098
- │ │ │ │
2099
- │ │ ▼ │
2100
- │ │ ┌──────────┐ │
2101
- │ │ │ Fixed │ │
2102
- │ │ └──────────┘ │
2103
- │ │ │ │
2104
- │ │ ▼ │
2105
- │ │ ┌──────────┐ ┌──────────┐ │
2106
- │ │ │ Verified │───▶│ Closed │ │
2107
- │ │ └──────────┘ └──────────┘ │
2108
- │ │ │ │
2109
- │ │ ▼ │
2110
- │ │ ┌──────────┐ │
2111
- │ └───────▶│ Reopened │ │
2112
- │ └──────────┘ │
2113
- │ │
2114
- │ Alternative paths: │
2115
- │ - New → Rejected (not a bug, duplicate) │
2116
- │ - Open → Deferred (postponed to later release) │
2117
- │ - Open → Won't Fix (accepted risk) │
2118
- │ │
2119
- └─────────────────────────────────────────────────────────────┘
2120
- ```
2121
-
2122
- ### Defect Report Template
2123
-
2124
- ```markdown
2125
- # Defect Report
2126
-
2127
- ## Summary
2128
- [One-line description of the defect]
2129
-
2130
- ## ID: BUG-1234
2131
- ## Priority: P1/P2/P3/P4
2132
- ## Severity: Critical/High/Medium/Low
2133
-
2134
- ## Environment
2135
- - **Version**: 2.3.1
2136
- - **OS**: Windows 11 / macOS 14.2
2137
- - **Browser**: Chrome 120
2138
- - **Device**: Desktop / Mobile
2139
-
2140
- ## Steps to Reproduce
2141
- 1. Navigate to login page
2142
- 2. Enter valid username "testuser"
2143
- 3. Enter invalid password "wrong123"
2144
- 4. Click "Login" button
2145
- 5. Observe error message
2146
-
2147
- ## Expected Result
2148
- Error message: "Invalid credentials. Please try again."
2149
-
2150
- ## Actual Result
2151
- Generic error: "Something went wrong."
115
+ ### Integration Testing (IT)
2152
116
 
2153
- ## Screenshots/Videos
2154
- [Attach relevant media]
117
+ **Characteristics**: Component integration, Real dependencies (often containerized), 1-10 seconds each
2155
118
 
2156
- ## Additional Information
2157
- - Occurs consistently (100% reproducible)
2158
- - Related to ticket: FEAT-456
2159
- - Log file: [attached]
2160
- ```
2161
-
2162
- ### Severity vs Priority
119
+ #### When Required
2163
120
 
2164
- | Severity | Description | Examples |
2165
- |----------|-------------|----------|
2166
- | Critical | System crash, data loss, security breach | App won't start, data corruption |
2167
- | High | Major feature broken, no workaround | Cannot complete checkout |
2168
- | Medium | Feature impaired but has workaround | Export fails, manual copy works |
2169
- | Low | Minor issue, cosmetic | Typo, alignment issue |
121
+ **Decision Rule**: If your unit test uses a wildcard matcher (`any()`, `It.IsAny<>`, `Arg.Any<>`) for a query/filter parameter, that functionality MUST have an integration test.
2170
122
 
2171
- | Priority | Description | Response Time |
2172
- |----------|-------------|---------------|
2173
- | P1 | Fix immediately | Same day |
2174
- | P2 | Fix before release | Within sprint |
2175
- | P3 | Fix when possible | Next sprint |
2176
- | P4 | Nice to fix | Backlog |
123
+ | Scenario | Reason |
124
+ |----------|--------|
125
+ | Query predicates | Mocks cannot verify filter expressions |
126
+ | Entity relationships | Verify foreign key correctness |
127
+ | Composite keys | In-memory DB may differ from real DB |
128
+ | Field mapping | DTO Entity transformations |
129
+ | Pagination | Row ordering and counting |
130
+ | Transactions | Rollback behavior |
2177
131
 
2178
- ### Defect Metrics
132
+ #### Scope
2179
133
 
2180
- ```
2181
- ┌─────────────────────────────────────────────────────────────┐
2182
- │ Key Defect Metrics │
2183
- ├─────────────────────────────────────────────────────────────┤
2184
- │ │
2185
- │ Defect Density = Total Defects / Size (KLOC or FP) │
2186
- │ - Industry average: 1-25 defects per KLOC │
2187
- │ │
2188
- │ Defect Detection Efficiency (DDE) = │
2189
- │ Defects found before release / Total defects × 100 │
2190
- │ - Target: > 90% │
2191
- │ │
2192
- │ Defect Removal Efficiency (DRE) = │
2193
- │ Defects removed / Defects injected × 100 │
2194
- │ - Target: > 95% │
2195
- │ │
2196
- │ Mean Time to Detect (MTTD) = Avg time from injection │
2197
- │ Mean Time to Resolve (MTTR) = Avg time from report to fix │
2198
- │ │
2199
- │ Defect Leakage Rate = Production defects / Total defects │
2200
- │ - Target: < 5% │
2201
- │ │
2202
- └─────────────────────────────────────────────────────────────┘
2203
- ```
134
+ | Include | Exclude |
135
+ |---------|---------|
136
+ | Database CRUD operations | Full user workflows |
137
+ | Repository + Database | Cross-service communication |
138
+ | Service + Repository | UI interactions |
139
+ | API endpoint + Service layer | |
140
+ | Message queue producers/consumers | |
141
+ | Cache read/write operations | |
2204
142
 
2205
- ### Root Cause Analysis
143
+ #### Naming Convention
2206
144
 
2207
145
  ```
2208
- ┌─────────────────────────────────────────────────────────────┐
2209
- │ Common Defect Root Causes │
2210
- ├─────────────────────────────────────────────────────────────┤
2211
- │ Requirements (35-40%) │
2212
- │ ├── Ambiguous requirements │
2213
- │ ├── Missing requirements │
2214
- │ └── Changed requirements │
2215
- ├─────────────────────────────────────────────────────────────┤
2216
- │ Design (20-25%) │
2217
- │ ├── Incorrect algorithm │
2218
- │ ├── Integration issues │
2219
- │ └── Performance not considered │
2220
- ├─────────────────────────────────────────────────────────────┤
2221
- │ Code (25-30%) │
2222
- │ ├── Logic errors │
2223
- │ ├── Boundary conditions │
2224
- │ └── Exception handling │
2225
- ├─────────────────────────────────────────────────────────────┤
2226
- │ Environment (10-15%) │
2227
- │ ├── Configuration issues │
2228
- │ ├── Third-party dependencies │
2229
- │ └── Data issues │
2230
- └─────────────────────────────────────────────────────────────┘
146
+ [ComponentName]IntegrationTests.[ext]
147
+ [ComponentName].integration.test.[ext]
148
+ [ComponentName].itest.[ext]
2231
149
  ```
2232
150
 
2233
151
  ---
2234
152
 
2235
- ## Test Process Management
2236
-
2237
- Structured approach to planning, monitoring, and controlling testing activities.
153
+ ### System Testing (ST)
2238
154
 
2239
- **Reference**: ISTQB CTFL v4.0 Chapter 5, ISO/IEC/IEEE 29119-2
155
+ **Characteristics**: Complete system, Production-like environment, Requirement-based
2240
156
 
2241
- ### Test Planning
157
+ #### Scope
2242
158
 
2243
- #### Test Plan Components (ISO/IEC/IEEE 29119-3)
159
+ | Include | Exclude |
160
+ |---------|---------|
161
+ | Complete API workflows | UI visual testing |
162
+ | Cross-service transactions | User journey simulations |
163
+ | Data flow through entire system | A/B testing scenarios |
164
+ | Security requirements | |
165
+ | Performance under load | |
166
+ | Error handling & recovery | |
2244
167
 
2245
- ```
2246
- ┌─────────────────────────────────────────────────────────────┐
2247
- │ Test Plan Structure │
2248
- ├─────────────────────────────────────────────────────────────┤
2249
- │ │
2250
- │ 1. Test Plan Identification │
2251
- │ - Project name, version, date │
2252
- │ │
2253
- │ 2. Introduction │
2254
- │ - Purpose, scope, objectives │
2255
- │ │
2256
- │ 3. Test Items │
2257
- │ - Features to be tested │
2258
- │ - Features not to be tested │
2259
- │ │
2260
- │ 4. Test Approach │
2261
- │ - Test levels and types │
2262
- │ - Test techniques │
2263
- │ - Entry/exit criteria │
2264
- │ │
2265
- │ 5. Test Environment │
2266
- │ - Hardware, software, tools │
2267
- │ - Test data requirements │
2268
- │ │
2269
- │ 6. Test Schedule │
2270
- │ - Milestones, dependencies │
2271
- │ │
2272
- │ 7. Roles and Responsibilities │
2273
- │ - Team structure, RACI matrix │
2274
- │ │
2275
- │ 8. Risks and Contingencies │
2276
- │ - Risk assessment, mitigation │
2277
- │ │
2278
- │ 9. Approvals │
2279
- │ - Sign-off requirements │
2280
- │ │
2281
- └─────────────────────────────────────────────────────────────┘
2282
- ```
2283
-
2284
- #### Entry and Exit Criteria
2285
-
2286
- | Criteria Type | Examples |
2287
- |---------------|----------|
2288
- | **Entry Criteria** (start testing) | Requirements approved, code complete, environment ready, test data available |
2289
- | **Exit Criteria** (stop testing) | All P1/P2 tests passed, coverage target met, no critical defects open |
2290
-
2291
- ```markdown
2292
- ## Entry Criteria for System Testing
2293
- - [ ] All integration tests passed
2294
- - [ ] Test environment deployed and verified
2295
- - [ ] Test data prepared and loaded
2296
- - [ ] Test cases reviewed and approved
2297
- - [ ] All blockers from IT resolved
2298
-
2299
- ## Exit Criteria for System Testing
2300
- - [ ] 100% of critical test cases executed
2301
- - [ ] 95% of high priority test cases passed
2302
- - [ ] No Critical or High severity defects open
2303
- - [ ] Code coverage > 80%
2304
- - [ ] Performance benchmarks met
2305
- ```
168
+ #### Types
2306
169
 
2307
- ### Test Monitoring and Control
2308
-
2309
- #### Key Progress Metrics
2310
-
2311
- ```
2312
- ┌─────────────────────────────────────────────────────────────┐
2313
- │ Test Progress Metrics │
2314
- ├─────────────────────────────────────────────────────────────┤
2315
- │ │
2316
- │ Execution Metrics: │
2317
- │ - Tests Planned: 500 │
2318
- │ - Tests Executed: 350 (70%) │
2319
- │ - Tests Passed: 320 (91% of executed) │
2320
- │ - Tests Failed: 30 (9% of executed) │
2321
- │ - Tests Blocked: 15 │
2322
- │ │
2323
- │ Defect Metrics: │
2324
- │ - Total Defects Found: 45 │
2325
- │ - Open Defects: 12 (3 Critical, 5 High, 4 Medium) │
2326
- │ - Fixed Defects: 28 │
2327
- │ - Verified Defects: 20 │
2328
- │ │
2329
- │ Coverage Metrics: │
2330
- │ - Requirements Coverage: 85% │
2331
- │ - Code Coverage: 78% │
2332
- │ - Risk Coverage: 90% │
2333
- │ │
2334
- └─────────────────────────────────────────────────────────────┘
2335
- ```
170
+ | Type | Description |
171
+ |------|-------------|
172
+ | Functional | Verify features work as specified |
173
+ | Performance | Load, stress, scalability testing |
174
+ | Security | Penetration, vulnerability scanning |
175
+ | Reliability | Failover, recovery, stability |
176
+ | Compatibility | Cross-platform, browser compatibility |
2336
177
 
2337
- #### Test Dashboard Example
178
+ #### Naming Convention
2338
179
 
2339
180
  ```
2340
- Test Progress Dashboard - Sprint 15
2341
- ═══════════════════════════════════════════════════════════════
2342
-
2343
- Test Execution Defect Status
2344
- █████████████████░░░░ 85% Open: ███░░░░░░░ 12
2345
- Fixed: ████████░░ 28
2346
- Passed │████████████████│ 320 Verified: ██████░░░░ 20
2347
- Failed │██ │ 30
2348
- Blocked │█ │ 15 Trend: ↓ (improving)
2349
-
2350
- Coverage Build Health
2351
- Requirements: ███████████░ 85% Last Build: ✓ Pass
2352
- Code: ████████░░░░ 78% Unit Tests: 245/245 ✓
2353
- Risk Items: █████████░░░ 90% Int Tests: 89/92 ⚠
2354
- Flaky Rate: 2.1%
181
+ [Feature]SystemTests.[ext]
182
+ [Feature].system.test.[ext]
183
+ [Feature]_st.[ext]
2355
184
  ```
2356
185
 
2357
- ### Test Completion
2358
-
2359
- #### Test Summary Report
2360
-
2361
- ```markdown
2362
- # Test Summary Report
2363
-
2364
- ## Project: [Project Name]
2365
- ## Version: [Version Number]
2366
- ## Test Period: [Start Date] - [End Date]
2367
-
2368
- ## Executive Summary
2369
- [Brief overview of testing activities and outcomes]
2370
-
2371
- ## Test Scope
2372
- - Features tested: [list]
2373
- - Features not tested: [list with justification]
2374
-
2375
- ## Test Results Summary
2376
-
2377
- | Test Level | Planned | Executed | Passed | Failed | Pass Rate |
2378
- |------------|---------|----------|--------|--------|-----------|
2379
- | Unit | 500 | 500 | 498 | 2 | 99.6% |
2380
- | Integration| 150 | 150 | 145 | 5 | 96.7% |
2381
- | System | 200 | 195 | 188 | 7 | 96.4% |
2382
- | E2E | 50 | 48 | 46 | 2 | 95.8% |
2383
-
2384
- ## Defect Summary
2385
-
2386
- | Severity | Found | Fixed | Open | Deferred |
2387
- |----------|-------|-------|------|----------|
2388
- | Critical | 2 | 2 | 0 | 0 |
2389
- | High | 8 | 7 | 1 | 0 |
2390
- | Medium | 15 | 12 | 2 | 1 |
2391
- | Low | 20 | 10 | 5 | 5 |
2392
-
2393
- ## Coverage Analysis
2394
- - Requirements coverage: 95%
2395
- - Code coverage: 82%
2396
- - Risk coverage: 100% (all high-risk items tested)
186
+ ---
2397
187
 
2398
- ## Open Issues and Risks
2399
- 1. [Issue 1]: Impact and mitigation
2400
- 2. [Issue 2]: Impact and mitigation
188
+ ### End-to-End Testing (E2E)
2401
189
 
2402
- ## Recommendations
2403
- 1. [Recommendation 1]
2404
- 2. [Recommendation 2]
190
+ **Characteristics**: User perspective, Full stack (UI → API → Database), Slowest (30s+ each)
2405
191
 
2406
- ## Conclusion
2407
- [Final assessment and release recommendation]
192
+ #### Scope
2408
193
 
2409
- ## Sign-off
2410
- - QA Lead: [Name, Date]
2411
- - Dev Lead: [Name, Date]
2412
- - Product Owner: [Name, Date]
2413
- ```
194
+ | Include | Exclude |
195
+ |---------|---------|
196
+ | Critical user journeys | Every possible user path |
197
+ | Login/Authentication flows | Edge cases (use UT/IT) |
198
+ | Core business transactions | Performance benchmarking |
199
+ | Cross-browser functionality | |
200
+ | Smoke tests for deployments | |
2414
201
 
2415
- ### Test Process Improvement
202
+ #### Naming Convention
2416
203
 
2417
204
  ```
2418
- ┌─────────────────────────────────────────────────────────────┐
2419
- │ Continuous Improvement Cycle │
2420
- ├─────────────────────────────────────────────────────────────┤
2421
- │ │
2422
- │ ┌───────────┐ │
2423
- │ │ Plan │ │
2424
- │ └─────┬─────┘ │
2425
- │ │ │
2426
- │ ▼ │
2427
- │ ┌───────────┐ ┌───────────┐ │
2428
- │ │ Act │◀───────▶│ Do │ │
2429
- │ └───────────┘ └─────┬─────┘ │
2430
- │ ▲ │ │
2431
- │ │ ▼ │
2432
- │ ┌─────┴─────┐ │
2433
- │ │ Check │ │
2434
- │ └───────────┘ │
2435
- │ │
2436
- │ Retrospective Questions: │
2437
- │ - What went well? │
2438
- │ - What could be improved? │
2439
- │ - What actions will we take? │
2440
- │ │
2441
- └─────────────────────────────────────────────────────────────┘
205
+ [UserJourney].e2e.[ext]
206
+ [Feature].e2e.spec.[ext]
207
+ e2e/[feature]/[scenario].[ext]
2442
208
  ```
2443
209
 
2444
210
  ---
2445
211
 
2446
212
  ## Test Doubles
2447
213
 
2448
- ### Types
2449
-
2450
- | Type | Purpose | Example Use |
214
+ | Type | Purpose | When to Use |
2451
215
  |------|---------|-------------|
2452
216
  | **Stub** | Returns predefined values | Fixed API responses |
2453
217
  | **Mock** | Verifies interactions | Verify method called |
@@ -2455,78 +219,34 @@ Risk Items: █████████░░░ 90% Int Tests: 89/92 ⚠
2455
219
  | **Spy** | Records calls, delegates to real | Partial mocking |
2456
220
  | **Dummy** | Placeholder, never used | Fill required parameters |
2457
221
 
2458
- ### Usage Guidelines
222
+ ### Usage by Test Level
2459
223
 
2460
- ```
2461
- ┌─────────────────────────────────────────────────────────┐
2462
- │ When to Use What │
2463
- ├─────────────────────────────────────────────────────────┤
2464
- │ Unit Tests (UT) │
2465
- │ ├── Use Mocks/Stubs for all external dependencies │
2466
- │ └── Verify interactions with Mocks │
2467
- ├─────────────────────────────────────────────────────────┤
2468
- │ Integration Tests (IT) │
2469
- │ ├── Use Fakes for databases (in-memory, containers) │
2470
- │ ├── Use Stubs for external APIs │
2471
- │ └── Minimize mocking - test real integrations │
2472
- ├─────────────────────────────────────────────────────────┤
2473
- │ System Tests (ST) │
2474
- │ ├── Use real components whenever possible │
2475
- │ ├── Use Fakes only for external third-party services │
2476
- │ └── No mocking within the system boundary │
2477
- ├─────────────────────────────────────────────────────────┤
2478
- │ E2E Tests │
2479
- │ ├── Use real everything │
2480
- │ └── Stub only external payment/email services │
2481
- └─────────────────────────────────────────────────────────┘
2482
- ```
224
+ | Level | Guidance |
225
+ |-------|----------|
226
+ | **UT** | Use Mocks/Stubs for all external dependencies |
227
+ | **IT** | Use Fakes for databases, Stubs for external APIs |
228
+ | **ST** | Use real components, Fakes only for external third-party |
229
+ | **E2E** | Use real everything; stub only external payment/email |
2483
230
 
2484
231
  ---
2485
232
 
2486
233
  ## Mock Limitations
2487
234
 
2488
- ### Query Predicate Verification
2489
-
2490
- **Problem**:
2491
- When mocking repository methods that accept query predicates (e.g., lambda expressions,
2492
- filter functions), using wildcard matchers like `any()` ignores the actual query logic,
2493
- allowing incorrect queries to pass unit tests.
235
+ **Problem**: Wildcard matchers (`any()`, `It.IsAny<>`) ignore actual query logic, allowing incorrect queries to pass.
2494
236
 
2495
- **Example**:
237
+ **Rule**: If mocking a method that accepts a query/filter/predicate parameter, you MUST have a corresponding integration test to verify the query logic.
2496
238
 
2497
239
  ```python
2498
- # Python Example
240
+ # Example - Python
2499
241
  # ❌ This test cannot verify query correctness
2500
242
  mock_repo.find.return_value = users
2501
- # Query could be wrong, test still passes
2502
243
 
2503
244
  # ✓ Add integration test to verify actual query
2504
245
  ```
2505
246
 
2506
- ```typescript
2507
- // TypeScript Example
2508
- // ❌ Jest mock ignores actual filter
2509
- jest.spyOn(repo, 'findBy').mockResolvedValue(users);
2510
-
2511
- // ✓ Verify with integration test
2512
- ```
2513
-
2514
- ```csharp
2515
- // C# Example
2516
- // ❌ Moq ignores the actual expression
2517
- _repo.Setup(r => r.FindAsync(It.IsAny<Expression<Func<User, bool>>>()))
2518
- .ReturnsAsync(users);
2519
-
2520
- // ✓ Verify with integration test or use It.Is<> to validate
2521
- ```
2522
-
2523
- **Rule of Thumb**:
2524
- If your unit test mocks a method that accepts a query/filter/predicate parameter,
2525
- you MUST have a corresponding integration test to verify the query logic.
2526
-
2527
247
  ---
2528
248
 
2529
- ## Test Data Management
249
+ ## Test Data Requirements
2530
250
 
2531
251
  ### Principles
2532
252
 
@@ -2535,294 +255,63 @@ you MUST have a corresponding integration test to verify the query logic.
2535
255
  3. **Determinism**: Tests don't depend on shared state
2536
256
  4. **Readability**: Test data clearly shows intent
2537
257
 
2538
- ### Patterns
2539
-
2540
- ```csharp
2541
- // Builder Pattern for Test Data
2542
- public class UserBuilder
2543
- {
2544
- private string _name = "Default User";
2545
- private string _email = "default@example.com";
2546
- private bool _isActive = true;
2547
-
2548
- public UserBuilder WithName(string name)
2549
- {
2550
- _name = name;
2551
- return this;
2552
- }
2553
-
2554
- public UserBuilder WithEmail(string email)
2555
- {
2556
- _email = email;
2557
- return this;
2558
- }
2559
-
2560
- public UserBuilder Inactive()
2561
- {
2562
- _isActive = false;
2563
- return this;
2564
- }
2565
-
2566
- public User Build() => new User
2567
- {
2568
- Name = _name,
2569
- Email = _email,
2570
- IsActive = _isActive
2571
- };
2572
- }
2573
-
2574
- // Usage in tests
2575
- var activeUser = new UserBuilder()
2576
- .WithName("Active User")
2577
- .Build();
2578
-
2579
- var inactiveUser = new UserBuilder()
2580
- .WithName("Inactive User")
2581
- .Inactive()
2582
- .Build();
2583
- ```
2584
-
2585
- ### Distinct Identifiers
2586
-
2587
- When entities have both a surrogate key (auto-generated ID) and a business identifier
2588
- (e.g., employee number, department code), test data MUST use different values for each.
258
+ ### Distinct Identifiers Rule
2589
259
 
2590
- **Problem**:
2591
- If test data uses identical values for both fields, field mapping errors go undetected.
260
+ When entities have both a surrogate key (auto-generated ID) and a business identifier, test data MUST use different values for each.
2592
261
 
2593
262
  ```python
2594
- # Python Example
2595
- # ❌ Wrong: id equals business_code
263
+ # Wrong: id equals business_code - mapping errors undetected
2596
264
  dept = Department(id=1, business_code=1)
2597
265
 
2598
266
  # ✓ Correct: distinct values catch mapping errors
2599
267
  dept = Department(id=1, business_code=1001)
2600
268
  ```
2601
269
 
2602
- ```csharp
2603
- // C# Example
2604
- // ❌ Wrong: Id equals DeptId - mapping errors go undetected
2605
- var dept = new Department { Id = 1, DeptId = 1 };
2606
-
2607
- // ✓ Correct: distinct values catch field mapping bugs
2608
- var dept = new Department { Id = 1, DeptId = 1001 };
2609
- ```
2610
-
2611
- **Validation**:
2612
- ```csharp
2613
- // C#
2614
- testData.Dept.Id.Should().NotBe(testData.Dept.DeptId,
2615
- "Test precondition: Id must differ from business identifier");
2616
- ```
2617
-
2618
- ### Composite Keys
270
+ ### Composite Keys Rule
2619
271
 
2620
272
  For entities with composite primary keys, ensure each record has a unique key combination.
2621
273
 
2622
- ```csharp
2623
- // C# Example
2624
- // Key collision - same (Id, SendTime) combination
2625
- var batch1 = new BatchRecord { Id = 0, SendTime = now };
2626
- var batch2 = new BatchRecord { Id = 0, SendTime = now }; // Conflict!
2627
-
2628
- // ✓ Unique combinations
2629
- var batch1 = new BatchRecord { Id = 0, SendTime = now.AddSeconds(1) };
2630
- var batch2 = new BatchRecord { Id = 0, SendTime = now.AddSeconds(2) };
2631
- ```
274
+ ```python
275
+ # ❌ Key collision
276
+ batch1 = BatchRecord(id=0, send_time=now)
277
+ batch2 = BatchRecord(id=0, send_time=now) # Conflict!
2632
278
 
2633
- **Tip**: Create helper functions that auto-generate unique composite keys.
2634
-
2635
- ```csharp
2636
- // C# Helper Example
2637
- private static int _timeOffset = 0;
2638
- public static BatchRecord CreateWithUniqueKey(DateTime baseTime)
2639
- {
2640
- return new BatchRecord
2641
- {
2642
- SendTime = baseTime.AddSeconds(Interlocked.Increment(ref _timeOffset))
2643
- };
2644
- }
279
+ # Unique combinations
280
+ batch1 = BatchRecord(id=0, send_time=now + timedelta(seconds=1))
281
+ batch2 = BatchRecord(id=0, send_time=now + timedelta(seconds=2))
2645
282
  ```
2646
283
 
2647
284
  ---
2648
285
 
2649
- ## Test Environment Isolation
286
+ ## Test Environment
2650
287
 
2651
- ### Purpose
288
+ ### Language-Specific Tools
2652
289
 
2653
- Ensure consistent, reproducible test results across development machines and CI/CD pipelines.
2654
-
2655
- ### Why It Matters
2656
-
2657
- - **Reproducibility**: Same tests produce same results everywhere
2658
- - **Isolation**: Project dependencies don't conflict with system or other projects
2659
- - **CI/CD Parity**: Local environment matches CI environment
2660
-
2661
- ### Language-Specific Virtual Environments
2662
-
2663
- | Language | Tools | Lock File |
2664
- |----------|-------|-----------|
2665
- | Python | venv, virtualenv, conda, poetry | requirements.txt, poetry.lock |
2666
- | Node.js | nvm, fnm + npm/yarn/pnpm | package-lock.json, yarn.lock |
2667
- | Ruby | rbenv, rvm, bundler | Gemfile.lock |
2668
- | Java | SDKMAN, jenv, Maven/Gradle | pom.xml, build.gradle.lock |
290
+ | Language | Version Manager | Lock File |
291
+ |----------|----------------|-----------|
292
+ | Python | venv, virtualenv, poetry | requirements.txt, poetry.lock |
293
+ | Node.js | nvm, fnm | package-lock.json, yarn.lock |
294
+ | Ruby | rbenv, rvm | Gemfile.lock |
295
+ | Java | SDKMAN, jenv | pom.xml, build.gradle.lock |
2669
296
  | .NET | dotnet SDK | packages.lock.json |
2670
297
  | Go | go mod | go.sum |
2671
298
  | Rust | rustup, cargo | Cargo.lock |
2672
299
 
2673
- #### Best Practices
300
+ ### Best Practices
2674
301
 
2675
302
  1. **Always use virtual environments** for development and testing
2676
303
  2. **Commit lock files** to version control
2677
304
  3. **Pin versions** in CI/CD pipelines
2678
305
  4. **Document required runtime versions** in README or .tool-versions
2679
306
 
2680
- #### Example: Python with venv
2681
-
2682
- ```bash
2683
- # Create virtual environment
2684
- python -m venv .venv
2685
-
2686
- # Activate
2687
- source .venv/bin/activate # Linux/macOS
2688
- .venv\Scripts\activate # Windows
2689
-
2690
- # Install dependencies
2691
- pip install -r requirements.txt
2692
-
2693
- # Run tests
2694
- pytest tests/
2695
- ```
2696
-
2697
- #### Example: Node.js with nvm
2698
-
2699
- ```bash
2700
- # Use project's Node version
2701
- nvm use
2702
-
2703
- # Install dependencies
2704
- npm ci
2705
-
2706
- # Run tests
2707
- npm test
2708
- ```
2709
-
2710
- ### Containerized Testing
2711
-
2712
- Use containers to provide consistent external dependencies (databases, message queues, etc.) for integration and system tests.
2713
-
2714
- #### When to Use
2715
-
2716
- | Test Level | Container Usage |
2717
- |------------|-----------------|
2718
- | UT (Unit Testing) | ❌ Not needed - use mocks |
2719
- | IT (Integration Testing) | ✅ Testcontainers for databases, caches |
2720
- | ST (System Testing) | ✅ Docker Compose for full environment |
2721
- | E2E (End-to-End) | ✅ Full containerized stack |
2722
-
2723
- #### Testcontainers
2724
-
2725
- Testcontainers provides lightweight, disposable containers for testing.
2726
-
2727
- ```csharp
2728
- // C# Example with Testcontainers
2729
- public class DatabaseIntegrationTests : IAsyncLifetime
2730
- {
2731
- private readonly PostgreSqlContainer _postgres = new PostgreSqlBuilder()
2732
- .WithImage("postgres:15")
2733
- .Build();
2734
-
2735
- public async Task InitializeAsync()
2736
- {
2737
- await _postgres.StartAsync();
2738
- }
2739
-
2740
- public async Task DisposeAsync()
2741
- {
2742
- await _postgres.DisposeAsync();
2743
- }
2744
-
2745
- [Fact]
2746
- public async Task Should_Connect_To_Database()
2747
- {
2748
- var connectionString = _postgres.GetConnectionString();
2749
- // Use connectionString for tests
2750
- }
2751
- }
2752
- ```
2753
-
2754
- ```python
2755
- # Python Example with Testcontainers
2756
- import pytest
2757
- from testcontainers.postgres import PostgresContainer
2758
-
2759
- @pytest.fixture(scope="module")
2760
- def postgres_container():
2761
- with PostgresContainer("postgres:15") as postgres:
2762
- yield postgres
2763
-
2764
- def test_database_connection(postgres_container):
2765
- connection_url = postgres_container.get_connection_url()
2766
- # Use connection_url for tests
2767
- ```
2768
-
2769
- #### Docker Compose for System Tests
2770
-
2771
- ```yaml
2772
- # docker-compose.test.yml
2773
- version: '3.8'
2774
- services:
2775
- app:
2776
- build: .
2777
- depends_on:
2778
- - db
2779
- - redis
2780
- - rabbitmq
2781
- environment:
2782
- - DATABASE_URL=postgres://test:test@db:5432/testdb
2783
- - REDIS_URL=redis://redis:6379
2784
- - RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672
2785
-
2786
- db:
2787
- image: postgres:15
2788
- environment:
2789
- POSTGRES_USER: test
2790
- POSTGRES_PASSWORD: test
2791
- POSTGRES_DB: testdb
2792
-
2793
- redis:
2794
- image: redis:7-alpine
2795
-
2796
- rabbitmq:
2797
- image: rabbitmq:3-management
2798
- ```
2799
-
2800
- ```bash
2801
- # Run system tests with Docker Compose
2802
- docker-compose -f docker-compose.test.yml up -d
2803
- npm run test:system
2804
- docker-compose -f docker-compose.test.yml down -v
2805
- ```
2806
-
2807
- ### Environment Parity Checklist
307
+ ### Container Usage by Test Level
2808
308
 
2809
- ```
2810
- ┌─────────────────────────────────────────────────────────────┐
2811
- │ Environment Parity Checklist │
2812
- ├─────────────────────────────────────────────────────────────┤
2813
- │ ✅ Same runtime version (Node, Python, etc.) locally & CI │
2814
- │ ✅ Same database version in containers & production │
2815
- │ ✅ Lock files committed and used in CI (npm ci, pip -r) │
2816
- │ ✅ Environment variables documented and consistent │
2817
- │ ✅ Container images tagged with specific versions │
2818
- │ ✅ .tool-versions or similar for runtime version mgmt │
2819
- ├─────────────────────────────────────────────────────────────┤
2820
- │ ❌ Using "latest" tags in production/CI │
2821
- │ ❌ Different DB versions between dev and CI │
2822
- │ ❌ Missing lock files in repository │
2823
- │ ❌ Hardcoded paths or machine-specific configurations │
2824
- └─────────────────────────────────────────────────────────────┘
2825
- ```
309
+ | Level | Container Usage |
310
+ |-------|-----------------|
311
+ | UT | Not needed - use mocks |
312
+ | IT | Testcontainers for databases, caches |
313
+ | ST | Docker Compose for full environment |
314
+ | E2E | Full containerized stack |
2826
315
 
2827
316
  ---
2828
317
 
@@ -2830,63 +319,22 @@ docker-compose -f docker-compose.test.yml down -v
2830
319
 
2831
320
  ### Test Execution Strategy
2832
321
 
2833
- ```yaml
2834
- # Example CI Pipeline
2835
- stages:
2836
- - unit-test # Run on every commit
2837
- - integration-test # Run on every commit
2838
- - system-test # Run on PR merge to main
2839
- - e2e-test # Run on release candidates
2840
-
2841
- unit-test:
2842
- stage: unit-test
2843
- script:
2844
- - npm run test:unit
2845
- timeout: 10m
2846
- rules:
2847
- - if: $CI_PIPELINE_SOURCE == "push"
2848
-
2849
- integration-test:
2850
- stage: integration-test
2851
- services:
2852
- - postgres:14
2853
- - redis:7
2854
- script:
2855
- - npm run test:integration
2856
- timeout: 30m
2857
- rules:
2858
- - if: $CI_PIPELINE_SOURCE == "push"
2859
-
2860
- system-test:
2861
- stage: system-test
2862
- environment: staging
2863
- script:
2864
- - npm run test:system
2865
- timeout: 2h
2866
- rules:
2867
- - if: $CI_COMMIT_BRANCH == "main"
2868
-
2869
- e2e-test:
2870
- stage: e2e-test
2871
- environment: staging
2872
- script:
2873
- - npm run test:e2e
2874
- timeout: 4h
2875
- rules:
2876
- - if: $CI_COMMIT_TAG
2877
- ```
2878
-
2879
- ### Test Reports
322
+ | Stage | When | Timeout |
323
+ |-------|------|---------|
324
+ | Unit Test | Every commit | 10 min |
325
+ | Integration Test | Every commit | 30 min |
326
+ | System Test | PR merge to main | 2 hours |
327
+ | E2E Test | Release candidates | 4 hours |
2880
328
 
2881
- Required metrics for each test level:
329
+ ### Required Metrics
2882
330
 
2883
331
  | Metric | UT | IT | ST | E2E |
2884
332
  |--------|----|----|----|----|
2885
- | Pass/Fail Count | | | | |
2886
- | Execution Time | | | | |
2887
- | Coverage % | | | ⚠️ | |
2888
- | Flaky Test Rate | | | | |
2889
- | Screenshots/Videos | | | ⚠️ | |
333
+ | Pass/Fail Count | Required | Required | Required | Required |
334
+ | Execution Time | Required | Required | Required | Required |
335
+ | Coverage % | Required | Required | Optional | Not needed |
336
+ | Flaky Test Rate | Required | Required | Required | Required |
337
+ | Screenshots/Videos | Not needed | Not needed | Optional | Required |
2890
338
 
2891
339
  ---
2892
340
 
@@ -2894,20 +342,10 @@ Required metrics for each test level:
2894
342
 
2895
343
  ### AAA Pattern
2896
344
 
2897
- ```csharp
2898
- [TestMethod]
2899
- public void MethodName_Scenario_ExpectedBehavior()
2900
- {
2901
- // Arrange - Set up test data and environment
2902
- var input = CreateTestInput();
2903
- var sut = new SystemUnderTest();
2904
-
2905
- // Act - Execute the behavior under test
2906
- var result = sut.Execute(input);
2907
-
2908
- // Assert - Verify the result
2909
- Assert.AreEqual(expected, result);
2910
- }
345
+ ```
346
+ // Arrange - Set up test data and environment
347
+ // Act - Execute the behavior under test
348
+ // Assert - Verify the result
2911
349
  ```
2912
350
 
2913
351
  ### FIRST Principles
@@ -2922,209 +360,89 @@ public void MethodName_Scenario_ExpectedBehavior()
2922
360
 
2923
361
  ### Anti-Patterns to Avoid
2924
362
 
2925
- ```
2926
- Test Interdependence
2927
- Tests that must run in specific order
2928
-
2929
- Flaky Tests
2930
- Tests that sometimes pass, sometimes fail
2931
-
2932
- ❌ Testing Implementation Details
2933
- Tests that break when refactoring
2934
-
2935
- ❌ Over-Mocking
2936
- Mocking so much that nothing real is tested
2937
-
2938
- ❌ Missing Assertions
2939
- Tests that verify nothing meaningful
2940
-
2941
- ❌ Magic Numbers/Strings
2942
- Unexplained values in test code
2943
-
2944
- ❌ Identical Test IDs
2945
- Using same values for surrogate and business keys
2946
- ```
363
+ - Test Interdependence (tests must run in specific order)
364
+ - Flaky Tests (sometimes pass, sometimes fail)
365
+ - Testing Implementation Details (tests break on refactoring)
366
+ - Over-Mocking (nothing real is tested)
367
+ - Missing Assertions (tests verify nothing meaningful)
368
+ - Magic Numbers/Strings (unexplained values)
369
+ - Identical Test IDs (same values for surrogate and business keys)
2947
370
 
2948
371
  ---
2949
372
 
2950
373
  ## Test Documentation Structure
2951
374
 
2952
- Improve test discoverability by maintaining a standardized documentation structure within test directories.
2953
-
2954
375
  ### tests/README.md Required Sections
2955
376
 
2956
- Every `tests/` directory SHOULD include a README.md with the following sections for discoverability.
377
+ Every `tests/` directory SHOULD include a README.md with:
2957
378
 
2958
379
  #### 1. Test Overview Table
2959
380
 
2960
- List all test types with count, technology stack, and execution environment.
2961
-
2962
- ```markdown
2963
381
  | Test Type | Count | Framework | Environment |
2964
382
  |-----------|-------|-----------|-------------|
2965
383
  | Unit Tests | 150 | Jest | Node.js |
2966
384
  | Integration Tests | 45 | Jest | Node.js + TestContainers |
2967
385
  | E2E Tests | 12 | Playwright | Browser |
2968
- ```
2969
386
 
2970
387
  #### 2. Current Status Section
2971
388
 
2972
- Display latest test execution results with clear pass/fail indicators.
2973
-
2974
- ```markdown
2975
- ## Current Status
2976
-
2977
389
  | Metric | Value | Target | Status |
2978
390
  |--------|-------|--------|--------|
2979
- | Pass Rate | 98.5% | 95% | |
2980
- | Line Coverage | 82% | 80% | |
2981
- | Branch Coverage | 75% | 70% | |
2982
-
2983
- **Last Run**: 2026-01-20 14:30 UTC
2984
- **Report**: [test-report-20260120-143000.md](results/test-report-20260120-143000.md)
2985
- ```
391
+ | Pass Rate | 98.5% | >= 95% | Pass |
392
+ | Line Coverage | 82% | >= 80% | Pass |
393
+ | Branch Coverage | 75% | >= 70% | Pass |
2986
394
 
2987
395
  #### 3. Report Links Section
2988
396
 
2989
- Include links to test reports, gap analysis, and coverage summary.
2990
-
2991
- ```markdown
2992
- ## Reports
2993
-
2994
397
  | Report Type | Location | Description |
2995
398
  |-------------|----------|-------------|
2996
- | Test Results | `results/` | Timestamped test execution reports |
2997
- | Coverage | `coverage/` | Code coverage reports (HTML, JSON) |
2998
- | Gap Analysis | `docs/gap-analysis.md` | Missing test coverage analysis |
2999
- ```
399
+ | Test Results | `results/` | Timestamped execution reports |
400
+ | Coverage | `coverage/` | Code coverage reports |
401
+ | Gap Analysis | `docs/gap-analysis.md` | Missing coverage analysis |
3000
402
 
3001
403
  ### Test Report Naming Convention
3002
404
 
3003
405
  | Item | Convention | Example |
3004
406
  |------|------------|---------|
3005
- | Report filename | `test-report-YYYYMMDD-HHMMSS.md` | `test-report-20260120-143000.md` |
3006
- | Report directory | `tests/results/` | `tests/results/test-report-*.md` |
3007
- | Coverage directory | `tests/coverage/` | `tests/coverage/lcov-report/` |
3008
- | Gap analysis | `tests/docs/gap-analysis.md` | - |
407
+ | Report filename | `test-report-YYYYMMDD-HHMMSS.md` | `test-report-20260129-143000.md` |
408
+ | Report directory | `tests/results/` | |
409
+ | Coverage directory | `tests/coverage/` | |
3009
410
 
3010
- ### Directory Structure Example
411
+ ### Directory Structure
3011
412
 
3012
413
  ```
3013
414
  tests/
3014
415
  ├── README.md # Test overview and status
3015
416
  ├── results/ # Test execution reports
3016
- │ ├── test-report-20260120-143000.md
3017
- │ └── test-report-20260119-090000.md
3018
417
  ├── coverage/ # Coverage reports
3019
- │ ├── lcov-report/
3020
- │ └── coverage-summary.json
3021
418
  ├── docs/ # Test documentation
3022
- │ └── gap-analysis.md
3023
419
  ├── unit/ # Unit tests
3024
420
  ├── integration/ # Integration tests
3025
421
  └── e2e/ # End-to-end tests
3026
422
  ```
3027
423
 
3028
- ### Benefits
3029
-
3030
- | Benefit | Description |
3031
- |---------|-------------|
3032
- | **Discoverability** | Developers can quickly find test status without searching |
3033
- | **Transparency** | Current quality metrics visible at a glance |
3034
- | **Traceability** | Historical reports enable trend analysis |
3035
- | **Onboarding** | New team members understand test structure immediately |
3036
-
3037
424
  ---
3038
425
 
3039
- ## Quick Reference Card
426
+ ## Coverage Targets Summary
3040
427
 
3041
- ```
3042
- ┌─────────────────────────────────────────────────────────────┐
3043
- │ Reference Standards │
3044
- ├─────────────────────────────────────────────────────────────┤
3045
- │ • ISTQB CTFL v4.0 - Testing certification │
3046
- │ • ISO/IEC/IEEE 29119 - Testing standards │
3047
- │ • SWEBOK v4.0 - Software engineering knowledge │
3048
- ├─────────────────────────────────────────────────────────────┤
3049
- │ Testing Fundamentals (SWEBOK) │
3050
- ├─────────────────────────────────────────────────────────────┤
3051
- │ Error → Fault → Failure (cause chain) │
3052
- │ Oracle Problem: How to know expected result? │
3053
- │ Limitation: Testing proves presence, not absence of bugs │
3054
- │ Testability: Controllability + Observability + Isolability │
3055
- ├─────────────────────────────────────────────────────────────┤
3056
- │ Testing Framework Options │
3057
- ├─────────────────────────────────────────────────────────────┤
3058
- │ ISTQB: UT → IT/SIT → ST → AT/UAT (Enterprise/Compliance) │
3059
- │ Industry: UT (70%) → IT (20%) → E2E (10%) (Agile/DevOps) │
3060
- ├─────────────────────────────────────────────────────────────┤
3061
- │ Testing Levels Summary │
3062
- ├──────────┬──────────────────────────────────────────────────┤
3063
- │ UT │ Single unit, isolated, mocked deps, < 100ms │
3064
- ├──────────┼──────────────────────────────────────────────────┤
3065
- │ IT/SIT │ Component integration, real DB, 1-10 sec │
3066
- ├──────────┼──────────────────────────────────────────────────┤
3067
- │ ST │ Full system, requirement-based (ISTQB only) │
3068
- ├──────────┼──────────────────────────────────────────────────┤
3069
- │ E2E │ User journeys, UI to DB, critical paths only │
3070
- ├──────────┼──────────────────────────────────────────────────┤
3071
- │ AT/UAT │ Business validation by end users (ISTQB only) │
3072
- ├──────────┴──────────────────────────────────────────────────┤
3073
- │ Test Types (ISTQB) │
3074
- ├─────────────────────────────────────────────────────────────┤
3075
- │ Functional: Verify system behavior (black-box) │
3076
- │ Non-Functional: Performance, security, usability │
3077
- │ Structural: Code coverage, white-box techniques │
3078
- │ Change-Related: Confirmation & regression testing │
3079
- ├─────────────────────────────────────────────────────────────┤
3080
- │ Test Design Techniques │
3081
- ├─────────────────────────────────────────────────────────────┤
3082
- │ Black-Box: EP, BVA, Decision Table, State Transition │
3083
- │ Pairwise Testing, Use Case Testing │
3084
- │ White-Box: Statement, Branch, Condition, MC/DC │
3085
- │ Data Flow (All-Defs, All-Uses) │
3086
- │ Experience: Exploratory, Error Guessing, Checklists │
3087
- ├─────────────────────────────────────────────────────────────┤
3088
- │ Coverage Targets │
3089
- ├─────────────────────────────────────────────────────────────┤
3090
- │ Line: 70% min / 85% recommended │
3091
- │ Branch: 60% min / 80% recommended │
3092
- │ Function: 80% min / 90% recommended │
3093
- │ Mutation Score: ≥ 80% for critical code │
3094
- ├─────────────────────────────────────────────────────────────┤
3095
- │ Specialized Testing Types │
3096
- ├─────────────────────────────────────────────────────────────┤
3097
- │ Contract: API compatibility for microservices │
3098
- │ Mutation: Test quality via code mutations │
3099
- │ Chaos: Resilience via failure injection │
3100
- │ Property-Based: Random inputs based on properties │
3101
- │ Visual Regression: UI screenshot comparison │
3102
- │ Accessibility: WCAG compliance validation │
3103
- │ Installation: Deployment verification │
3104
- │ Recovery: Failure recovery (RTO/RPO) │
3105
- │ Configuration: Multi-config testing │
3106
- │ Usability: User experience (SUS, NPS) │
3107
- ├─────────────────────────────────────────────────────────────┤
3108
- │ Test-Related Measures │
3109
- ├─────────────────────────────────────────────────────────────┤
3110
- │ Fault Density = Faults / KLOC (target: < 1) │
3111
- │ Defect Leakage = Prod defects / Total defects (< 5%) │
3112
- │ Fault Seeding: Insert known faults to measure detection │
3113
- │ Reliability Growth: Track defect rate over time │
3114
- └─────────────────────────────────────────────────────────────┘
3115
- ```
428
+ | Metric | Minimum | Recommended |
429
+ |--------|---------|-------------|
430
+ | Line | 70% | 85% |
431
+ | Branch | 60% | 80% |
432
+ | Function | 80% | 90% |
433
+ | Mutation Score | - | >= 80% (critical code) |
3116
434
 
3117
435
  ---
3118
436
 
3119
437
  ## Related Standards
3120
438
 
439
+ - [Testing Theory Knowledge Base](../skills/testing-guide/testing-theory.md) - Educational content, examples, techniques
3121
440
  - [Test-Driven Development](test-driven-development.md) - TDD/BDD/ATDD methodology
3122
- - [Test Completeness Dimensions](test-completeness-dimensions.md) - 7-dimension test coverage
441
+ - [Test Completeness Dimensions](test-completeness-dimensions.md) - 8-dimension test coverage
3123
442
  - [Spec-Driven Development](spec-driven-development.md) - SDD workflow integration
3124
- - [Anti-Hallucination Standard](anti-hallucination.md)
3125
443
  - [Code Check-in Standards](checkin-standards.md)
3126
444
  - [Code Review Checklist](code-review-checklist.md)
3127
- - [Commit Message Guide](commit-message-guide.md)
445
+ - [Deployment Standards](deployment-standards.md) - Test requirements for deployment readiness
3128
446
 
3129
447
  ---
3130
448
 
@@ -3132,13 +450,14 @@ tests/
3132
450
 
3133
451
  | Version | Date | Changes |
3134
452
  |---------|------|---------|
3135
- | 2.2.0 | 2026-01-20 | Added Test Documentation Structure section: tests/README.md required sections, test report naming conventions, directory structure example. |
3136
- | 2.1.0 | 2026-01-05 | Added SWEBOK v4.0 reference. New sections: Testing Fundamentals (Faults vs Failures, Oracle Problem, Theoretical Limitations, Testability, Test Adequacy), Test-Related Measures (Fault Density, Reliability Growth, Fault Seeding). Expanded: Test Design Techniques (Pairwise Testing, Data Flow Testing), Specialized Testing (Installation, Alpha/Beta, Recovery, Configuration, Back-to-Back, Usability). Updated Quick Reference Card. |
3137
- | 2.0.0 | 2026-01-05 | Major update aligned with ISTQB CTFL v4.0 and ISO/IEC/IEEE 29119. Added: Static Testing (reviews, static analysis), Test Design Techniques (EP, BVA, Decision Table, State Transition, Use Case, Statement/Branch/Condition/MC/DC coverage), Experience-Based Testing (exploratory, error guessing, checklists), Risk-Based Testing (risk assessment, prioritization), Specialized Testing (contract, mutation, chaos, property-based, visual regression, accessibility), Defect Management (lifecycle, metrics, root cause), Test Process Management (planning, monitoring, completion) |
3138
- | 1.3.0 | 2025-12-29 | Add: Testing Framework Selection (ISTQB vs Industry Pyramid), IT/SIT abbreviation clarification, source references |
3139
- | 1.2.0 | 2025-12-19 | Add: Mock Limitations section, When Integration Tests Are Required, Distinct Identifiers, Composite Keys test data patterns |
3140
- | 1.1.1 | 2025-12-11 | Improved: System test example to use generic domain concepts instead of specific business terminology |
3141
- | 1.1.0 | 2025-12-05 | Add test environment isolation section (venv, containers) |
453
+ | 3.0.0 | 2026-01-29 | **Major refactor**: Split into Rules (this file) and Theory (testing-theory.md). Reduced from 141KB/3185 lines to ~12KB/350 lines. All educational content moved to skills/testing-guide/testing-theory.md. Rules-only format optimized for AI agent consumption. |
454
+ | 2.2.0 | 2026-01-20 | Added Test Documentation Structure section |
455
+ | 2.1.0 | 2026-01-05 | Added SWEBOK v4.0 reference, Testing Fundamentals, Test-Related Measures |
456
+ | 2.0.0 | 2026-01-05 | Major update aligned with ISTQB CTFL v4.0 and ISO/IEC/IEEE 29119 |
457
+ | 1.3.0 | 2025-12-29 | Add Testing Framework Selection, IT/SIT abbreviation clarification |
458
+ | 1.2.0 | 2025-12-19 | Add Mock Limitations, Integration Test requirements, Test Data patterns |
459
+ | 1.1.1 | 2025-12-11 | Improved System test example with generic domain concepts |
460
+ | 1.1.0 | 2025-12-05 | Add test environment isolation section |
3142
461
  | 1.0.0 | 2025-12-05 | Initial testing standards with UT/IT/ST/E2E coverage |
3143
462
 
3144
463
  ---