universal-dev-standards 3.5.1-beta.1 → 3.5.1-beta.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (433) hide show
  1. package/bin/uds.js +2 -0
  2. package/bundled/core/ai-instruction-standards.md +205 -0
  3. package/bundled/core/anti-hallucination.md +684 -0
  4. package/bundled/core/changelog-standards.md +556 -0
  5. package/bundled/core/checkin-standards.md +935 -0
  6. package/bundled/core/code-review-checklist.md +684 -0
  7. package/bundled/core/commit-message-guide.md +915 -0
  8. package/bundled/core/documentation-structure.md +1117 -0
  9. package/bundled/core/documentation-writing-standards.md +487 -0
  10. package/bundled/core/error-code-standards.md +382 -0
  11. package/bundled/core/git-workflow.md +859 -0
  12. package/bundled/core/logging-standards.md +323 -0
  13. package/bundled/core/project-structure.md +354 -0
  14. package/bundled/core/refactoring-standards.md +636 -0
  15. package/bundled/core/spec-driven-development.md +207 -0
  16. package/bundled/core/test-completeness-dimensions.md +536 -0
  17. package/bundled/core/test-driven-development.md +995 -0
  18. package/bundled/core/testing-standards.md +3061 -0
  19. package/bundled/core/versioning.md +902 -0
  20. package/bundled/locales/README.md +88 -0
  21. package/bundled/locales/zh-CN/CHANGELOG.md +705 -0
  22. package/bundled/locales/zh-CN/CLAUDE.md +213 -0
  23. package/bundled/locales/zh-CN/MAINTENANCE.md +689 -0
  24. package/bundled/locales/zh-CN/README.md +707 -0
  25. package/bundled/locales/zh-CN/STANDARDS-MAPPING.md +177 -0
  26. package/bundled/locales/zh-CN/adoption/ADOPTION-GUIDE.md +392 -0
  27. package/bundled/locales/zh-CN/adoption/STATIC-DYNAMIC-GUIDE.md +302 -0
  28. package/bundled/locales/zh-CN/adoption/checklists/enterprise.md +332 -0
  29. package/bundled/locales/zh-CN/adoption/checklists/minimal.md +141 -0
  30. package/bundled/locales/zh-CN/adoption/checklists/recommended.md +272 -0
  31. package/bundled/locales/zh-CN/ai/MAINTENANCE.md +739 -0
  32. package/bundled/locales/zh-CN/ai/options/changelog/auto-generated.ai.yaml +76 -0
  33. package/bundled/locales/zh-CN/ai/options/changelog/keep-a-changelog.ai.yaml +99 -0
  34. package/bundled/locales/zh-CN/ai/options/code-review/automated-review.ai.yaml +120 -0
  35. package/bundled/locales/zh-CN/ai/options/code-review/pair-programming.ai.yaml +109 -0
  36. package/bundled/locales/zh-CN/ai/options/code-review/pr-review.ai.yaml +104 -0
  37. package/bundled/locales/zh-CN/ai/options/commit-message/bilingual.ai.yaml +105 -0
  38. package/bundled/locales/zh-CN/ai/options/commit-message/english.ai.yaml +79 -0
  39. package/bundled/locales/zh-CN/ai/options/commit-message/traditional-chinese.ai.yaml +100 -0
  40. package/bundled/locales/zh-CN/ai/options/documentation/api-docs.ai.yaml +140 -0
  41. package/bundled/locales/zh-CN/ai/options/documentation/markdown-docs.ai.yaml +89 -0
  42. package/bundled/locales/zh-CN/ai/options/documentation/wiki-style.ai.yaml +119 -0
  43. package/bundled/locales/zh-CN/ai/options/git-workflow/gitflow.ai.yaml +133 -0
  44. package/bundled/locales/zh-CN/ai/options/git-workflow/github-flow.ai.yaml +73 -0
  45. package/bundled/locales/zh-CN/ai/options/git-workflow/merge-commit.ai.yaml +88 -0
  46. package/bundled/locales/zh-CN/ai/options/git-workflow/rebase-ff.ai.yaml +113 -0
  47. package/bundled/locales/zh-CN/ai/options/git-workflow/squash-merge.ai.yaml +85 -0
  48. package/bundled/locales/zh-CN/ai/options/git-workflow/trunk-based.ai.yaml +117 -0
  49. package/bundled/locales/zh-CN/ai/options/project-structure/dotnet.ai.yaml +108 -0
  50. package/bundled/locales/zh-CN/ai/options/project-structure/go.ai.yaml +115 -0
  51. package/bundled/locales/zh-CN/ai/options/project-structure/java.ai.yaml +113 -0
  52. package/bundled/locales/zh-CN/ai/options/project-structure/kotlin.ai.yaml +123 -0
  53. package/bundled/locales/zh-CN/ai/options/project-structure/nodejs.ai.yaml +101 -0
  54. package/bundled/locales/zh-CN/ai/options/project-structure/php.ai.yaml +147 -0
  55. package/bundled/locales/zh-CN/ai/options/project-structure/python.ai.yaml +116 -0
  56. package/bundled/locales/zh-CN/ai/options/project-structure/ruby.ai.yaml +140 -0
  57. package/bundled/locales/zh-CN/ai/options/project-structure/rust.ai.yaml +117 -0
  58. package/bundled/locales/zh-CN/ai/options/project-structure/swift.ai.yaml +143 -0
  59. package/bundled/locales/zh-CN/ai/options/testing/contract-testing.ai.yaml +254 -0
  60. package/bundled/locales/zh-CN/ai/options/testing/e2e-testing.ai.yaml +116 -0
  61. package/bundled/locales/zh-CN/ai/options/testing/industry-pyramid.ai.yaml +144 -0
  62. package/bundled/locales/zh-CN/ai/options/testing/integration-testing.ai.yaml +90 -0
  63. package/bundled/locales/zh-CN/ai/options/testing/istqb-framework.ai.yaml +108 -0
  64. package/bundled/locales/zh-CN/ai/options/testing/performance-testing.ai.yaml +272 -0
  65. package/bundled/locales/zh-CN/ai/options/testing/security-testing.ai.yaml +160 -0
  66. package/bundled/locales/zh-CN/ai/options/testing/system-testing.ai.yaml +101 -0
  67. package/bundled/locales/zh-CN/ai/options/testing/unit-testing.ai.yaml +82 -0
  68. package/bundled/locales/zh-CN/ai/standards/anti-hallucination.ai.yaml +145 -0
  69. package/bundled/locales/zh-CN/ai/standards/changelog.ai.yaml +146 -0
  70. package/bundled/locales/zh-CN/ai/standards/checkin-standards.ai.yaml +170 -0
  71. package/bundled/locales/zh-CN/ai/standards/code-review.ai.yaml +148 -0
  72. package/bundled/locales/zh-CN/ai/standards/commit-message.ai.yaml +175 -0
  73. package/bundled/locales/zh-CN/ai/standards/documentation-structure.ai.yaml +124 -0
  74. package/bundled/locales/zh-CN/ai/standards/documentation-writing-standards.ai.yaml +190 -0
  75. package/bundled/locales/zh-CN/ai/standards/error-codes.ai.yaml +139 -0
  76. package/bundled/locales/zh-CN/ai/standards/git-workflow.ai.yaml +95 -0
  77. package/bundled/locales/zh-CN/ai/standards/logging.ai.yaml +128 -0
  78. package/bundled/locales/zh-CN/ai/standards/project-structure.ai.yaml +134 -0
  79. package/bundled/locales/zh-CN/ai/standards/spec-driven-development.ai.yaml +169 -0
  80. package/bundled/locales/zh-CN/ai/standards/test-completeness-dimensions.ai.yaml +220 -0
  81. package/bundled/locales/zh-CN/ai/standards/testing.ai.yaml +137 -0
  82. package/bundled/locales/zh-CN/ai/standards/versioning.ai.yaml +211 -0
  83. package/bundled/locales/zh-CN/core/ai-instruction-standards.md +213 -0
  84. package/bundled/locales/zh-CN/core/anti-hallucination.md +691 -0
  85. package/bundled/locales/zh-CN/core/changelog-standards.md +147 -0
  86. package/bundled/locales/zh-CN/core/checkin-standards.md +943 -0
  87. package/bundled/locales/zh-CN/core/code-review-guide.md +693 -0
  88. package/bundled/locales/zh-CN/core/commit-message-guide.md +129 -0
  89. package/bundled/locales/zh-CN/core/documentation-structure.md +173 -0
  90. package/bundled/locales/zh-CN/core/documentation-writing-standards.md +495 -0
  91. package/bundled/locales/zh-CN/core/error-code-standards.md +180 -0
  92. package/bundled/locales/zh-CN/core/git-workflow.md +193 -0
  93. package/bundled/locales/zh-CN/core/logging-standards.md +174 -0
  94. package/bundled/locales/zh-CN/core/project-structure.md +184 -0
  95. package/bundled/locales/zh-CN/core/refactoring-standards.md +642 -0
  96. package/bundled/locales/zh-CN/core/spec-driven-development.md +215 -0
  97. package/bundled/locales/zh-CN/core/test-completeness-dimensions.md +544 -0
  98. package/bundled/locales/zh-CN/core/test-driven-development.md +1002 -0
  99. package/bundled/locales/zh-CN/core/testing-standards.md +170 -0
  100. package/bundled/locales/zh-CN/core/versioning.md +177 -0
  101. package/bundled/locales/zh-CN/docs/AI-AGENT-ROADMAP.md +303 -0
  102. package/bundled/locales/zh-CN/docs/CLI-INIT-OPTIONS.md +990 -0
  103. package/bundled/locales/zh-CN/docs/OPERATION-WORKFLOW.md +1064 -0
  104. package/bundled/locales/zh-CN/docs/USAGE-MODES-COMPARISON.md +333 -0
  105. package/bundled/locales/zh-CN/docs/WINDOWS-GUIDE.md +215 -0
  106. package/bundled/locales/zh-CN/integrations/codex/AGENTS.md +115 -0
  107. package/bundled/locales/zh-CN/integrations/codex/README.md +67 -0
  108. package/bundled/locales/zh-CN/integrations/gemini-cli/GEMINI.md +102 -0
  109. package/bundled/locales/zh-CN/integrations/gemini-cli/README.md +140 -0
  110. package/bundled/locales/zh-CN/integrations/github-copilot/COPILOT-CHAT-REFERENCE.md +269 -0
  111. package/bundled/locales/zh-CN/integrations/github-copilot/README.md +167 -0
  112. package/bundled/locales/zh-CN/integrations/github-copilot/copilot-instructions.md +263 -0
  113. package/bundled/locales/zh-CN/integrations/github-copilot/skills-mapping.md +192 -0
  114. package/bundled/locales/zh-CN/integrations/google-antigravity/INSTRUCTIONS.md +61 -0
  115. package/bundled/locales/zh-CN/integrations/google-antigravity/README.md +84 -0
  116. package/bundled/locales/zh-CN/integrations/opencode/AGENTS.md +111 -0
  117. package/bundled/locales/zh-CN/integrations/opencode/README.md +176 -0
  118. package/bundled/locales/zh-CN/integrations/opencode/skills-mapping.md +396 -0
  119. package/bundled/locales/zh-CN/integrations/openspec/AGENTS.md +301 -0
  120. package/bundled/locales/zh-CN/integrations/openspec/README.md +43 -0
  121. package/bundled/locales/zh-CN/integrations/spec-kit/AGENTS.md +184 -0
  122. package/bundled/locales/zh-CN/integrations/spec-kit/README.md +43 -0
  123. package/bundled/locales/zh-CN/options/commit-message/bilingual.md +161 -0
  124. package/bundled/locales/zh-CN/options/commit-message/english.md +137 -0
  125. package/bundled/locales/zh-CN/options/commit-message/traditional-chinese.md +162 -0
  126. package/bundled/locales/zh-CN/options/git-workflow/gitflow.md +147 -0
  127. package/bundled/locales/zh-CN/options/git-workflow/github-flow.md +124 -0
  128. package/bundled/locales/zh-CN/options/git-workflow/merge-commit.md +124 -0
  129. package/bundled/locales/zh-CN/options/git-workflow/rebase-ff.md +160 -0
  130. package/bundled/locales/zh-CN/options/git-workflow/squash-merge.md +117 -0
  131. package/bundled/locales/zh-CN/options/git-workflow/trunk-based.md +125 -0
  132. package/bundled/locales/zh-CN/options/project-structure/dotnet.md +183 -0
  133. package/bundled/locales/zh-CN/options/project-structure/go.md +226 -0
  134. package/bundled/locales/zh-CN/options/project-structure/java.md +213 -0
  135. package/bundled/locales/zh-CN/options/project-structure/nodejs.md +185 -0
  136. package/bundled/locales/zh-CN/options/project-structure/python.md +229 -0
  137. package/bundled/locales/zh-CN/options/testing/e2e-testing.md +207 -0
  138. package/bundled/locales/zh-CN/options/testing/integration-testing.md +230 -0
  139. package/bundled/locales/zh-CN/options/testing/system-testing.md +183 -0
  140. package/bundled/locales/zh-CN/options/testing/unit-testing.md +165 -0
  141. package/bundled/locales/zh-CN/skills/INTEGRATION-GUIDE.md +218 -0
  142. package/bundled/locales/zh-CN/skills/README.md +134 -0
  143. package/bundled/locales/zh-CN/skills/_shared/README.md +68 -0
  144. package/bundled/locales/zh-CN/skills/claude-code/CONTRIBUTING.template.md +151 -0
  145. package/bundled/locales/zh-CN/skills/claude-code/README.md +174 -0
  146. package/bundled/locales/zh-CN/skills/claude-code/ai-collaboration-standards/SKILL.md +175 -0
  147. package/bundled/locales/zh-CN/skills/claude-code/ai-collaboration-standards/anti-hallucination.md +223 -0
  148. package/bundled/locales/zh-CN/skills/claude-code/ai-collaboration-standards/certainty-labels.md +132 -0
  149. package/bundled/locales/zh-CN/skills/claude-code/changelog-guide/SKILL.md +237 -0
  150. package/bundled/locales/zh-CN/skills/claude-code/checkin-assistant/SKILL.md +407 -0
  151. package/bundled/locales/zh-CN/skills/claude-code/code-review-assistant/SKILL.md +154 -0
  152. package/bundled/locales/zh-CN/skills/claude-code/code-review-assistant/checkin-checklist.md +257 -0
  153. package/bundled/locales/zh-CN/skills/claude-code/code-review-assistant/review-checklist.md +246 -0
  154. package/bundled/locales/zh-CN/skills/claude-code/commands/bdd.md +142 -0
  155. package/bundled/locales/zh-CN/skills/claude-code/commands/methodology.md +274 -0
  156. package/bundled/locales/zh-CN/skills/claude-code/commit-standards/SKILL.md +191 -0
  157. package/bundled/locales/zh-CN/skills/claude-code/commit-standards/conventional-commits.md +264 -0
  158. package/bundled/locales/zh-CN/skills/claude-code/commit-standards/language-options.md +172 -0
  159. package/bundled/locales/zh-CN/skills/claude-code/documentation-guide/SKILL.md +421 -0
  160. package/bundled/locales/zh-CN/skills/claude-code/documentation-guide/documentation-structure.md +357 -0
  161. package/bundled/locales/zh-CN/skills/claude-code/documentation-guide/readme-template.md +412 -0
  162. package/bundled/locales/zh-CN/skills/claude-code/error-code-guide/SKILL.md +269 -0
  163. package/bundled/locales/zh-CN/skills/claude-code/git-workflow-guide/SKILL.md +218 -0
  164. package/bundled/locales/zh-CN/skills/claude-code/git-workflow-guide/branch-naming.md +220 -0
  165. package/bundled/locales/zh-CN/skills/claude-code/git-workflow-guide/git-workflow.md +321 -0
  166. package/bundled/locales/zh-CN/skills/claude-code/logging-guide/SKILL.md +285 -0
  167. package/bundled/locales/zh-CN/skills/claude-code/methodology-system/SKILL.md +131 -0
  168. package/bundled/locales/zh-CN/skills/claude-code/methodology-system/create-methodology.md +350 -0
  169. package/bundled/locales/zh-CN/skills/claude-code/methodology-system/runtime.md +279 -0
  170. package/bundled/locales/zh-CN/skills/claude-code/project-structure-guide/SKILL.md +143 -0
  171. package/bundled/locales/zh-CN/skills/claude-code/project-structure-guide/language-patterns.md +271 -0
  172. package/bundled/locales/zh-CN/skills/claude-code/refactoring-assistant/SKILL.md +162 -0
  173. package/bundled/locales/zh-CN/skills/claude-code/release-standards/SKILL.md +191 -0
  174. package/bundled/locales/zh-CN/skills/claude-code/release-standards/changelog-format.md +247 -0
  175. package/bundled/locales/zh-CN/skills/claude-code/release-standards/release-workflow.md +345 -0
  176. package/bundled/locales/zh-CN/skills/claude-code/release-standards/semantic-versioning.md +250 -0
  177. package/bundled/locales/zh-CN/skills/claude-code/requirement-assistant/SKILL.md +227 -0
  178. package/bundled/locales/zh-CN/skills/claude-code/requirement-assistant/requirement-checklist.md +325 -0
  179. package/bundled/locales/zh-CN/skills/claude-code/requirement-assistant/requirement-writing.md +399 -0
  180. package/bundled/locales/zh-CN/skills/claude-code/spec-driven-dev/SKILL.md +243 -0
  181. package/bundled/locales/zh-CN/skills/claude-code/tdd-assistant/SKILL.md +332 -0
  182. package/bundled/locales/zh-CN/skills/claude-code/tdd-assistant/language-examples.md +639 -0
  183. package/bundled/locales/zh-CN/skills/claude-code/tdd-assistant/tdd-workflow.md +486 -0
  184. package/bundled/locales/zh-CN/skills/claude-code/test-coverage-assistant/SKILL.md +282 -0
  185. package/bundled/locales/zh-CN/skills/claude-code/testing-guide/SKILL.md +234 -0
  186. package/bundled/locales/zh-CN/skills/claude-code/testing-guide/testing-pyramid.md +448 -0
  187. package/bundled/locales/zh-CN/skills/cline/README.md +58 -0
  188. package/bundled/locales/zh-CN/skills/copilot/README.md +61 -0
  189. package/bundled/locales/zh-CN/skills/copilot/copilot-instructions.md +79 -0
  190. package/bundled/locales/zh-CN/skills/cursor/README.md +58 -0
  191. package/bundled/locales/zh-CN/skills/windsurf/README.md +59 -0
  192. package/bundled/locales/zh-TW/CHANGELOG.md +707 -0
  193. package/bundled/locales/zh-TW/CLAUDE.md +213 -0
  194. package/bundled/locales/zh-TW/MAINTENANCE.md +683 -0
  195. package/bundled/locales/zh-TW/README.md +687 -0
  196. package/bundled/locales/zh-TW/STANDARDS-MAPPING.md +177 -0
  197. package/bundled/locales/zh-TW/adoption/ADOPTION-GUIDE.md +392 -0
  198. package/bundled/locales/zh-TW/adoption/STATIC-DYNAMIC-GUIDE.md +299 -0
  199. package/bundled/locales/zh-TW/adoption/checklists/enterprise.md +332 -0
  200. package/bundled/locales/zh-TW/adoption/checklists/minimal.md +141 -0
  201. package/bundled/locales/zh-TW/adoption/checklists/recommended.md +272 -0
  202. package/bundled/locales/zh-TW/ai/MAINTENANCE.md +754 -0
  203. package/bundled/locales/zh-TW/ai/options/changelog/auto-generated.ai.yaml +76 -0
  204. package/bundled/locales/zh-TW/ai/options/changelog/keep-a-changelog.ai.yaml +99 -0
  205. package/bundled/locales/zh-TW/ai/options/code-review/automated-review.ai.yaml +120 -0
  206. package/bundled/locales/zh-TW/ai/options/code-review/pair-programming.ai.yaml +109 -0
  207. package/bundled/locales/zh-TW/ai/options/code-review/pr-review.ai.yaml +104 -0
  208. package/bundled/locales/zh-TW/ai/options/commit-message/bilingual.ai.yaml +105 -0
  209. package/bundled/locales/zh-TW/ai/options/commit-message/english.ai.yaml +79 -0
  210. package/bundled/locales/zh-TW/ai/options/commit-message/traditional-chinese.ai.yaml +100 -0
  211. package/bundled/locales/zh-TW/ai/options/documentation/api-docs.ai.yaml +140 -0
  212. package/bundled/locales/zh-TW/ai/options/documentation/markdown-docs.ai.yaml +89 -0
  213. package/bundled/locales/zh-TW/ai/options/documentation/wiki-style.ai.yaml +119 -0
  214. package/bundled/locales/zh-TW/ai/options/git-workflow/gitflow.ai.yaml +133 -0
  215. package/bundled/locales/zh-TW/ai/options/git-workflow/github-flow.ai.yaml +73 -0
  216. package/bundled/locales/zh-TW/ai/options/git-workflow/merge-commit.ai.yaml +88 -0
  217. package/bundled/locales/zh-TW/ai/options/git-workflow/rebase-ff.ai.yaml +113 -0
  218. package/bundled/locales/zh-TW/ai/options/git-workflow/squash-merge.ai.yaml +85 -0
  219. package/bundled/locales/zh-TW/ai/options/git-workflow/trunk-based.ai.yaml +117 -0
  220. package/bundled/locales/zh-TW/ai/options/project-structure/dotnet.ai.yaml +108 -0
  221. package/bundled/locales/zh-TW/ai/options/project-structure/go.ai.yaml +115 -0
  222. package/bundled/locales/zh-TW/ai/options/project-structure/java.ai.yaml +113 -0
  223. package/bundled/locales/zh-TW/ai/options/project-structure/kotlin.ai.yaml +123 -0
  224. package/bundled/locales/zh-TW/ai/options/project-structure/nodejs.ai.yaml +101 -0
  225. package/bundled/locales/zh-TW/ai/options/project-structure/php.ai.yaml +147 -0
  226. package/bundled/locales/zh-TW/ai/options/project-structure/python.ai.yaml +116 -0
  227. package/bundled/locales/zh-TW/ai/options/project-structure/ruby.ai.yaml +140 -0
  228. package/bundled/locales/zh-TW/ai/options/project-structure/rust.ai.yaml +117 -0
  229. package/bundled/locales/zh-TW/ai/options/project-structure/swift.ai.yaml +143 -0
  230. package/bundled/locales/zh-TW/ai/options/testing/contract-testing.ai.yaml +254 -0
  231. package/bundled/locales/zh-TW/ai/options/testing/e2e-testing.ai.yaml +116 -0
  232. package/bundled/locales/zh-TW/ai/options/testing/industry-pyramid.ai.yaml +144 -0
  233. package/bundled/locales/zh-TW/ai/options/testing/integration-testing.ai.yaml +90 -0
  234. package/bundled/locales/zh-TW/ai/options/testing/istqb-framework.ai.yaml +108 -0
  235. package/bundled/locales/zh-TW/ai/options/testing/performance-testing.ai.yaml +272 -0
  236. package/bundled/locales/zh-TW/ai/options/testing/security-testing.ai.yaml +160 -0
  237. package/bundled/locales/zh-TW/ai/options/testing/system-testing.ai.yaml +101 -0
  238. package/bundled/locales/zh-TW/ai/options/testing/unit-testing.ai.yaml +82 -0
  239. package/bundled/locales/zh-TW/ai/standards/anti-hallucination.ai.yaml +145 -0
  240. package/bundled/locales/zh-TW/ai/standards/changelog.ai.yaml +146 -0
  241. package/bundled/locales/zh-TW/ai/standards/checkin-standards.ai.yaml +170 -0
  242. package/bundled/locales/zh-TW/ai/standards/code-review.ai.yaml +148 -0
  243. package/bundled/locales/zh-TW/ai/standards/commit-message.ai.yaml +175 -0
  244. package/bundled/locales/zh-TW/ai/standards/documentation-structure.ai.yaml +124 -0
  245. package/bundled/locales/zh-TW/ai/standards/documentation-writing-standards.ai.yaml +190 -0
  246. package/bundled/locales/zh-TW/ai/standards/error-codes.ai.yaml +139 -0
  247. package/bundled/locales/zh-TW/ai/standards/git-workflow.ai.yaml +95 -0
  248. package/bundled/locales/zh-TW/ai/standards/logging.ai.yaml +128 -0
  249. package/bundled/locales/zh-TW/ai/standards/project-structure.ai.yaml +134 -0
  250. package/bundled/locales/zh-TW/ai/standards/spec-driven-development.ai.yaml +169 -0
  251. package/bundled/locales/zh-TW/ai/standards/test-completeness-dimensions.ai.yaml +220 -0
  252. package/bundled/locales/zh-TW/ai/standards/testing.ai.yaml +137 -0
  253. package/bundled/locales/zh-TW/ai/standards/versioning.ai.yaml +211 -0
  254. package/bundled/locales/zh-TW/core/ai-instruction-standards.md +213 -0
  255. package/bundled/locales/zh-TW/core/anti-hallucination.md +691 -0
  256. package/bundled/locales/zh-TW/core/changelog-standards.md +564 -0
  257. package/bundled/locales/zh-TW/core/checkin-standards.md +943 -0
  258. package/bundled/locales/zh-TW/core/code-review-checklist.md +693 -0
  259. package/bundled/locales/zh-TW/core/commit-message-guide.md +809 -0
  260. package/bundled/locales/zh-TW/core/documentation-structure.md +1125 -0
  261. package/bundled/locales/zh-TW/core/documentation-writing-standards.md +495 -0
  262. package/bundled/locales/zh-TW/core/error-code-standards.md +384 -0
  263. package/bundled/locales/zh-TW/core/git-workflow.md +860 -0
  264. package/bundled/locales/zh-TW/core/logging-standards.md +325 -0
  265. package/bundled/locales/zh-TW/core/project-structure.md +362 -0
  266. package/bundled/locales/zh-TW/core/refactoring-standards.md +642 -0
  267. package/bundled/locales/zh-TW/core/spec-driven-development.md +215 -0
  268. package/bundled/locales/zh-TW/core/test-completeness-dimensions.md +544 -0
  269. package/bundled/locales/zh-TW/core/test-driven-development.md +1004 -0
  270. package/bundled/locales/zh-TW/core/testing-standards.md +2158 -0
  271. package/bundled/locales/zh-TW/core/versioning.md +909 -0
  272. package/bundled/locales/zh-TW/docs/AI-AGENT-ROADMAP.md +303 -0
  273. package/bundled/locales/zh-TW/docs/CLI-INIT-OPTIONS.md +990 -0
  274. package/bundled/locales/zh-TW/docs/OPERATION-WORKFLOW.md +1064 -0
  275. package/bundled/locales/zh-TW/docs/USAGE-MODES-COMPARISON.md +333 -0
  276. package/bundled/locales/zh-TW/docs/WINDOWS-GUIDE.md +215 -0
  277. package/bundled/locales/zh-TW/integrations/codex/AGENTS.md +115 -0
  278. package/bundled/locales/zh-TW/integrations/codex/README.md +113 -0
  279. package/bundled/locales/zh-TW/integrations/gemini-cli/GEMINI.md +102 -0
  280. package/bundled/locales/zh-TW/integrations/gemini-cli/README.md +140 -0
  281. package/bundled/locales/zh-TW/integrations/github-copilot/COPILOT-CHAT-REFERENCE.md +269 -0
  282. package/bundled/locales/zh-TW/integrations/github-copilot/README.md +167 -0
  283. package/bundled/locales/zh-TW/integrations/github-copilot/copilot-instructions.md +263 -0
  284. package/bundled/locales/zh-TW/integrations/github-copilot/skills-mapping.md +192 -0
  285. package/bundled/locales/zh-TW/integrations/google-antigravity/INSTRUCTIONS.md +61 -0
  286. package/bundled/locales/zh-TW/integrations/google-antigravity/README.md +84 -0
  287. package/bundled/locales/zh-TW/integrations/opencode/AGENTS.md +111 -0
  288. package/bundled/locales/zh-TW/integrations/opencode/README.md +176 -0
  289. package/bundled/locales/zh-TW/integrations/opencode/skills-mapping.md +396 -0
  290. package/bundled/locales/zh-TW/integrations/openspec/AGENTS.md +301 -0
  291. package/bundled/locales/zh-TW/integrations/openspec/README.md +43 -0
  292. package/bundled/locales/zh-TW/integrations/spec-kit/AGENTS.md +184 -0
  293. package/bundled/locales/zh-TW/integrations/spec-kit/README.md +43 -0
  294. package/bundled/locales/zh-TW/options/commit-message/bilingual.md +161 -0
  295. package/bundled/locales/zh-TW/options/commit-message/english.md +137 -0
  296. package/bundled/locales/zh-TW/options/commit-message/traditional-chinese.md +162 -0
  297. package/bundled/locales/zh-TW/options/git-workflow/gitflow.md +147 -0
  298. package/bundled/locales/zh-TW/options/git-workflow/github-flow.md +124 -0
  299. package/bundled/locales/zh-TW/options/git-workflow/merge-commit.md +124 -0
  300. package/bundled/locales/zh-TW/options/git-workflow/rebase-ff.md +160 -0
  301. package/bundled/locales/zh-TW/options/git-workflow/squash-merge.md +117 -0
  302. package/bundled/locales/zh-TW/options/git-workflow/trunk-based.md +125 -0
  303. package/bundled/locales/zh-TW/options/project-structure/dotnet.md +183 -0
  304. package/bundled/locales/zh-TW/options/project-structure/go.md +226 -0
  305. package/bundled/locales/zh-TW/options/project-structure/java.md +213 -0
  306. package/bundled/locales/zh-TW/options/project-structure/nodejs.md +185 -0
  307. package/bundled/locales/zh-TW/options/project-structure/python.md +229 -0
  308. package/bundled/locales/zh-TW/options/testing/e2e-testing.md +207 -0
  309. package/bundled/locales/zh-TW/options/testing/integration-testing.md +230 -0
  310. package/bundled/locales/zh-TW/options/testing/system-testing.md +183 -0
  311. package/bundled/locales/zh-TW/options/testing/unit-testing.md +165 -0
  312. package/bundled/locales/zh-TW/skills/INTEGRATION-GUIDE.md +218 -0
  313. package/bundled/locales/zh-TW/skills/README.md +132 -0
  314. package/bundled/locales/zh-TW/skills/_shared/README.md +68 -0
  315. package/bundled/locales/zh-TW/skills/claude-code/CONTRIBUTING.template.md +151 -0
  316. package/bundled/locales/zh-TW/skills/claude-code/README.md +174 -0
  317. package/bundled/locales/zh-TW/skills/claude-code/ai-collaboration-standards/SKILL.md +175 -0
  318. package/bundled/locales/zh-TW/skills/claude-code/ai-collaboration-standards/anti-hallucination.md +223 -0
  319. package/bundled/locales/zh-TW/skills/claude-code/ai-collaboration-standards/certainty-labels.md +132 -0
  320. package/bundled/locales/zh-TW/skills/claude-code/changelog-guide/SKILL.md +237 -0
  321. package/bundled/locales/zh-TW/skills/claude-code/checkin-assistant/SKILL.md +407 -0
  322. package/bundled/locales/zh-TW/skills/claude-code/code-review-assistant/SKILL.md +154 -0
  323. package/bundled/locales/zh-TW/skills/claude-code/code-review-assistant/checkin-checklist.md +257 -0
  324. package/bundled/locales/zh-TW/skills/claude-code/code-review-assistant/review-checklist.md +246 -0
  325. package/bundled/locales/zh-TW/skills/claude-code/commands/bdd.md +142 -0
  326. package/bundled/locales/zh-TW/skills/claude-code/commands/methodology.md +274 -0
  327. package/bundled/locales/zh-TW/skills/claude-code/commit-standards/SKILL.md +191 -0
  328. package/bundled/locales/zh-TW/skills/claude-code/commit-standards/conventional-commits.md +264 -0
  329. package/bundled/locales/zh-TW/skills/claude-code/commit-standards/language-options.md +172 -0
  330. package/bundled/locales/zh-TW/skills/claude-code/documentation-guide/SKILL.md +421 -0
  331. package/bundled/locales/zh-TW/skills/claude-code/documentation-guide/documentation-structure.md +357 -0
  332. package/bundled/locales/zh-TW/skills/claude-code/documentation-guide/readme-template.md +412 -0
  333. package/bundled/locales/zh-TW/skills/claude-code/error-code-guide/SKILL.md +269 -0
  334. package/bundled/locales/zh-TW/skills/claude-code/git-workflow-guide/SKILL.md +218 -0
  335. package/bundled/locales/zh-TW/skills/claude-code/git-workflow-guide/branch-naming.md +220 -0
  336. package/bundled/locales/zh-TW/skills/claude-code/git-workflow-guide/git-workflow.md +321 -0
  337. package/bundled/locales/zh-TW/skills/claude-code/logging-guide/SKILL.md +285 -0
  338. package/bundled/locales/zh-TW/skills/claude-code/methodology-system/SKILL.md +131 -0
  339. package/bundled/locales/zh-TW/skills/claude-code/methodology-system/create-methodology.md +350 -0
  340. package/bundled/locales/zh-TW/skills/claude-code/methodology-system/runtime.md +279 -0
  341. package/bundled/locales/zh-TW/skills/claude-code/project-structure-guide/SKILL.md +143 -0
  342. package/bundled/locales/zh-TW/skills/claude-code/project-structure-guide/language-patterns.md +271 -0
  343. package/bundled/locales/zh-TW/skills/claude-code/refactoring-assistant/SKILL.md +162 -0
  344. package/bundled/locales/zh-TW/skills/claude-code/release-standards/SKILL.md +191 -0
  345. package/bundled/locales/zh-TW/skills/claude-code/release-standards/changelog-format.md +247 -0
  346. package/bundled/locales/zh-TW/skills/claude-code/release-standards/release-workflow.md +345 -0
  347. package/bundled/locales/zh-TW/skills/claude-code/release-standards/semantic-versioning.md +250 -0
  348. package/bundled/locales/zh-TW/skills/claude-code/requirement-assistant/SKILL.md +227 -0
  349. package/bundled/locales/zh-TW/skills/claude-code/requirement-assistant/requirement-checklist.md +325 -0
  350. package/bundled/locales/zh-TW/skills/claude-code/requirement-assistant/requirement-writing.md +399 -0
  351. package/bundled/locales/zh-TW/skills/claude-code/spec-driven-dev/SKILL.md +243 -0
  352. package/bundled/locales/zh-TW/skills/claude-code/tdd-assistant/SKILL.md +332 -0
  353. package/bundled/locales/zh-TW/skills/claude-code/tdd-assistant/language-examples.md +639 -0
  354. package/bundled/locales/zh-TW/skills/claude-code/tdd-assistant/tdd-workflow.md +486 -0
  355. package/bundled/locales/zh-TW/skills/claude-code/test-coverage-assistant/SKILL.md +282 -0
  356. package/bundled/locales/zh-TW/skills/claude-code/testing-guide/SKILL.md +234 -0
  357. package/bundled/locales/zh-TW/skills/claude-code/testing-guide/testing-pyramid.md +448 -0
  358. package/bundled/locales/zh-TW/skills/cline/README.md +58 -0
  359. package/bundled/locales/zh-TW/skills/copilot/README.md +61 -0
  360. package/bundled/locales/zh-TW/skills/copilot/copilot-instructions.md +79 -0
  361. package/bundled/locales/zh-TW/skills/cursor/README.md +58 -0
  362. package/bundled/locales/zh-TW/skills/windsurf/README.md +59 -0
  363. package/bundled/skills/claude-code/CONTRIBUTING.template.md +141 -0
  364. package/bundled/skills/claude-code/README.md +196 -0
  365. package/bundled/skills/claude-code/ai/standards/checkin.ai.yaml +21 -0
  366. package/bundled/skills/claude-code/ai/standards/commit.ai.yaml +20 -0
  367. package/bundled/skills/claude-code/ai/standards/refactoring.ai.yaml +34 -0
  368. package/bundled/skills/claude-code/ai/standards/testing.ai.yaml +41 -0
  369. package/bundled/skills/claude-code/ai-collaboration-standards/SKILL.md +175 -0
  370. package/bundled/skills/claude-code/ai-collaboration-standards/anti-hallucination.md +215 -0
  371. package/bundled/skills/claude-code/ai-collaboration-standards/certainty-labels.md +124 -0
  372. package/bundled/skills/claude-code/changelog-guide/SKILL.md +232 -0
  373. package/bundled/skills/claude-code/checkin-assistant/SKILL.md +402 -0
  374. package/bundled/skills/claude-code/code-review-assistant/SKILL.md +220 -0
  375. package/bundled/skills/claude-code/code-review-assistant/checkin-checklist.md +249 -0
  376. package/bundled/skills/claude-code/code-review-assistant/review-checklist.md +238 -0
  377. package/bundled/skills/claude-code/commands/README.md +78 -0
  378. package/bundled/skills/claude-code/commands/bdd.md +142 -0
  379. package/bundled/skills/claude-code/commands/changelog.md +57 -0
  380. package/bundled/skills/claude-code/commands/check.md +91 -0
  381. package/bundled/skills/claude-code/commands/commit.md +48 -0
  382. package/bundled/skills/claude-code/commands/config.md +97 -0
  383. package/bundled/skills/claude-code/commands/coverage.md +58 -0
  384. package/bundled/skills/claude-code/commands/docs.md +75 -0
  385. package/bundled/skills/claude-code/commands/init.md +88 -0
  386. package/bundled/skills/claude-code/commands/methodology.md +268 -0
  387. package/bundled/skills/claude-code/commands/release.md +50 -0
  388. package/bundled/skills/claude-code/commands/requirement.md +54 -0
  389. package/bundled/skills/claude-code/commands/review.md +50 -0
  390. package/bundled/skills/claude-code/commands/spec.md +69 -0
  391. package/bundled/skills/claude-code/commands/tdd.md +86 -0
  392. package/bundled/skills/claude-code/commands/update.md +122 -0
  393. package/bundled/skills/claude-code/commit-standards/SKILL.md +249 -0
  394. package/bundled/skills/claude-code/commit-standards/conventional-commits.md +256 -0
  395. package/bundled/skills/claude-code/commit-standards/language-options.md +164 -0
  396. package/bundled/skills/claude-code/documentation-guide/SKILL.md +416 -0
  397. package/bundled/skills/claude-code/documentation-guide/documentation-structure.md +349 -0
  398. package/bundled/skills/claude-code/documentation-guide/readme-template.md +404 -0
  399. package/bundled/skills/claude-code/error-code-guide/SKILL.md +264 -0
  400. package/bundled/skills/claude-code/git-workflow-guide/SKILL.md +226 -0
  401. package/bundled/skills/claude-code/git-workflow-guide/branch-naming.md +212 -0
  402. package/bundled/skills/claude-code/git-workflow-guide/git-workflow.md +313 -0
  403. package/bundled/skills/claude-code/logging-guide/SKILL.md +280 -0
  404. package/bundled/skills/claude-code/methodology-system/SKILL.md +215 -0
  405. package/bundled/skills/claude-code/methodology-system/create-methodology.md +450 -0
  406. package/bundled/skills/claude-code/methodology-system/runtime.md +271 -0
  407. package/bundled/skills/claude-code/project-structure-guide/SKILL.md +143 -0
  408. package/bundled/skills/claude-code/project-structure-guide/language-patterns.md +263 -0
  409. package/bundled/skills/claude-code/refactoring-assistant/SKILL.md +205 -0
  410. package/bundled/skills/claude-code/release-standards/SKILL.md +191 -0
  411. package/bundled/skills/claude-code/release-standards/changelog-format.md +239 -0
  412. package/bundled/skills/claude-code/release-standards/release-workflow.md +337 -0
  413. package/bundled/skills/claude-code/release-standards/semantic-versioning.md +242 -0
  414. package/bundled/skills/claude-code/requirement-assistant/SKILL.md +222 -0
  415. package/bundled/skills/claude-code/requirement-assistant/requirement-checklist.md +317 -0
  416. package/bundled/skills/claude-code/requirement-assistant/requirement-writing.md +391 -0
  417. package/bundled/skills/claude-code/spec-driven-dev/SKILL.md +238 -0
  418. package/bundled/skills/claude-code/tdd-assistant/SKILL.md +384 -0
  419. package/bundled/skills/claude-code/tdd-assistant/language-examples.md +1276 -0
  420. package/bundled/skills/claude-code/tdd-assistant/tdd-workflow.md +659 -0
  421. package/bundled/skills/claude-code/test-coverage-assistant/SKILL.md +277 -0
  422. package/bundled/skills/claude-code/testing-guide/SKILL.md +317 -0
  423. package/bundled/skills/claude-code/testing-guide/testing-pyramid.md +440 -0
  424. package/package.json +4 -2
  425. package/src/commands/check.js +38 -6
  426. package/src/commands/init.js +20 -6
  427. package/src/commands/list.js +7 -2
  428. package/src/commands/skills.js +0 -2
  429. package/src/commands/update.js +271 -0
  430. package/src/i18n/messages.js +161 -11
  431. package/src/utils/copier.js +63 -10
  432. package/src/utils/skills-installer.js +47 -11
  433. package/standards-registry.json +6 -3
@@ -0,0 +1,3061 @@
1
+ # Testing Standards
2
+
3
+ > **Language**: English | [繁體中文](../locales/zh-TW/core/testing-standards.md)
4
+
5
+ **Version**: 2.1.0
6
+ **Last Updated**: 2026-01-05
7
+ **Applicability**: All software projects
8
+
9
+ ---
10
+
11
+ ## Purpose
12
+
13
+ This standard defines testing conventions and best practices to ensure software quality through systematic testing at multiple levels.
14
+
15
+ **Reference Standards**:
16
+ - [ISTQB CTFL v4.0](https://istqb.org/certifications/certified-tester-foundation-level-ctfl-v4-0/)
17
+ - [ISO/IEC/IEEE 29119](https://www.iso.org/standard/81291.html)
18
+ - [SWEBOK v4.0](https://www.computer.org/education/bodies-of-knowledge/software-engineering) - IEEE Computer Society
19
+
20
+ ---
21
+
22
+ ## Table of Contents
23
+
24
+ 1. [Testing Fundamentals](#testing-fundamentals) ⭐ NEW
25
+ 2. [Testing Framework Selection](#testing-framework-selection)
26
+ 3. [Testing Pyramid](#testing-pyramid-default-industry-standard)
27
+ 4. [Test Levels](#unit-testing-ut) (UT, IT, ST, E2E)
28
+ 5. [Static Testing](#static-testing)
29
+ 6. [Test Design Techniques](#test-design-techniques)
30
+ 7. [Experience-Based Testing](#experience-based-testing)
31
+ 8. [Risk-Based Testing](#risk-based-testing)
32
+ 9. [Specialized Testing](#specialized-testing)
33
+ 10. [Test-Related Measures](#test-related-measures) ⭐ NEW
34
+ 11. [Defect Management](#defect-management)
35
+ 12. [Test Process Management](#test-process-management)
36
+ 13. [Test Doubles](#test-doubles)
37
+ 14. [Test Data Management](#test-data-management)
38
+ 15. [Test Environment Isolation](#test-environment-isolation)
39
+ 16. [CI/CD Integration](#cicd-integration)
40
+ 17. [Best Practices](#best-practices)
41
+
42
+ ---
43
+
44
+ ## Testing Fundamentals
45
+
46
+ Understanding the theoretical foundations of software testing is essential for effective test design and execution.
47
+
48
+ **Reference**: SWEBOK v4.0 Chapter 5
49
+
50
+ ### Faults vs Failures
51
+
52
+ ```
53
+ ┌─────────────────────────────────────────────────────────────┐
54
+ │ Faults, Errors, and Failures │
55
+ ├─────────────────────────────────────────────────────────────┤
56
+ │ │
57
+ │ Error (Mistake) │
58
+ │ │ │
59
+ │ ▼ introduced during development │
60
+ │ ┌─────────┐ │
61
+ │ │ Fault │ (Defect/Bug in the code) │
62
+ │ └─────────┘ │
63
+ │ │ │
64
+ │ ▼ when executed │
65
+ │ ┌─────────┐ │
66
+ │ │ Failure │ (Observable incorrect behavior) │
67
+ │ └─────────┘ │
68
+ │ │
69
+ │ Key Points: │
70
+ │ • A fault may exist without causing a failure │
71
+ │ • A failure requires a fault to be executed │
72
+ │ • Multiple faults can cause the same failure │
73
+ │ • Testing finds failures; debugging finds faults │
74
+ │ │
75
+ └─────────────────────────────────────────────────────────────┘
76
+ ```
77
+
78
+ ### The Oracle Problem
79
+
80
+ An **oracle** is any mechanism that determines whether observed program behavior is correct. The oracle problem addresses the challenge of knowing expected outcomes.
81
+
82
+ ```
83
+ ┌─────────────────────────────────────────────────────────────┐
84
+ │ Oracle Types │
85
+ ├─────────────────────────────────────────────────────────────┤
86
+ │ │
87
+ │ Human Oracle │
88
+ │ └── Manual verification by domain expert │
89
+ │ │
90
+ │ Specification-Based Oracle │
91
+ │ └── Compare against formal requirements │
92
+ │ │
93
+ │ Reference Implementation │
94
+ │ └── Compare against known-correct implementation │
95
+ │ │
96
+ │ Metamorphic Oracle │
97
+ │ └── Verify relationships between outputs │
98
+ │ (e.g., sort(sort(x)) == sort(x)) │
99
+ │ │
100
+ │ Statistical Oracle │
101
+ │ └── Validate against expected distributions │
102
+ │ │
103
+ │ Implicit Oracle │
104
+ │ └── No crash, no exception, no timeout │
105
+ │ │
106
+ └─────────────────────────────────────────────────────────────┘
107
+ ```
108
+
109
+ ### Theoretical Limitations of Testing
110
+
111
+ > "Program testing can be used to show the presence of bugs, but never to show their absence." — Edsger W. Dijkstra
112
+
113
+ ```
114
+ ┌─────────────────────────────────────────────────────────────┐
115
+ │ Testing Limitations │
116
+ ├─────────────────────────────────────────────────────────────┤
117
+ │ │
118
+ │ Exhaustive Testing is Impossible │
119
+ │ ├── Infinite input domain │
120
+ │ ├── State space explosion │
121
+ │ └── Timing and concurrency variations │
122
+ │ │
123
+ │ Infeasible Paths │
124
+ │ ├── Some code paths cannot be executed by any input │
125
+ │ ├── Dead code may appear reachable in CFG │
126
+ │ └── 100% path coverage may be mathematically impossible │
127
+ │ │
128
+ │ Halting Problem │
129
+ │ └── Cannot algorithmically determine if all tests halt │
130
+ │ │
131
+ │ Implications: │
132
+ │ • Testing is sampling, not proof │
133
+ │ • Risk-based prioritization is essential │
134
+ │ • Combine testing with formal methods where critical │
135
+ │ │
136
+ └─────────────────────────────────────────────────────────────┘
137
+ ```
138
+
139
+ ### Testability
140
+
141
+ Testability measures how easily software can be tested effectively.
142
+
143
+ | Factor | Description | How to Improve |
144
+ |--------|-------------|----------------|
145
+ | **Controllability** | Ability to set system state for testing | Provide test hooks, dependency injection |
146
+ | **Observability** | Ability to observe test results | Add logging, expose internal state |
147
+ | **Isolability** | Ability to test components independently | Modular design, clear interfaces |
148
+ | **Understandability** | Clarity of expected behavior | Good documentation, clear specifications |
149
+ | **Stability** | Low rate of change | Freeze features before testing phase |
150
+ | **Simplicity** | Low complexity | Reduce cyclomatic complexity |
151
+
152
+ ### Test Adequacy Criteria
153
+
154
+ Test adequacy criteria answer: "Have we tested enough?"
155
+
156
+ ```
157
+ ┌─────────────────────────────────────────────────────────────┐
158
+ │ Test Adequacy Criteria │
159
+ ├─────────────────────────────────────────────────────────────┤
160
+ │ │
161
+ │ Coverage-Based Criteria │
162
+ │ ├── Statement Coverage: % of statements executed │
163
+ │ ├── Branch Coverage: % of branches taken │
164
+ │ ├── Condition Coverage: % of conditions evaluated │
165
+ │ └── MC/DC: Modified condition/decision coverage │
166
+ │ │
167
+ │ Fault-Based Criteria │
168
+ │ ├── Mutation Score: % of mutants killed │
169
+ │ └── Fault Seeding: % of seeded faults found │
170
+ │ │
171
+ │ Requirements-Based Criteria │
172
+ │ └── Requirements Coverage: % of requirements tested │
173
+ │ │
174
+ │ Risk-Based Criteria │
175
+ │ └── Risk Coverage: % of high-risk items tested │
176
+ │ │
177
+ │ Stopping Rules: │
178
+ │ • Achieved target coverage level │
179
+ │ • Defect discovery rate drops below threshold │
180
+ │ • Budget/time exhausted (with documented risk) │
181
+ │ │
182
+ └─────────────────────────────────────────────────────────────┘
183
+ ```
184
+
185
+ ---
186
+
187
+ ## Testing Framework Selection
188
+
189
+ Choose a testing framework based on your project needs. This standard supports two approaches:
190
+
191
+ ### Option A: ISTQB Standard Framework
192
+
193
+ Best for enterprise projects, certification needs, and formal QA processes.
194
+
195
+ **Reference**: [ISTQB Glossary v4.0](https://glossary.istqb.org)
196
+
197
+ | Level | Abbreviation | Purpose |
198
+ |-------|--------------|---------|
199
+ | Unit Testing | UT | Verify individual code units |
200
+ | Integration Testing | IT/SIT | Verify component interactions |
201
+ | System Testing | ST | Verify system meets requirements |
202
+ | Acceptance Testing | AT/UAT | Verify system meets business needs |
203
+
204
+ **When to choose ISTQB**:
205
+ - Enterprise projects with formal QA processes
206
+ - Projects requiring certification or compliance
207
+ - Organizations with dedicated QA teams
208
+ - Projects with strict audit requirements
209
+
210
+ ### Option B: Industry Testing Pyramid
211
+
212
+ Best for agile development, CI/CD optimization, and rapid iteration.
213
+
214
+ **Reference**: [Martin Fowler's Testing Pyramid](https://martinfowler.com/bliki/TestPyramid.html), [Google Testing Blog](https://testing.googleblog.com)
215
+
216
+ | Level | Abbreviation | Ratio | Purpose |
217
+ |-------|--------------|-------|---------|
218
+ | Unit Testing | UT | 70% | Isolated component tests |
219
+ | Integration Testing | IT/SIT* | 20% | Component interaction tests |
220
+ | E2E Testing | E2E | 10% | User workflow tests |
221
+
222
+ *Note on Integration Testing abbreviation:
223
+ - **IT** (Integration Testing): Common in Agile/DevOps communities
224
+ - **SIT** (System Integration Testing): Common in Enterprise/ISTQB contexts
225
+ - Both terms refer to the same testing level
226
+
227
+ **When to choose Industry Pyramid**:
228
+ - Agile/Scrum development teams
229
+ - CI/CD focused environments
230
+ - Small to medium projects with rapid iteration
231
+ - DevOps practices
232
+
233
+ ---
234
+
235
+ ## Testing Pyramid (Default: Industry Standard)
236
+
237
+ ```
238
+ ┌─────────┐
239
+ │ E2E │ ← 10% (Fewer, slower, expensive)
240
+ ─┴─────────┴─
241
+ ┌─────────────┐
242
+ │ IT/SIT │ ← 20% (Integration Testing)
243
+ ─┴─────────────┴─
244
+ ┌─────────────────┐
245
+ │ UT │ ← 70% (Unit Testing - Foundation)
246
+ └─────────────────┘
247
+ ```
248
+
249
+ ### Recommended Ratio (Industry Pyramid)
250
+
251
+ | Level | Abbreviation | Percentage | Execution Time |
252
+ |-------|--------------|------------|----------------|
253
+ | Unit Testing | UT | 70% | < 10 min total |
254
+ | Integration Testing | IT/SIT | 20% | < 30 min total |
255
+ | E2E Testing | E2E | 10% | < 2 hours total |
256
+
257
+ ### ISTQB 4-Level Structure (Alternative)
258
+
259
+ | Level | Abbreviation | Performed By | Focus |
260
+ |-------|--------------|--------------|-------|
261
+ | Unit Testing | UT | Developers | Code correctness |
262
+ | Integration Testing | IT/SIT | Developers/QA | Interface contracts |
263
+ | System Testing | ST | QA Team | Requirements verification |
264
+ | Acceptance Testing | AT/UAT | End Users | Business validation |
265
+
266
+ ---
267
+
268
+ ## Unit Testing (UT)
269
+
270
+ ### Definition
271
+
272
+ Tests individual functions, methods, or classes in isolation from external dependencies.
273
+
274
+ ### Characteristics
275
+
276
+ - **Isolated**: No database, network, or file system access
277
+ - **Fast**: Each test < 100ms
278
+ - **Deterministic**: Same input always produces same output
279
+
280
+ ### Scope
281
+
282
+ ```
283
+ ┌─────────────────────────────────────────┐
284
+ │ Unit Under Test │
285
+ ├─────────────────────────────────────────┤
286
+ │ ✅ Single function/method │
287
+ │ ✅ Single class │
288
+ │ ✅ Pure business logic │
289
+ │ ✅ Data transformations │
290
+ │ ✅ Validation rules │
291
+ ├─────────────────────────────────────────┤
292
+ │ ❌ Database queries │
293
+ │ ❌ External API calls │
294
+ │ ❌ File I/O operations │
295
+ │ ❌ Multi-class interactions │
296
+ └─────────────────────────────────────────┘
297
+ ```
298
+
299
+ ### Naming Convention
300
+
301
+ **File Naming**:
302
+ ```
303
+ [ClassName]Tests.[ext]
304
+ [ClassName].test.[ext]
305
+ [ClassName].spec.[ext]
306
+
307
+ Examples:
308
+ UserService.test.ts
309
+ UserServiceTests.cs
310
+ user_service_test.py
311
+ user_service_test.go
312
+ ```
313
+
314
+ **Method Naming**:
315
+ ```
316
+ [MethodName]_[Scenario]_[ExpectedResult]
317
+ should_[ExpectedBehavior]_when_[Condition]
318
+ test_[method]_[scenario]_[expected]
319
+
320
+ Examples:
321
+ CalculateTotal_WithDiscount_ReturnsDiscountedPrice()
322
+ should_return_null_when_user_not_found()
323
+ test_validate_email_invalid_format_returns_false()
324
+ ```
325
+
326
+ ### Coverage Guidelines
327
+
328
+ | Metric | Minimum | Recommended |
329
+ |--------|---------|-------------|
330
+ | Line Coverage | 70% | 85% |
331
+ | Branch Coverage | 60% | 80% |
332
+ | Function Coverage | 80% | 90% |
333
+
334
+ ### Example
335
+
336
+ ```csharp
337
+ // C# Example
338
+ [TestClass]
339
+ public class UserValidatorTests
340
+ {
341
+ private UserValidator _validator;
342
+
343
+ [TestInitialize]
344
+ public void Setup()
345
+ {
346
+ _validator = new UserValidator();
347
+ }
348
+
349
+ [TestMethod]
350
+ public void ValidateEmail_ValidFormat_ReturnsTrue()
351
+ {
352
+ // Arrange
353
+ var email = "user@example.com";
354
+
355
+ // Act
356
+ var result = _validator.ValidateEmail(email);
357
+
358
+ // Assert
359
+ Assert.IsTrue(result);
360
+ }
361
+
362
+ [TestMethod]
363
+ public void ValidateEmail_InvalidFormat_ReturnsFalse()
364
+ {
365
+ // Arrange
366
+ var email = "invalid-email";
367
+
368
+ // Act
369
+ var result = _validator.ValidateEmail(email);
370
+
371
+ // Assert
372
+ Assert.IsFalse(result);
373
+ }
374
+ }
375
+ ```
376
+
377
+ ```typescript
378
+ // TypeScript Example
379
+ describe('UserValidator', () => {
380
+ let validator: UserValidator;
381
+
382
+ beforeEach(() => {
383
+ validator = new UserValidator();
384
+ });
385
+
386
+ describe('validateEmail', () => {
387
+ it('should return true for valid email format', () => {
388
+ const result = validator.validateEmail('user@example.com');
389
+ expect(result).toBe(true);
390
+ });
391
+
392
+ it('should return false for invalid email format', () => {
393
+ const result = validator.validateEmail('invalid-email');
394
+ expect(result).toBe(false);
395
+ });
396
+ });
397
+ });
398
+ ```
399
+
400
+ ---
401
+
402
+ ## Integration Testing (IT)
403
+
404
+ ### Definition
405
+
406
+ Tests interactions between multiple components, modules, or external systems.
407
+
408
+ ### When Integration Tests Are Required
409
+
410
+ | Scenario | Reason |
411
+ |----------|--------|
412
+ | Query predicates | Mocks cannot verify filter expressions |
413
+ | Entity relationships | Verify foreign key correctness |
414
+ | Composite keys | In-memory DB may differ from real DB |
415
+ | Field mapping | DTO ↔ Entity transformations |
416
+ | Pagination | Row ordering and counting |
417
+ | Transactions | Rollback behavior |
418
+
419
+ **Decision Rule**:
420
+ If your unit test uses a wildcard matcher (`any()`, `It.IsAny<>`, `Arg.Any<>`)
421
+ for a query/filter parameter, that functionality MUST have an integration test.
422
+
423
+ ### Characteristics
424
+
425
+ - **Component Integration**: Tests module boundaries
426
+ - **Real Dependencies**: Uses actual databases, APIs (often containerized)
427
+ - **Slower**: Each test typically 1-10 seconds
428
+
429
+ ### Scope
430
+
431
+ ```
432
+ ┌─────────────────────────────────────────┐
433
+ │ Integration Test Scope │
434
+ ├─────────────────────────────────────────┤
435
+ │ ✅ Database CRUD operations │
436
+ │ ✅ Repository + Database │
437
+ │ ✅ Service + Repository │
438
+ │ ✅ API endpoint + Service layer │
439
+ │ ✅ Message queue producers/consumers │
440
+ │ ✅ Cache read/write operations │
441
+ ├─────────────────────────────────────────┤
442
+ │ ❌ Full user workflows │
443
+ │ ❌ Cross-service communication │
444
+ │ ❌ UI interactions │
445
+ └─────────────────────────────────────────┘
446
+ ```
447
+
448
+ ### Naming Convention
449
+
450
+ **File Naming**:
451
+ ```
452
+ [ComponentName]IntegrationTests.[ext]
453
+ [ComponentName].integration.test.[ext]
454
+ [ComponentName].itest.[ext]
455
+
456
+ Examples:
457
+ UserRepositoryIntegrationTests.cs
458
+ user-service.integration.test.ts
459
+ user_repository_itest.py
460
+ ```
461
+
462
+ **Method Naming**:
463
+ ```
464
+ [Operation]_[Context]_[ExpectedOutcome]
465
+
466
+ Examples:
467
+ CreateUser_WithValidData_PersistsToDatabase()
468
+ GetUserById_ExistingUser_ReturnsUserFromDatabase()
469
+ SendMessage_ToQueue_ConsumerReceivesMessage()
470
+ ```
471
+
472
+ ### Test Fixtures
473
+
474
+ ```csharp
475
+ // C# Integration Test Example with Test Database
476
+ [TestClass]
477
+ public class UserRepositoryIntegrationTests
478
+ {
479
+ private TestDbContext _dbContext;
480
+ private UserRepository _repository;
481
+
482
+ [TestInitialize]
483
+ public async Task Setup()
484
+ {
485
+ // Use test database (e.g., SQLite in-memory or Testcontainers)
486
+ _dbContext = TestDbContextFactory.Create();
487
+ _repository = new UserRepository(_dbContext);
488
+ await _dbContext.Database.EnsureCreatedAsync();
489
+ }
490
+
491
+ [TestCleanup]
492
+ public async Task Cleanup()
493
+ {
494
+ await _dbContext.DisposeAsync();
495
+ }
496
+
497
+ [TestMethod]
498
+ public async Task CreateUser_WithValidData_PersistsToDatabase()
499
+ {
500
+ // Arrange
501
+ var user = new User { Name = "Test User", Email = "test@example.com" };
502
+
503
+ // Act
504
+ await _repository.CreateAsync(user);
505
+ var savedUser = await _repository.GetByIdAsync(user.Id);
506
+
507
+ // Assert
508
+ Assert.IsNotNull(savedUser);
509
+ Assert.AreEqual("Test User", savedUser.Name);
510
+ }
511
+ }
512
+ ```
513
+
514
+ ---
515
+
516
+ ## System Testing (ST)
517
+
518
+ ### Definition
519
+
520
+ Tests the complete integrated system to verify it meets specified requirements.
521
+
522
+ ### Characteristics
523
+
524
+ - **Complete System**: All components deployed and integrated
525
+ - **Requirement-Based**: Tests against functional specifications
526
+ - **Production-Like**: Uses environment similar to production
527
+
528
+ ### Scope
529
+
530
+ ```
531
+ ┌─────────────────────────────────────────┐
532
+ │ System Test Scope │
533
+ ├─────────────────────────────────────────┤
534
+ │ ✅ Complete API workflows │
535
+ │ ✅ Cross-service transactions │
536
+ │ ✅ Data flow through entire system │
537
+ │ ✅ Security requirements │
538
+ │ ✅ Performance under load │
539
+ │ ✅ Error handling & recovery │
540
+ │ ✅ Configuration validation │
541
+ ├─────────────────────────────────────────┤
542
+ │ ❌ UI visual testing │
543
+ │ ❌ User journey simulations │
544
+ │ ❌ A/B testing scenarios │
545
+ └─────────────────────────────────────────┘
546
+ ```
547
+
548
+ ### Types of System Tests
549
+
550
+ | Type | Description |
551
+ |------|-------------|
552
+ | Functional | Verify features work as specified |
553
+ | Performance | Load, stress, scalability testing |
554
+ | Security | Penetration, vulnerability scanning |
555
+ | Reliability | Failover, recovery, stability |
556
+ | Compatibility | Cross-platform, browser compatibility |
557
+
558
+ ### Naming Convention
559
+
560
+ **File Naming**:
561
+ ```
562
+ [Feature]SystemTests.[ext]
563
+ [Feature].system.test.[ext]
564
+ [Feature]_st.[ext]
565
+
566
+ Examples:
567
+ OrderProcessingSystemTests.cs
568
+ authentication.system.test.ts
569
+ payment_processing_st.py
570
+ ```
571
+
572
+ ### Example
573
+
574
+ ```csharp
575
+ // System Test Example: Complete Resource Processing Flow
576
+ // Note: Replace {Resource}, {Item}, {Action} with your domain concepts
577
+ [TestClass]
578
+ public class ResourceProcessingSystemTests
579
+ {
580
+ private HttpClient _client;
581
+ private TestEnvironment _env;
582
+
583
+ [TestInitialize]
584
+ public async Task Setup()
585
+ {
586
+ _env = await TestEnvironment.CreateAsync();
587
+ _client = _env.CreateAuthenticatedClient();
588
+ }
589
+
590
+ [TestMethod]
591
+ public async Task ProcessResource_CompleteFlow_CompletedSuccessfully()
592
+ {
593
+ // Arrange: Create test data
594
+ var item = await _env.CreateTestItem(value: 100);
595
+ var user = await _env.CreateTestUser();
596
+
597
+ // Act: Execute complete processing flow
598
+ // Step 1: Create request
599
+ var requestResponse = await _client.PostAsync("/api/requests",
600
+ new { itemId = item.Id, quantity = 2 });
601
+ Assert.AreEqual(HttpStatusCode.OK, requestResponse.StatusCode);
602
+
603
+ // Step 2: Submit processing
604
+ var processResponse = await _client.PostAsync("/api/processes",
605
+ new { requestId = requestResponse.RequestId, userId = user.Id });
606
+ var process = await processResponse.Content.ReadAsAsync<Process>();
607
+ Assert.AreEqual(HttpStatusCode.Created, processResponse.StatusCode);
608
+
609
+ // Step 3: Confirm completion
610
+ var confirmResponse = await _client.PostAsync($"/api/processes/{process.Id}/confirm",
611
+ new { confirmationType = "standard", amount = 200 });
612
+ Assert.AreEqual(HttpStatusCode.OK, confirmResponse.StatusCode);
613
+
614
+ // Assert: Verify final state
615
+ var finalProcess = await _client.GetAsync($"/api/processes/{process.Id}");
616
+ var result = await finalProcess.Content.ReadAsAsync<Process>();
617
+
618
+ Assert.AreEqual(ProcessStatus.Completed, result.Status);
619
+ Assert.AreEqual(200, result.TotalAmount);
620
+ Assert.IsNotNull(result.Confirmation);
621
+ }
622
+ }
623
+ ```
624
+
625
+ ---
626
+
627
+ ## End-to-End Testing (E2E)
628
+
629
+ ### Definition
630
+
631
+ Tests complete user workflows from the user interface through all system layers.
632
+
633
+ ### Characteristics
634
+
635
+ - **User Perspective**: Simulates real user interactions
636
+ - **Full Stack**: UI → API → Database → External Services
637
+ - **Slowest**: Each test typically 30 seconds to several minutes
638
+
639
+ ### Scope
640
+
641
+ ```
642
+ ┌─────────────────────────────────────────┐
643
+ │ E2E Test Scope │
644
+ ├─────────────────────────────────────────┤
645
+ │ ✅ Critical user journeys │
646
+ │ ✅ Login/Authentication flows │
647
+ │ ✅ Core business transactions │
648
+ │ ✅ Cross-browser functionality │
649
+ │ ✅ Smoke tests for deployments │
650
+ ├─────────────────────────────────────────┤
651
+ │ ❌ Every possible user path │
652
+ │ ❌ Edge cases (use UT/IT) │
653
+ │ ❌ Performance benchmarking │
654
+ └─────────────────────────────────────────┘
655
+ ```
656
+
657
+ ### Naming Convention
658
+
659
+ **File Naming**:
660
+ ```
661
+ [UserJourney].e2e.[ext]
662
+ [Feature].e2e.spec.[ext]
663
+ e2e/[feature]/[scenario].[ext]
664
+
665
+ Examples:
666
+ user-registration.e2e.ts
667
+ checkout-flow.e2e.spec.ts
668
+ e2e/authentication/login.spec.ts
669
+ ```
670
+
671
+ ### Example
672
+
673
+ ```typescript
674
+ // Playwright E2E Test Example
675
+ import { test, expect } from '@playwright/test';
676
+
677
+ test.describe('User Registration Journey', () => {
678
+ test('should complete registration and login successfully', async ({ page }) => {
679
+ // Step 1: Navigate to registration page
680
+ await page.goto('/register');
681
+
682
+ // Step 2: Fill registration form
683
+ await page.fill('[data-testid="email"]', 'newuser@example.com');
684
+ await page.fill('[data-testid="password"]', 'SecurePass123!');
685
+ await page.fill('[data-testid="confirm-password"]', 'SecurePass123!');
686
+ await page.click('[data-testid="register-button"]');
687
+
688
+ // Step 3: Verify registration success
689
+ await expect(page.locator('[data-testid="success-message"]'))
690
+ .toContainText('Registration successful');
691
+
692
+ // Step 4: Login with new account
693
+ await page.goto('/login');
694
+ await page.fill('[data-testid="email"]', 'newuser@example.com');
695
+ await page.fill('[data-testid="password"]', 'SecurePass123!');
696
+ await page.click('[data-testid="login-button"]');
697
+
698
+ // Step 5: Verify login success and dashboard redirect
699
+ await expect(page).toHaveURL('/dashboard');
700
+ await expect(page.locator('[data-testid="welcome-message"]'))
701
+ .toContainText('Welcome, newuser@example.com');
702
+ });
703
+ });
704
+ ```
705
+
706
+ ---
707
+
708
+ ## Static Testing
709
+
710
+ Static testing examines work products (code, documents, requirements) without executing the software. It complements dynamic testing by finding defects earlier in the SDLC.
711
+
712
+ **Reference**: ISTQB CTFL v4.0 Chapter 3, ISO/IEC/IEEE 29119-4
713
+
714
+ ### Types of Static Testing
715
+
716
+ ```
717
+ ┌─────────────────────────────────────────────────────────────┐
718
+ │ Static Testing Types │
719
+ ├─────────────────────────────────────────────────────────────┤
720
+ │ Reviews (Manual) │
721
+ │ ├── Informal Review: Ad-hoc, no formal process │
722
+ │ ├── Walkthrough: Author-led, educational purpose │
723
+ │ ├── Technical Review: Peer-led, find defects │
724
+ │ └── Inspection: Formal, metrics-driven, most rigorous │
725
+ ├─────────────────────────────────────────────────────────────┤
726
+ │ Static Analysis (Tool-based) │
727
+ │ ├── Code Analysis: Linters, complexity analyzers │
728
+ │ ├── Security Analysis: SAST tools (SonarQube, Checkmarx) │
729
+ │ ├── Architecture Analysis: Dependency checks │
730
+ │ └── Data Flow Analysis: Variable usage patterns │
731
+ └─────────────────────────────────────────────────────────────┘
732
+ ```
733
+
734
+ ### Review Process (ISTQB)
735
+
736
+ | Phase | Activities |
737
+ |-------|------------|
738
+ | Planning | Define scope, entry criteria, roles |
739
+ | Initiate Review | Distribute work products |
740
+ | Individual Review | Each reviewer examines independently |
741
+ | Issue Communication | Discuss findings in meeting |
742
+ | Fixing & Reporting | Author fixes, metrics collected |
743
+
744
+ ### Static Analysis Tools by Language
745
+
746
+ | Language | Linting | Security | Complexity |
747
+ |----------|---------|----------|------------|
748
+ | JavaScript/TypeScript | ESLint, Biome | npm audit, Snyk | ESLint complexity rules |
749
+ | Python | Pylint, Ruff, Flake8 | Bandit, Safety | Radon |
750
+ | Java | Checkstyle, PMD | SpotBugs, OWASP DC | JaCoCo |
751
+ | C# | StyleCop, Roslyn | Security Code Scan | NDepend |
752
+ | Go | golangci-lint | gosec | gocyclo |
753
+
754
+ ### When to Use Static Testing
755
+
756
+ | Work Product | Review Type | Tools |
757
+ |--------------|-------------|-------|
758
+ | Requirements | Inspection, Walkthrough | - |
759
+ | Design Documents | Technical Review | Architecture tools |
760
+ | Code | Technical Review, Tool Analysis | Linters, SAST |
761
+ | Test Plans | Walkthrough | - |
762
+ | User Documentation | Walkthrough | Spell/grammar checkers |
763
+
764
+ ### Code Review Checklist
765
+
766
+ ```
767
+ ┌─────────────────────────────────────────────────────────────┐
768
+ │ Code Review Focus Areas │
769
+ ├─────────────────────────────────────────────────────────────┤
770
+ │ Functionality │
771
+ │ ├── Does the code do what it's supposed to do? │
772
+ │ ├── Are edge cases handled? │
773
+ │ └── Is error handling appropriate? │
774
+ ├─────────────────────────────────────────────────────────────┤
775
+ │ Maintainability │
776
+ │ ├── Is the code readable and well-organized? │
777
+ │ ├── Are names meaningful? │
778
+ │ └── Is complexity manageable? │
779
+ ├─────────────────────────────────────────────────────────────┤
780
+ │ Security │
781
+ │ ├── Input validation present? │
782
+ │ ├── No hardcoded secrets? │
783
+ │ └── SQL injection / XSS prevention? │
784
+ ├─────────────────────────────────────────────────────────────┤
785
+ │ Performance │
786
+ │ ├── No obvious inefficiencies? │
787
+ │ ├── Resource cleanup (connections, files)? │
788
+ │ └── Appropriate data structures? │
789
+ └─────────────────────────────────────────────────────────────┘
790
+ ```
791
+
792
+ ### Integration with CI/CD
793
+
794
+ ```yaml
795
+ # Example: Static analysis in CI pipeline
796
+ static-analysis:
797
+ stage: test
798
+ script:
799
+ - npm run lint # Linting
800
+ - npm run lint:security # Security analysis
801
+ - npx sonarqube-scanner # Quality gates
802
+ rules:
803
+ - if: $CI_PIPELINE_SOURCE == "push"
804
+ ```
805
+
806
+ ---
807
+
808
+ ## Test Design Techniques
809
+
810
+ Test design techniques are methods for deriving and selecting test cases. ISTQB and ISO/IEC/IEEE 29119 categorize them into three main approaches.
811
+
812
+ **Reference**: ISTQB CTFL v4.0 Chapter 4, ISO/IEC/IEEE 29119-4
813
+
814
+ ### Technique Categories
815
+
816
+ ```
817
+ ┌─────────────────────────────────────────────────────────────┐
818
+ │ Test Design Technique Categories │
819
+ ├─────────────────────────────────────────────────────────────┤
820
+ │ Specification-Based (Black-Box) │
821
+ │ └── Derive tests from requirements/specifications │
822
+ ├─────────────────────────────────────────────────────────────┤
823
+ │ Structure-Based (White-Box) │
824
+ │ └── Derive tests from internal code structure │
825
+ ├─────────────────────────────────────────────────────────────┤
826
+ │ Experience-Based │
827
+ │ └── Derive tests from tester knowledge and intuition │
828
+ └─────────────────────────────────────────────────────────────┘
829
+ ```
830
+
831
+ ### Specification-Based Techniques (Black-Box)
832
+
833
+ #### 1. Equivalence Partitioning (EP)
834
+
835
+ Divides input data into partitions where all values should be treated the same by the system.
836
+
837
+ ```
838
+ Input: Age (valid range: 18-65)
839
+
840
+ ┌─────────────┬─────────────┬─────────────┐
841
+ │ Invalid │ Valid │ Invalid │
842
+ │ < 18 │ 18 - 65 │ > 65 │
843
+ ├─────────────┼─────────────┼─────────────┤
844
+ │ Partition 1│ Partition 2│ Partition 3│
845
+ │ Test: 10 │ Test: 30 │ Test: 70 │
846
+ └─────────────┴─────────────┴─────────────┘
847
+
848
+ Coverage: At least 1 test case per partition
849
+ ```
850
+
851
+ ```python
852
+ # Python Example: Equivalence Partitioning
853
+ def test_age_validation():
854
+ validator = AgeValidator(min_age=18, max_age=65)
855
+
856
+ # Partition 1: Below minimum (invalid)
857
+ assert validator.is_valid(10) == False
858
+
859
+ # Partition 2: Valid range
860
+ assert validator.is_valid(30) == True
861
+
862
+ # Partition 3: Above maximum (invalid)
863
+ assert validator.is_valid(70) == False
864
+ ```
865
+
866
+ #### 2. Boundary Value Analysis (BVA)
867
+
868
+ Tests at the boundaries of equivalence partitions where defects are most likely.
869
+
870
+ ```
871
+ Input: Quantity (valid range: 1-100)
872
+
873
+ 0 1 2 ... 99 100 101
874
+ │ │ │ │ │ │
875
+ ▼ ▼ ▼ ▼ ▼ ▼
876
+ Invalid Valid Valid Valid Valid Invalid
877
+ (boundary)(boundary) (boundary)(boundary)
878
+
879
+ Test Values: 0, 1, 2, 99, 100, 101
880
+ ```
881
+
882
+ ```typescript
883
+ // TypeScript Example: Boundary Value Analysis
884
+ describe('Quantity Validation', () => {
885
+ const validator = new QuantityValidator(1, 100);
886
+
887
+ // Lower boundary
888
+ it('rejects 0 (below minimum)', () => {
889
+ expect(validator.isValid(0)).toBe(false);
890
+ });
891
+
892
+ it('accepts 1 (minimum boundary)', () => {
893
+ expect(validator.isValid(1)).toBe(true);
894
+ });
895
+
896
+ it('accepts 2 (above minimum)', () => {
897
+ expect(validator.isValid(2)).toBe(true);
898
+ });
899
+
900
+ // Upper boundary
901
+ it('accepts 99 (below maximum)', () => {
902
+ expect(validator.isValid(99)).toBe(true);
903
+ });
904
+
905
+ it('accepts 100 (maximum boundary)', () => {
906
+ expect(validator.isValid(100)).toBe(true);
907
+ });
908
+
909
+ it('rejects 101 (above maximum)', () => {
910
+ expect(validator.isValid(101)).toBe(false);
911
+ });
912
+ });
913
+ ```
914
+
915
+ #### 3. Decision Table Testing
916
+
917
+ Captures complex business rules with multiple conditions.
918
+
919
+ ```
920
+ Discount Rules:
921
+ - Premium member: 20% discount
922
+ - Order > $100: 10% discount
923
+ - Both conditions: 25% discount (not cumulative)
924
+
925
+ ┌─────────────────┬──────┬──────┬──────┬──────┐
926
+ │ Conditions │ TC1 │ TC2 │ TC3 │ TC4 │
927
+ ├─────────────────┼──────┼──────┼──────┼──────┤
928
+ │ Premium Member │ N │ Y │ N │ Y │
929
+ │ Order > $100 │ N │ N │ Y │ Y │
930
+ ├─────────────────┼──────┼──────┼──────┼──────┤
931
+ │ Actions │ │ │ │ │
932
+ ├─────────────────┼──────┼──────┼──────┼──────┤
933
+ │ Discount % │ 0% │ 20% │ 10% │ 25% │
934
+ └─────────────────┴──────┴──────┴──────┴──────┘
935
+ ```
936
+
937
+ ```csharp
938
+ // C# Example: Decision Table Testing
939
+ [TestClass]
940
+ public class DiscountCalculatorTests
941
+ {
942
+ private DiscountCalculator _calculator;
943
+
944
+ [TestInitialize]
945
+ public void Setup() => _calculator = new DiscountCalculator();
946
+
947
+ [TestMethod]
948
+ public void Calculate_NotPremium_SmallOrder_NoDiscount()
949
+ {
950
+ var result = _calculator.Calculate(isPremium: false, orderAmount: 50);
951
+ Assert.AreEqual(0, result.DiscountPercent);
952
+ }
953
+
954
+ [TestMethod]
955
+ public void Calculate_Premium_SmallOrder_20PercentDiscount()
956
+ {
957
+ var result = _calculator.Calculate(isPremium: true, orderAmount: 50);
958
+ Assert.AreEqual(20, result.DiscountPercent);
959
+ }
960
+
961
+ [TestMethod]
962
+ public void Calculate_NotPremium_LargeOrder_10PercentDiscount()
963
+ {
964
+ var result = _calculator.Calculate(isPremium: false, orderAmount: 150);
965
+ Assert.AreEqual(10, result.DiscountPercent);
966
+ }
967
+
968
+ [TestMethod]
969
+ public void Calculate_Premium_LargeOrder_25PercentDiscount()
970
+ {
971
+ var result = _calculator.Calculate(isPremium: true, orderAmount: 150);
972
+ Assert.AreEqual(25, result.DiscountPercent);
973
+ }
974
+ }
975
+ ```
976
+
977
+ #### 4. State Transition Testing
978
+
979
+ Tests behavior changes based on system states and transitions.
980
+
981
+ ```
982
+ Order State Machine:
983
+
984
+ ┌─────────┐ place() ┌───────────┐ pay() ┌────────┐
985
+ │ Draft │──────────────▶│ Pending │───────────▶│ Paid │
986
+ └─────────┘ └───────────┘ └────────┘
987
+ │ │ │
988
+ │ cancel() │ cancel() │ ship()
989
+ ▼ ▼ ▼
990
+ ┌───────────┐ ┌───────────┐ ┌──────────┐
991
+ │ Cancelled │ │ Cancelled │ │ Shipped │
992
+ └───────────┘ └───────────┘ └──────────┘
993
+
994
+ State Transition Table:
995
+ ┌───────────┬─────────┬───────────┬───────────┬───────────┐
996
+ │ Current │ place() │ pay() │ ship() │ cancel() │
997
+ ├───────────┼─────────┼───────────┼───────────┼───────────┤
998
+ │ Draft │ Pending │ Invalid │ Invalid │ Cancelled │
999
+ │ Pending │ Invalid │ Paid │ Invalid │ Cancelled │
1000
+ │ Paid │ Invalid │ Invalid │ Shipped │ Invalid │
1001
+ │ Shipped │ Invalid │ Invalid │ Invalid │ Invalid │
1002
+ │ Cancelled │ Invalid │ Invalid │ Invalid │ Invalid │
1003
+ └───────────┴─────────┴───────────┴───────────┴───────────┘
1004
+ ```
1005
+
1006
+ ```python
1007
+ # Python Example: State Transition Testing
1008
+ class TestOrderStateMachine:
1009
+ def test_draft_to_pending_on_place(self):
1010
+ order = Order(state="draft")
1011
+ order.place()
1012
+ assert order.state == "pending"
1013
+
1014
+ def test_pending_to_paid_on_pay(self):
1015
+ order = Order(state="pending")
1016
+ order.pay()
1017
+ assert order.state == "paid"
1018
+
1019
+ def test_paid_to_shipped_on_ship(self):
1020
+ order = Order(state="paid")
1021
+ order.ship()
1022
+ assert order.state == "shipped"
1023
+
1024
+ def test_invalid_transition_pay_from_draft(self):
1025
+ order = Order(state="draft")
1026
+ with pytest.raises(InvalidTransitionError):
1027
+ order.pay()
1028
+
1029
+ def test_invalid_transition_cancel_from_shipped(self):
1030
+ order = Order(state="shipped")
1031
+ with pytest.raises(InvalidTransitionError):
1032
+ order.cancel()
1033
+ ```
1034
+
1035
+ #### 5. Use Case Testing
1036
+
1037
+ Tests complete user scenarios from start to finish.
1038
+
1039
+ ```
1040
+ Use Case: User Login
1041
+
1042
+ Primary Flow:
1043
+ 1. User enters username
1044
+ 2. User enters password
1045
+ 3. User clicks login
1046
+ 4. System validates credentials
1047
+ 5. System redirects to dashboard
1048
+
1049
+ Alternative Flows:
1050
+ A1. Invalid credentials → Show error, remain on login
1051
+ A2. Account locked → Show locked message
1052
+ A3. Password expired → Redirect to password change
1053
+
1054
+ Test Cases:
1055
+ - TC1: Valid credentials → Dashboard (Primary)
1056
+ - TC2: Invalid password → Error message (A1)
1057
+ - TC3: Invalid username → Error message (A1)
1058
+ - TC4: Locked account → Locked message (A2)
1059
+ - TC5: Expired password → Password change page (A3)
1060
+ ```
1061
+
1062
+ ### Structure-Based Techniques (White-Box)
1063
+
1064
+ #### 1. Statement Coverage
1065
+
1066
+ Ensure every statement is executed at least once.
1067
+
1068
+ ```
1069
+ Code:
1070
+ 1 function categorize(score) {
1071
+ 2 let result;
1072
+ 3 if (score >= 90) {
1073
+ 4 result = 'A';
1074
+ 5 } else if (score >= 70) {
1075
+ 6 result = 'B';
1076
+ 7 } else {
1077
+ 8 result = 'C';
1078
+ 9 }
1079
+ 10 return result;
1080
+ 11 }
1081
+
1082
+ 100% Statement Coverage requires tests:
1083
+ - Test 1: score = 95 → executes lines 1-4, 10
1084
+ - Test 2: score = 80 → executes lines 1-3, 5-6, 10
1085
+ - Test 3: score = 50 → executes lines 1-3, 5, 7-10
1086
+ ```
1087
+
1088
+ #### 2. Branch Coverage
1089
+
1090
+ Ensure every branch (decision outcome) is executed.
1091
+
1092
+ ```
1093
+ Code:
1094
+ if (a > 0 && b > 0) {
1095
+ doSomething();
1096
+ }
1097
+
1098
+ Branches:
1099
+ - True branch: a > 0 AND b > 0 is true
1100
+ - False branch: a > 0 AND b > 0 is false
1101
+
1102
+ 100% Branch Coverage:
1103
+ - Test 1: a = 1, b = 1 → True branch
1104
+ - Test 2: a = 0, b = 1 → False branch (or a = 1, b = 0)
1105
+ ```
1106
+
1107
+ #### 3. Condition Coverage
1108
+
1109
+ Ensure each condition in a decision is evaluated to both true and false.
1110
+
1111
+ ```
1112
+ Code:
1113
+ if (a > 0 && b > 0) { ... }
1114
+
1115
+ Conditions: (a > 0), (b > 0)
1116
+
1117
+ 100% Condition Coverage:
1118
+ - Test 1: a = 1, b = 1 → (a > 0) = true, (b > 0) = true
1119
+ - Test 2: a = 0, b = 0 → (a > 0) = false, (b > 0) = false
1120
+
1121
+ Note: Does not guarantee branch coverage!
1122
+ ```
1123
+
1124
+ #### 4. MC/DC (Modified Condition/Decision Coverage)
1125
+
1126
+ Each condition independently affects the decision outcome. Required for safety-critical systems.
1127
+
1128
+ ```
1129
+ Code:
1130
+ if (a && b) { ... }
1131
+
1132
+ MC/DC Requirements:
1133
+ 1. Each condition evaluated to true and false
1134
+ 2. Each condition independently affects outcome
1135
+
1136
+ Test Cases:
1137
+ - Test 1: a = true, b = true → Decision = true
1138
+ - Test 2: a = false, b = true → Decision = false (a changed outcome)
1139
+ - Test 3: a = true, b = false → Decision = false (b changed outcome)
1140
+ ```
1141
+
1142
+ ### Additional Specification-Based Techniques
1143
+
1144
+ #### 6. Pairwise Testing (Combinatorial Testing)
1145
+
1146
+ Tests all possible pairs of input parameter values, reducing test cases while maintaining good coverage.
1147
+
1148
+ **Reference**: SWEBOK v4.0 - Input Domain-Based Techniques
1149
+
1150
+ ```
1151
+ Example: Login Form with 3 parameters, each with 3 values
1152
+
1153
+ Parameters:
1154
+ - Browser: Chrome, Firefox, Safari
1155
+ - OS: Windows, macOS, Linux
1156
+ - Language: English, Spanish, French
1157
+
1158
+ Full Combinatorial: 3 × 3 × 3 = 27 test cases
1159
+ Pairwise Coverage: 9 test cases (covers all pairs)
1160
+
1161
+ ┌──────┬──────────┬─────────┬──────────┐
1162
+ │ TC │ Browser │ OS │ Language │
1163
+ ├──────┼──────────┼─────────┼──────────┤
1164
+ │ 1 │ Chrome │ Windows │ English │
1165
+ │ 2 │ Chrome │ macOS │ Spanish │
1166
+ │ 3 │ Chrome │ Linux │ French │
1167
+ │ 4 │ Firefox │ Windows │ Spanish │
1168
+ │ 5 │ Firefox │ macOS │ French │
1169
+ │ 6 │ Firefox │ Linux │ English │
1170
+ │ 7 │ Safari │ Windows │ French │
1171
+ │ 8 │ Safari │ macOS │ English │
1172
+ │ 9 │ Safari │ Linux │ Spanish │
1173
+ └──────┴──────────┴─────────┴──────────┘
1174
+ ```
1175
+
1176
+ **Tools**: PICT (Microsoft), AllPairs, Jenny
1177
+
1178
+ ```bash
1179
+ # Using PICT (Pairwise Independent Combinatorial Testing)
1180
+ # Input file: params.txt
1181
+ # Browser: Chrome, Firefox, Safari
1182
+ # OS: Windows, macOS, Linux
1183
+ # Language: English, Spanish, French
1184
+
1185
+ pict params.txt > test_cases.txt
1186
+ ```
1187
+
1188
+ #### 7. Data Flow Testing
1189
+
1190
+ Tests the flow of data through a program by tracking variable definitions and uses.
1191
+
1192
+ **Reference**: SWEBOK v4.0 - Code-Based Techniques
1193
+
1194
+ ```
1195
+ ┌─────────────────────────────────────────────────────────────┐
1196
+ │ Data Flow Terminology │
1197
+ ├─────────────────────────────────────────────────────────────┤
1198
+ │ │
1199
+ │ Definition (def): Variable is assigned a value │
1200
+ │ Use (use): Variable's value is accessed │
1201
+ │ - c-use: Computational use (in calculation) │
1202
+ │ - p-use: Predicate use (in condition) │
1203
+ │ │
1204
+ │ def-use pair: Path from definition to use │
1205
+ │ def-clear path: No redefinition between def and use │
1206
+ │ │
1207
+ │ Example: │
1208
+ │ 1: x = 5 ← def(x) │
1209
+ │ 2: y = x + 1 ← c-use(x), def(y) │
1210
+ │ 3: if (x > 0) ← p-use(x) │
1211
+ │ 4: z = y * 2 ← c-use(y), def(z) │
1212
+ │ │
1213
+ └─────────────────────────────────────────────────────────────┘
1214
+ ```
1215
+
1216
+ **Coverage Criteria**:
1217
+
1218
+ | Criterion | Requirement | Strength |
1219
+ |-----------|-------------|----------|
1220
+ | All-Defs | Every def reaches at least one use | Weak |
1221
+ | All-Uses | Every def-use pair is exercised | Medium |
1222
+ | All-DU-Paths | All def-clear paths from def to use | Strong |
1223
+
1224
+ ```python
1225
+ # Python Example: Data Flow Testing
1226
+ def calculate_discount(price, quantity, is_member):
1227
+ # def: total
1228
+ total = price * quantity
1229
+
1230
+ # def: discount
1231
+ if is_member: # p-use: is_member
1232
+ discount = 0.1 # def: discount (path 1)
1233
+ else:
1234
+ discount = 0.0 # def: discount (path 2)
1235
+
1236
+ # c-use: total, discount
1237
+ if total > 100: # p-use: total
1238
+ discount += 0.05 # c-use: discount, def: discount
1239
+
1240
+ # c-use: total, discount
1241
+ return total * (1 - discount)
1242
+
1243
+ # Test cases for All-Uses coverage:
1244
+ # TC1: is_member=True, total>100 → exercises def@line4, use@line10
1245
+ # TC2: is_member=False, total>100 → exercises def@line6, use@line10
1246
+ # TC3: is_member=True, total<=100 → exercises def@line4, use@line12
1247
+ # TC4: is_member=False, total<=100 → exercises def@line6, use@line12
1248
+ ```
1249
+
1250
+ ### Technique Selection Guide
1251
+
1252
+ | Technique | Best For | Test Level |
1253
+ |-----------|----------|------------|
1254
+ | Equivalence Partitioning | Input validation, ranges | UT, IT |
1255
+ | Boundary Value Analysis | Numeric limits, dates | UT, IT |
1256
+ | Decision Table | Complex business rules | UT, IT, ST |
1257
+ | State Transition | Workflows, status changes | IT, ST |
1258
+ | Use Case Testing | User scenarios | ST, E2E |
1259
+ | Pairwise Testing | Multi-parameter combinations | IT, ST |
1260
+ | Data Flow Testing | Variable lifecycle verification | UT |
1261
+ | Statement Coverage | Basic code coverage | UT |
1262
+ | Branch Coverage | Decision logic | UT |
1263
+ | MC/DC | Safety-critical systems | UT |
1264
+
1265
+ ---
1266
+
1267
+ ## Experience-Based Testing
1268
+
1269
+ Experience-based testing leverages the tester's knowledge, skills, and intuition to design and execute tests. It complements systematic techniques.
1270
+
1271
+ **Reference**: ISTQB CTFL v4.0 Section 4.4
1272
+
1273
+ ### Types of Experience-Based Testing
1274
+
1275
+ #### 1. Exploratory Testing
1276
+
1277
+ Simultaneous test design, execution, and learning. The tester explores the system without predefined scripts.
1278
+
1279
+ ```
1280
+ ┌─────────────────────────────────────────────────────────────┐
1281
+ │ Exploratory Testing Session │
1282
+ ├─────────────────────────────────────────────────────────────┤
1283
+ │ Charter: "Explore the checkout process focusing on │
1284
+ │ payment failure scenarios" │
1285
+ ├─────────────────────────────────────────────────────────────┤
1286
+ │ Time-box: 60 minutes │
1287
+ ├─────────────────────────────────────────────────────────────┤
1288
+ │ Notes: │
1289
+ │ - Tested expired credit card → Got generic error │
1290
+ │ - Tested insufficient funds → Same generic error │
1291
+ │ - Found: No specific error messages for different failures │
1292
+ │ - Bug: Retry with same card doesn't clear previous error │
1293
+ ├─────────────────────────────────────────────────────────────┤
1294
+ │ Bugs Found: 2 │
1295
+ │ Areas for Further Testing: Error message specificity │
1296
+ └─────────────────────────────────────────────────────────────┘
1297
+ ```
1298
+
1299
+ **Session-Based Test Management (SBTM)**:
1300
+
1301
+ | Element | Description |
1302
+ |---------|-------------|
1303
+ | Charter | Mission statement for the session |
1304
+ | Time-box | Fixed duration (typically 60-90 min) |
1305
+ | Session Notes | Observations, questions, bugs found |
1306
+ | Debrief | Review findings with team |
1307
+
1308
+ #### 2. Error Guessing
1309
+
1310
+ Anticipate defects based on experience with similar systems or common mistakes.
1311
+
1312
+ ```
1313
+ Common Error Categories to Guess:
1314
+
1315
+ ┌─────────────────────────────────────────────────────────────┐
1316
+ │ Input Errors │
1317
+ │ ├── Empty/null inputs │
1318
+ │ ├── Very long strings │
1319
+ │ ├── Special characters (', ", <, >, &, etc.) │
1320
+ │ ├── Unicode/emoji characters │
1321
+ │ └── Negative numbers where positive expected │
1322
+ ├─────────────────────────────────────────────────────────────┤
1323
+ │ Boundary Errors │
1324
+ │ ├── Off-by-one errors │
1325
+ │ ├── Integer overflow/underflow │
1326
+ │ ├── Date boundaries (leap years, month ends) │
1327
+ │ └── Array index out of bounds │
1328
+ ├─────────────────────────────────────────────────────────────┤
1329
+ │ State Errors │
1330
+ │ ├── Operations on deleted records │
1331
+ │ ├── Concurrent modifications │
1332
+ │ ├── Session timeout during operation │
1333
+ │ └── Browser back button after submission │
1334
+ ├─────────────────────────────────────────────────────────────┤
1335
+ │ Environment Errors │
1336
+ │ ├── Network timeout/disconnection │
1337
+ │ ├── Low disk space │
1338
+ │ ├── Database connection pool exhaustion │
1339
+ │ └── Time zone differences │
1340
+ └─────────────────────────────────────────────────────────────┘
1341
+ ```
1342
+
1343
+ #### 3. Checklist-Based Testing
1344
+
1345
+ Use predefined checklists based on experience and standards.
1346
+
1347
+ ```markdown
1348
+ # API Endpoint Testing Checklist
1349
+
1350
+ ## Authentication & Authorization
1351
+ - [ ] Endpoint rejects unauthenticated requests
1352
+ - [ ] Endpoint rejects expired tokens
1353
+ - [ ] Endpoint enforces role-based access
1354
+ - [ ] Different user roles see appropriate data
1355
+
1356
+ ## Input Validation
1357
+ - [ ] Required fields are validated
1358
+ - [ ] Data types are enforced
1359
+ - [ ] String length limits enforced
1360
+ - [ ] Malicious input sanitized (XSS, SQL injection)
1361
+
1362
+ ## Response Handling
1363
+ - [ ] Success responses have correct status codes
1364
+ - [ ] Error responses have meaningful messages
1365
+ - [ ] Response format matches API documentation
1366
+ - [ ] Pagination works correctly
1367
+
1368
+ ## Performance
1369
+ - [ ] Response time under load is acceptable
1370
+ - [ ] No N+1 query problems
1371
+ - [ ] Large datasets handled gracefully
1372
+ ```
1373
+
1374
+ ### When to Use Experience-Based Testing
1375
+
1376
+ | Scenario | Recommended Technique |
1377
+ |----------|----------------------|
1378
+ | New feature exploration | Exploratory Testing |
1379
+ | Legacy system with no documentation | Exploratory Testing |
1380
+ | Known problematic areas | Error Guessing |
1381
+ | Regression testing | Checklist-Based |
1382
+ | Time pressure / quick validation | Error Guessing |
1383
+ | Compliance verification | Checklist-Based |
1384
+
1385
+ ### Combining with Systematic Techniques
1386
+
1387
+ ```
1388
+ ┌─────────────────────────────────────────────────────────────┐
1389
+ │ Optimal Testing Approach │
1390
+ ├─────────────────────────────────────────────────────────────┤
1391
+ │ │
1392
+ │ Systematic Techniques Experience-Based │
1393
+ │ (70-80% of effort) (20-30% of effort) │
1394
+ │ │
1395
+ │ ┌──────────────────┐ ┌──────────────────┐ │
1396
+ │ │ Equivalence Part.│ │ Exploratory │ │
1397
+ │ │ Boundary Value │ + │ Error Guessing │ │
1398
+ │ │ Decision Tables │ │ Checklists │ │
1399
+ │ └──────────────────┘ └──────────────────┘ │
1400
+ │ │
1401
+ │ Provides: Provides: │
1402
+ │ - Coverage - Edge cases │
1403
+ │ - Repeatability - Real-world scenarios │
1404
+ │ - Documentation - Quick feedback │
1405
+ │ │
1406
+ └─────────────────────────────────────────────────────────────┘
1407
+ ```
1408
+
1409
+ ---
1410
+
1411
+ ## Risk-Based Testing
1412
+
1413
+ Risk-based testing prioritizes testing efforts based on the likelihood and impact of potential failures.
1414
+
1415
+ **Reference**: ISTQB CTFL v4.0 Section 5.2
1416
+
1417
+ ### Risk Assessment
1418
+
1419
+ #### Risk Formula
1420
+
1421
+ ```
1422
+ Risk Level = Likelihood × Impact
1423
+
1424
+ ┌─────────────────────────────────────────────────────────────┐
1425
+ │ Risk Matrix │
1426
+ ├─────────────┬───────────────────────────────────────────────┤
1427
+ │ │ Impact │
1428
+ │ Likelihood │ Low (1) Medium (2) High (3) │
1429
+ ├─────────────┼───────────────────────────────────────────────┤
1430
+ │ High (3) │ Medium(3) High(6) Critical(9) │
1431
+ │ Medium (2) │ Low(2) Medium(4) High(6) │
1432
+ │ Low (1) │ Low(1) Low(2) Medium(3) │
1433
+ └─────────────┴───────────────────────────────────────────────┘
1434
+ ```
1435
+
1436
+ #### Risk Factors
1437
+
1438
+ **Likelihood Factors** (Technical Risk):
1439
+ - Complexity of the feature
1440
+ - New/unfamiliar technology
1441
+ - Code quality metrics
1442
+ - Developer experience
1443
+ - Frequency of changes
1444
+ - Integration complexity
1445
+
1446
+ **Impact Factors** (Business Risk):
1447
+ - Revenue impact
1448
+ - User base affected
1449
+ - Regulatory compliance
1450
+ - Brand reputation
1451
+ - Data security/privacy
1452
+ - Operational criticality
1453
+
1454
+ ### Risk-Based Test Prioritization
1455
+
1456
+ ```
1457
+ Example: E-commerce Application
1458
+
1459
+ ┌──────────────────────┬─────────┬─────────┬───────┬──────────┐
1460
+ │ Feature │Likelihood│ Impact │ Risk │ Priority │
1461
+ ├──────────────────────┼─────────┼─────────┼───────┼──────────┤
1462
+ │ Payment Processing │ 2 │ 3 │ 6 │ 1 │
1463
+ │ User Authentication │ 2 │ 3 │ 6 │ 2 │
1464
+ │ Shopping Cart │ 2 │ 2 │ 4 │ 3 │
1465
+ │ Product Search │ 1 │ 2 │ 2 │ 4 │
1466
+ │ Product Reviews │ 1 │ 1 │ 1 │ 5 │
1467
+ │ Wishlist │ 1 │ 1 │ 1 │ 6 │
1468
+ └──────────────────────┴─────────┴─────────┴───────┴──────────┘
1469
+
1470
+ Test Effort Allocation:
1471
+ - Critical (Risk 6+): 50% of test effort, most thorough coverage
1472
+ - Medium (Risk 3-5): 30% of test effort, standard coverage
1473
+ - Low (Risk 1-2): 20% of test effort, basic coverage
1474
+ ```
1475
+
1476
+ ### Risk-Based Test Planning
1477
+
1478
+ ```markdown
1479
+ # Risk-Based Test Plan Template
1480
+
1481
+ ## 1. Risk Identification
1482
+ List all features/components and their risk factors.
1483
+
1484
+ ## 2. Risk Analysis
1485
+ Calculate risk scores using Likelihood × Impact.
1486
+
1487
+ ## 3. Test Prioritization
1488
+ | Priority | Features | Test Depth |
1489
+ |----------|----------|------------|
1490
+ | P1 | Payment, Auth | Full coverage, all techniques |
1491
+ | P2 | Cart, Checkout | Standard coverage |
1492
+ | P3 | Search, Browse | Basic happy path |
1493
+ | P4 | Reviews, Wishlist | Minimal, smoke tests |
1494
+
1495
+ ## 4. Risk Mitigation
1496
+ - P1 features: 100% automated tests, manual exploratory
1497
+ - P2 features: 80% automated, sample manual tests
1498
+ - P3 features: Key scenarios automated
1499
+ - P4 features: Basic smoke tests only
1500
+
1501
+ ## 5. Residual Risk Acceptance
1502
+ Document accepted risks for low-priority features.
1503
+ ```
1504
+
1505
+ ### Continuous Risk Assessment
1506
+
1507
+ ```
1508
+ ┌─────────────────────────────────────────────────────────────┐
1509
+ │ Continuous Risk Re-evaluation │
1510
+ ├─────────────────────────────────────────────────────────────┤
1511
+ │ │
1512
+ │ Sprint Start ──▶ During Sprint ──▶ Sprint End │
1513
+ │ │ │ │ │
1514
+ │ ▼ ▼ ▼ │
1515
+ │ Initial Risk Update based on: Review: │
1516
+ │ Assessment - Defects found - Actual vs Expected │
1517
+ │ - Code changes - Adjust for next │
1518
+ │ - New requirements sprint │
1519
+ │ │
1520
+ └─────────────────────────────────────────────────────────────┘
1521
+ ```
1522
+
1523
+ ---
1524
+
1525
+ ## Specialized Testing
1526
+
1527
+ Modern software development requires specialized testing types beyond traditional categories.
1528
+
1529
+ ### Contract Testing
1530
+
1531
+ Verifies that services communicate correctly according to agreed contracts. Essential for microservices architectures.
1532
+
1533
+ ```
1534
+ ┌─────────────────────────────────────────────────────────────┐
1535
+ │ Consumer-Driven Contract Testing │
1536
+ ├─────────────────────────────────────────────────────────────┤
1537
+ │ │
1538
+ │ Consumer Provider │
1539
+ │ (Frontend) (API) │
1540
+ │ │ │ │
1541
+ │ │ 1. Define expectations │ │
1542
+ │ │─────────────────────────────▶ │ │
1543
+ │ │ │ │
1544
+ │ │ 2. Generate contract │ │
1545
+ │ │ (Pact file) │ │
1546
+ │ │ │ │
1547
+ │ │ 3. Share contract │ │
1548
+ │ │─────────────────────────────▶ │ │
1549
+ │ │ │ │
1550
+ │ │ 4. Provider verifies │
1551
+ │ │ against contract │
1552
+ │ │ │ │
1553
+ └─────────────────────────────────────────────────────────────┘
1554
+ ```
1555
+
1556
+ **Tools**: Pact, Spring Cloud Contract, Postman
1557
+
1558
+ ```typescript
1559
+ // Consumer Test (Pact.js Example)
1560
+ describe('User API Contract', () => {
1561
+ it('returns user by ID', async () => {
1562
+ await provider.addInteraction({
1563
+ state: 'user with ID 123 exists',
1564
+ uponReceiving: 'a request for user 123',
1565
+ withRequest: {
1566
+ method: 'GET',
1567
+ path: '/users/123'
1568
+ },
1569
+ willRespondWith: {
1570
+ status: 200,
1571
+ body: {
1572
+ id: '123',
1573
+ name: Matchers.string('John Doe'),
1574
+ email: Matchers.email()
1575
+ }
1576
+ }
1577
+ });
1578
+
1579
+ const user = await userClient.getUser('123');
1580
+ expect(user.id).toBe('123');
1581
+ });
1582
+ });
1583
+ ```
1584
+
1585
+ ### Mutation Testing
1586
+
1587
+ Evaluates test suite quality by introducing small changes (mutants) to the code and checking if tests detect them.
1588
+
1589
+ ```
1590
+ ┌─────────────────────────────────────────────────────────────┐
1591
+ │ Mutation Testing │
1592
+ ├─────────────────────────────────────────────────────────────┤
1593
+ │ │
1594
+ │ Original Code: Mutant (change operator): │
1595
+ │ if (a > b) if (a >= b) ← Boundary mutant │
1596
+ │ if (a > b) if (a < b) ← Negation mutant │
1597
+ │ return a + b; return a - b; ← Arithmetic mutant │
1598
+ │ return true; return false; ← Return value mutant │
1599
+ │ │
1600
+ ├─────────────────────────────────────────────────────────────┤
1601
+ │ Mutation Score = (Killed Mutants / Total Mutants) × 100 │
1602
+ │ │
1603
+ │ - Killed: Test failed → Good, test detected the change │
1604
+ │ - Survived: Test passed → Bad, test missed the defect │
1605
+ │ │
1606
+ │ Target: > 80% mutation score for critical code │
1607
+ └─────────────────────────────────────────────────────────────┘
1608
+ ```
1609
+
1610
+ **Tools**: Stryker (JS/TS), PITest (Java), mutmut (Python)
1611
+
1612
+ ```bash
1613
+ # Run mutation testing with Stryker
1614
+ npx stryker run
1615
+
1616
+ # Example output:
1617
+ # Mutation score: 85.7%
1618
+ # Killed: 180 Survived: 30 No coverage: 10
1619
+ ```
1620
+
1621
+ ### Chaos Engineering
1622
+
1623
+ Proactively tests system resilience by injecting failures in production or staging environments.
1624
+
1625
+ ```
1626
+ ┌─────────────────────────────────────────────────────────────┐
1627
+ │ Chaos Engineering Principles │
1628
+ ├─────────────────────────────────────────────────────────────┤
1629
+ │ │
1630
+ │ 1. Define "steady state" (normal behavior metrics) │
1631
+ │ 2. Hypothesize steady state continues during chaos │
1632
+ │ 3. Introduce real-world events: │
1633
+ │ - Server failures │
1634
+ │ - Network latency/partition │
1635
+ │ - Resource exhaustion │
1636
+ │ - Clock skew │
1637
+ │ 4. Try to disprove the hypothesis │
1638
+ │ 5. Minimize blast radius (start small) │
1639
+ │ │
1640
+ └─────────────────────────────────────────────────────────────┘
1641
+ ```
1642
+
1643
+ **Tools**: Chaos Monkey, Gremlin, Litmus, Chaos Toolkit
1644
+
1645
+ ```yaml
1646
+ # Chaos Toolkit Experiment Example
1647
+ title: "Service resilience when database is slow"
1648
+ description: "Verify the service degrades gracefully"
1649
+
1650
+ steady-state-hypothesis:
1651
+ title: "Service responds within SLA"
1652
+ probes:
1653
+ - name: "service-responds"
1654
+ type: probe
1655
+ provider:
1656
+ type: http
1657
+ url: "http://service/health"
1658
+ timeout: 3
1659
+
1660
+ method:
1661
+ - name: "inject-db-latency"
1662
+ type: action
1663
+ provider:
1664
+ type: process
1665
+ path: "tc"
1666
+ arguments: "qdisc add dev eth0 root netem delay 500ms"
1667
+ pauses:
1668
+ after: 30
1669
+
1670
+ rollbacks:
1671
+ - name: "remove-latency"
1672
+ type: action
1673
+ provider:
1674
+ type: process
1675
+ path: "tc"
1676
+ arguments: "qdisc del dev eth0 root"
1677
+ ```
1678
+
1679
+ ### Property-Based Testing
1680
+
1681
+ Generates random test inputs based on properties the code should satisfy.
1682
+
1683
+ ```python
1684
+ # Python Example with Hypothesis
1685
+ from hypothesis import given, strategies as st
1686
+
1687
+ # Property: Sorting should be idempotent
1688
+ @given(st.lists(st.integers()))
1689
+ def test_sort_idempotent(xs):
1690
+ sorted_once = sorted(xs)
1691
+ sorted_twice = sorted(sorted_once)
1692
+ assert sorted_once == sorted_twice
1693
+
1694
+ # Property: Sorted list should have same elements
1695
+ @given(st.lists(st.integers()))
1696
+ def test_sort_preserves_elements(xs):
1697
+ sorted_xs = sorted(xs)
1698
+ assert sorted(xs) == sorted(sorted_xs)
1699
+ assert len(xs) == len(sorted_xs)
1700
+
1701
+ # Property: JSON encode/decode roundtrip
1702
+ @given(st.dictionaries(st.text(), st.integers()))
1703
+ def test_json_roundtrip(d):
1704
+ assert json.loads(json.dumps(d)) == d
1705
+ ```
1706
+
1707
+ **Tools**: Hypothesis (Python), fast-check (JS/TS), QuickCheck (Haskell), FsCheck (C#)
1708
+
1709
+ ### Visual Regression Testing
1710
+
1711
+ Detects unintended visual changes in UI components.
1712
+
1713
+ ```
1714
+ ┌─────────────────────────────────────────────────────────────┐
1715
+ │ Visual Regression Testing │
1716
+ ├─────────────────────────────────────────────────────────────┤
1717
+ │ │
1718
+ │ 1. Capture baseline screenshots │
1719
+ │ 2. Run tests, capture new screenshots │
1720
+ │ 3. Compare pixel-by-pixel or perceptually │
1721
+ │ 4. Flag differences for review │
1722
+ │ │
1723
+ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
1724
+ │ │ Baseline │ → │ Current │ → │ Diff │ │
1725
+ │ │ Screenshot │ │ Screenshot │ │ Report │ │
1726
+ │ └─────────────┘ └─────────────┘ └─────────────┘ │
1727
+ │ │
1728
+ └─────────────────────────────────────────────────────────────┘
1729
+ ```
1730
+
1731
+ **Tools**: Percy, Chromatic, BackstopJS, Playwright visual comparisons
1732
+
1733
+ ```typescript
1734
+ // Playwright Visual Regression Example
1735
+ test('homepage visual regression', async ({ page }) => {
1736
+ await page.goto('/');
1737
+ await expect(page).toHaveScreenshot('homepage.png', {
1738
+ maxDiffPixels: 100 // Allow minor differences
1739
+ });
1740
+ });
1741
+
1742
+ test('button component visual regression', async ({ page }) => {
1743
+ await page.goto('/components/button');
1744
+ const button = page.locator('[data-testid="primary-button"]');
1745
+ await expect(button).toHaveScreenshot('primary-button.png');
1746
+ });
1747
+ ```
1748
+
1749
+ ### Accessibility Testing (a11y)
1750
+
1751
+ Ensures applications are usable by people with disabilities.
1752
+
1753
+ ```
1754
+ ┌─────────────────────────────────────────────────────────────┐
1755
+ │ Accessibility Testing Checklist │
1756
+ ├─────────────────────────────────────────────────────────────┤
1757
+ │ WCAG 2.1 Guidelines │
1758
+ │ ├── Perceivable │
1759
+ │ │ ├── Alt text for images │
1760
+ │ │ ├── Captions for video │
1761
+ │ │ └── Sufficient color contrast │
1762
+ │ ├── Operable │
1763
+ │ │ ├── Keyboard navigation │
1764
+ │ │ ├── Focus indicators visible │
1765
+ │ │ └── No keyboard traps │
1766
+ │ ├── Understandable │
1767
+ │ │ ├── Clear labels and instructions │
1768
+ │ │ ├── Predictable navigation │
1769
+ │ │ └── Error identification │
1770
+ │ └── Robust │
1771
+ │ ├── Valid HTML │
1772
+ │ └── ARIA attributes correct │
1773
+ └─────────────────────────────────────────────────────────────┘
1774
+ ```
1775
+
1776
+ **Tools**: axe-core, Pa11y, WAVE, Lighthouse
1777
+
1778
+ ```typescript
1779
+ // Playwright + axe-core Example
1780
+ import AxeBuilder from '@axe-core/playwright';
1781
+
1782
+ test('homepage accessibility', async ({ page }) => {
1783
+ await page.goto('/');
1784
+
1785
+ const results = await new AxeBuilder({ page })
1786
+ .withTags(['wcag2a', 'wcag2aa'])
1787
+ .analyze();
1788
+
1789
+ expect(results.violations).toEqual([]);
1790
+ });
1791
+ ```
1792
+
1793
+ ### Additional Specialized Testing Types (SWEBOK)
1794
+
1795
+ The following test types are defined in SWEBOK v4.0 as test objectives:
1796
+
1797
+ #### Installation Testing
1798
+
1799
+ Verifies that the software installs correctly in the target environment.
1800
+
1801
+ ```
1802
+ ┌─────────────────────────────────────────────────────────────┐
1803
+ │ Installation Testing Checklist │
1804
+ ├─────────────────────────────────────────────────────────────┤
1805
+ │ │
1806
+ │ Pre-Installation │
1807
+ │ ├── Prerequisites verified (OS, runtime, dependencies) │
1808
+ │ ├── Disk space requirements met │
1809
+ │ └── Permissions and access rights confirmed │
1810
+ │ │
1811
+ │ Installation Process │
1812
+ │ ├── Fresh installation succeeds │
1813
+ │ ├── Upgrade from previous version succeeds │
1814
+ │ ├── Custom installation options work │
1815
+ │ └── Silent/unattended installation works │
1816
+ │ │
1817
+ │ Post-Installation │
1818
+ │ ├── Application launches correctly │
1819
+ │ ├── Configuration files created properly │
1820
+ │ ├── Registry/system settings correct │
1821
+ │ └── Uninstallation removes all components │
1822
+ │ │
1823
+ └─────────────────────────────────────────────────────────────┘
1824
+ ```
1825
+
1826
+ #### Alpha and Beta Testing
1827
+
1828
+ | Phase | Participants | Environment | Purpose |
1829
+ |-------|--------------|-------------|---------|
1830
+ | **Alpha** | Internal users, developers | Development site | Early validation, major issues |
1831
+ | **Beta** | External users, customers | Customer site | Real-world validation, feedback |
1832
+ | **Open Beta** | General public | Various | Wide exposure, stress testing |
1833
+
1834
+ #### Recovery Testing
1835
+
1836
+ Tests the system's ability to recover from failures.
1837
+
1838
+ ```
1839
+ ┌─────────────────────────────────────────────────────────────┐
1840
+ │ Recovery Testing Scenarios │
1841
+ ├─────────────────────────────────────────────────────────────┤
1842
+ │ │
1843
+ │ Hardware Failures │
1844
+ │ ├── Power outage during transaction │
1845
+ │ ├── Disk failure and RAID recovery │
1846
+ │ └── Network disconnection and reconnection │
1847
+ │ │
1848
+ │ Software Failures │
1849
+ │ ├── Application crash recovery │
1850
+ │ ├── Database connection loss and reconnect │
1851
+ │ └── Service restart behavior │
1852
+ │ │
1853
+ │ Data Recovery │
1854
+ │ ├── Backup and restore procedures │
1855
+ │ ├── Transaction rollback │
1856
+ │ └── Point-in-time recovery │
1857
+ │ │
1858
+ │ Metrics: │
1859
+ │ • Recovery Time Objective (RTO): Max acceptable downtime │
1860
+ │ • Recovery Point Objective (RPO): Max acceptable data loss │
1861
+ │ │
1862
+ └─────────────────────────────────────────────────────────────┘
1863
+ ```
1864
+
1865
+ #### Configuration Testing
1866
+
1867
+ Tests the system under different configurations.
1868
+
1869
+ ```
1870
+ Configuration Matrix Example:
1871
+
1872
+ ┌──────────────────┬─────────┬─────────┬─────────┐
1873
+ │ Configuration │ Config A│ Config B│ Config C│
1874
+ ├──────────────────┼─────────┼─────────┼─────────┤
1875
+ │ Memory │ 4 GB │ 8 GB │ 16 GB │
1876
+ │ Database │ SQLite │ MySQL │ PostgreSQL│
1877
+ │ Cache │ None │ Redis │ Memcached│
1878
+ │ Load Balancer │ None │ Nginx │ HAProxy │
1879
+ └──────────────────┴─────────┴─────────┴─────────┘
1880
+
1881
+ Test each supported configuration combination.
1882
+ ```
1883
+
1884
+ #### Back-to-Back Testing
1885
+
1886
+ Compares outputs of multiple implementations with identical inputs.
1887
+
1888
+ ```
1889
+ ┌─────────────────────────────────────────────────────────────┐
1890
+ │ Back-to-Back Testing │
1891
+ ├─────────────────────────────────────────────────────────────┤
1892
+ │ │
1893
+ │ Use Cases: │
1894
+ │ ├── Comparing new version against old version │
1895
+ │ ├── Comparing optimized code against reference impl │
1896
+ │ ├── Cross-platform consistency verification │
1897
+ │ └── Validating refactored code behavior │
1898
+ │ │
1899
+ │ Process: │
1900
+ │ ┌─────────┐ ┌─────────────┐ ┌──────────┐ │
1901
+ │ │ Input │────▶│ Version A │────▶│ Output A │──┐ │
1902
+ │ │ Data │ └─────────────┘ └──────────┘ │ │
1903
+ │ │ │ ┌─────────────┐ ┌──────────┐ ▼ │
1904
+ │ │ │────▶│ Version B │────▶│ Output B │─▶Compare │
1905
+ │ └─────────┘ └─────────────┘ └──────────┘ │
1906
+ │ │
1907
+ └─────────────────────────────────────────────────────────────┘
1908
+ ```
1909
+
1910
+ #### Usability Testing
1911
+
1912
+ Evaluates how easily users can learn and use the system.
1913
+
1914
+ ```
1915
+ ┌─────────────────────────────────────────────────────────────┐
1916
+ │ Usability Testing Methods │
1917
+ ├─────────────────────────────────────────────────────────────┤
1918
+ │ │
1919
+ │ Observational Methods │
1920
+ │ ├── Think-aloud protocol │
1921
+ │ ├── Eye tracking │
1922
+ │ └── Session recording │
1923
+ │ │
1924
+ │ Task-Based Methods │
1925
+ │ ├── Task completion rate │
1926
+ │ ├── Time on task │
1927
+ │ └── Error rate │
1928
+ │ │
1929
+ │ Survey Methods │
1930
+ │ ├── System Usability Scale (SUS) │
1931
+ │ ├── Net Promoter Score (NPS) │
1932
+ │ └── Post-task questionnaires │
1933
+ │ │
1934
+ │ Key Metrics: │
1935
+ │ • Learnability: Time to complete first task │
1936
+ │ • Efficiency: Time to complete after learning │
1937
+ │ • Memorability: Performance after period of non-use │
1938
+ │ • Satisfaction: User rating scores │
1939
+ │ │
1940
+ └─────────────────────────────────────────────────────────────┘
1941
+ ```
1942
+
1943
+ ### Specialized Testing Summary
1944
+
1945
+ | Type | Purpose | When to Use |
1946
+ |------|---------|-------------|
1947
+ | Contract Testing | API compatibility | Microservices |
1948
+ | Mutation Testing | Test quality assessment | Critical code paths |
1949
+ | Chaos Engineering | Resilience validation | Production systems |
1950
+ | Property-Based | Edge case discovery | Algorithms, parsers |
1951
+ | Visual Regression | UI consistency | Frontend changes |
1952
+ | Accessibility | Inclusive design | All user-facing apps |
1953
+ | Installation | Deployment verification | Release candidates |
1954
+ | Alpha/Beta | Early user validation | Pre-release phases |
1955
+ | Recovery | Failure recovery | Business-critical systems |
1956
+ | Configuration | Multi-config support | Cross-platform apps |
1957
+ | Back-to-Back | Version comparison | Migrations, refactoring |
1958
+ | Usability | User experience | User-facing applications |
1959
+
1960
+ ---
1961
+
1962
+ ## Test-Related Measures
1963
+
1964
+ Quantitative measures for evaluating both the software under test and the quality of testing itself.
1965
+
1966
+ **Reference**: SWEBOK v4.0 Section 5.4
1967
+
1968
+ ### Program Evaluation Measures
1969
+
1970
+ #### Fault Density
1971
+
1972
+ ```
1973
+ Fault Density = Total Faults / Size
1974
+
1975
+ Where Size can be:
1976
+ - KLOC (Thousands of Lines of Code)
1977
+ - Function Points
1978
+ - Story Points
1979
+
1980
+ Industry Benchmarks:
1981
+ ┌─────────────────────┬────────────────────────────┐
1982
+ │ Quality Level │ Faults per KLOC │
1983
+ ├─────────────────────┼────────────────────────────┤
1984
+ │ Typical │ 1 - 25 │
1985
+ │ Good │ 0.5 - 1 │
1986
+ │ Excellent │ < 0.5 │
1987
+ │ Safety-Critical │ < 0.1 │
1988
+ └─────────────────────┴────────────────────────────┘
1989
+ ```
1990
+
1991
+ #### Reliability Growth Models
1992
+
1993
+ Track defect discovery rate over time to predict remaining defects.
1994
+
1995
+ ```
1996
+ ┌─────────────────────────────────────────────────────────────┐
1997
+ │ Reliability Growth Curve │
1998
+ ├─────────────────────────────────────────────────────────────┤
1999
+ │ │
2000
+ │ Defects │
2001
+ │ Found │
2002
+ │ │ ╭──────────────────────────────── │
2003
+ │ │ ╭─╯ │
2004
+ │ │ ╭╯ │
2005
+ │ │ ╭╯ │
2006
+ │ │╭╯ │
2007
+ │ ├╯ │
2008
+ │ └──────────────────────────────────────▶ Time │
2009
+ │ │
2010
+ │ As testing progresses: │
2011
+ │ • Defect discovery rate decreases │
2012
+ │ • Curve approaches asymptote (total defects) │
2013
+ │ • Remaining defects can be estimated │
2014
+ │ │
2015
+ │ Common Models: │
2016
+ │ • Goel-Okumoto: Exponential │
2017
+ │ • Musa: Logarithmic │
2018
+ │ • Jelinski-Moranda: De-eutrophication │
2019
+ │ │
2020
+ └─────────────────────────────────────────────────────────────┘
2021
+ ```
2022
+
2023
+ ### Test Evaluation Measures
2024
+
2025
+ #### Fault Seeding
2026
+
2027
+ Insert known faults to measure test effectiveness.
2028
+
2029
+ ```
2030
+ Fault Detection Effectiveness = Seeded Faults Found / Total Seeded Faults
2031
+
2032
+ If tests find 80% of seeded faults, estimate:
2033
+ - Similar detection rate for real faults
2034
+ - Remaining faults ≈ Found faults / Detection rate
2035
+
2036
+ Example:
2037
+ - Seeded 20 faults, tests found 16 (80%)
2038
+ - Found 40 real faults during testing
2039
+ - Estimated total real faults ≈ 40 / 0.8 = 50
2040
+ - Estimated remaining ≈ 10 faults
2041
+ ```
2042
+
2043
+ #### Coverage Metrics Summary
2044
+
2045
+ | Metric | Formula | Target |
2046
+ |--------|---------|--------|
2047
+ | Statement Coverage | Executed statements / Total statements | ≥ 70% |
2048
+ | Branch Coverage | Executed branches / Total branches | ≥ 60% |
2049
+ | Condition Coverage | Conditions evaluated T & F / Total conditions | ≥ 80% |
2050
+ | MC/DC | Conditions independently affecting decision | 100% (safety-critical) |
2051
+ | Requirements Coverage | Tested requirements / Total requirements | 100% |
2052
+ | Mutation Score | Killed mutants / Total mutants | ≥ 80% |
2053
+
2054
+ ### Testing Process Metrics
2055
+
2056
+ ```
2057
+ ┌─────────────────────────────────────────────────────────────┐
2058
+ │ Key Testing Metrics │
2059
+ ├─────────────────────────────────────────────────────────────┤
2060
+ │ │
2061
+ │ Efficiency Metrics │
2062
+ │ ├── Test Case Productivity = Tests created / Effort │
2063
+ │ ├── Defect Detection Rate = Defects found / Test hours │
2064
+ │ └── Automation Rate = Automated tests / Total tests │
2065
+ │ │
2066
+ │ Effectiveness Metrics │
2067
+ │ ├── Defect Leakage = Prod defects / Total defects │
2068
+ │ ├── Test Effectiveness = Defects found / Defects present │
2069
+ │ └── Phase Containment = Defects found in phase / Injected │
2070
+ │ │
2071
+ │ Progress Metrics │
2072
+ │ ├── Test Execution Rate = Tests run / Tests planned │
2073
+ │ ├── Pass Rate = Tests passed / Tests executed │
2074
+ │ └── Blocking Rate = Blocked tests / Total tests │
2075
+ │ │
2076
+ └─────────────────────────────────────────────────────────────┘
2077
+ ```
2078
+
2079
+ ---
2080
+
2081
+ ## Defect Management
2082
+
2083
+ Systematic tracking and management of defects throughout the testing lifecycle.
2084
+
2085
+ **Reference**: ISTQB CTFL v4.0 Section 5.5, ISO/IEC/IEEE 29119-3
2086
+
2087
+ ### Defect Lifecycle
2088
+
2089
+ ```
2090
+ ┌─────────────────────────────────────────────────────────────┐
2091
+ │ Defect Lifecycle │
2092
+ ├─────────────────────────────────────────────────────────────┤
2093
+ │ │
2094
+ │ ┌────────┐ ┌────────┐ ┌──────────┐ │
2095
+ │ │ New │───▶│ Open │───▶│ In Progress│ │
2096
+ │ └────────┘ └────────┘ └──────────┘ │
2097
+ │ │ │ │
2098
+ │ │ ▼ │
2099
+ │ │ ┌──────────┐ │
2100
+ │ │ │ Fixed │ │
2101
+ │ │ └──────────┘ │
2102
+ │ │ │ │
2103
+ │ │ ▼ │
2104
+ │ │ ┌──────────┐ ┌──────────┐ │
2105
+ │ │ │ Verified │───▶│ Closed │ │
2106
+ │ │ └──────────┘ └──────────┘ │
2107
+ │ │ │ │
2108
+ │ │ ▼ │
2109
+ │ │ ┌──────────┐ │
2110
+ │ └───────▶│ Reopened │ │
2111
+ │ └──────────┘ │
2112
+ │ │
2113
+ │ Alternative paths: │
2114
+ │ - New → Rejected (not a bug, duplicate) │
2115
+ │ - Open → Deferred (postponed to later release) │
2116
+ │ - Open → Won't Fix (accepted risk) │
2117
+ │ │
2118
+ └─────────────────────────────────────────────────────────────┘
2119
+ ```
2120
+
2121
+ ### Defect Report Template
2122
+
2123
+ ```markdown
2124
+ # Defect Report
2125
+
2126
+ ## Summary
2127
+ [One-line description of the defect]
2128
+
2129
+ ## ID: BUG-1234
2130
+ ## Priority: P1/P2/P3/P4
2131
+ ## Severity: Critical/High/Medium/Low
2132
+
2133
+ ## Environment
2134
+ - **Version**: 2.3.1
2135
+ - **OS**: Windows 11 / macOS 14.2
2136
+ - **Browser**: Chrome 120
2137
+ - **Device**: Desktop / Mobile
2138
+
2139
+ ## Steps to Reproduce
2140
+ 1. Navigate to login page
2141
+ 2. Enter valid username "testuser"
2142
+ 3. Enter invalid password "wrong123"
2143
+ 4. Click "Login" button
2144
+ 5. Observe error message
2145
+
2146
+ ## Expected Result
2147
+ Error message: "Invalid credentials. Please try again."
2148
+
2149
+ ## Actual Result
2150
+ Generic error: "Something went wrong."
2151
+
2152
+ ## Screenshots/Videos
2153
+ [Attach relevant media]
2154
+
2155
+ ## Additional Information
2156
+ - Occurs consistently (100% reproducible)
2157
+ - Related to ticket: FEAT-456
2158
+ - Log file: [attached]
2159
+ ```
2160
+
2161
+ ### Severity vs Priority
2162
+
2163
+ | Severity | Description | Examples |
2164
+ |----------|-------------|----------|
2165
+ | Critical | System crash, data loss, security breach | App won't start, data corruption |
2166
+ | High | Major feature broken, no workaround | Cannot complete checkout |
2167
+ | Medium | Feature impaired but has workaround | Export fails, manual copy works |
2168
+ | Low | Minor issue, cosmetic | Typo, alignment issue |
2169
+
2170
+ | Priority | Description | Response Time |
2171
+ |----------|-------------|---------------|
2172
+ | P1 | Fix immediately | Same day |
2173
+ | P2 | Fix before release | Within sprint |
2174
+ | P3 | Fix when possible | Next sprint |
2175
+ | P4 | Nice to fix | Backlog |
2176
+
2177
+ ### Defect Metrics
2178
+
2179
+ ```
2180
+ ┌─────────────────────────────────────────────────────────────┐
2181
+ │ Key Defect Metrics │
2182
+ ├─────────────────────────────────────────────────────────────┤
2183
+ │ │
2184
+ │ Defect Density = Total Defects / Size (KLOC or FP) │
2185
+ │ - Industry average: 1-25 defects per KLOC │
2186
+ │ │
2187
+ │ Defect Detection Efficiency (DDE) = │
2188
+ │ Defects found before release / Total defects × 100 │
2189
+ │ - Target: > 90% │
2190
+ │ │
2191
+ │ Defect Removal Efficiency (DRE) = │
2192
+ │ Defects removed / Defects injected × 100 │
2193
+ │ - Target: > 95% │
2194
+ │ │
2195
+ │ Mean Time to Detect (MTTD) = Avg time from injection │
2196
+ │ Mean Time to Resolve (MTTR) = Avg time from report to fix │
2197
+ │ │
2198
+ │ Defect Leakage Rate = Production defects / Total defects │
2199
+ │ - Target: < 5% │
2200
+ │ │
2201
+ └─────────────────────────────────────────────────────────────┘
2202
+ ```
2203
+
2204
+ ### Root Cause Analysis
2205
+
2206
+ ```
2207
+ ┌─────────────────────────────────────────────────────────────┐
2208
+ │ Common Defect Root Causes │
2209
+ ├─────────────────────────────────────────────────────────────┤
2210
+ │ Requirements (35-40%) │
2211
+ │ ├── Ambiguous requirements │
2212
+ │ ├── Missing requirements │
2213
+ │ └── Changed requirements │
2214
+ ├─────────────────────────────────────────────────────────────┤
2215
+ │ Design (20-25%) │
2216
+ │ ├── Incorrect algorithm │
2217
+ │ ├── Integration issues │
2218
+ │ └── Performance not considered │
2219
+ ├─────────────────────────────────────────────────────────────┤
2220
+ │ Code (25-30%) │
2221
+ │ ├── Logic errors │
2222
+ │ ├── Boundary conditions │
2223
+ │ └── Exception handling │
2224
+ ├─────────────────────────────────────────────────────────────┤
2225
+ │ Environment (10-15%) │
2226
+ │ ├── Configuration issues │
2227
+ │ ├── Third-party dependencies │
2228
+ │ └── Data issues │
2229
+ └─────────────────────────────────────────────────────────────┘
2230
+ ```
2231
+
2232
+ ---
2233
+
2234
+ ## Test Process Management
2235
+
2236
+ Structured approach to planning, monitoring, and controlling testing activities.
2237
+
2238
+ **Reference**: ISTQB CTFL v4.0 Chapter 5, ISO/IEC/IEEE 29119-2
2239
+
2240
+ ### Test Planning
2241
+
2242
+ #### Test Plan Components (ISO/IEC/IEEE 29119-3)
2243
+
2244
+ ```
2245
+ ┌─────────────────────────────────────────────────────────────┐
2246
+ │ Test Plan Structure │
2247
+ ├─────────────────────────────────────────────────────────────┤
2248
+ │ │
2249
+ │ 1. Test Plan Identification │
2250
+ │ - Project name, version, date │
2251
+ │ │
2252
+ │ 2. Introduction │
2253
+ │ - Purpose, scope, objectives │
2254
+ │ │
2255
+ │ 3. Test Items │
2256
+ │ - Features to be tested │
2257
+ │ - Features not to be tested │
2258
+ │ │
2259
+ │ 4. Test Approach │
2260
+ │ - Test levels and types │
2261
+ │ - Test techniques │
2262
+ │ - Entry/exit criteria │
2263
+ │ │
2264
+ │ 5. Test Environment │
2265
+ │ - Hardware, software, tools │
2266
+ │ - Test data requirements │
2267
+ │ │
2268
+ │ 6. Test Schedule │
2269
+ │ - Milestones, dependencies │
2270
+ │ │
2271
+ │ 7. Roles and Responsibilities │
2272
+ │ - Team structure, RACI matrix │
2273
+ │ │
2274
+ │ 8. Risks and Contingencies │
2275
+ │ - Risk assessment, mitigation │
2276
+ │ │
2277
+ │ 9. Approvals │
2278
+ │ - Sign-off requirements │
2279
+ │ │
2280
+ └─────────────────────────────────────────────────────────────┘
2281
+ ```
2282
+
2283
+ #### Entry and Exit Criteria
2284
+
2285
+ | Criteria Type | Examples |
2286
+ |---------------|----------|
2287
+ | **Entry Criteria** (start testing) | Requirements approved, code complete, environment ready, test data available |
2288
+ | **Exit Criteria** (stop testing) | All P1/P2 tests passed, coverage target met, no critical defects open |
2289
+
2290
+ ```markdown
2291
+ ## Entry Criteria for System Testing
2292
+ - [ ] All integration tests passed
2293
+ - [ ] Test environment deployed and verified
2294
+ - [ ] Test data prepared and loaded
2295
+ - [ ] Test cases reviewed and approved
2296
+ - [ ] All blockers from IT resolved
2297
+
2298
+ ## Exit Criteria for System Testing
2299
+ - [ ] 100% of critical test cases executed
2300
+ - [ ] 95% of high priority test cases passed
2301
+ - [ ] No Critical or High severity defects open
2302
+ - [ ] Code coverage > 80%
2303
+ - [ ] Performance benchmarks met
2304
+ ```
2305
+
2306
+ ### Test Monitoring and Control
2307
+
2308
+ #### Key Progress Metrics
2309
+
2310
+ ```
2311
+ ┌─────────────────────────────────────────────────────────────┐
2312
+ │ Test Progress Metrics │
2313
+ ├─────────────────────────────────────────────────────────────┤
2314
+ │ │
2315
+ │ Execution Metrics: │
2316
+ │ - Tests Planned: 500 │
2317
+ │ - Tests Executed: 350 (70%) │
2318
+ │ - Tests Passed: 320 (91% of executed) │
2319
+ │ - Tests Failed: 30 (9% of executed) │
2320
+ │ - Tests Blocked: 15 │
2321
+ │ │
2322
+ │ Defect Metrics: │
2323
+ │ - Total Defects Found: 45 │
2324
+ │ - Open Defects: 12 (3 Critical, 5 High, 4 Medium) │
2325
+ │ - Fixed Defects: 28 │
2326
+ │ - Verified Defects: 20 │
2327
+ │ │
2328
+ │ Coverage Metrics: │
2329
+ │ - Requirements Coverage: 85% │
2330
+ │ - Code Coverage: 78% │
2331
+ │ - Risk Coverage: 90% │
2332
+ │ │
2333
+ └─────────────────────────────────────────────────────────────┘
2334
+ ```
2335
+
2336
+ #### Test Dashboard Example
2337
+
2338
+ ```
2339
+ Test Progress Dashboard - Sprint 15
2340
+ ═══════════════════════════════════════════════════════════════
2341
+
2342
+ Test Execution Defect Status
2343
+ █████████████████░░░░ 85% Open: ███░░░░░░░ 12
2344
+ Fixed: ████████░░ 28
2345
+ Passed │████████████████│ 320 Verified: ██████░░░░ 20
2346
+ Failed │██ │ 30
2347
+ Blocked │█ │ 15 Trend: ↓ (improving)
2348
+
2349
+ Coverage Build Health
2350
+ Requirements: ███████████░ 85% Last Build: ✓ Pass
2351
+ Code: ████████░░░░ 78% Unit Tests: 245/245 ✓
2352
+ Risk Items: █████████░░░ 90% Int Tests: 89/92 ⚠
2353
+ Flaky Rate: 2.1%
2354
+ ```
2355
+
2356
+ ### Test Completion
2357
+
2358
+ #### Test Summary Report
2359
+
2360
+ ```markdown
2361
+ # Test Summary Report
2362
+
2363
+ ## Project: [Project Name]
2364
+ ## Version: [Version Number]
2365
+ ## Test Period: [Start Date] - [End Date]
2366
+
2367
+ ## Executive Summary
2368
+ [Brief overview of testing activities and outcomes]
2369
+
2370
+ ## Test Scope
2371
+ - Features tested: [list]
2372
+ - Features not tested: [list with justification]
2373
+
2374
+ ## Test Results Summary
2375
+
2376
+ | Test Level | Planned | Executed | Passed | Failed | Pass Rate |
2377
+ |------------|---------|----------|--------|--------|-----------|
2378
+ | Unit | 500 | 500 | 498 | 2 | 99.6% |
2379
+ | Integration| 150 | 150 | 145 | 5 | 96.7% |
2380
+ | System | 200 | 195 | 188 | 7 | 96.4% |
2381
+ | E2E | 50 | 48 | 46 | 2 | 95.8% |
2382
+
2383
+ ## Defect Summary
2384
+
2385
+ | Severity | Found | Fixed | Open | Deferred |
2386
+ |----------|-------|-------|------|----------|
2387
+ | Critical | 2 | 2 | 0 | 0 |
2388
+ | High | 8 | 7 | 1 | 0 |
2389
+ | Medium | 15 | 12 | 2 | 1 |
2390
+ | Low | 20 | 10 | 5 | 5 |
2391
+
2392
+ ## Coverage Analysis
2393
+ - Requirements coverage: 95%
2394
+ - Code coverage: 82%
2395
+ - Risk coverage: 100% (all high-risk items tested)
2396
+
2397
+ ## Open Issues and Risks
2398
+ 1. [Issue 1]: Impact and mitigation
2399
+ 2. [Issue 2]: Impact and mitigation
2400
+
2401
+ ## Recommendations
2402
+ 1. [Recommendation 1]
2403
+ 2. [Recommendation 2]
2404
+
2405
+ ## Conclusion
2406
+ [Final assessment and release recommendation]
2407
+
2408
+ ## Sign-off
2409
+ - QA Lead: [Name, Date]
2410
+ - Dev Lead: [Name, Date]
2411
+ - Product Owner: [Name, Date]
2412
+ ```
2413
+
2414
+ ### Test Process Improvement
2415
+
2416
+ ```
2417
+ ┌─────────────────────────────────────────────────────────────┐
2418
+ │ Continuous Improvement Cycle │
2419
+ ├─────────────────────────────────────────────────────────────┤
2420
+ │ │
2421
+ │ ┌───────────┐ │
2422
+ │ │ Plan │ │
2423
+ │ └─────┬─────┘ │
2424
+ │ │ │
2425
+ │ ▼ │
2426
+ │ ┌───────────┐ ┌───────────┐ │
2427
+ │ │ Act │◀───────▶│ Do │ │
2428
+ │ └───────────┘ └─────┬─────┘ │
2429
+ │ ▲ │ │
2430
+ │ │ ▼ │
2431
+ │ ┌─────┴─────┐ │
2432
+ │ │ Check │ │
2433
+ │ └───────────┘ │
2434
+ │ │
2435
+ │ Retrospective Questions: │
2436
+ │ - What went well? │
2437
+ │ - What could be improved? │
2438
+ │ - What actions will we take? │
2439
+ │ │
2440
+ └─────────────────────────────────────────────────────────────┘
2441
+ ```
2442
+
2443
+ ---
2444
+
2445
+ ## Test Doubles
2446
+
2447
+ ### Types
2448
+
2449
+ | Type | Purpose | Example Use |
2450
+ |------|---------|-------------|
2451
+ | **Stub** | Returns predefined values | Fixed API responses |
2452
+ | **Mock** | Verifies interactions | Verify method called |
2453
+ | **Fake** | Simplified implementation | In-memory database |
2454
+ | **Spy** | Records calls, delegates to real | Partial mocking |
2455
+ | **Dummy** | Placeholder, never used | Fill required parameters |
2456
+
2457
+ ### Usage Guidelines
2458
+
2459
+ ```
2460
+ ┌─────────────────────────────────────────────────────────┐
2461
+ │ When to Use What │
2462
+ ├─────────────────────────────────────────────────────────┤
2463
+ │ Unit Tests (UT) │
2464
+ │ ├── Use Mocks/Stubs for all external dependencies │
2465
+ │ └── Verify interactions with Mocks │
2466
+ ├─────────────────────────────────────────────────────────┤
2467
+ │ Integration Tests (IT) │
2468
+ │ ├── Use Fakes for databases (in-memory, containers) │
2469
+ │ ├── Use Stubs for external APIs │
2470
+ │ └── Minimize mocking - test real integrations │
2471
+ ├─────────────────────────────────────────────────────────┤
2472
+ │ System Tests (ST) │
2473
+ │ ├── Use real components whenever possible │
2474
+ │ ├── Use Fakes only for external third-party services │
2475
+ │ └── No mocking within the system boundary │
2476
+ ├─────────────────────────────────────────────────────────┤
2477
+ │ E2E Tests │
2478
+ │ ├── Use real everything │
2479
+ │ └── Stub only external payment/email services │
2480
+ └─────────────────────────────────────────────────────────┘
2481
+ ```
2482
+
2483
+ ---
2484
+
2485
+ ## Mock Limitations
2486
+
2487
+ ### Query Predicate Verification
2488
+
2489
+ **Problem**:
2490
+ When mocking repository methods that accept query predicates (e.g., lambda expressions,
2491
+ filter functions), using wildcard matchers like `any()` ignores the actual query logic,
2492
+ allowing incorrect queries to pass unit tests.
2493
+
2494
+ **Example**:
2495
+
2496
+ ```python
2497
+ # Python Example
2498
+ # ❌ This test cannot verify query correctness
2499
+ mock_repo.find.return_value = users
2500
+ # Query could be wrong, test still passes
2501
+
2502
+ # ✓ Add integration test to verify actual query
2503
+ ```
2504
+
2505
+ ```typescript
2506
+ // TypeScript Example
2507
+ // ❌ Jest mock ignores actual filter
2508
+ jest.spyOn(repo, 'findBy').mockResolvedValue(users);
2509
+
2510
+ // ✓ Verify with integration test
2511
+ ```
2512
+
2513
+ ```csharp
2514
+ // C# Example
2515
+ // ❌ Moq ignores the actual expression
2516
+ _repo.Setup(r => r.FindAsync(It.IsAny<Expression<Func<User, bool>>>()))
2517
+ .ReturnsAsync(users);
2518
+
2519
+ // ✓ Verify with integration test or use It.Is<> to validate
2520
+ ```
2521
+
2522
+ **Rule of Thumb**:
2523
+ If your unit test mocks a method that accepts a query/filter/predicate parameter,
2524
+ you MUST have a corresponding integration test to verify the query logic.
2525
+
2526
+ ---
2527
+
2528
+ ## Test Data Management
2529
+
2530
+ ### Principles
2531
+
2532
+ 1. **Isolation**: Each test manages its own data
2533
+ 2. **Cleanup**: Tests clean up after themselves
2534
+ 3. **Determinism**: Tests don't depend on shared state
2535
+ 4. **Readability**: Test data clearly shows intent
2536
+
2537
+ ### Patterns
2538
+
2539
+ ```csharp
2540
+ // Builder Pattern for Test Data
2541
+ public class UserBuilder
2542
+ {
2543
+ private string _name = "Default User";
2544
+ private string _email = "default@example.com";
2545
+ private bool _isActive = true;
2546
+
2547
+ public UserBuilder WithName(string name)
2548
+ {
2549
+ _name = name;
2550
+ return this;
2551
+ }
2552
+
2553
+ public UserBuilder WithEmail(string email)
2554
+ {
2555
+ _email = email;
2556
+ return this;
2557
+ }
2558
+
2559
+ public UserBuilder Inactive()
2560
+ {
2561
+ _isActive = false;
2562
+ return this;
2563
+ }
2564
+
2565
+ public User Build() => new User
2566
+ {
2567
+ Name = _name,
2568
+ Email = _email,
2569
+ IsActive = _isActive
2570
+ };
2571
+ }
2572
+
2573
+ // Usage in tests
2574
+ var activeUser = new UserBuilder()
2575
+ .WithName("Active User")
2576
+ .Build();
2577
+
2578
+ var inactiveUser = new UserBuilder()
2579
+ .WithName("Inactive User")
2580
+ .Inactive()
2581
+ .Build();
2582
+ ```
2583
+
2584
+ ### Distinct Identifiers
2585
+
2586
+ When entities have both a surrogate key (auto-generated ID) and a business identifier
2587
+ (e.g., employee number, department code), test data MUST use different values for each.
2588
+
2589
+ **Problem**:
2590
+ If test data uses identical values for both fields, field mapping errors go undetected.
2591
+
2592
+ ```python
2593
+ # Python Example
2594
+ # ❌ Wrong: id equals business_code
2595
+ dept = Department(id=1, business_code=1)
2596
+
2597
+ # ✓ Correct: distinct values catch mapping errors
2598
+ dept = Department(id=1, business_code=1001)
2599
+ ```
2600
+
2601
+ ```csharp
2602
+ // C# Example
2603
+ // ❌ Wrong: Id equals DeptId - mapping errors go undetected
2604
+ var dept = new Department { Id = 1, DeptId = 1 };
2605
+
2606
+ // ✓ Correct: distinct values catch field mapping bugs
2607
+ var dept = new Department { Id = 1, DeptId = 1001 };
2608
+ ```
2609
+
2610
+ **Validation**:
2611
+ ```csharp
2612
+ // C#
2613
+ testData.Dept.Id.Should().NotBe(testData.Dept.DeptId,
2614
+ "Test precondition: Id must differ from business identifier");
2615
+ ```
2616
+
2617
+ ### Composite Keys
2618
+
2619
+ For entities with composite primary keys, ensure each record has a unique key combination.
2620
+
2621
+ ```csharp
2622
+ // C# Example
2623
+ // ❌ Key collision - same (Id, SendTime) combination
2624
+ var batch1 = new BatchRecord { Id = 0, SendTime = now };
2625
+ var batch2 = new BatchRecord { Id = 0, SendTime = now }; // Conflict!
2626
+
2627
+ // ✓ Unique combinations
2628
+ var batch1 = new BatchRecord { Id = 0, SendTime = now.AddSeconds(1) };
2629
+ var batch2 = new BatchRecord { Id = 0, SendTime = now.AddSeconds(2) };
2630
+ ```
2631
+
2632
+ **Tip**: Create helper functions that auto-generate unique composite keys.
2633
+
2634
+ ```csharp
2635
+ // C# Helper Example
2636
+ private static int _timeOffset = 0;
2637
+ public static BatchRecord CreateWithUniqueKey(DateTime baseTime)
2638
+ {
2639
+ return new BatchRecord
2640
+ {
2641
+ SendTime = baseTime.AddSeconds(Interlocked.Increment(ref _timeOffset))
2642
+ };
2643
+ }
2644
+ ```
2645
+
2646
+ ---
2647
+
2648
+ ## Test Environment Isolation
2649
+
2650
+ ### Purpose
2651
+
2652
+ Ensure consistent, reproducible test results across development machines and CI/CD pipelines.
2653
+
2654
+ ### Why It Matters
2655
+
2656
+ - **Reproducibility**: Same tests produce same results everywhere
2657
+ - **Isolation**: Project dependencies don't conflict with system or other projects
2658
+ - **CI/CD Parity**: Local environment matches CI environment
2659
+
2660
+ ### Language-Specific Virtual Environments
2661
+
2662
+ | Language | Tools | Lock File |
2663
+ |----------|-------|-----------|
2664
+ | Python | venv, virtualenv, conda, poetry | requirements.txt, poetry.lock |
2665
+ | Node.js | nvm, fnm + npm/yarn/pnpm | package-lock.json, yarn.lock |
2666
+ | Ruby | rbenv, rvm, bundler | Gemfile.lock |
2667
+ | Java | SDKMAN, jenv, Maven/Gradle | pom.xml, build.gradle.lock |
2668
+ | .NET | dotnet SDK | packages.lock.json |
2669
+ | Go | go mod | go.sum |
2670
+ | Rust | rustup, cargo | Cargo.lock |
2671
+
2672
+ #### Best Practices
2673
+
2674
+ 1. **Always use virtual environments** for development and testing
2675
+ 2. **Commit lock files** to version control
2676
+ 3. **Pin versions** in CI/CD pipelines
2677
+ 4. **Document required runtime versions** in README or .tool-versions
2678
+
2679
+ #### Example: Python with venv
2680
+
2681
+ ```bash
2682
+ # Create virtual environment
2683
+ python -m venv .venv
2684
+
2685
+ # Activate
2686
+ source .venv/bin/activate # Linux/macOS
2687
+ .venv\Scripts\activate # Windows
2688
+
2689
+ # Install dependencies
2690
+ pip install -r requirements.txt
2691
+
2692
+ # Run tests
2693
+ pytest tests/
2694
+ ```
2695
+
2696
+ #### Example: Node.js with nvm
2697
+
2698
+ ```bash
2699
+ # Use project's Node version
2700
+ nvm use
2701
+
2702
+ # Install dependencies
2703
+ npm ci
2704
+
2705
+ # Run tests
2706
+ npm test
2707
+ ```
2708
+
2709
+ ### Containerized Testing
2710
+
2711
+ Use containers to provide consistent external dependencies (databases, message queues, etc.) for integration and system tests.
2712
+
2713
+ #### When to Use
2714
+
2715
+ | Test Level | Container Usage |
2716
+ |------------|-----------------|
2717
+ | UT (Unit Testing) | ❌ Not needed - use mocks |
2718
+ | IT (Integration Testing) | ✅ Testcontainers for databases, caches |
2719
+ | ST (System Testing) | ✅ Docker Compose for full environment |
2720
+ | E2E (End-to-End) | ✅ Full containerized stack |
2721
+
2722
+ #### Testcontainers
2723
+
2724
+ Testcontainers provides lightweight, disposable containers for testing.
2725
+
2726
+ ```csharp
2727
+ // C# Example with Testcontainers
2728
+ public class DatabaseIntegrationTests : IAsyncLifetime
2729
+ {
2730
+ private readonly PostgreSqlContainer _postgres = new PostgreSqlBuilder()
2731
+ .WithImage("postgres:15")
2732
+ .Build();
2733
+
2734
+ public async Task InitializeAsync()
2735
+ {
2736
+ await _postgres.StartAsync();
2737
+ }
2738
+
2739
+ public async Task DisposeAsync()
2740
+ {
2741
+ await _postgres.DisposeAsync();
2742
+ }
2743
+
2744
+ [Fact]
2745
+ public async Task Should_Connect_To_Database()
2746
+ {
2747
+ var connectionString = _postgres.GetConnectionString();
2748
+ // Use connectionString for tests
2749
+ }
2750
+ }
2751
+ ```
2752
+
2753
+ ```python
2754
+ # Python Example with Testcontainers
2755
+ import pytest
2756
+ from testcontainers.postgres import PostgresContainer
2757
+
2758
+ @pytest.fixture(scope="module")
2759
+ def postgres_container():
2760
+ with PostgresContainer("postgres:15") as postgres:
2761
+ yield postgres
2762
+
2763
+ def test_database_connection(postgres_container):
2764
+ connection_url = postgres_container.get_connection_url()
2765
+ # Use connection_url for tests
2766
+ ```
2767
+
2768
+ #### Docker Compose for System Tests
2769
+
2770
+ ```yaml
2771
+ # docker-compose.test.yml
2772
+ version: '3.8'
2773
+ services:
2774
+ app:
2775
+ build: .
2776
+ depends_on:
2777
+ - db
2778
+ - redis
2779
+ - rabbitmq
2780
+ environment:
2781
+ - DATABASE_URL=postgres://test:test@db:5432/testdb
2782
+ - REDIS_URL=redis://redis:6379
2783
+ - RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672
2784
+
2785
+ db:
2786
+ image: postgres:15
2787
+ environment:
2788
+ POSTGRES_USER: test
2789
+ POSTGRES_PASSWORD: test
2790
+ POSTGRES_DB: testdb
2791
+
2792
+ redis:
2793
+ image: redis:7-alpine
2794
+
2795
+ rabbitmq:
2796
+ image: rabbitmq:3-management
2797
+ ```
2798
+
2799
+ ```bash
2800
+ # Run system tests with Docker Compose
2801
+ docker-compose -f docker-compose.test.yml up -d
2802
+ npm run test:system
2803
+ docker-compose -f docker-compose.test.yml down -v
2804
+ ```
2805
+
2806
+ ### Environment Parity Checklist
2807
+
2808
+ ```
2809
+ ┌─────────────────────────────────────────────────────────────┐
2810
+ │ Environment Parity Checklist │
2811
+ ├─────────────────────────────────────────────────────────────┤
2812
+ │ ✅ Same runtime version (Node, Python, etc.) locally & CI │
2813
+ │ ✅ Same database version in containers & production │
2814
+ │ ✅ Lock files committed and used in CI (npm ci, pip -r) │
2815
+ │ ✅ Environment variables documented and consistent │
2816
+ │ ✅ Container images tagged with specific versions │
2817
+ │ ✅ .tool-versions or similar for runtime version mgmt │
2818
+ ├─────────────────────────────────────────────────────────────┤
2819
+ │ ❌ Using "latest" tags in production/CI │
2820
+ │ ❌ Different DB versions between dev and CI │
2821
+ │ ❌ Missing lock files in repository │
2822
+ │ ❌ Hardcoded paths or machine-specific configurations │
2823
+ └─────────────────────────────────────────────────────────────┘
2824
+ ```
2825
+
2826
+ ---
2827
+
2828
+ ## CI/CD Integration
2829
+
2830
+ ### Test Execution Strategy
2831
+
2832
+ ```yaml
2833
+ # Example CI Pipeline
2834
+ stages:
2835
+ - unit-test # Run on every commit
2836
+ - integration-test # Run on every commit
2837
+ - system-test # Run on PR merge to main
2838
+ - e2e-test # Run on release candidates
2839
+
2840
+ unit-test:
2841
+ stage: unit-test
2842
+ script:
2843
+ - npm run test:unit
2844
+ timeout: 10m
2845
+ rules:
2846
+ - if: $CI_PIPELINE_SOURCE == "push"
2847
+
2848
+ integration-test:
2849
+ stage: integration-test
2850
+ services:
2851
+ - postgres:14
2852
+ - redis:7
2853
+ script:
2854
+ - npm run test:integration
2855
+ timeout: 30m
2856
+ rules:
2857
+ - if: $CI_PIPELINE_SOURCE == "push"
2858
+
2859
+ system-test:
2860
+ stage: system-test
2861
+ environment: staging
2862
+ script:
2863
+ - npm run test:system
2864
+ timeout: 2h
2865
+ rules:
2866
+ - if: $CI_COMMIT_BRANCH == "main"
2867
+
2868
+ e2e-test:
2869
+ stage: e2e-test
2870
+ environment: staging
2871
+ script:
2872
+ - npm run test:e2e
2873
+ timeout: 4h
2874
+ rules:
2875
+ - if: $CI_COMMIT_TAG
2876
+ ```
2877
+
2878
+ ### Test Reports
2879
+
2880
+ Required metrics for each test level:
2881
+
2882
+ | Metric | UT | IT | ST | E2E |
2883
+ |--------|----|----|----|----|
2884
+ | Pass/Fail Count | ✅ | ✅ | ✅ | ✅ |
2885
+ | Execution Time | ✅ | ✅ | ✅ | ✅ |
2886
+ | Coverage % | ✅ | ✅ | ⚠️ | ❌ |
2887
+ | Flaky Test Rate | ✅ | ✅ | ✅ | ✅ |
2888
+ | Screenshots/Videos | ❌ | ❌ | ⚠️ | ✅ |
2889
+
2890
+ ---
2891
+
2892
+ ## Best Practices
2893
+
2894
+ ### AAA Pattern
2895
+
2896
+ ```csharp
2897
+ [TestMethod]
2898
+ public void MethodName_Scenario_ExpectedBehavior()
2899
+ {
2900
+ // Arrange - Set up test data and environment
2901
+ var input = CreateTestInput();
2902
+ var sut = new SystemUnderTest();
2903
+
2904
+ // Act - Execute the behavior under test
2905
+ var result = sut.Execute(input);
2906
+
2907
+ // Assert - Verify the result
2908
+ Assert.AreEqual(expected, result);
2909
+ }
2910
+ ```
2911
+
2912
+ ### FIRST Principles
2913
+
2914
+ | Principle | Description |
2915
+ |-----------|-------------|
2916
+ | **F**ast | Tests run quickly |
2917
+ | **I**ndependent | Tests don't affect each other |
2918
+ | **R**epeatable | Same result every time |
2919
+ | **S**elf-validating | Clear pass/fail |
2920
+ | **T**imely | Written with production code |
2921
+
2922
+ ### Anti-Patterns to Avoid
2923
+
2924
+ ```
2925
+ ❌ Test Interdependence
2926
+ Tests that must run in specific order
2927
+
2928
+ ❌ Flaky Tests
2929
+ Tests that sometimes pass, sometimes fail
2930
+
2931
+ ❌ Testing Implementation Details
2932
+ Tests that break when refactoring
2933
+
2934
+ ❌ Over-Mocking
2935
+ Mocking so much that nothing real is tested
2936
+
2937
+ ❌ Missing Assertions
2938
+ Tests that verify nothing meaningful
2939
+
2940
+ ❌ Magic Numbers/Strings
2941
+ Unexplained values in test code
2942
+
2943
+ ❌ Identical Test IDs
2944
+ Using same values for surrogate and business keys
2945
+ ```
2946
+
2947
+ ---
2948
+
2949
+ ## Quick Reference Card
2950
+
2951
+ ```
2952
+ ┌─────────────────────────────────────────────────────────────┐
2953
+ │ Reference Standards │
2954
+ ├─────────────────────────────────────────────────────────────┤
2955
+ │ • ISTQB CTFL v4.0 - Testing certification │
2956
+ │ • ISO/IEC/IEEE 29119 - Testing standards │
2957
+ │ • SWEBOK v4.0 - Software engineering knowledge │
2958
+ ├─────────────────────────────────────────────────────────────┤
2959
+ │ Testing Fundamentals (SWEBOK) │
2960
+ ├─────────────────────────────────────────────────────────────┤
2961
+ │ Error → Fault → Failure (cause chain) │
2962
+ │ Oracle Problem: How to know expected result? │
2963
+ │ Limitation: Testing proves presence, not absence of bugs │
2964
+ │ Testability: Controllability + Observability + Isolability │
2965
+ ├─────────────────────────────────────────────────────────────┤
2966
+ │ Testing Framework Options │
2967
+ ├─────────────────────────────────────────────────────────────┤
2968
+ │ ISTQB: UT → IT/SIT → ST → AT/UAT (Enterprise/Compliance) │
2969
+ │ Industry: UT (70%) → IT (20%) → E2E (10%) (Agile/DevOps) │
2970
+ ├─────────────────────────────────────────────────────────────┤
2971
+ │ Testing Levels Summary │
2972
+ ├──────────┬──────────────────────────────────────────────────┤
2973
+ │ UT │ Single unit, isolated, mocked deps, < 100ms │
2974
+ ├──────────┼──────────────────────────────────────────────────┤
2975
+ │ IT/SIT │ Component integration, real DB, 1-10 sec │
2976
+ ├──────────┼──────────────────────────────────────────────────┤
2977
+ │ ST │ Full system, requirement-based (ISTQB only) │
2978
+ ├──────────┼──────────────────────────────────────────────────┤
2979
+ │ E2E │ User journeys, UI to DB, critical paths only │
2980
+ ├──────────┼──────────────────────────────────────────────────┤
2981
+ │ AT/UAT │ Business validation by end users (ISTQB only) │
2982
+ ├──────────┴──────────────────────────────────────────────────┤
2983
+ │ Test Types (ISTQB) │
2984
+ ├─────────────────────────────────────────────────────────────┤
2985
+ │ Functional: Verify system behavior (black-box) │
2986
+ │ Non-Functional: Performance, security, usability │
2987
+ │ Structural: Code coverage, white-box techniques │
2988
+ │ Change-Related: Confirmation & regression testing │
2989
+ ├─────────────────────────────────────────────────────────────┤
2990
+ │ Test Design Techniques │
2991
+ ├─────────────────────────────────────────────────────────────┤
2992
+ │ Black-Box: EP, BVA, Decision Table, State Transition │
2993
+ │ Pairwise Testing, Use Case Testing │
2994
+ │ White-Box: Statement, Branch, Condition, MC/DC │
2995
+ │ Data Flow (All-Defs, All-Uses) │
2996
+ │ Experience: Exploratory, Error Guessing, Checklists │
2997
+ ├─────────────────────────────────────────────────────────────┤
2998
+ │ Coverage Targets │
2999
+ ├─────────────────────────────────────────────────────────────┤
3000
+ │ Line: 70% min / 85% recommended │
3001
+ │ Branch: 60% min / 80% recommended │
3002
+ │ Function: 80% min / 90% recommended │
3003
+ │ Mutation Score: ≥ 80% for critical code │
3004
+ ├─────────────────────────────────────────────────────────────┤
3005
+ │ Specialized Testing Types │
3006
+ ├─────────────────────────────────────────────────────────────┤
3007
+ │ Contract: API compatibility for microservices │
3008
+ │ Mutation: Test quality via code mutations │
3009
+ │ Chaos: Resilience via failure injection │
3010
+ │ Property-Based: Random inputs based on properties │
3011
+ │ Visual Regression: UI screenshot comparison │
3012
+ │ Accessibility: WCAG compliance validation │
3013
+ │ Installation: Deployment verification │
3014
+ │ Recovery: Failure recovery (RTO/RPO) │
3015
+ │ Configuration: Multi-config testing │
3016
+ │ Usability: User experience (SUS, NPS) │
3017
+ ├─────────────────────────────────────────────────────────────┤
3018
+ │ Test-Related Measures │
3019
+ ├─────────────────────────────────────────────────────────────┤
3020
+ │ Fault Density = Faults / KLOC (target: < 1) │
3021
+ │ Defect Leakage = Prod defects / Total defects (< 5%) │
3022
+ │ Fault Seeding: Insert known faults to measure detection │
3023
+ │ Reliability Growth: Track defect rate over time │
3024
+ └─────────────────────────────────────────────────────────────┘
3025
+ ```
3026
+
3027
+ ---
3028
+
3029
+ ## Related Standards
3030
+
3031
+ - [Test-Driven Development](test-driven-development.md) - TDD/BDD/ATDD methodology
3032
+ - [Test Completeness Dimensions](test-completeness-dimensions.md) - 7-dimension test coverage
3033
+ - [Spec-Driven Development](spec-driven-development.md) - SDD workflow integration
3034
+ - [Anti-Hallucination Standard](anti-hallucination.md)
3035
+ - [Code Check-in Standards](checkin-standards.md)
3036
+ - [Code Review Checklist](code-review-checklist.md)
3037
+ - [Commit Message Guide](commit-message-guide.md)
3038
+
3039
+ ---
3040
+
3041
+ ## Version History
3042
+
3043
+ | Version | Date | Changes |
3044
+ |---------|------|---------|
3045
+ | 2.1.0 | 2026-01-05 | Added SWEBOK v4.0 reference. New sections: Testing Fundamentals (Faults vs Failures, Oracle Problem, Theoretical Limitations, Testability, Test Adequacy), Test-Related Measures (Fault Density, Reliability Growth, Fault Seeding). Expanded: Test Design Techniques (Pairwise Testing, Data Flow Testing), Specialized Testing (Installation, Alpha/Beta, Recovery, Configuration, Back-to-Back, Usability). Updated Quick Reference Card. |
3046
+ | 2.0.0 | 2026-01-05 | Major update aligned with ISTQB CTFL v4.0 and ISO/IEC/IEEE 29119. Added: Static Testing (reviews, static analysis), Test Design Techniques (EP, BVA, Decision Table, State Transition, Use Case, Statement/Branch/Condition/MC/DC coverage), Experience-Based Testing (exploratory, error guessing, checklists), Risk-Based Testing (risk assessment, prioritization), Specialized Testing (contract, mutation, chaos, property-based, visual regression, accessibility), Defect Management (lifecycle, metrics, root cause), Test Process Management (planning, monitoring, completion) |
3047
+ | 1.3.0 | 2025-12-29 | Add: Testing Framework Selection (ISTQB vs Industry Pyramid), IT/SIT abbreviation clarification, source references |
3048
+ | 1.2.0 | 2025-12-19 | Add: Mock Limitations section, When Integration Tests Are Required, Distinct Identifiers, Composite Keys test data patterns |
3049
+ | 1.1.1 | 2025-12-11 | Improved: System test example to use generic domain concepts instead of specific business terminology |
3050
+ | 1.1.0 | 2025-12-05 | Add test environment isolation section (venv, containers) |
3051
+ | 1.0.0 | 2025-12-05 | Initial testing standards with UT/IT/ST/E2E coverage |
3052
+
3053
+ ---
3054
+
3055
+ ## License
3056
+
3057
+ This standard is released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
3058
+
3059
+ ---
3060
+
3061
+ **Maintainer**: Development Team