universal-dev-standards 4.2.0 → 5.0.0-beta.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/bin/uds.js +165 -5
- package/bundled/core/acceptance-test-driven-development.md +19 -721
- package/bundled/core/accessibility-standards.md +862 -0
- package/bundled/core/ai-agreement-standards.md +95 -0
- package/bundled/core/ai-friendly-architecture.md +2 -0
- package/bundled/core/ai-instruction-standards.md +2 -0
- package/bundled/core/anti-hallucination.md +90 -637
- package/bundled/core/behavior-driven-development.md +20 -735
- package/bundled/core/changelog-standards.md +4 -0
- package/bundled/core/checkin-standards.md +21 -4
- package/bundled/core/code-review-checklist.md +3 -0
- package/bundled/core/commit-message-guide.md +14 -3
- package/bundled/core/deployment-standards.md +296 -0
- package/bundled/core/developer-memory.md +568 -0
- package/bundled/core/documentation-structure.md +114 -18
- package/bundled/core/documentation-writing-standards.md +272 -2
- package/bundled/core/error-code-standards.md +3 -0
- package/bundled/core/forward-derivation-standards.md +134 -8
- package/bundled/core/git-workflow.md +72 -686
- package/bundled/core/guides/anti-hallucination-guide.md +517 -0
- package/bundled/core/guides/git-workflow-guide.md +954 -0
- package/bundled/core/guides/performance-guide.md +508 -0
- package/bundled/core/guides/refactoring-guide.md +608 -0
- package/bundled/core/guides/security-guide.md +485 -0
- package/bundled/core/logging-standards.md +337 -2
- package/bundled/core/performance-standards.md +231 -0
- package/bundled/core/project-context-memory.md +180 -0
- package/bundled/core/project-structure.md +315 -2
- package/bundled/core/refactoring-standards.md +81 -761
- package/bundled/core/requirement-engineering.md +39 -0
- package/bundled/core/reverse-engineering-standards.md +6 -4
- package/bundled/core/security-standards.md +244 -0
- package/bundled/core/spec-driven-development.md +20 -319
- package/bundled/core/test-completeness-dimensions.md +213 -11
- package/bundled/core/test-driven-development.md +22 -978
- package/bundled/core/testing-standards.md +222 -2903
- package/bundled/core/versioning.md +16 -55
- package/bundled/core/virtual-organization-standards.md +85 -0
- package/bundled/locales/README.md +4 -4
- package/bundled/locales/zh-CN/CHANGELOG.md +18 -18
- package/bundled/locales/zh-CN/CLAUDE.md +3 -3
- package/bundled/locales/zh-CN/MAINTENANCE.md +7 -7
- package/bundled/locales/zh-CN/README.md +2 -2
- package/bundled/locales/zh-CN/STANDARDS-MAPPING.md +1 -1
- package/bundled/locales/zh-CN/adoption/ADOPTION-GUIDE.md +3 -3
- package/bundled/locales/zh-CN/adoption/DAILY-WORKFLOW-GUIDE.md +66 -7
- package/bundled/locales/zh-CN/adoption/STATIC-DYNAMIC-GUIDE.md +22 -22
- package/bundled/locales/zh-CN/adoption/checklists/enterprise.md +5 -5
- package/bundled/locales/zh-CN/adoption/checklists/minimal.md +4 -4
- package/bundled/locales/zh-CN/adoption/checklists/recommended.md +5 -5
- package/bundled/locales/zh-CN/ai/MAINTENANCE.md +13 -13
- package/bundled/locales/zh-CN/core/ai-instruction-standards.md +1 -1
- package/bundled/locales/zh-CN/core/anti-hallucination.md +39 -3
- package/bundled/locales/zh-CN/core/checkin-standards.md +18 -5
- package/bundled/locales/zh-CN/core/commit-message-guide.md +1 -1
- package/bundled/locales/zh-CN/core/developer-memory.md +576 -0
- package/bundled/locales/zh-CN/core/documentation-structure.md +43 -5
- package/bundled/locales/zh-CN/core/documentation-writing-standards.md +146 -5
- package/bundled/locales/zh-CN/core/git-workflow.md +179 -4
- package/bundled/locales/zh-CN/core/logging-standards.md +177 -4
- package/bundled/locales/zh-CN/core/project-structure.md +117 -5
- package/bundled/locales/zh-CN/core/refactoring-standards.md +3 -3
- package/bundled/locales/zh-CN/core/reverse-engineering-standards.md +6 -6
- package/bundled/locales/zh-CN/core/spec-driven-development.md +516 -14
- package/bundled/locales/zh-CN/core/test-completeness-dimensions.md +155 -24
- package/bundled/locales/zh-CN/core/test-driven-development.md +71 -20
- package/bundled/locales/zh-CN/core/testing-standards.md +4 -4
- package/bundled/locales/zh-CN/core/versioning.md +1 -1
- package/bundled/locales/zh-CN/docs/AI-AGENT-ROADMAP.md +90 -51
- package/bundled/locales/zh-CN/docs/CHEATSHEET.md +184 -0
- package/bundled/locales/zh-CN/docs/CLI-FLOW.md +239 -0
- package/bundled/locales/zh-CN/docs/FEATURE-REFERENCE.md +320 -0
- package/bundled/locales/zh-CN/docs/OPERATION-WORKFLOW.md +43 -43
- package/bundled/locales/zh-CN/docs/STANDARDS-REFERENCE.md +440 -0
- package/bundled/locales/zh-CN/docs/USAGE-MODES-COMPARISON.md +1 -1
- package/bundled/locales/zh-CN/docs/WINDOWS-GUIDE.md +1 -1
- package/bundled/locales/zh-CN/integrations/codex/README.md +3 -3
- package/bundled/locales/zh-CN/integrations/gemini-cli/README.md +3 -3
- package/bundled/locales/zh-CN/integrations/github-copilot/README.md +5 -5
- package/bundled/locales/zh-CN/integrations/github-copilot/copilot-instructions.md +9 -9
- package/bundled/locales/zh-CN/integrations/github-copilot/skills-mapping.md +18 -6
- package/bundled/locales/zh-CN/integrations/google-antigravity/README.md +2 -2
- package/bundled/locales/zh-CN/integrations/opencode/README.md +3 -3
- package/bundled/locales/zh-CN/integrations/opencode/skills-mapping.md +22 -10
- package/bundled/locales/zh-CN/integrations/openspec/README.md +1 -1
- package/bundled/locales/zh-CN/integrations/spec-kit/AGENTS.md +3 -3
- package/bundled/locales/zh-CN/integrations/spec-kit/README.md +1 -1
- package/bundled/locales/zh-CN/options/commit-message/bilingual.md +1 -1
- package/bundled/locales/zh-CN/options/commit-message/english.md +1 -1
- package/bundled/locales/zh-CN/options/commit-message/traditional-chinese.md +1 -1
- package/bundled/locales/zh-CN/options/git-workflow/gitflow.md +1 -1
- package/bundled/locales/zh-CN/options/git-workflow/github-flow.md +1 -1
- package/bundled/locales/zh-CN/options/git-workflow/merge-commit.md +1 -1
- package/bundled/locales/zh-CN/options/git-workflow/rebase-ff.md +1 -1
- package/bundled/locales/zh-CN/options/git-workflow/squash-merge.md +1 -1
- package/bundled/locales/zh-CN/options/git-workflow/trunk-based.md +1 -1
- package/bundled/locales/zh-CN/options/project-structure/dotnet.md +1 -1
- package/bundled/locales/zh-CN/options/project-structure/go.md +1 -1
- package/bundled/locales/zh-CN/options/project-structure/java.md +1 -1
- package/bundled/locales/zh-CN/options/project-structure/nodejs.md +1 -1
- package/bundled/locales/zh-CN/options/project-structure/python.md +1 -1
- package/bundled/locales/zh-CN/options/testing/e2e-testing.md +1 -1
- package/bundled/locales/zh-CN/options/testing/integration-testing.md +1 -1
- package/bundled/locales/zh-CN/options/testing/system-testing.md +1 -1
- package/bundled/locales/zh-CN/options/testing/unit-testing.md +1 -1
- package/bundled/locales/zh-CN/skills/{claude-code/CONTRIBUTING.template.md → CONTRIBUTING.template.md} +2 -2
- package/bundled/locales/zh-CN/skills/README.md +131 -81
- package/bundled/locales/zh-CN/skills/agents/README.md +281 -0
- package/bundled/locales/zh-CN/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/SKILL.md +10 -10
- package/bundled/locales/zh-CN/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/anti-hallucination.md +8 -8
- package/bundled/locales/zh-CN/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/certainty-labels.md +8 -8
- package/bundled/locales/zh-CN/skills/ai-friendly-architecture/SKILL.md +221 -0
- package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +214 -0
- package/bundled/locales/zh-CN/skills/atdd-assistant/SKILL.md +71 -0
- package/bundled/locales/zh-CN/skills/atdd-assistant/acceptance-criteria-guide.md +428 -0
- package/bundled/locales/zh-CN/skills/atdd-assistant/atdd-workflow.md +484 -0
- package/bundled/locales/zh-CN/skills/atdd-assistant/guide.md +368 -0
- package/bundled/locales/zh-CN/skills/bdd-assistant/SKILL.md +68 -0
- package/bundled/locales/zh-CN/skills/bdd-assistant/bdd-workflow.md +406 -0
- package/bundled/locales/zh-CN/skills/bdd-assistant/gherkin-guide.md +454 -0
- package/bundled/locales/zh-CN/skills/bdd-assistant/guide.md +324 -0
- package/bundled/locales/zh-CN/skills/changelog-guide/SKILL.md +64 -0
- package/bundled/locales/zh-CN/skills/{claude-code/changelog-guide/SKILL.md → changelog-guide/guide.md} +6 -7
- package/bundled/locales/zh-CN/skills/checkin-assistant/SKILL.md +56 -0
- package/bundled/locales/zh-CN/skills/{claude-code/checkin-assistant/SKILL.md → checkin-assistant/guide.md} +6 -7
- package/bundled/locales/zh-CN/skills/code-review-assistant/SKILL.md +56 -0
- package/bundled/locales/zh-CN/skills/{claude-code/code-review-assistant → code-review-assistant}/checkin-checklist.md +4 -4
- package/bundled/locales/zh-CN/skills/{claude-code/code-review-assistant/SKILL.md → code-review-assistant/guide.md} +5 -6
- package/bundled/locales/zh-CN/skills/{claude-code/code-review-assistant → code-review-assistant}/review-checklist.md +4 -4
- package/bundled/locales/zh-CN/skills/{claude-code/commands → commands}/bdd.md +2 -2
- package/bundled/locales/zh-CN/skills/{claude-code/commands → commands}/methodology.md +3 -3
- package/bundled/locales/zh-CN/skills/{claude-code/commands → commands}/refactor.md +3 -3
- package/bundled/locales/zh-CN/skills/commit-standards/SKILL.md +61 -0
- package/bundled/locales/zh-CN/skills/{claude-code/commit-standards → commit-standards}/conventional-commits.md +4 -4
- package/bundled/locales/zh-CN/skills/{claude-code/commit-standards/SKILL.md → commit-standards/guide.md} +5 -6
- package/bundled/locales/zh-CN/skills/{claude-code/commit-standards → commit-standards}/language-options.md +3 -3
- package/bundled/locales/zh-CN/skills/docs-generator/SKILL.md +59 -0
- package/bundled/locales/zh-CN/skills/docs-generator/guide.md +262 -0
- package/bundled/locales/zh-CN/skills/{claude-code/documentation-guide → documentation-guide}/SKILL.md +7 -8
- package/bundled/locales/zh-CN/skills/{claude-code/documentation-guide → documentation-guide}/documentation-structure.md +4 -4
- package/bundled/locales/zh-CN/skills/{claude-code/documentation-guide → documentation-guide}/readme-template.md +3 -3
- package/bundled/locales/zh-CN/skills/{claude-code/error-code-guide → error-code-guide}/SKILL.md +6 -6
- package/bundled/locales/zh-CN/skills/forward-derivation/SKILL.md +62 -0
- package/bundled/locales/zh-CN/skills/forward-derivation/guide.md +130 -0
- package/bundled/locales/zh-CN/skills/{claude-code/git-workflow-guide → git-workflow-guide}/SKILL.md +6 -6
- package/bundled/locales/zh-CN/skills/{claude-code/git-workflow-guide → git-workflow-guide}/branch-naming.md +3 -3
- package/bundled/locales/zh-CN/skills/{claude-code/git-workflow-guide → git-workflow-guide}/git-workflow.md +4 -4
- package/bundled/locales/zh-CN/skills/{claude-code/logging-guide → logging-guide}/SKILL.md +6 -6
- package/bundled/locales/zh-CN/skills/methodology-system/SKILL.md +78 -0
- package/bundled/locales/zh-CN/skills/{claude-code/methodology-system → methodology-system}/create-methodology.md +2 -2
- package/bundled/locales/zh-CN/skills/{claude-code/methodology-system/SKILL.md → methodology-system/guide.md} +2 -2
- package/bundled/locales/zh-CN/skills/{claude-code/methodology-system → methodology-system}/runtime.md +3 -3
- package/bundled/locales/zh-CN/skills/project-discovery/SKILL.md +66 -0
- package/bundled/locales/zh-CN/skills/project-discovery/guide.md +371 -0
- package/bundled/locales/zh-CN/skills/{claude-code/project-structure-guide → project-structure-guide}/SKILL.md +4 -4
- package/bundled/locales/zh-CN/skills/{claude-code/project-structure-guide → project-structure-guide}/language-patterns.md +1 -1
- package/bundled/locales/zh-CN/skills/refactoring-assistant/SKILL.md +67 -0
- package/bundled/locales/zh-CN/skills/{claude-code/refactoring-assistant/SKILL.md → refactoring-assistant/guide.md} +7 -8
- package/bundled/locales/zh-CN/skills/release-standards/SKILL.md +64 -0
- package/bundled/locales/zh-CN/skills/{claude-code/release-standards → release-standards}/changelog-format.md +3 -3
- package/bundled/locales/zh-CN/skills/{claude-code/release-standards/SKILL.md → release-standards/guide.md} +5 -6
- package/bundled/locales/zh-CN/skills/{claude-code/release-standards → release-standards}/release-workflow.md +7 -7
- package/bundled/locales/zh-CN/skills/{claude-code/release-standards → release-standards}/semantic-versioning.md +3 -3
- package/bundled/locales/zh-CN/skills/requirement-assistant/SKILL.md +67 -0
- package/bundled/locales/zh-CN/skills/{claude-code/requirement-assistant/SKILL.md → requirement-assistant/guide.md} +4 -5
- package/bundled/locales/zh-CN/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-checklist.md +3 -3
- package/bundled/locales/zh-CN/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-writing.md +3 -3
- package/bundled/locales/zh-CN/skills/reverse-engineer/SKILL.md +67 -0
- package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer → reverse-engineer}/bdd-extraction.md +2 -2
- package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer/SKILL.md → reverse-engineer/guide.md} +17 -18
- package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer → reverse-engineer}/tdd-analysis.md +2 -2
- package/bundled/locales/zh-CN/skills/{claude-code/reverse-engineer → reverse-engineer}/workflow.md +3 -3
- package/bundled/locales/zh-CN/skills/spec-driven-dev/SKILL.md +75 -0
- package/bundled/locales/zh-CN/skills/{claude-code/spec-driven-dev/SKILL.md → spec-driven-dev/guide.md} +10 -11
- package/bundled/locales/zh-CN/skills/tdd-assistant/SKILL.md +66 -0
- package/bundled/locales/zh-CN/skills/{claude-code/tdd-assistant/SKILL.md → tdd-assistant/guide.md} +12 -16
- package/bundled/locales/zh-CN/skills/{claude-code/tdd-assistant → tdd-assistant}/language-examples.md +3 -3
- package/bundled/locales/zh-CN/skills/{claude-code/tdd-assistant → tdd-assistant}/tdd-workflow.md +3 -3
- package/bundled/locales/zh-CN/skills/test-coverage-assistant/SKILL.md +65 -0
- package/bundled/locales/zh-CN/skills/test-coverage-assistant/guide.md +284 -0
- package/bundled/locales/zh-CN/skills/{claude-code/testing-guide → testing-guide}/SKILL.md +8 -8
- package/bundled/locales/zh-CN/skills/{claude-code/testing-guide → testing-guide}/testing-pyramid.md +4 -4
- package/bundled/locales/zh-CN/skills/{cline → tools/cline}/README.md +1 -1
- package/bundled/locales/zh-CN/skills/{copilot → tools/copilot}/README.md +1 -1
- package/bundled/locales/zh-CN/skills/{copilot → tools/copilot}/copilot-instructions.md +1 -1
- package/bundled/locales/zh-CN/skills/{cursor → tools/cursor}/README.md +1 -1
- package/bundled/locales/zh-CN/skills/{windsurf → tools/windsurf}/README.md +1 -1
- package/bundled/locales/zh-TW/CHANGELOG.md +16 -16
- package/bundled/locales/zh-TW/CLAUDE.md +3 -3
- package/bundled/locales/zh-TW/MAINTENANCE.md +7 -7
- package/bundled/locales/zh-TW/README.md +13 -7
- package/bundled/locales/zh-TW/STANDARDS-MAPPING.md +1 -1
- package/bundled/locales/zh-TW/adoption/ADOPTION-GUIDE.md +4 -4
- package/bundled/locales/zh-TW/adoption/DAILY-WORKFLOW-GUIDE.md +66 -7
- package/bundled/locales/zh-TW/adoption/STATIC-DYNAMIC-GUIDE.md +22 -22
- package/bundled/locales/zh-TW/adoption/checklists/enterprise.md +5 -5
- package/bundled/locales/zh-TW/adoption/checklists/minimal.md +4 -4
- package/bundled/locales/zh-TW/adoption/checklists/recommended.md +5 -5
- package/bundled/locales/zh-TW/ai/MAINTENANCE.md +13 -13
- package/bundled/locales/zh-TW/ai/standards/anti-hallucination.ai.yaml +3 -2
- package/bundled/locales/zh-TW/ai/standards/git-workflow.ai.yaml +4 -3
- package/bundled/locales/zh-TW/ai/standards/spec-driven-development.ai.yaml +3 -3
- package/bundled/locales/zh-TW/ai/standards/testing.ai.yaml +1 -0
- package/bundled/locales/zh-TW/core/ai-instruction-standards.md +1 -1
- package/bundled/locales/zh-TW/core/anti-hallucination.md +38 -701
- package/bundled/locales/zh-TW/core/changelog-standards.md +1 -1
- package/bundled/locales/zh-TW/core/checkin-standards.md +18 -5
- package/bundled/locales/zh-TW/core/code-review-checklist.md +1 -1
- package/bundled/locales/zh-TW/core/commit-message-guide.md +1 -1
- package/bundled/locales/zh-TW/core/developer-memory.md +576 -0
- package/bundled/locales/zh-TW/core/documentation-structure.md +115 -22
- package/bundled/locales/zh-TW/core/documentation-writing-standards.md +126 -5
- package/bundled/locales/zh-TW/core/forward-derivation-standards.md +2 -2
- package/bundled/locales/zh-TW/core/git-workflow.md +38 -812
- package/bundled/locales/zh-TW/core/guides/anti-hallucination-guide.md +779 -0
- package/bundled/locales/zh-TW/core/guides/git-workflow-guide.md +1016 -0
- package/bundled/locales/zh-TW/core/guides/refactoring-guide.md +864 -0
- package/bundled/locales/zh-TW/core/guides/testing-guide.md +2280 -0
- package/bundled/locales/zh-TW/core/logging-standards.md +159 -4
- package/bundled/locales/zh-TW/core/project-structure.md +102 -9
- package/bundled/locales/zh-TW/core/refactoring-standards.md +37 -823
- package/bundled/locales/zh-TW/core/reverse-engineering-standards.md +6 -6
- package/bundled/locales/zh-TW/core/spec-driven-development.md +17 -268
- package/bundled/locales/zh-TW/core/test-completeness-dimensions.md +128 -14
- package/bundled/locales/zh-TW/core/test-driven-development.md +20 -983
- package/bundled/locales/zh-TW/core/testing-standards.md +120 -2105
- package/bundled/locales/zh-TW/core/versioning.md +13 -55
- package/bundled/locales/zh-TW/docs/AI-AGENT-ROADMAP.md +90 -51
- package/bundled/locales/zh-TW/docs/CHEATSHEET.md +184 -0
- package/bundled/locales/zh-TW/docs/CLI-FLOW.md +239 -0
- package/bundled/locales/zh-TW/docs/FEATURE-REFERENCE.md +320 -0
- package/bundled/locales/zh-TW/docs/MIGRATION-V5.md +71 -0
- package/bundled/locales/zh-TW/docs/OPERATION-WORKFLOW.md +42 -42
- package/bundled/locales/zh-TW/docs/STANDARDS-REFERENCE.md +440 -0
- package/bundled/locales/zh-TW/docs/USAGE-MODES-COMPARISON.md +1 -1
- package/bundled/locales/zh-TW/docs/WINDOWS-GUIDE.md +1 -1
- package/bundled/locales/zh-TW/integrations/claude-code/README.md +44 -0
- package/bundled/locales/zh-TW/integrations/github-copilot/copilot-instructions.md +3 -3
- package/bundled/locales/zh-TW/integrations/github-copilot/skills-mapping.md +18 -6
- package/bundled/locales/zh-TW/integrations/opencode/skills-mapping.md +22 -10
- package/bundled/locales/zh-TW/methodologies/guides/sdd-guide.md +338 -0
- package/bundled/locales/zh-TW/methodologies/guides/tdd-guide.md +1055 -0
- package/bundled/locales/zh-TW/options/commit-message/bilingual.md +1 -1
- package/bundled/locales/zh-TW/options/commit-message/english.md +1 -1
- package/bundled/locales/zh-TW/options/commit-message/traditional-chinese.md +1 -1
- package/bundled/locales/zh-TW/options/git-workflow/gitflow.md +1 -1
- package/bundled/locales/zh-TW/options/git-workflow/github-flow.md +1 -1
- package/bundled/locales/zh-TW/options/git-workflow/merge-commit.md +1 -1
- package/bundled/locales/zh-TW/options/git-workflow/rebase-ff.md +1 -1
- package/bundled/locales/zh-TW/options/git-workflow/squash-merge.md +1 -1
- package/bundled/locales/zh-TW/options/git-workflow/trunk-based.md +1 -1
- package/bundled/locales/zh-TW/options/project-structure/dotnet.md +1 -1
- package/bundled/locales/zh-TW/options/project-structure/go.md +1 -1
- package/bundled/locales/zh-TW/options/project-structure/java.md +1 -1
- package/bundled/locales/zh-TW/options/project-structure/nodejs.md +1 -1
- package/bundled/locales/zh-TW/options/project-structure/python.md +1 -1
- package/bundled/locales/zh-TW/options/testing/e2e-testing.md +1 -1
- package/bundled/locales/zh-TW/options/testing/integration-testing.md +1 -1
- package/bundled/locales/zh-TW/options/testing/system-testing.md +1 -1
- package/bundled/locales/zh-TW/options/testing/unit-testing.md +1 -1
- package/bundled/locales/zh-TW/skills/{claude-code/CONTRIBUTING.template.md → CONTRIBUTING.template.md} +2 -2
- package/bundled/locales/zh-TW/skills/README.md +131 -79
- package/bundled/locales/zh-TW/skills/agents/README.md +281 -0
- package/bundled/locales/zh-TW/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/SKILL.md +7 -7
- package/bundled/locales/zh-TW/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/anti-hallucination.md +8 -8
- package/bundled/locales/zh-TW/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/certainty-labels.md +8 -8
- package/bundled/locales/zh-TW/skills/ai-friendly-architecture/SKILL.md +221 -0
- package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +214 -0
- package/bundled/locales/zh-TW/skills/atdd-assistant/SKILL.md +71 -0
- package/bundled/locales/zh-TW/skills/atdd-assistant/acceptance-criteria-guide.md +428 -0
- package/bundled/locales/zh-TW/skills/atdd-assistant/atdd-workflow.md +484 -0
- package/bundled/locales/zh-TW/skills/atdd-assistant/guide.md +368 -0
- package/bundled/locales/zh-TW/skills/bdd-assistant/SKILL.md +68 -0
- package/bundled/locales/zh-TW/skills/bdd-assistant/bdd-workflow.md +406 -0
- package/bundled/locales/zh-TW/skills/bdd-assistant/gherkin-guide.md +454 -0
- package/bundled/locales/zh-TW/skills/bdd-assistant/guide.md +324 -0
- package/bundled/locales/zh-TW/skills/changelog-guide/SKILL.md +64 -0
- package/bundled/locales/zh-TW/skills/{claude-code/changelog-guide/SKILL.md → changelog-guide/guide.md} +6 -7
- package/bundled/locales/zh-TW/skills/checkin-assistant/SKILL.md +56 -0
- package/bundled/locales/zh-TW/skills/{claude-code/checkin-assistant/SKILL.md → checkin-assistant/guide.md} +6 -7
- package/bundled/locales/zh-TW/skills/code-review-assistant/SKILL.md +56 -0
- package/bundled/locales/zh-TW/skills/{claude-code/code-review-assistant → code-review-assistant}/checkin-checklist.md +4 -4
- package/bundled/locales/zh-TW/skills/{claude-code/code-review-assistant/SKILL.md → code-review-assistant/guide.md} +2 -3
- package/bundled/locales/zh-TW/skills/{claude-code/code-review-assistant → code-review-assistant}/review-checklist.md +4 -4
- package/bundled/locales/zh-TW/skills/{claude-code/commands → commands}/bdd.md +2 -2
- package/bundled/locales/zh-TW/skills/{claude-code/commands → commands}/methodology.md +3 -3
- package/bundled/locales/zh-TW/skills/{claude-code/commands → commands}/refactor.md +3 -3
- package/bundled/locales/zh-TW/skills/commit-standards/SKILL.md +61 -0
- package/bundled/locales/zh-TW/skills/{claude-code/commit-standards → commit-standards}/conventional-commits.md +4 -4
- package/bundled/locales/zh-TW/skills/{claude-code/commit-standards/SKILL.md → commit-standards/guide.md} +5 -6
- package/bundled/locales/zh-TW/skills/{claude-code/commit-standards → commit-standards}/language-options.md +3 -3
- package/bundled/locales/zh-TW/skills/docs-generator/SKILL.md +59 -0
- package/bundled/locales/zh-TW/skills/docs-generator/guide.md +262 -0
- package/bundled/locales/zh-TW/skills/{claude-code/documentation-guide → documentation-guide}/SKILL.md +7 -8
- package/bundled/locales/zh-TW/skills/{claude-code/documentation-guide → documentation-guide}/documentation-structure.md +4 -4
- package/bundled/locales/zh-TW/skills/{claude-code/documentation-guide → documentation-guide}/readme-template.md +3 -3
- package/bundled/locales/zh-TW/skills/{claude-code/error-code-guide → error-code-guide}/SKILL.md +6 -6
- package/bundled/locales/zh-TW/skills/forward-derivation/SKILL.md +62 -0
- package/bundled/locales/zh-TW/skills/{claude-code/forward-derivation/SKILL.md → forward-derivation/guide.md} +24 -19
- package/bundled/locales/zh-TW/skills/{claude-code/git-workflow-guide → git-workflow-guide}/SKILL.md +3 -3
- package/bundled/locales/zh-TW/skills/{claude-code/git-workflow-guide → git-workflow-guide}/branch-naming.md +3 -3
- package/bundled/locales/zh-TW/skills/{claude-code/git-workflow-guide → git-workflow-guide}/git-workflow.md +4 -4
- package/bundled/locales/zh-TW/skills/{claude-code/logging-guide → logging-guide}/SKILL.md +6 -6
- package/bundled/locales/zh-TW/skills/methodology-system/SKILL.md +78 -0
- package/bundled/locales/zh-TW/skills/{claude-code/methodology-system → methodology-system}/create-methodology.md +2 -2
- package/bundled/locales/zh-TW/skills/{claude-code/methodology-system/SKILL.md → methodology-system/guide.md} +2 -2
- package/bundled/locales/zh-TW/skills/{claude-code/methodology-system → methodology-system}/runtime.md +3 -3
- package/bundled/locales/zh-TW/skills/project-discovery/SKILL.md +66 -0
- package/bundled/locales/zh-TW/skills/project-discovery/guide.md +371 -0
- package/bundled/locales/zh-TW/skills/project-discovery/workflow.md +574 -0
- package/bundled/locales/zh-TW/skills/{claude-code/project-structure-guide → project-structure-guide}/SKILL.md +4 -4
- package/bundled/locales/zh-TW/skills/{claude-code/project-structure-guide → project-structure-guide}/language-patterns.md +1 -1
- package/bundled/locales/zh-TW/skills/refactoring-assistant/SKILL.md +67 -0
- package/bundled/locales/zh-TW/skills/{claude-code/refactoring-assistant/SKILL.md → refactoring-assistant/guide.md} +7 -8
- package/bundled/locales/zh-TW/skills/release-standards/SKILL.md +64 -0
- package/bundled/locales/zh-TW/skills/{claude-code/release-standards → release-standards}/changelog-format.md +3 -3
- package/bundled/locales/zh-TW/skills/{claude-code/release-standards/SKILL.md → release-standards/guide.md} +2 -3
- package/bundled/locales/zh-TW/skills/{claude-code/release-standards → release-standards}/release-workflow.md +60 -28
- package/bundled/locales/zh-TW/skills/{claude-code/release-standards → release-standards}/semantic-versioning.md +3 -3
- package/bundled/locales/zh-TW/skills/requirement-assistant/SKILL.md +67 -0
- package/bundled/locales/zh-TW/skills/{claude-code/requirement-assistant/SKILL.md → requirement-assistant/guide.md} +2 -3
- package/bundled/locales/zh-TW/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-checklist.md +3 -3
- package/bundled/locales/zh-TW/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-writing.md +3 -3
- package/bundled/locales/zh-TW/skills/reverse-engineer/SKILL.md +67 -0
- package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer → reverse-engineer}/bdd-extraction.md +2 -2
- package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer/SKILL.md → reverse-engineer/guide.md} +17 -18
- package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer → reverse-engineer}/tdd-analysis.md +2 -2
- package/bundled/locales/zh-TW/skills/{claude-code/reverse-engineer → reverse-engineer}/workflow.md +3 -3
- package/bundled/locales/zh-TW/skills/spec-driven-dev/SKILL.md +75 -0
- package/bundled/locales/zh-TW/skills/{claude-code/spec-driven-dev/SKILL.md → spec-driven-dev/guide.md} +9 -10
- package/bundled/locales/zh-TW/skills/tdd-assistant/SKILL.md +66 -0
- package/bundled/locales/zh-TW/skills/{claude-code/tdd-assistant/SKILL.md → tdd-assistant/guide.md} +9 -10
- package/bundled/locales/zh-TW/skills/{claude-code/tdd-assistant → tdd-assistant}/language-examples.md +3 -3
- package/bundled/locales/zh-TW/skills/{claude-code/tdd-assistant → tdd-assistant}/tdd-workflow.md +3 -3
- package/bundled/locales/zh-TW/skills/test-coverage-assistant/SKILL.md +65 -0
- package/bundled/locales/zh-TW/skills/{claude-code/test-coverage-assistant/SKILL.md → test-coverage-assistant/guide.md} +27 -24
- package/bundled/locales/zh-TW/skills/{claude-code/testing-guide → testing-guide}/SKILL.md +6 -6
- package/bundled/locales/zh-TW/skills/{claude-code/testing-guide → testing-guide}/testing-pyramid.md +4 -4
- package/bundled/locales/zh-TW/skills/{cline → tools/cline}/README.md +1 -1
- package/bundled/locales/zh-TW/skills/{copilot → tools/copilot}/README.md +1 -1
- package/bundled/locales/zh-TW/skills/{copilot → tools/copilot}/copilot-instructions.md +1 -1
- package/bundled/locales/zh-TW/skills/{cursor → tools/cursor}/README.md +1 -1
- package/bundled/locales/zh-TW/skills/{windsurf → tools/windsurf}/README.md +1 -1
- package/bundled/skills/INTEGRATION-GUIDE.md +210 -0
- package/bundled/skills/README.md +95 -0
- package/bundled/skills/_shared/README.md +60 -0
- package/bundled/skills/{claude-code/agents → agents}/README.md +2 -2
- package/bundled/skills/{claude-code/agents → agents}/code-architect.md +1 -1
- package/bundled/skills/{claude-code/agents → agents}/doc-writer.md +1 -1
- package/bundled/skills/{claude-code/agents → agents}/reviewer.md +1 -1
- package/bundled/skills/{claude-code/agents → agents}/spec-analyst.md +1 -1
- package/bundled/skills/{claude-code/agents → agents}/test-specialist.md +1 -1
- package/bundled/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/SKILL.md +28 -5
- package/bundled/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/anti-hallucination.md +24 -5
- package/bundled/skills/{claude-code/ai-collaboration-standards → ai-collaboration-standards}/certainty-labels.md +31 -6
- package/bundled/skills/ai-friendly-architecture/SKILL.md +216 -0
- package/bundled/skills/ai-instruction-standards/SKILL.md +209 -0
- package/bundled/skills/atdd-assistant/SKILL.md +74 -0
- package/bundled/skills/{claude-code/atdd-assistant → atdd-assistant}/acceptance-criteria-guide.md +1 -1
- package/bundled/skills/{claude-code/atdd-assistant → atdd-assistant}/atdd-workflow.md +1 -1
- package/bundled/skills/{claude-code/atdd-assistant/SKILL.md → atdd-assistant/guide.md} +11 -11
- package/bundled/skills/bdd-assistant/SKILL.md +70 -0
- package/bundled/skills/{claude-code/bdd-assistant → bdd-assistant}/bdd-workflow.md +1 -1
- package/bundled/skills/{claude-code/bdd-assistant → bdd-assistant}/gherkin-guide.md +1 -1
- package/bundled/skills/{claude-code/bdd-assistant/SKILL.md → bdd-assistant/guide.md} +11 -11
- package/bundled/skills/changelog-guide/SKILL.md +66 -0
- package/bundled/skills/{claude-code/changelog-guide/SKILL.md → changelog-guide/guide.md} +8 -6
- package/bundled/skills/checkin-assistant/SKILL.md +52 -0
- package/bundled/skills/{claude-code/checkin-assistant/SKILL.md → checkin-assistant/guide.md} +6 -6
- package/bundled/skills/code-review-assistant/SKILL.md +53 -0
- package/bundled/skills/{claude-code/code-review-assistant → code-review-assistant}/checkin-checklist.md +3 -3
- package/bundled/skills/{claude-code/code-review-assistant/SKILL.md → code-review-assistant/guide.md} +7 -5
- package/bundled/skills/{claude-code/code-review-assistant → code-review-assistant}/review-checklist.md +3 -3
- package/bundled/skills/commands/COMMAND-FAMILY-OVERVIEW.md +290 -0
- package/bundled/skills/{claude-code/commands → commands}/README.md +13 -3
- package/bundled/skills/{claude-code/commands → commands}/atdd.md +3 -3
- package/bundled/skills/{claude-code/commands → commands}/bdd.md +2 -2
- package/bundled/skills/commands/changelog.md +58 -0
- package/bundled/skills/{claude-code/commands → commands}/check.md +1 -1
- package/bundled/skills/commands/checkin.md +53 -0
- package/bundled/skills/{claude-code/commands → commands}/commit.md +3 -3
- package/bundled/skills/{claude-code/commands → commands}/config.md +1 -1
- package/bundled/skills/{claude-code/commands → commands}/coverage.md +5 -4
- package/bundled/skills/commands/derive-all.md +53 -0
- package/bundled/skills/commands/derive-atdd.md +49 -0
- package/bundled/skills/commands/derive-bdd.md +47 -0
- package/bundled/skills/commands/derive-tdd.md +48 -0
- package/bundled/skills/commands/derive.md +72 -0
- package/bundled/skills/commands/discover.md +80 -0
- package/bundled/skills/commands/docgen.md +61 -0
- package/bundled/skills/commands/docs.md +59 -0
- package/bundled/skills/commands/guide.md +44 -0
- package/bundled/skills/{claude-code/commands → commands}/init.md +2 -2
- package/bundled/skills/{claude-code/commands → commands}/methodology.md +32 -16
- package/bundled/skills/{claude-code/commands → commands}/refactor.md +2 -2
- package/bundled/skills/commands/release.md +56 -0
- package/bundled/skills/{claude-code/commands → commands}/requirement.md +3 -3
- package/bundled/skills/commands/reverse-bdd.md +47 -0
- package/bundled/skills/commands/reverse-sdd.md +51 -0
- package/bundled/skills/commands/reverse-tdd.md +51 -0
- package/bundled/skills/commands/reverse.md +63 -0
- package/bundled/skills/{claude-code/commands → commands}/review.md +3 -3
- package/bundled/skills/commands/sdd.md +135 -0
- package/bundled/skills/{claude-code/commands → commands}/tdd.md +3 -3
- package/bundled/skills/{claude-code/commands → commands}/update.md +2 -2
- package/bundled/skills/commit-standards/SKILL.md +60 -0
- package/bundled/skills/{claude-code/commit-standards → commit-standards}/conventional-commits.md +3 -3
- package/bundled/skills/{claude-code/commit-standards/SKILL.md → commit-standards/guide.md} +17 -5
- package/bundled/skills/{claude-code/commit-standards → commit-standards}/language-options.md +2 -2
- package/bundled/skills/docs-generator/SKILL.md +69 -0
- package/bundled/skills/docs-generator/guide.md +258 -0
- package/bundled/skills/{claude-code/documentation-guide → documentation-guide}/SKILL.md +7 -7
- package/bundled/skills/{claude-code/documentation-guide → documentation-guide}/documentation-structure.md +3 -3
- package/bundled/skills/{claude-code/documentation-guide → documentation-guide}/readme-template.md +2 -2
- package/bundled/skills/{claude-code/error-code-guide → error-code-guide}/SKILL.md +7 -5
- package/bundled/skills/forward-derivation/SKILL.md +61 -0
- package/bundled/skills/{claude-code/forward-derivation/SKILL.md → forward-derivation/guide.md} +44 -29
- package/bundled/skills/{claude-code/git-workflow-guide → git-workflow-guide}/SKILL.md +7 -5
- package/bundled/skills/{claude-code/git-workflow-guide → git-workflow-guide}/branch-naming.md +2 -2
- package/bundled/skills/{claude-code/git-workflow-guide → git-workflow-guide}/git-workflow.md +3 -3
- package/bundled/skills/{claude-code/logging-guide → logging-guide}/SKILL.md +7 -5
- package/bundled/skills/methodology-system/SKILL.md +75 -0
- package/bundled/skills/{claude-code/methodology-system → methodology-system}/create-methodology.md +1 -1
- package/bundled/skills/{claude-code/methodology-system/SKILL.md → methodology-system/guide.md} +51 -16
- package/bundled/skills/methodology-system/integrated-flow.md +454 -0
- package/bundled/skills/{claude-code/methodology-system → methodology-system}/runtime.md +2 -2
- package/bundled/skills/project-discovery/SKILL.md +63 -0
- package/bundled/skills/project-discovery/guide.md +367 -0
- package/bundled/skills/project-discovery/workflow.md +566 -0
- package/bundled/skills/{claude-code/project-structure-guide → project-structure-guide}/SKILL.md +14 -4
- package/bundled/skills/refactoring-assistant/SKILL.md +78 -0
- package/bundled/skills/{claude-code/refactoring-assistant/SKILL.md → refactoring-assistant/guide.md} +7 -7
- package/bundled/skills/release-standards/SKILL.md +61 -0
- package/bundled/skills/{claude-code/release-standards → release-standards}/changelog-format.md +2 -2
- package/bundled/skills/{claude-code/release-standards/SKILL.md → release-standards/guide.md} +7 -5
- package/bundled/skills/{claude-code/release-standards → release-standards}/release-workflow.md +59 -24
- package/bundled/skills/{claude-code/release-standards → release-standards}/semantic-versioning.md +2 -2
- package/bundled/skills/requirement-assistant/SKILL.md +66 -0
- package/bundled/skills/{claude-code/requirement-assistant/SKILL.md → requirement-assistant/guide.md} +14 -6
- package/bundled/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-checklist.md +2 -2
- package/bundled/skills/{claude-code/requirement-assistant → requirement-assistant}/requirement-writing.md +2 -2
- package/bundled/skills/reverse-engineer/SKILL.md +64 -0
- package/bundled/skills/{claude-code/reverse-engineer → reverse-engineer}/bdd-extraction.md +1 -1
- package/bundled/skills/{claude-code/reverse-engineer/SKILL.md → reverse-engineer/guide.md} +21 -18
- package/bundled/skills/{claude-code/reverse-engineer → reverse-engineer}/tdd-analysis.md +1 -1
- package/bundled/skills/{claude-code/reverse-engineer → reverse-engineer}/workflow.md +2 -2
- package/bundled/skills/spec-driven-dev/SKILL.md +80 -0
- package/bundled/skills/{claude-code/spec-driven-dev/SKILL.md → spec-driven-dev/guide.md} +50 -8
- package/bundled/skills/tdd-assistant/SKILL.md +65 -0
- package/bundled/skills/{claude-code/tdd-assistant/SKILL.md → tdd-assistant/guide.md} +10 -10
- package/bundled/skills/{claude-code/tdd-assistant → tdd-assistant}/language-examples.md +3 -3
- package/bundled/skills/{claude-code/tdd-assistant → tdd-assistant}/tdd-workflow.md +2 -2
- package/bundled/skills/test-coverage-assistant/SKILL.md +63 -0
- package/bundled/skills/{claude-code/test-coverage-assistant/SKILL.md → test-coverage-assistant/guide.md} +39 -24
- package/bundled/skills/{claude-code/testing-guide → testing-guide}/SKILL.md +12 -7
- package/bundled/skills/{claude-code/testing-guide → testing-guide}/testing-pyramid.md +3 -3
- package/bundled/skills/testing-guide/testing-theory.md +2291 -0
- package/bundled/skills/tools/cline/.clinerules +79 -0
- package/bundled/skills/tools/cline/README.md +50 -0
- package/bundled/skills/tools/copilot/README.md +99 -0
- package/bundled/skills/tools/copilot/copilot-instructions.md +71 -0
- package/bundled/skills/tools/cursor/.cursorrules +101 -0
- package/bundled/skills/tools/cursor/README.md +50 -0
- package/bundled/skills/tools/windsurf/.windsurfrules +76 -0
- package/bundled/skills/tools/windsurf/README.md +51 -0
- package/bundled/skills/{claude-code/workflows → workflows}/README.md +1 -1
- package/bundled/skills/workflows/release.workflow.yaml +893 -0
- package/package.json +4 -1
- package/src/commands/check.js +195 -130
- package/src/commands/config.js +1262 -0
- package/src/commands/configure.js +3 -950
- package/src/commands/fix.js +53 -0
- package/src/commands/hitl.js +19 -0
- package/src/commands/init.js +320 -1030
- package/src/commands/list.js +1 -1
- package/src/commands/simulate.js +61 -0
- package/src/commands/spec.js +288 -0
- package/src/commands/start.js +373 -0
- package/src/commands/sweep.js +142 -0
- package/src/commands/update.js +136 -19
- package/src/commands/workflow.js +265 -1
- package/src/config/ai-agent-paths.js +89 -29
- package/src/core/constants.js +7 -1
- package/src/core/manifest.js +1 -0
- package/src/flows/init-flow.js +243 -0
- package/src/hitl/checkpoint.js +80 -0
- package/src/hitl/classifier.js +54 -0
- package/src/hitl/manager.js +88 -0
- package/src/i18n/messages.js +907 -99
- package/src/index.js +1 -0
- package/src/installers/integration-installer.js +237 -0
- package/src/installers/manifest-installer.js +131 -0
- package/src/installers/skills-installer.js +286 -0
- package/src/installers/standards-installer.js +193 -0
- package/src/missions/MissionManager.js +501 -0
- package/src/prompts/init.js +228 -130
- package/src/prompts/integrations.js +13 -1
- package/src/schemas/standard.schema.json +117 -0
- package/src/utils/agents-installer.js +2 -2
- package/src/utils/config-loader.js +47 -0
- package/src/utils/config-manager.js +108 -0
- package/src/utils/config-merger.js +47 -0
- package/src/utils/github.js +22 -18
- package/src/utils/inquirer-patch.js +98 -0
- package/src/utils/integration-generator.js +392 -58
- package/src/utils/reference-sync.js +69 -19
- package/src/utils/skills-installer.js +6 -3
- package/src/utils/standard-fixer.js +141 -0
- package/src/utils/standard-validator.js +329 -0
- package/src/utils/workflow-executor.js +886 -0
- package/src/utils/workflow-state.js +404 -0
- package/src/utils/workflows-installer.js +2 -2
- package/src/vibe/auto-sweep.js +349 -0
- package/src/vibe/micro-spec.js +428 -0
- package/standards-registry.json +278 -102
- package/bundled/locales/zh-CN/skills/claude-code/README.md +0 -174
- package/bundled/locales/zh-CN/skills/claude-code/test-coverage-assistant/SKILL.md +0 -282
- package/bundled/locales/zh-TW/skills/claude-code/README.md +0 -174
- package/bundled/skills/claude-code/README.md +0 -215
- package/bundled/skills/claude-code/commands/changelog.md +0 -57
- package/bundled/skills/claude-code/commands/derive-all.md +0 -214
- package/bundled/skills/claude-code/commands/derive-atdd.md +0 -177
- package/bundled/skills/claude-code/commands/derive-bdd.md +0 -158
- package/bundled/skills/claude-code/commands/derive-tdd.md +0 -259
- package/bundled/skills/claude-code/commands/docs.md +0 -75
- package/bundled/skills/claude-code/commands/release.md +0 -50
- package/bundled/skills/claude-code/commands/reverse-bdd.md +0 -264
- package/bundled/skills/claude-code/commands/reverse-spec.md +0 -185
- package/bundled/skills/claude-code/commands/reverse-tdd.md +0 -351
- package/bundled/skills/claude-code/commands/spec.md +0 -69
- package/bundled/skills/claude-code/methodology-system/integrated-flow.md +0 -322
- /package/bundled/skills/{claude-code/CONTRIBUTING.template.md → CONTRIBUTING.template.md} +0 -0
- /package/bundled/skills/{claude-code/ai → ai}/standards/checkin.ai.yaml +0 -0
- /package/bundled/skills/{claude-code/ai → ai}/standards/commit.ai.yaml +0 -0
- /package/bundled/skills/{claude-code/ai → ai}/standards/refactoring.ai.yaml +0 -0
- /package/bundled/skills/{claude-code/ai → ai}/standards/testing.ai.yaml +0 -0
- /package/bundled/skills/{claude-code/project-structure-guide → project-structure-guide}/language-patterns.md +0 -0
- /package/bundled/skills/{claude-code/workflows → workflows}/code-review.workflow.yaml +0 -0
- /package/bundled/skills/{claude-code/workflows → workflows}/feature-dev.workflow.yaml +0 -0
- /package/bundled/skills/{claude-code/workflows → workflows}/integrated-flow.workflow.yaml +0 -0
- /package/bundled/skills/{claude-code/workflows → workflows}/large-codebase-analysis.workflow.yaml +0 -0
|
@@ -2,2452 +2,216 @@
|
|
|
2
2
|
|
|
3
3
|
> **Language**: English | [繁體中文](../locales/zh-TW/core/testing-standards.md)
|
|
4
4
|
|
|
5
|
-
**Version**:
|
|
6
|
-
**Last Updated**: 2026-01-
|
|
5
|
+
**Version**: 3.0.0
|
|
6
|
+
**Last Updated**: 2026-01-29
|
|
7
7
|
**Applicability**: All software projects
|
|
8
|
+
**Scope**: universal
|
|
9
|
+
**Industry Standards**: ISTQB CTFL v4.0, ISO/IEC/IEEE 29119
|
|
10
|
+
**References**: [istqb.org](https://istqb.org/)
|
|
8
11
|
|
|
9
12
|
---
|
|
10
13
|
|
|
11
14
|
## Purpose
|
|
12
15
|
|
|
13
|
-
This standard defines testing conventions and
|
|
16
|
+
This standard defines actionable testing rules and conventions for AI agents and developers. For theoretical foundations, educational content, and detailed examples, see [Testing Theory Knowledge Base](../skills/testing-guide/testing-theory.md).
|
|
14
17
|
|
|
15
18
|
**Reference Standards**:
|
|
16
19
|
- [ISTQB CTFL v4.0](https://istqb.org/certifications/certified-tester-foundation-level-ctfl-v4-0/)
|
|
17
20
|
- [ISO/IEC/IEEE 29119](https://www.iso.org/standard/81291.html)
|
|
18
|
-
- [SWEBOK v4.0](https://www.computer.org/education/bodies-of-knowledge/software-engineering)
|
|
21
|
+
- [SWEBOK v4.0](https://www.computer.org/education/bodies-of-knowledge/software-engineering)
|
|
19
22
|
|
|
20
23
|
---
|
|
21
24
|
|
|
22
|
-
##
|
|
23
|
-
|
|
24
|
-
1. [Testing Fundamentals](#testing-fundamentals) ⭐ NEW
|
|
25
|
-
2. [Testing Framework Selection](#testing-framework-selection)
|
|
26
|
-
3. [Testing Pyramid](#testing-pyramid-default-industry-standard)
|
|
27
|
-
4. [Test Levels](#unit-testing-ut) (UT, IT, ST, E2E)
|
|
28
|
-
5. [Static Testing](#static-testing)
|
|
29
|
-
6. [Test Design Techniques](#test-design-techniques)
|
|
30
|
-
7. [Experience-Based Testing](#experience-based-testing)
|
|
31
|
-
8. [Risk-Based Testing](#risk-based-testing)
|
|
32
|
-
9. [Specialized Testing](#specialized-testing)
|
|
33
|
-
10. [Test-Related Measures](#test-related-measures) ⭐ NEW
|
|
34
|
-
11. [Defect Management](#defect-management)
|
|
35
|
-
12. [Test Process Management](#test-process-management)
|
|
36
|
-
13. [Test Doubles](#test-doubles)
|
|
37
|
-
14. [Test Data Management](#test-data-management)
|
|
38
|
-
15. [Test Environment Isolation](#test-environment-isolation)
|
|
39
|
-
16. [CI/CD Integration](#cicd-integration)
|
|
40
|
-
17. [Best Practices](#best-practices)
|
|
41
|
-
18. [Test Documentation Structure](#test-documentation-structure) ⭐ NEW
|
|
25
|
+
## Glossary
|
|
42
26
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
**
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
```
|
|
54
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
55
|
-
│ Faults, Errors, and Failures │
|
|
56
|
-
├─────────────────────────────────────────────────────────────┤
|
|
57
|
-
│ │
|
|
58
|
-
│ Error (Mistake) │
|
|
59
|
-
│ │ │
|
|
60
|
-
│ ▼ introduced during development │
|
|
61
|
-
│ ┌─────────┐ │
|
|
62
|
-
│ │ Fault │ (Defect/Bug in the code) │
|
|
63
|
-
│ └─────────┘ │
|
|
64
|
-
│ │ │
|
|
65
|
-
│ ▼ when executed │
|
|
66
|
-
│ ┌─────────┐ │
|
|
67
|
-
│ │ Failure │ (Observable incorrect behavior) │
|
|
68
|
-
│ └─────────┘ │
|
|
69
|
-
│ │
|
|
70
|
-
│ Key Points: │
|
|
71
|
-
│ • A fault may exist without causing a failure │
|
|
72
|
-
│ • A failure requires a fault to be executed │
|
|
73
|
-
│ • Multiple faults can cause the same failure │
|
|
74
|
-
│ • Testing finds failures; debugging finds faults │
|
|
75
|
-
│ │
|
|
76
|
-
└─────────────────────────────────────────────────────────────┘
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
### The Oracle Problem
|
|
80
|
-
|
|
81
|
-
An **oracle** is any mechanism that determines whether observed program behavior is correct. The oracle problem addresses the challenge of knowing expected outcomes.
|
|
82
|
-
|
|
83
|
-
```
|
|
84
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
85
|
-
│ Oracle Types │
|
|
86
|
-
├─────────────────────────────────────────────────────────────┤
|
|
87
|
-
│ │
|
|
88
|
-
│ Human Oracle │
|
|
89
|
-
│ └── Manual verification by domain expert │
|
|
90
|
-
│ │
|
|
91
|
-
│ Specification-Based Oracle │
|
|
92
|
-
│ └── Compare against formal requirements │
|
|
93
|
-
│ │
|
|
94
|
-
│ Reference Implementation │
|
|
95
|
-
│ └── Compare against known-correct implementation │
|
|
96
|
-
│ │
|
|
97
|
-
│ Metamorphic Oracle │
|
|
98
|
-
│ └── Verify relationships between outputs │
|
|
99
|
-
│ (e.g., sort(sort(x)) == sort(x)) │
|
|
100
|
-
│ │
|
|
101
|
-
│ Statistical Oracle │
|
|
102
|
-
│ └── Validate against expected distributions │
|
|
103
|
-
│ │
|
|
104
|
-
│ Implicit Oracle │
|
|
105
|
-
│ └── No crash, no exception, no timeout │
|
|
106
|
-
│ │
|
|
107
|
-
└─────────────────────────────────────────────────────────────┘
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
### Theoretical Limitations of Testing
|
|
111
|
-
|
|
112
|
-
> "Program testing can be used to show the presence of bugs, but never to show their absence." — Edsger W. Dijkstra
|
|
113
|
-
|
|
114
|
-
```
|
|
115
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
116
|
-
│ Testing Limitations │
|
|
117
|
-
├─────────────────────────────────────────────────────────────┤
|
|
118
|
-
│ │
|
|
119
|
-
│ Exhaustive Testing is Impossible │
|
|
120
|
-
│ ├── Infinite input domain │
|
|
121
|
-
│ ├── State space explosion │
|
|
122
|
-
│ └── Timing and concurrency variations │
|
|
123
|
-
│ │
|
|
124
|
-
│ Infeasible Paths │
|
|
125
|
-
│ ├── Some code paths cannot be executed by any input │
|
|
126
|
-
│ ├── Dead code may appear reachable in CFG │
|
|
127
|
-
│ └── 100% path coverage may be mathematically impossible │
|
|
128
|
-
│ │
|
|
129
|
-
│ Halting Problem │
|
|
130
|
-
│ └── Cannot algorithmically determine if all tests halt │
|
|
131
|
-
│ │
|
|
132
|
-
│ Implications: │
|
|
133
|
-
│ • Testing is sampling, not proof │
|
|
134
|
-
│ • Risk-based prioritization is essential │
|
|
135
|
-
│ • Combine testing with formal methods where critical │
|
|
136
|
-
│ │
|
|
137
|
-
└─────────────────────────────────────────────────────────────┘
|
|
138
|
-
```
|
|
139
|
-
|
|
140
|
-
### Testability
|
|
141
|
-
|
|
142
|
-
Testability measures how easily software can be tested effectively.
|
|
143
|
-
|
|
144
|
-
| Factor | Description | How to Improve |
|
|
145
|
-
|--------|-------------|----------------|
|
|
146
|
-
| **Controllability** | Ability to set system state for testing | Provide test hooks, dependency injection |
|
|
147
|
-
| **Observability** | Ability to observe test results | Add logging, expose internal state |
|
|
148
|
-
| **Isolability** | Ability to test components independently | Modular design, clear interfaces |
|
|
149
|
-
| **Understandability** | Clarity of expected behavior | Good documentation, clear specifications |
|
|
150
|
-
| **Stability** | Low rate of change | Freeze features before testing phase |
|
|
151
|
-
| **Simplicity** | Low complexity | Reduce cyclomatic complexity |
|
|
152
|
-
|
|
153
|
-
### Test Adequacy Criteria
|
|
154
|
-
|
|
155
|
-
Test adequacy criteria answer: "Have we tested enough?"
|
|
156
|
-
|
|
157
|
-
```
|
|
158
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
159
|
-
│ Test Adequacy Criteria │
|
|
160
|
-
├─────────────────────────────────────────────────────────────┤
|
|
161
|
-
│ │
|
|
162
|
-
│ Coverage-Based Criteria │
|
|
163
|
-
│ ├── Statement Coverage: % of statements executed │
|
|
164
|
-
│ ├── Branch Coverage: % of branches taken │
|
|
165
|
-
│ ├── Condition Coverage: % of conditions evaluated │
|
|
166
|
-
│ └── MC/DC: Modified condition/decision coverage │
|
|
167
|
-
│ │
|
|
168
|
-
│ Fault-Based Criteria │
|
|
169
|
-
│ ├── Mutation Score: % of mutants killed │
|
|
170
|
-
│ └── Fault Seeding: % of seeded faults found │
|
|
171
|
-
│ │
|
|
172
|
-
│ Requirements-Based Criteria │
|
|
173
|
-
│ └── Requirements Coverage: % of requirements tested │
|
|
174
|
-
│ │
|
|
175
|
-
│ Risk-Based Criteria │
|
|
176
|
-
│ └── Risk Coverage: % of high-risk items tested │
|
|
177
|
-
│ │
|
|
178
|
-
│ Stopping Rules: │
|
|
179
|
-
│ • Achieved target coverage level │
|
|
180
|
-
│ • Defect discovery rate drops below threshold │
|
|
181
|
-
│ • Budget/time exhausted (with documented risk) │
|
|
182
|
-
│ │
|
|
183
|
-
└─────────────────────────────────────────────────────────────┘
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
---
|
|
187
|
-
|
|
188
|
-
## Testing Framework Selection
|
|
189
|
-
|
|
190
|
-
Choose a testing framework based on your project needs. This standard supports two approaches:
|
|
191
|
-
|
|
192
|
-
### Option A: ISTQB Standard Framework
|
|
193
|
-
|
|
194
|
-
Best for enterprise projects, certification needs, and formal QA processes.
|
|
195
|
-
|
|
196
|
-
**Reference**: [ISTQB Glossary v4.0](https://glossary.istqb.org)
|
|
197
|
-
|
|
198
|
-
| Level | Abbreviation | Purpose |
|
|
199
|
-
|-------|--------------|---------|
|
|
200
|
-
| Unit Testing | UT | Verify individual code units |
|
|
201
|
-
| Integration Testing | IT/SIT | Verify component interactions |
|
|
202
|
-
| System Testing | ST | Verify system meets requirements |
|
|
203
|
-
| Acceptance Testing | AT/UAT | Verify system meets business needs |
|
|
204
|
-
|
|
205
|
-
**When to choose ISTQB**:
|
|
206
|
-
- Enterprise projects with formal QA processes
|
|
207
|
-
- Projects requiring certification or compliance
|
|
208
|
-
- Organizations with dedicated QA teams
|
|
209
|
-
- Projects with strict audit requirements
|
|
210
|
-
|
|
211
|
-
### Option B: Industry Testing Pyramid
|
|
212
|
-
|
|
213
|
-
Best for agile development, CI/CD optimization, and rapid iteration.
|
|
214
|
-
|
|
215
|
-
**Reference**: [Martin Fowler's Testing Pyramid](https://martinfowler.com/bliki/TestPyramid.html), [Google Testing Blog](https://testing.googleblog.com)
|
|
216
|
-
|
|
217
|
-
| Level | Abbreviation | Ratio | Purpose |
|
|
218
|
-
|-------|--------------|-------|---------|
|
|
219
|
-
| Unit Testing | UT | 70% | Isolated component tests |
|
|
220
|
-
| Integration Testing | IT/SIT* | 20% | Component interaction tests |
|
|
221
|
-
| E2E Testing | E2E | 10% | User workflow tests |
|
|
222
|
-
|
|
223
|
-
*Note on Integration Testing abbreviation:
|
|
224
|
-
- **IT** (Integration Testing): Common in Agile/DevOps communities
|
|
225
|
-
- **SIT** (System Integration Testing): Common in Enterprise/ISTQB contexts
|
|
226
|
-
- Both terms refer to the same testing level
|
|
227
|
-
|
|
228
|
-
**When to choose Industry Pyramid**:
|
|
229
|
-
- Agile/Scrum development teams
|
|
230
|
-
- CI/CD focused environments
|
|
231
|
-
- Small to medium projects with rapid iteration
|
|
232
|
-
- DevOps practices
|
|
233
|
-
|
|
234
|
-
---
|
|
235
|
-
|
|
236
|
-
## Testing Pyramid (Default: Industry Standard)
|
|
237
|
-
|
|
238
|
-
```
|
|
239
|
-
┌─────────┐
|
|
240
|
-
│ E2E │ ← 10% (Fewer, slower, expensive)
|
|
241
|
-
─┴─────────┴─
|
|
242
|
-
┌─────────────┐
|
|
243
|
-
│ IT/SIT │ ← 20% (Integration Testing)
|
|
244
|
-
─┴─────────────┴─
|
|
245
|
-
┌─────────────────┐
|
|
246
|
-
│ UT │ ← 70% (Unit Testing - Foundation)
|
|
247
|
-
└─────────────────┘
|
|
248
|
-
```
|
|
249
|
-
|
|
250
|
-
### Recommended Ratio (Industry Pyramid)
|
|
251
|
-
|
|
252
|
-
| Level | Abbreviation | Percentage | Execution Time |
|
|
253
|
-
|-------|--------------|------------|----------------|
|
|
254
|
-
| Unit Testing | UT | 70% | < 10 min total |
|
|
255
|
-
| Integration Testing | IT/SIT | 20% | < 30 min total |
|
|
256
|
-
| E2E Testing | E2E | 10% | < 2 hours total |
|
|
257
|
-
|
|
258
|
-
### ISTQB 4-Level Structure (Alternative)
|
|
259
|
-
|
|
260
|
-
| Level | Abbreviation | Performed By | Focus |
|
|
261
|
-
|-------|--------------|--------------|-------|
|
|
262
|
-
| Unit Testing | UT | Developers | Code correctness |
|
|
263
|
-
| Integration Testing | IT/SIT | Developers/QA | Interface contracts |
|
|
264
|
-
| System Testing | ST | QA Team | Requirements verification |
|
|
265
|
-
| Acceptance Testing | AT/UAT | End Users | Business validation |
|
|
266
|
-
|
|
267
|
-
---
|
|
268
|
-
|
|
269
|
-
## Unit Testing (UT)
|
|
270
|
-
|
|
271
|
-
### Definition
|
|
272
|
-
|
|
273
|
-
Tests individual functions, methods, or classes in isolation from external dependencies.
|
|
274
|
-
|
|
275
|
-
### Characteristics
|
|
276
|
-
|
|
277
|
-
- **Isolated**: No database, network, or file system access
|
|
278
|
-
- **Fast**: Each test < 100ms
|
|
279
|
-
- **Deterministic**: Same input always produces same output
|
|
280
|
-
|
|
281
|
-
### Scope
|
|
282
|
-
|
|
283
|
-
```
|
|
284
|
-
┌─────────────────────────────────────────┐
|
|
285
|
-
│ Unit Under Test │
|
|
286
|
-
├─────────────────────────────────────────┤
|
|
287
|
-
│ ✅ Single function/method │
|
|
288
|
-
│ ✅ Single class │
|
|
289
|
-
│ ✅ Pure business logic │
|
|
290
|
-
│ ✅ Data transformations │
|
|
291
|
-
│ ✅ Validation rules │
|
|
292
|
-
├─────────────────────────────────────────┤
|
|
293
|
-
│ ❌ Database queries │
|
|
294
|
-
│ ❌ External API calls │
|
|
295
|
-
│ ❌ File I/O operations │
|
|
296
|
-
│ ❌ Multi-class interactions │
|
|
297
|
-
└─────────────────────────────────────────┘
|
|
298
|
-
```
|
|
299
|
-
|
|
300
|
-
### Naming Convention
|
|
301
|
-
|
|
302
|
-
**File Naming**:
|
|
303
|
-
```
|
|
304
|
-
[ClassName]Tests.[ext]
|
|
305
|
-
[ClassName].test.[ext]
|
|
306
|
-
[ClassName].spec.[ext]
|
|
307
|
-
|
|
308
|
-
Examples:
|
|
309
|
-
UserService.test.ts
|
|
310
|
-
UserServiceTests.cs
|
|
311
|
-
user_service_test.py
|
|
312
|
-
user_service_test.go
|
|
313
|
-
```
|
|
314
|
-
|
|
315
|
-
**Method Naming**:
|
|
316
|
-
```
|
|
317
|
-
[MethodName]_[Scenario]_[ExpectedResult]
|
|
318
|
-
should_[ExpectedBehavior]_when_[Condition]
|
|
319
|
-
test_[method]_[scenario]_[expected]
|
|
320
|
-
|
|
321
|
-
Examples:
|
|
322
|
-
CalculateTotal_WithDiscount_ReturnsDiscountedPrice()
|
|
323
|
-
should_return_null_when_user_not_found()
|
|
324
|
-
test_validate_email_invalid_format_returns_false()
|
|
325
|
-
```
|
|
326
|
-
|
|
327
|
-
### Coverage Guidelines
|
|
328
|
-
|
|
329
|
-
| Metric | Minimum | Recommended |
|
|
330
|
-
|--------|---------|-------------|
|
|
331
|
-
| Line Coverage | 70% | 85% |
|
|
332
|
-
| Branch Coverage | 60% | 80% |
|
|
333
|
-
| Function Coverage | 80% | 90% |
|
|
334
|
-
|
|
335
|
-
### Example
|
|
336
|
-
|
|
337
|
-
```csharp
|
|
338
|
-
// C# Example
|
|
339
|
-
[TestClass]
|
|
340
|
-
public class UserValidatorTests
|
|
341
|
-
{
|
|
342
|
-
private UserValidator _validator;
|
|
343
|
-
|
|
344
|
-
[TestInitialize]
|
|
345
|
-
public void Setup()
|
|
346
|
-
{
|
|
347
|
-
_validator = new UserValidator();
|
|
348
|
-
}
|
|
349
|
-
|
|
350
|
-
[TestMethod]
|
|
351
|
-
public void ValidateEmail_ValidFormat_ReturnsTrue()
|
|
352
|
-
{
|
|
353
|
-
// Arrange
|
|
354
|
-
var email = "user@example.com";
|
|
355
|
-
|
|
356
|
-
// Act
|
|
357
|
-
var result = _validator.ValidateEmail(email);
|
|
358
|
-
|
|
359
|
-
// Assert
|
|
360
|
-
Assert.IsTrue(result);
|
|
361
|
-
}
|
|
362
|
-
|
|
363
|
-
[TestMethod]
|
|
364
|
-
public void ValidateEmail_InvalidFormat_ReturnsFalse()
|
|
365
|
-
{
|
|
366
|
-
// Arrange
|
|
367
|
-
var email = "invalid-email";
|
|
368
|
-
|
|
369
|
-
// Act
|
|
370
|
-
var result = _validator.ValidateEmail(email);
|
|
371
|
-
|
|
372
|
-
// Assert
|
|
373
|
-
Assert.IsFalse(result);
|
|
374
|
-
}
|
|
375
|
-
}
|
|
376
|
-
```
|
|
377
|
-
|
|
378
|
-
```typescript
|
|
379
|
-
// TypeScript Example
|
|
380
|
-
describe('UserValidator', () => {
|
|
381
|
-
let validator: UserValidator;
|
|
382
|
-
|
|
383
|
-
beforeEach(() => {
|
|
384
|
-
validator = new UserValidator();
|
|
385
|
-
});
|
|
386
|
-
|
|
387
|
-
describe('validateEmail', () => {
|
|
388
|
-
it('should return true for valid email format', () => {
|
|
389
|
-
const result = validator.validateEmail('user@example.com');
|
|
390
|
-
expect(result).toBe(true);
|
|
391
|
-
});
|
|
392
|
-
|
|
393
|
-
it('should return false for invalid email format', () => {
|
|
394
|
-
const result = validator.validateEmail('invalid-email');
|
|
395
|
-
expect(result).toBe(false);
|
|
396
|
-
});
|
|
397
|
-
});
|
|
398
|
-
});
|
|
399
|
-
```
|
|
400
|
-
|
|
401
|
-
---
|
|
402
|
-
|
|
403
|
-
## Integration Testing (IT)
|
|
404
|
-
|
|
405
|
-
### Definition
|
|
406
|
-
|
|
407
|
-
Tests interactions between multiple components, modules, or external systems.
|
|
408
|
-
|
|
409
|
-
### When Integration Tests Are Required
|
|
410
|
-
|
|
411
|
-
| Scenario | Reason |
|
|
412
|
-
|----------|--------|
|
|
413
|
-
| Query predicates | Mocks cannot verify filter expressions |
|
|
414
|
-
| Entity relationships | Verify foreign key correctness |
|
|
415
|
-
| Composite keys | In-memory DB may differ from real DB |
|
|
416
|
-
| Field mapping | DTO ↔ Entity transformations |
|
|
417
|
-
| Pagination | Row ordering and counting |
|
|
418
|
-
| Transactions | Rollback behavior |
|
|
419
|
-
|
|
420
|
-
**Decision Rule**:
|
|
421
|
-
If your unit test uses a wildcard matcher (`any()`, `It.IsAny<>`, `Arg.Any<>`)
|
|
422
|
-
for a query/filter parameter, that functionality MUST have an integration test.
|
|
423
|
-
|
|
424
|
-
### Characteristics
|
|
425
|
-
|
|
426
|
-
- **Component Integration**: Tests module boundaries
|
|
427
|
-
- **Real Dependencies**: Uses actual databases, APIs (often containerized)
|
|
428
|
-
- **Slower**: Each test typically 1-10 seconds
|
|
429
|
-
|
|
430
|
-
### Scope
|
|
431
|
-
|
|
432
|
-
```
|
|
433
|
-
┌─────────────────────────────────────────┐
|
|
434
|
-
│ Integration Test Scope │
|
|
435
|
-
├─────────────────────────────────────────┤
|
|
436
|
-
│ ✅ Database CRUD operations │
|
|
437
|
-
│ ✅ Repository + Database │
|
|
438
|
-
│ ✅ Service + Repository │
|
|
439
|
-
│ ✅ API endpoint + Service layer │
|
|
440
|
-
│ ✅ Message queue producers/consumers │
|
|
441
|
-
│ ✅ Cache read/write operations │
|
|
442
|
-
├─────────────────────────────────────────┤
|
|
443
|
-
│ ❌ Full user workflows │
|
|
444
|
-
│ ❌ Cross-service communication │
|
|
445
|
-
│ ❌ UI interactions │
|
|
446
|
-
└─────────────────────────────────────────┘
|
|
447
|
-
```
|
|
448
|
-
|
|
449
|
-
### Naming Convention
|
|
450
|
-
|
|
451
|
-
**File Naming**:
|
|
452
|
-
```
|
|
453
|
-
[ComponentName]IntegrationTests.[ext]
|
|
454
|
-
[ComponentName].integration.test.[ext]
|
|
455
|
-
[ComponentName].itest.[ext]
|
|
456
|
-
|
|
457
|
-
Examples:
|
|
458
|
-
UserRepositoryIntegrationTests.cs
|
|
459
|
-
user-service.integration.test.ts
|
|
460
|
-
user_repository_itest.py
|
|
461
|
-
```
|
|
462
|
-
|
|
463
|
-
**Method Naming**:
|
|
464
|
-
```
|
|
465
|
-
[Operation]_[Context]_[ExpectedOutcome]
|
|
466
|
-
|
|
467
|
-
Examples:
|
|
468
|
-
CreateUser_WithValidData_PersistsToDatabase()
|
|
469
|
-
GetUserById_ExistingUser_ReturnsUserFromDatabase()
|
|
470
|
-
SendMessage_ToQueue_ConsumerReceivesMessage()
|
|
471
|
-
```
|
|
472
|
-
|
|
473
|
-
### Test Fixtures
|
|
474
|
-
|
|
475
|
-
```csharp
|
|
476
|
-
// C# Integration Test Example with Test Database
|
|
477
|
-
[TestClass]
|
|
478
|
-
public class UserRepositoryIntegrationTests
|
|
479
|
-
{
|
|
480
|
-
private TestDbContext _dbContext;
|
|
481
|
-
private UserRepository _repository;
|
|
482
|
-
|
|
483
|
-
[TestInitialize]
|
|
484
|
-
public async Task Setup()
|
|
485
|
-
{
|
|
486
|
-
// Use test database (e.g., SQLite in-memory or Testcontainers)
|
|
487
|
-
_dbContext = TestDbContextFactory.Create();
|
|
488
|
-
_repository = new UserRepository(_dbContext);
|
|
489
|
-
await _dbContext.Database.EnsureCreatedAsync();
|
|
490
|
-
}
|
|
491
|
-
|
|
492
|
-
[TestCleanup]
|
|
493
|
-
public async Task Cleanup()
|
|
494
|
-
{
|
|
495
|
-
await _dbContext.DisposeAsync();
|
|
496
|
-
}
|
|
497
|
-
|
|
498
|
-
[TestMethod]
|
|
499
|
-
public async Task CreateUser_WithValidData_PersistsToDatabase()
|
|
500
|
-
{
|
|
501
|
-
// Arrange
|
|
502
|
-
var user = new User { Name = "Test User", Email = "test@example.com" };
|
|
503
|
-
|
|
504
|
-
// Act
|
|
505
|
-
await _repository.CreateAsync(user);
|
|
506
|
-
var savedUser = await _repository.GetByIdAsync(user.Id);
|
|
507
|
-
|
|
508
|
-
// Assert
|
|
509
|
-
Assert.IsNotNull(savedUser);
|
|
510
|
-
Assert.AreEqual("Test User", savedUser.Name);
|
|
511
|
-
}
|
|
512
|
-
}
|
|
513
|
-
```
|
|
514
|
-
|
|
515
|
-
---
|
|
516
|
-
|
|
517
|
-
## System Testing (ST)
|
|
518
|
-
|
|
519
|
-
### Definition
|
|
520
|
-
|
|
521
|
-
Tests the complete integrated system to verify it meets specified requirements.
|
|
522
|
-
|
|
523
|
-
### Characteristics
|
|
524
|
-
|
|
525
|
-
- **Complete System**: All components deployed and integrated
|
|
526
|
-
- **Requirement-Based**: Tests against functional specifications
|
|
527
|
-
- **Production-Like**: Uses environment similar to production
|
|
528
|
-
|
|
529
|
-
### Scope
|
|
530
|
-
|
|
531
|
-
```
|
|
532
|
-
┌─────────────────────────────────────────┐
|
|
533
|
-
│ System Test Scope │
|
|
534
|
-
├─────────────────────────────────────────┤
|
|
535
|
-
│ ✅ Complete API workflows │
|
|
536
|
-
│ ✅ Cross-service transactions │
|
|
537
|
-
│ ✅ Data flow through entire system │
|
|
538
|
-
│ ✅ Security requirements │
|
|
539
|
-
│ ✅ Performance under load │
|
|
540
|
-
│ ✅ Error handling & recovery │
|
|
541
|
-
│ ✅ Configuration validation │
|
|
542
|
-
├─────────────────────────────────────────┤
|
|
543
|
-
│ ❌ UI visual testing │
|
|
544
|
-
│ ❌ User journey simulations │
|
|
545
|
-
│ ❌ A/B testing scenarios │
|
|
546
|
-
└─────────────────────────────────────────┘
|
|
547
|
-
```
|
|
548
|
-
|
|
549
|
-
### Types of System Tests
|
|
550
|
-
|
|
551
|
-
| Type | Description |
|
|
552
|
-
|------|-------------|
|
|
553
|
-
| Functional | Verify features work as specified |
|
|
554
|
-
| Performance | Load, stress, scalability testing |
|
|
555
|
-
| Security | Penetration, vulnerability scanning |
|
|
556
|
-
| Reliability | Failover, recovery, stability |
|
|
557
|
-
| Compatibility | Cross-platform, browser compatibility |
|
|
558
|
-
|
|
559
|
-
### Naming Convention
|
|
560
|
-
|
|
561
|
-
**File Naming**:
|
|
562
|
-
```
|
|
563
|
-
[Feature]SystemTests.[ext]
|
|
564
|
-
[Feature].system.test.[ext]
|
|
565
|
-
[Feature]_st.[ext]
|
|
566
|
-
|
|
567
|
-
Examples:
|
|
568
|
-
OrderProcessingSystemTests.cs
|
|
569
|
-
authentication.system.test.ts
|
|
570
|
-
payment_processing_st.py
|
|
571
|
-
```
|
|
572
|
-
|
|
573
|
-
### Example
|
|
574
|
-
|
|
575
|
-
```csharp
|
|
576
|
-
// System Test Example: Complete Resource Processing Flow
|
|
577
|
-
// Note: Replace {Resource}, {Item}, {Action} with your domain concepts
|
|
578
|
-
[TestClass]
|
|
579
|
-
public class ResourceProcessingSystemTests
|
|
580
|
-
{
|
|
581
|
-
private HttpClient _client;
|
|
582
|
-
private TestEnvironment _env;
|
|
583
|
-
|
|
584
|
-
[TestInitialize]
|
|
585
|
-
public async Task Setup()
|
|
586
|
-
{
|
|
587
|
-
_env = await TestEnvironment.CreateAsync();
|
|
588
|
-
_client = _env.CreateAuthenticatedClient();
|
|
589
|
-
}
|
|
590
|
-
|
|
591
|
-
[TestMethod]
|
|
592
|
-
public async Task ProcessResource_CompleteFlow_CompletedSuccessfully()
|
|
593
|
-
{
|
|
594
|
-
// Arrange: Create test data
|
|
595
|
-
var item = await _env.CreateTestItem(value: 100);
|
|
596
|
-
var user = await _env.CreateTestUser();
|
|
597
|
-
|
|
598
|
-
// Act: Execute complete processing flow
|
|
599
|
-
// Step 1: Create request
|
|
600
|
-
var requestResponse = await _client.PostAsync("/api/requests",
|
|
601
|
-
new { itemId = item.Id, quantity = 2 });
|
|
602
|
-
Assert.AreEqual(HttpStatusCode.OK, requestResponse.StatusCode);
|
|
603
|
-
|
|
604
|
-
// Step 2: Submit processing
|
|
605
|
-
var processResponse = await _client.PostAsync("/api/processes",
|
|
606
|
-
new { requestId = requestResponse.RequestId, userId = user.Id });
|
|
607
|
-
var process = await processResponse.Content.ReadAsAsync<Process>();
|
|
608
|
-
Assert.AreEqual(HttpStatusCode.Created, processResponse.StatusCode);
|
|
609
|
-
|
|
610
|
-
// Step 3: Confirm completion
|
|
611
|
-
var confirmResponse = await _client.PostAsync($"/api/processes/{process.Id}/confirm",
|
|
612
|
-
new { confirmationType = "standard", amount = 200 });
|
|
613
|
-
Assert.AreEqual(HttpStatusCode.OK, confirmResponse.StatusCode);
|
|
614
|
-
|
|
615
|
-
// Assert: Verify final state
|
|
616
|
-
var finalProcess = await _client.GetAsync($"/api/processes/{process.Id}");
|
|
617
|
-
var result = await finalProcess.Content.ReadAsAsync<Process>();
|
|
618
|
-
|
|
619
|
-
Assert.AreEqual(ProcessStatus.Completed, result.Status);
|
|
620
|
-
Assert.AreEqual(200, result.TotalAmount);
|
|
621
|
-
Assert.IsNotNull(result.Confirmation);
|
|
622
|
-
}
|
|
623
|
-
}
|
|
624
|
-
```
|
|
625
|
-
|
|
626
|
-
---
|
|
627
|
-
|
|
628
|
-
## End-to-End Testing (E2E)
|
|
629
|
-
|
|
630
|
-
### Definition
|
|
631
|
-
|
|
632
|
-
Tests complete user workflows from the user interface through all system layers.
|
|
633
|
-
|
|
634
|
-
### Characteristics
|
|
635
|
-
|
|
636
|
-
- **User Perspective**: Simulates real user interactions
|
|
637
|
-
- **Full Stack**: UI → API → Database → External Services
|
|
638
|
-
- **Slowest**: Each test typically 30 seconds to several minutes
|
|
639
|
-
|
|
640
|
-
### Scope
|
|
641
|
-
|
|
642
|
-
```
|
|
643
|
-
┌─────────────────────────────────────────┐
|
|
644
|
-
│ E2E Test Scope │
|
|
645
|
-
├─────────────────────────────────────────┤
|
|
646
|
-
│ ✅ Critical user journeys │
|
|
647
|
-
│ ✅ Login/Authentication flows │
|
|
648
|
-
│ ✅ Core business transactions │
|
|
649
|
-
│ ✅ Cross-browser functionality │
|
|
650
|
-
│ ✅ Smoke tests for deployments │
|
|
651
|
-
├─────────────────────────────────────────┤
|
|
652
|
-
│ ❌ Every possible user path │
|
|
653
|
-
│ ❌ Edge cases (use UT/IT) │
|
|
654
|
-
│ ❌ Performance benchmarking │
|
|
655
|
-
└─────────────────────────────────────────┘
|
|
656
|
-
```
|
|
657
|
-
|
|
658
|
-
### Naming Convention
|
|
659
|
-
|
|
660
|
-
**File Naming**:
|
|
661
|
-
```
|
|
662
|
-
[UserJourney].e2e.[ext]
|
|
663
|
-
[Feature].e2e.spec.[ext]
|
|
664
|
-
e2e/[feature]/[scenario].[ext]
|
|
665
|
-
|
|
666
|
-
Examples:
|
|
667
|
-
user-registration.e2e.ts
|
|
668
|
-
checkout-flow.e2e.spec.ts
|
|
669
|
-
e2e/authentication/login.spec.ts
|
|
670
|
-
```
|
|
671
|
-
|
|
672
|
-
### Example
|
|
673
|
-
|
|
674
|
-
```typescript
|
|
675
|
-
// Playwright E2E Test Example
|
|
676
|
-
import { test, expect } from '@playwright/test';
|
|
677
|
-
|
|
678
|
-
test.describe('User Registration Journey', () => {
|
|
679
|
-
test('should complete registration and login successfully', async ({ page }) => {
|
|
680
|
-
// Step 1: Navigate to registration page
|
|
681
|
-
await page.goto('/register');
|
|
682
|
-
|
|
683
|
-
// Step 2: Fill registration form
|
|
684
|
-
await page.fill('[data-testid="email"]', 'newuser@example.com');
|
|
685
|
-
await page.fill('[data-testid="password"]', 'SecurePass123!');
|
|
686
|
-
await page.fill('[data-testid="confirm-password"]', 'SecurePass123!');
|
|
687
|
-
await page.click('[data-testid="register-button"]');
|
|
688
|
-
|
|
689
|
-
// Step 3: Verify registration success
|
|
690
|
-
await expect(page.locator('[data-testid="success-message"]'))
|
|
691
|
-
.toContainText('Registration successful');
|
|
692
|
-
|
|
693
|
-
// Step 4: Login with new account
|
|
694
|
-
await page.goto('/login');
|
|
695
|
-
await page.fill('[data-testid="email"]', 'newuser@example.com');
|
|
696
|
-
await page.fill('[data-testid="password"]', 'SecurePass123!');
|
|
697
|
-
await page.click('[data-testid="login-button"]');
|
|
698
|
-
|
|
699
|
-
// Step 5: Verify login success and dashboard redirect
|
|
700
|
-
await expect(page).toHaveURL('/dashboard');
|
|
701
|
-
await expect(page.locator('[data-testid="welcome-message"]'))
|
|
702
|
-
.toContainText('Welcome, newuser@example.com');
|
|
703
|
-
});
|
|
704
|
-
});
|
|
705
|
-
```
|
|
706
|
-
|
|
707
|
-
---
|
|
708
|
-
|
|
709
|
-
## Static Testing
|
|
710
|
-
|
|
711
|
-
Static testing examines work products (code, documents, requirements) without executing the software. It complements dynamic testing by finding defects earlier in the SDLC.
|
|
712
|
-
|
|
713
|
-
**Reference**: ISTQB CTFL v4.0 Chapter 3, ISO/IEC/IEEE 29119-4
|
|
714
|
-
|
|
715
|
-
### Types of Static Testing
|
|
716
|
-
|
|
717
|
-
```
|
|
718
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
719
|
-
│ Static Testing Types │
|
|
720
|
-
├─────────────────────────────────────────────────────────────┤
|
|
721
|
-
│ Reviews (Manual) │
|
|
722
|
-
│ ├── Informal Review: Ad-hoc, no formal process │
|
|
723
|
-
│ ├── Walkthrough: Author-led, educational purpose │
|
|
724
|
-
│ ├── Technical Review: Peer-led, find defects │
|
|
725
|
-
│ └── Inspection: Formal, metrics-driven, most rigorous │
|
|
726
|
-
├─────────────────────────────────────────────────────────────┤
|
|
727
|
-
│ Static Analysis (Tool-based) │
|
|
728
|
-
│ ├── Code Analysis: Linters, complexity analyzers │
|
|
729
|
-
│ ├── Security Analysis: SAST tools (SonarQube, Checkmarx) │
|
|
730
|
-
│ ├── Architecture Analysis: Dependency checks │
|
|
731
|
-
│ └── Data Flow Analysis: Variable usage patterns │
|
|
732
|
-
└─────────────────────────────────────────────────────────────┘
|
|
733
|
-
```
|
|
734
|
-
|
|
735
|
-
### Review Process (ISTQB)
|
|
736
|
-
|
|
737
|
-
| Phase | Activities |
|
|
738
|
-
|-------|------------|
|
|
739
|
-
| Planning | Define scope, entry criteria, roles |
|
|
740
|
-
| Initiate Review | Distribute work products |
|
|
741
|
-
| Individual Review | Each reviewer examines independently |
|
|
742
|
-
| Issue Communication | Discuss findings in meeting |
|
|
743
|
-
| Fixing & Reporting | Author fixes, metrics collected |
|
|
744
|
-
|
|
745
|
-
### Static Analysis Tools by Language
|
|
746
|
-
|
|
747
|
-
| Language | Linting | Security | Complexity |
|
|
748
|
-
|----------|---------|----------|------------|
|
|
749
|
-
| JavaScript/TypeScript | ESLint, Biome | npm audit, Snyk | ESLint complexity rules |
|
|
750
|
-
| Python | Pylint, Ruff, Flake8 | Bandit, Safety | Radon |
|
|
751
|
-
| Java | Checkstyle, PMD | SpotBugs, OWASP DC | JaCoCo |
|
|
752
|
-
| C# | StyleCop, Roslyn | Security Code Scan | NDepend |
|
|
753
|
-
| Go | golangci-lint | gosec | gocyclo |
|
|
754
|
-
|
|
755
|
-
### When to Use Static Testing
|
|
756
|
-
|
|
757
|
-
| Work Product | Review Type | Tools |
|
|
758
|
-
|--------------|-------------|-------|
|
|
759
|
-
| Requirements | Inspection, Walkthrough | - |
|
|
760
|
-
| Design Documents | Technical Review | Architecture tools |
|
|
761
|
-
| Code | Technical Review, Tool Analysis | Linters, SAST |
|
|
762
|
-
| Test Plans | Walkthrough | - |
|
|
763
|
-
| User Documentation | Walkthrough | Spell/grammar checkers |
|
|
764
|
-
|
|
765
|
-
### Code Review Checklist
|
|
766
|
-
|
|
767
|
-
```
|
|
768
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
769
|
-
│ Code Review Focus Areas │
|
|
770
|
-
├─────────────────────────────────────────────────────────────┤
|
|
771
|
-
│ Functionality │
|
|
772
|
-
│ ├── Does the code do what it's supposed to do? │
|
|
773
|
-
│ ├── Are edge cases handled? │
|
|
774
|
-
│ └── Is error handling appropriate? │
|
|
775
|
-
├─────────────────────────────────────────────────────────────┤
|
|
776
|
-
│ Maintainability │
|
|
777
|
-
│ ├── Is the code readable and well-organized? │
|
|
778
|
-
│ ├── Are names meaningful? │
|
|
779
|
-
│ └── Is complexity manageable? │
|
|
780
|
-
├─────────────────────────────────────────────────────────────┤
|
|
781
|
-
│ Security │
|
|
782
|
-
│ ├── Input validation present? │
|
|
783
|
-
│ ├── No hardcoded secrets? │
|
|
784
|
-
│ └── SQL injection / XSS prevention? │
|
|
785
|
-
├─────────────────────────────────────────────────────────────┤
|
|
786
|
-
│ Performance │
|
|
787
|
-
│ ├── No obvious inefficiencies? │
|
|
788
|
-
│ ├── Resource cleanup (connections, files)? │
|
|
789
|
-
│ └── Appropriate data structures? │
|
|
790
|
-
└─────────────────────────────────────────────────────────────┘
|
|
791
|
-
```
|
|
792
|
-
|
|
793
|
-
### Integration with CI/CD
|
|
794
|
-
|
|
795
|
-
```yaml
|
|
796
|
-
# Example: Static analysis in CI pipeline
|
|
797
|
-
static-analysis:
|
|
798
|
-
stage: test
|
|
799
|
-
script:
|
|
800
|
-
- npm run lint # Linting
|
|
801
|
-
- npm run lint:security # Security analysis
|
|
802
|
-
- npx sonarqube-scanner # Quality gates
|
|
803
|
-
rules:
|
|
804
|
-
- if: $CI_PIPELINE_SOURCE == "push"
|
|
805
|
-
```
|
|
806
|
-
|
|
807
|
-
---
|
|
808
|
-
|
|
809
|
-
## Test Design Techniques
|
|
810
|
-
|
|
811
|
-
Test design techniques are methods for deriving and selecting test cases. ISTQB and ISO/IEC/IEEE 29119 categorize them into three main approaches.
|
|
812
|
-
|
|
813
|
-
**Reference**: ISTQB CTFL v4.0 Chapter 4, ISO/IEC/IEEE 29119-4
|
|
814
|
-
|
|
815
|
-
### Technique Categories
|
|
816
|
-
|
|
817
|
-
```
|
|
818
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
819
|
-
│ Test Design Technique Categories │
|
|
820
|
-
├─────────────────────────────────────────────────────────────┤
|
|
821
|
-
│ Specification-Based (Black-Box) │
|
|
822
|
-
│ └── Derive tests from requirements/specifications │
|
|
823
|
-
├─────────────────────────────────────────────────────────────┤
|
|
824
|
-
│ Structure-Based (White-Box) │
|
|
825
|
-
│ └── Derive tests from internal code structure │
|
|
826
|
-
├─────────────────────────────────────────────────────────────┤
|
|
827
|
-
│ Experience-Based │
|
|
828
|
-
│ └── Derive tests from tester knowledge and intuition │
|
|
829
|
-
└─────────────────────────────────────────────────────────────┘
|
|
830
|
-
```
|
|
831
|
-
|
|
832
|
-
### Specification-Based Techniques (Black-Box)
|
|
833
|
-
|
|
834
|
-
#### 1. Equivalence Partitioning (EP)
|
|
835
|
-
|
|
836
|
-
Divides input data into partitions where all values should be treated the same by the system.
|
|
837
|
-
|
|
838
|
-
```
|
|
839
|
-
Input: Age (valid range: 18-65)
|
|
840
|
-
|
|
841
|
-
┌─────────────┬─────────────┬─────────────┐
|
|
842
|
-
│ Invalid │ Valid │ Invalid │
|
|
843
|
-
│ < 18 │ 18 - 65 │ > 65 │
|
|
844
|
-
├─────────────┼─────────────┼─────────────┤
|
|
845
|
-
│ Partition 1│ Partition 2│ Partition 3│
|
|
846
|
-
│ Test: 10 │ Test: 30 │ Test: 70 │
|
|
847
|
-
└─────────────┴─────────────┴─────────────┘
|
|
848
|
-
|
|
849
|
-
Coverage: At least 1 test case per partition
|
|
850
|
-
```
|
|
851
|
-
|
|
852
|
-
```python
|
|
853
|
-
# Python Example: Equivalence Partitioning
|
|
854
|
-
def test_age_validation():
|
|
855
|
-
validator = AgeValidator(min_age=18, max_age=65)
|
|
856
|
-
|
|
857
|
-
# Partition 1: Below minimum (invalid)
|
|
858
|
-
assert validator.is_valid(10) == False
|
|
859
|
-
|
|
860
|
-
# Partition 2: Valid range
|
|
861
|
-
assert validator.is_valid(30) == True
|
|
862
|
-
|
|
863
|
-
# Partition 3: Above maximum (invalid)
|
|
864
|
-
assert validator.is_valid(70) == False
|
|
865
|
-
```
|
|
866
|
-
|
|
867
|
-
#### 2. Boundary Value Analysis (BVA)
|
|
868
|
-
|
|
869
|
-
Tests at the boundaries of equivalence partitions where defects are most likely.
|
|
870
|
-
|
|
871
|
-
```
|
|
872
|
-
Input: Quantity (valid range: 1-100)
|
|
873
|
-
|
|
874
|
-
0 1 2 ... 99 100 101
|
|
875
|
-
│ │ │ │ │ │
|
|
876
|
-
▼ ▼ ▼ ▼ ▼ ▼
|
|
877
|
-
Invalid Valid Valid Valid Valid Invalid
|
|
878
|
-
(boundary)(boundary) (boundary)(boundary)
|
|
879
|
-
|
|
880
|
-
Test Values: 0, 1, 2, 99, 100, 101
|
|
881
|
-
```
|
|
882
|
-
|
|
883
|
-
```typescript
|
|
884
|
-
// TypeScript Example: Boundary Value Analysis
|
|
885
|
-
describe('Quantity Validation', () => {
|
|
886
|
-
const validator = new QuantityValidator(1, 100);
|
|
887
|
-
|
|
888
|
-
// Lower boundary
|
|
889
|
-
it('rejects 0 (below minimum)', () => {
|
|
890
|
-
expect(validator.isValid(0)).toBe(false);
|
|
891
|
-
});
|
|
892
|
-
|
|
893
|
-
it('accepts 1 (minimum boundary)', () => {
|
|
894
|
-
expect(validator.isValid(1)).toBe(true);
|
|
895
|
-
});
|
|
896
|
-
|
|
897
|
-
it('accepts 2 (above minimum)', () => {
|
|
898
|
-
expect(validator.isValid(2)).toBe(true);
|
|
899
|
-
});
|
|
900
|
-
|
|
901
|
-
// Upper boundary
|
|
902
|
-
it('accepts 99 (below maximum)', () => {
|
|
903
|
-
expect(validator.isValid(99)).toBe(true);
|
|
904
|
-
});
|
|
905
|
-
|
|
906
|
-
it('accepts 100 (maximum boundary)', () => {
|
|
907
|
-
expect(validator.isValid(100)).toBe(true);
|
|
908
|
-
});
|
|
909
|
-
|
|
910
|
-
it('rejects 101 (above maximum)', () => {
|
|
911
|
-
expect(validator.isValid(101)).toBe(false);
|
|
912
|
-
});
|
|
913
|
-
});
|
|
914
|
-
```
|
|
915
|
-
|
|
916
|
-
#### 3. Decision Table Testing
|
|
917
|
-
|
|
918
|
-
Captures complex business rules with multiple conditions.
|
|
919
|
-
|
|
920
|
-
```
|
|
921
|
-
Discount Rules:
|
|
922
|
-
- Premium member: 20% discount
|
|
923
|
-
- Order > $100: 10% discount
|
|
924
|
-
- Both conditions: 25% discount (not cumulative)
|
|
925
|
-
|
|
926
|
-
┌─────────────────┬──────┬──────┬──────┬──────┐
|
|
927
|
-
│ Conditions │ TC1 │ TC2 │ TC3 │ TC4 │
|
|
928
|
-
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
929
|
-
│ Premium Member │ N │ Y │ N │ Y │
|
|
930
|
-
│ Order > $100 │ N │ N │ Y │ Y │
|
|
931
|
-
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
932
|
-
│ Actions │ │ │ │ │
|
|
933
|
-
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
934
|
-
│ Discount % │ 0% │ 20% │ 10% │ 25% │
|
|
935
|
-
└─────────────────┴──────┴──────┴──────┴──────┘
|
|
936
|
-
```
|
|
937
|
-
|
|
938
|
-
```csharp
|
|
939
|
-
// C# Example: Decision Table Testing
|
|
940
|
-
[TestClass]
|
|
941
|
-
public class DiscountCalculatorTests
|
|
942
|
-
{
|
|
943
|
-
private DiscountCalculator _calculator;
|
|
944
|
-
|
|
945
|
-
[TestInitialize]
|
|
946
|
-
public void Setup() => _calculator = new DiscountCalculator();
|
|
947
|
-
|
|
948
|
-
[TestMethod]
|
|
949
|
-
public void Calculate_NotPremium_SmallOrder_NoDiscount()
|
|
950
|
-
{
|
|
951
|
-
var result = _calculator.Calculate(isPremium: false, orderAmount: 50);
|
|
952
|
-
Assert.AreEqual(0, result.DiscountPercent);
|
|
953
|
-
}
|
|
954
|
-
|
|
955
|
-
[TestMethod]
|
|
956
|
-
public void Calculate_Premium_SmallOrder_20PercentDiscount()
|
|
957
|
-
{
|
|
958
|
-
var result = _calculator.Calculate(isPremium: true, orderAmount: 50);
|
|
959
|
-
Assert.AreEqual(20, result.DiscountPercent);
|
|
960
|
-
}
|
|
961
|
-
|
|
962
|
-
[TestMethod]
|
|
963
|
-
public void Calculate_NotPremium_LargeOrder_10PercentDiscount()
|
|
964
|
-
{
|
|
965
|
-
var result = _calculator.Calculate(isPremium: false, orderAmount: 150);
|
|
966
|
-
Assert.AreEqual(10, result.DiscountPercent);
|
|
967
|
-
}
|
|
968
|
-
|
|
969
|
-
[TestMethod]
|
|
970
|
-
public void Calculate_Premium_LargeOrder_25PercentDiscount()
|
|
971
|
-
{
|
|
972
|
-
var result = _calculator.Calculate(isPremium: true, orderAmount: 150);
|
|
973
|
-
Assert.AreEqual(25, result.DiscountPercent);
|
|
974
|
-
}
|
|
975
|
-
}
|
|
976
|
-
```
|
|
977
|
-
|
|
978
|
-
#### 4. State Transition Testing
|
|
979
|
-
|
|
980
|
-
Tests behavior changes based on system states and transitions.
|
|
981
|
-
|
|
982
|
-
```
|
|
983
|
-
Order State Machine:
|
|
984
|
-
|
|
985
|
-
┌─────────┐ place() ┌───────────┐ pay() ┌────────┐
|
|
986
|
-
│ Draft │──────────────▶│ Pending │───────────▶│ Paid │
|
|
987
|
-
└─────────┘ └───────────┘ └────────┘
|
|
988
|
-
│ │ │
|
|
989
|
-
│ cancel() │ cancel() │ ship()
|
|
990
|
-
▼ ▼ ▼
|
|
991
|
-
┌───────────┐ ┌───────────┐ ┌──────────┐
|
|
992
|
-
│ Cancelled │ │ Cancelled │ │ Shipped │
|
|
993
|
-
└───────────┘ └───────────┘ └──────────┘
|
|
994
|
-
|
|
995
|
-
State Transition Table:
|
|
996
|
-
┌───────────┬─────────┬───────────┬───────────┬───────────┐
|
|
997
|
-
│ Current │ place() │ pay() │ ship() │ cancel() │
|
|
998
|
-
├───────────┼─────────┼───────────┼───────────┼───────────┤
|
|
999
|
-
│ Draft │ Pending │ Invalid │ Invalid │ Cancelled │
|
|
1000
|
-
│ Pending │ Invalid │ Paid │ Invalid │ Cancelled │
|
|
1001
|
-
│ Paid │ Invalid │ Invalid │ Shipped │ Invalid │
|
|
1002
|
-
│ Shipped │ Invalid │ Invalid │ Invalid │ Invalid │
|
|
1003
|
-
│ Cancelled │ Invalid │ Invalid │ Invalid │ Invalid │
|
|
1004
|
-
└───────────┴─────────┴───────────┴───────────┴───────────┘
|
|
1005
|
-
```
|
|
1006
|
-
|
|
1007
|
-
```python
|
|
1008
|
-
# Python Example: State Transition Testing
|
|
1009
|
-
class TestOrderStateMachine:
|
|
1010
|
-
def test_draft_to_pending_on_place(self):
|
|
1011
|
-
order = Order(state="draft")
|
|
1012
|
-
order.place()
|
|
1013
|
-
assert order.state == "pending"
|
|
1014
|
-
|
|
1015
|
-
def test_pending_to_paid_on_pay(self):
|
|
1016
|
-
order = Order(state="pending")
|
|
1017
|
-
order.pay()
|
|
1018
|
-
assert order.state == "paid"
|
|
1019
|
-
|
|
1020
|
-
def test_paid_to_shipped_on_ship(self):
|
|
1021
|
-
order = Order(state="paid")
|
|
1022
|
-
order.ship()
|
|
1023
|
-
assert order.state == "shipped"
|
|
1024
|
-
|
|
1025
|
-
def test_invalid_transition_pay_from_draft(self):
|
|
1026
|
-
order = Order(state="draft")
|
|
1027
|
-
with pytest.raises(InvalidTransitionError):
|
|
1028
|
-
order.pay()
|
|
1029
|
-
|
|
1030
|
-
def test_invalid_transition_cancel_from_shipped(self):
|
|
1031
|
-
order = Order(state="shipped")
|
|
1032
|
-
with pytest.raises(InvalidTransitionError):
|
|
1033
|
-
order.cancel()
|
|
1034
|
-
```
|
|
1035
|
-
|
|
1036
|
-
#### 5. Use Case Testing
|
|
1037
|
-
|
|
1038
|
-
Tests complete user scenarios from start to finish.
|
|
1039
|
-
|
|
1040
|
-
```
|
|
1041
|
-
Use Case: User Login
|
|
1042
|
-
|
|
1043
|
-
Primary Flow:
|
|
1044
|
-
1. User enters username
|
|
1045
|
-
2. User enters password
|
|
1046
|
-
3. User clicks login
|
|
1047
|
-
4. System validates credentials
|
|
1048
|
-
5. System redirects to dashboard
|
|
1049
|
-
|
|
1050
|
-
Alternative Flows:
|
|
1051
|
-
A1. Invalid credentials → Show error, remain on login
|
|
1052
|
-
A2. Account locked → Show locked message
|
|
1053
|
-
A3. Password expired → Redirect to password change
|
|
1054
|
-
|
|
1055
|
-
Test Cases:
|
|
1056
|
-
- TC1: Valid credentials → Dashboard (Primary)
|
|
1057
|
-
- TC2: Invalid password → Error message (A1)
|
|
1058
|
-
- TC3: Invalid username → Error message (A1)
|
|
1059
|
-
- TC4: Locked account → Locked message (A2)
|
|
1060
|
-
- TC5: Expired password → Password change page (A3)
|
|
1061
|
-
```
|
|
1062
|
-
|
|
1063
|
-
### Structure-Based Techniques (White-Box)
|
|
1064
|
-
|
|
1065
|
-
#### 1. Statement Coverage
|
|
1066
|
-
|
|
1067
|
-
Ensure every statement is executed at least once.
|
|
1068
|
-
|
|
1069
|
-
```
|
|
1070
|
-
Code:
|
|
1071
|
-
1 function categorize(score) {
|
|
1072
|
-
2 let result;
|
|
1073
|
-
3 if (score >= 90) {
|
|
1074
|
-
4 result = 'A';
|
|
1075
|
-
5 } else if (score >= 70) {
|
|
1076
|
-
6 result = 'B';
|
|
1077
|
-
7 } else {
|
|
1078
|
-
8 result = 'C';
|
|
1079
|
-
9 }
|
|
1080
|
-
10 return result;
|
|
1081
|
-
11 }
|
|
1082
|
-
|
|
1083
|
-
100% Statement Coverage requires tests:
|
|
1084
|
-
- Test 1: score = 95 → executes lines 1-4, 10
|
|
1085
|
-
- Test 2: score = 80 → executes lines 1-3, 5-6, 10
|
|
1086
|
-
- Test 3: score = 50 → executes lines 1-3, 5, 7-10
|
|
1087
|
-
```
|
|
1088
|
-
|
|
1089
|
-
#### 2. Branch Coverage
|
|
1090
|
-
|
|
1091
|
-
Ensure every branch (decision outcome) is executed.
|
|
1092
|
-
|
|
1093
|
-
```
|
|
1094
|
-
Code:
|
|
1095
|
-
if (a > 0 && b > 0) {
|
|
1096
|
-
doSomething();
|
|
1097
|
-
}
|
|
1098
|
-
|
|
1099
|
-
Branches:
|
|
1100
|
-
- True branch: a > 0 AND b > 0 is true
|
|
1101
|
-
- False branch: a > 0 AND b > 0 is false
|
|
1102
|
-
|
|
1103
|
-
100% Branch Coverage:
|
|
1104
|
-
- Test 1: a = 1, b = 1 → True branch
|
|
1105
|
-
- Test 2: a = 0, b = 1 → False branch (or a = 1, b = 0)
|
|
1106
|
-
```
|
|
1107
|
-
|
|
1108
|
-
#### 3. Condition Coverage
|
|
1109
|
-
|
|
1110
|
-
Ensure each condition in a decision is evaluated to both true and false.
|
|
1111
|
-
|
|
1112
|
-
```
|
|
1113
|
-
Code:
|
|
1114
|
-
if (a > 0 && b > 0) { ... }
|
|
1115
|
-
|
|
1116
|
-
Conditions: (a > 0), (b > 0)
|
|
1117
|
-
|
|
1118
|
-
100% Condition Coverage:
|
|
1119
|
-
- Test 1: a = 1, b = 1 → (a > 0) = true, (b > 0) = true
|
|
1120
|
-
- Test 2: a = 0, b = 0 → (a > 0) = false, (b > 0) = false
|
|
1121
|
-
|
|
1122
|
-
Note: Does not guarantee branch coverage!
|
|
1123
|
-
```
|
|
1124
|
-
|
|
1125
|
-
#### 4. MC/DC (Modified Condition/Decision Coverage)
|
|
1126
|
-
|
|
1127
|
-
Each condition independently affects the decision outcome. Required for safety-critical systems.
|
|
1128
|
-
|
|
1129
|
-
```
|
|
1130
|
-
Code:
|
|
1131
|
-
if (a && b) { ... }
|
|
1132
|
-
|
|
1133
|
-
MC/DC Requirements:
|
|
1134
|
-
1. Each condition evaluated to true and false
|
|
1135
|
-
2. Each condition independently affects outcome
|
|
1136
|
-
|
|
1137
|
-
Test Cases:
|
|
1138
|
-
- Test 1: a = true, b = true → Decision = true
|
|
1139
|
-
- Test 2: a = false, b = true → Decision = false (a changed outcome)
|
|
1140
|
-
- Test 3: a = true, b = false → Decision = false (b changed outcome)
|
|
1141
|
-
```
|
|
1142
|
-
|
|
1143
|
-
### Additional Specification-Based Techniques
|
|
1144
|
-
|
|
1145
|
-
#### 6. Pairwise Testing (Combinatorial Testing)
|
|
1146
|
-
|
|
1147
|
-
Tests all possible pairs of input parameter values, reducing test cases while maintaining good coverage.
|
|
1148
|
-
|
|
1149
|
-
**Reference**: SWEBOK v4.0 - Input Domain-Based Techniques
|
|
1150
|
-
|
|
1151
|
-
```
|
|
1152
|
-
Example: Login Form with 3 parameters, each with 3 values
|
|
1153
|
-
|
|
1154
|
-
Parameters:
|
|
1155
|
-
- Browser: Chrome, Firefox, Safari
|
|
1156
|
-
- OS: Windows, macOS, Linux
|
|
1157
|
-
- Language: English, Spanish, French
|
|
1158
|
-
|
|
1159
|
-
Full Combinatorial: 3 × 3 × 3 = 27 test cases
|
|
1160
|
-
Pairwise Coverage: 9 test cases (covers all pairs)
|
|
1161
|
-
|
|
1162
|
-
┌──────┬──────────┬─────────┬──────────┐
|
|
1163
|
-
│ TC │ Browser │ OS │ Language │
|
|
1164
|
-
├──────┼──────────┼─────────┼──────────┤
|
|
1165
|
-
│ 1 │ Chrome │ Windows │ English │
|
|
1166
|
-
│ 2 │ Chrome │ macOS │ Spanish │
|
|
1167
|
-
│ 3 │ Chrome │ Linux │ French │
|
|
1168
|
-
│ 4 │ Firefox │ Windows │ Spanish │
|
|
1169
|
-
│ 5 │ Firefox │ macOS │ French │
|
|
1170
|
-
│ 6 │ Firefox │ Linux │ English │
|
|
1171
|
-
│ 7 │ Safari │ Windows │ French │
|
|
1172
|
-
│ 8 │ Safari │ macOS │ English │
|
|
1173
|
-
│ 9 │ Safari │ Linux │ Spanish │
|
|
1174
|
-
└──────┴──────────┴─────────┴──────────┘
|
|
1175
|
-
```
|
|
1176
|
-
|
|
1177
|
-
**Tools**: PICT (Microsoft), AllPairs, Jenny
|
|
1178
|
-
|
|
1179
|
-
```bash
|
|
1180
|
-
# Using PICT (Pairwise Independent Combinatorial Testing)
|
|
1181
|
-
# Input file: params.txt
|
|
1182
|
-
# Browser: Chrome, Firefox, Safari
|
|
1183
|
-
# OS: Windows, macOS, Linux
|
|
1184
|
-
# Language: English, Spanish, French
|
|
1185
|
-
|
|
1186
|
-
pict params.txt > test_cases.txt
|
|
1187
|
-
```
|
|
1188
|
-
|
|
1189
|
-
#### 7. Data Flow Testing
|
|
1190
|
-
|
|
1191
|
-
Tests the flow of data through a program by tracking variable definitions and uses.
|
|
1192
|
-
|
|
1193
|
-
**Reference**: SWEBOK v4.0 - Code-Based Techniques
|
|
1194
|
-
|
|
1195
|
-
```
|
|
1196
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1197
|
-
│ Data Flow Terminology │
|
|
1198
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1199
|
-
│ │
|
|
1200
|
-
│ Definition (def): Variable is assigned a value │
|
|
1201
|
-
│ Use (use): Variable's value is accessed │
|
|
1202
|
-
│ - c-use: Computational use (in calculation) │
|
|
1203
|
-
│ - p-use: Predicate use (in condition) │
|
|
1204
|
-
│ │
|
|
1205
|
-
│ def-use pair: Path from definition to use │
|
|
1206
|
-
│ def-clear path: No redefinition between def and use │
|
|
1207
|
-
│ │
|
|
1208
|
-
│ Example: │
|
|
1209
|
-
│ 1: x = 5 ← def(x) │
|
|
1210
|
-
│ 2: y = x + 1 ← c-use(x), def(y) │
|
|
1211
|
-
│ 3: if (x > 0) ← p-use(x) │
|
|
1212
|
-
│ 4: z = y * 2 ← c-use(y), def(z) │
|
|
1213
|
-
│ │
|
|
1214
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1215
|
-
```
|
|
1216
|
-
|
|
1217
|
-
**Coverage Criteria**:
|
|
1218
|
-
|
|
1219
|
-
| Criterion | Requirement | Strength |
|
|
1220
|
-
|-----------|-------------|----------|
|
|
1221
|
-
| All-Defs | Every def reaches at least one use | Weak |
|
|
1222
|
-
| All-Uses | Every def-use pair is exercised | Medium |
|
|
1223
|
-
| All-DU-Paths | All def-clear paths from def to use | Strong |
|
|
1224
|
-
|
|
1225
|
-
```python
|
|
1226
|
-
# Python Example: Data Flow Testing
|
|
1227
|
-
def calculate_discount(price, quantity, is_member):
|
|
1228
|
-
# def: total
|
|
1229
|
-
total = price * quantity
|
|
1230
|
-
|
|
1231
|
-
# def: discount
|
|
1232
|
-
if is_member: # p-use: is_member
|
|
1233
|
-
discount = 0.1 # def: discount (path 1)
|
|
1234
|
-
else:
|
|
1235
|
-
discount = 0.0 # def: discount (path 2)
|
|
1236
|
-
|
|
1237
|
-
# c-use: total, discount
|
|
1238
|
-
if total > 100: # p-use: total
|
|
1239
|
-
discount += 0.05 # c-use: discount, def: discount
|
|
1240
|
-
|
|
1241
|
-
# c-use: total, discount
|
|
1242
|
-
return total * (1 - discount)
|
|
1243
|
-
|
|
1244
|
-
# Test cases for All-Uses coverage:
|
|
1245
|
-
# TC1: is_member=True, total>100 → exercises def@line4, use@line10
|
|
1246
|
-
# TC2: is_member=False, total>100 → exercises def@line6, use@line10
|
|
1247
|
-
# TC3: is_member=True, total<=100 → exercises def@line4, use@line12
|
|
1248
|
-
# TC4: is_member=False, total<=100 → exercises def@line6, use@line12
|
|
1249
|
-
```
|
|
1250
|
-
|
|
1251
|
-
### Technique Selection Guide
|
|
1252
|
-
|
|
1253
|
-
| Technique | Best For | Test Level |
|
|
1254
|
-
|-----------|----------|------------|
|
|
1255
|
-
| Equivalence Partitioning | Input validation, ranges | UT, IT |
|
|
1256
|
-
| Boundary Value Analysis | Numeric limits, dates | UT, IT |
|
|
1257
|
-
| Decision Table | Complex business rules | UT, IT, ST |
|
|
1258
|
-
| State Transition | Workflows, status changes | IT, ST |
|
|
1259
|
-
| Use Case Testing | User scenarios | ST, E2E |
|
|
1260
|
-
| Pairwise Testing | Multi-parameter combinations | IT, ST |
|
|
1261
|
-
| Data Flow Testing | Variable lifecycle verification | UT |
|
|
1262
|
-
| Statement Coverage | Basic code coverage | UT |
|
|
1263
|
-
| Branch Coverage | Decision logic | UT |
|
|
1264
|
-
| MC/DC | Safety-critical systems | UT |
|
|
1265
|
-
|
|
1266
|
-
---
|
|
1267
|
-
|
|
1268
|
-
## Experience-Based Testing
|
|
1269
|
-
|
|
1270
|
-
Experience-based testing leverages the tester's knowledge, skills, and intuition to design and execute tests. It complements systematic techniques.
|
|
1271
|
-
|
|
1272
|
-
**Reference**: ISTQB CTFL v4.0 Section 4.4
|
|
1273
|
-
|
|
1274
|
-
### Types of Experience-Based Testing
|
|
1275
|
-
|
|
1276
|
-
#### 1. Exploratory Testing
|
|
1277
|
-
|
|
1278
|
-
Simultaneous test design, execution, and learning. The tester explores the system without predefined scripts.
|
|
1279
|
-
|
|
1280
|
-
```
|
|
1281
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1282
|
-
│ Exploratory Testing Session │
|
|
1283
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1284
|
-
│ Charter: "Explore the checkout process focusing on │
|
|
1285
|
-
│ payment failure scenarios" │
|
|
1286
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1287
|
-
│ Time-box: 60 minutes │
|
|
1288
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1289
|
-
│ Notes: │
|
|
1290
|
-
│ - Tested expired credit card → Got generic error │
|
|
1291
|
-
│ - Tested insufficient funds → Same generic error │
|
|
1292
|
-
│ - Found: No specific error messages for different failures │
|
|
1293
|
-
│ - Bug: Retry with same card doesn't clear previous error │
|
|
1294
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1295
|
-
│ Bugs Found: 2 │
|
|
1296
|
-
│ Areas for Further Testing: Error message specificity │
|
|
1297
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1298
|
-
```
|
|
1299
|
-
|
|
1300
|
-
**Session-Based Test Management (SBTM)**:
|
|
1301
|
-
|
|
1302
|
-
| Element | Description |
|
|
1303
|
-
|---------|-------------|
|
|
1304
|
-
| Charter | Mission statement for the session |
|
|
1305
|
-
| Time-box | Fixed duration (typically 60-90 min) |
|
|
1306
|
-
| Session Notes | Observations, questions, bugs found |
|
|
1307
|
-
| Debrief | Review findings with team |
|
|
1308
|
-
|
|
1309
|
-
#### 2. Error Guessing
|
|
1310
|
-
|
|
1311
|
-
Anticipate defects based on experience with similar systems or common mistakes.
|
|
1312
|
-
|
|
1313
|
-
```
|
|
1314
|
-
Common Error Categories to Guess:
|
|
1315
|
-
|
|
1316
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1317
|
-
│ Input Errors │
|
|
1318
|
-
│ ├── Empty/null inputs │
|
|
1319
|
-
│ ├── Very long strings │
|
|
1320
|
-
│ ├── Special characters (', ", <, >, &, etc.) │
|
|
1321
|
-
│ ├── Unicode/emoji characters │
|
|
1322
|
-
│ └── Negative numbers where positive expected │
|
|
1323
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1324
|
-
│ Boundary Errors │
|
|
1325
|
-
│ ├── Off-by-one errors │
|
|
1326
|
-
│ ├── Integer overflow/underflow │
|
|
1327
|
-
│ ├── Date boundaries (leap years, month ends) │
|
|
1328
|
-
│ └── Array index out of bounds │
|
|
1329
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1330
|
-
│ State Errors │
|
|
1331
|
-
│ ├── Operations on deleted records │
|
|
1332
|
-
│ ├── Concurrent modifications │
|
|
1333
|
-
│ ├── Session timeout during operation │
|
|
1334
|
-
│ └── Browser back button after submission │
|
|
1335
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1336
|
-
│ Environment Errors │
|
|
1337
|
-
│ ├── Network timeout/disconnection │
|
|
1338
|
-
│ ├── Low disk space │
|
|
1339
|
-
│ ├── Database connection pool exhaustion │
|
|
1340
|
-
│ └── Time zone differences │
|
|
1341
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1342
|
-
```
|
|
1343
|
-
|
|
1344
|
-
#### 3. Checklist-Based Testing
|
|
1345
|
-
|
|
1346
|
-
Use predefined checklists based on experience and standards.
|
|
1347
|
-
|
|
1348
|
-
```markdown
|
|
1349
|
-
# API Endpoint Testing Checklist
|
|
1350
|
-
|
|
1351
|
-
## Authentication & Authorization
|
|
1352
|
-
- [ ] Endpoint rejects unauthenticated requests
|
|
1353
|
-
- [ ] Endpoint rejects expired tokens
|
|
1354
|
-
- [ ] Endpoint enforces role-based access
|
|
1355
|
-
- [ ] Different user roles see appropriate data
|
|
1356
|
-
|
|
1357
|
-
## Input Validation
|
|
1358
|
-
- [ ] Required fields are validated
|
|
1359
|
-
- [ ] Data types are enforced
|
|
1360
|
-
- [ ] String length limits enforced
|
|
1361
|
-
- [ ] Malicious input sanitized (XSS, SQL injection)
|
|
1362
|
-
|
|
1363
|
-
## Response Handling
|
|
1364
|
-
- [ ] Success responses have correct status codes
|
|
1365
|
-
- [ ] Error responses have meaningful messages
|
|
1366
|
-
- [ ] Response format matches API documentation
|
|
1367
|
-
- [ ] Pagination works correctly
|
|
1368
|
-
|
|
1369
|
-
## Performance
|
|
1370
|
-
- [ ] Response time under load is acceptable
|
|
1371
|
-
- [ ] No N+1 query problems
|
|
1372
|
-
- [ ] Large datasets handled gracefully
|
|
1373
|
-
```
|
|
1374
|
-
|
|
1375
|
-
### When to Use Experience-Based Testing
|
|
1376
|
-
|
|
1377
|
-
| Scenario | Recommended Technique |
|
|
1378
|
-
|----------|----------------------|
|
|
1379
|
-
| New feature exploration | Exploratory Testing |
|
|
1380
|
-
| Legacy system with no documentation | Exploratory Testing |
|
|
1381
|
-
| Known problematic areas | Error Guessing |
|
|
1382
|
-
| Regression testing | Checklist-Based |
|
|
1383
|
-
| Time pressure / quick validation | Error Guessing |
|
|
1384
|
-
| Compliance verification | Checklist-Based |
|
|
1385
|
-
|
|
1386
|
-
### Combining with Systematic Techniques
|
|
1387
|
-
|
|
1388
|
-
```
|
|
1389
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1390
|
-
│ Optimal Testing Approach │
|
|
1391
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1392
|
-
│ │
|
|
1393
|
-
│ Systematic Techniques Experience-Based │
|
|
1394
|
-
│ (70-80% of effort) (20-30% of effort) │
|
|
1395
|
-
│ │
|
|
1396
|
-
│ ┌──────────────────┐ ┌──────────────────┐ │
|
|
1397
|
-
│ │ Equivalence Part.│ │ Exploratory │ │
|
|
1398
|
-
│ │ Boundary Value │ + │ Error Guessing │ │
|
|
1399
|
-
│ │ Decision Tables │ │ Checklists │ │
|
|
1400
|
-
│ └──────────────────┘ └──────────────────┘ │
|
|
1401
|
-
│ │
|
|
1402
|
-
│ Provides: Provides: │
|
|
1403
|
-
│ - Coverage - Edge cases │
|
|
1404
|
-
│ - Repeatability - Real-world scenarios │
|
|
1405
|
-
│ - Documentation - Quick feedback │
|
|
1406
|
-
│ │
|
|
1407
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1408
|
-
```
|
|
1409
|
-
|
|
1410
|
-
---
|
|
1411
|
-
|
|
1412
|
-
## Risk-Based Testing
|
|
1413
|
-
|
|
1414
|
-
Risk-based testing prioritizes testing efforts based on the likelihood and impact of potential failures.
|
|
1415
|
-
|
|
1416
|
-
**Reference**: ISTQB CTFL v4.0 Section 5.2
|
|
1417
|
-
|
|
1418
|
-
### Risk Assessment
|
|
1419
|
-
|
|
1420
|
-
#### Risk Formula
|
|
1421
|
-
|
|
1422
|
-
```
|
|
1423
|
-
Risk Level = Likelihood × Impact
|
|
1424
|
-
|
|
1425
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1426
|
-
│ Risk Matrix │
|
|
1427
|
-
├─────────────┬───────────────────────────────────────────────┤
|
|
1428
|
-
│ │ Impact │
|
|
1429
|
-
│ Likelihood │ Low (1) Medium (2) High (3) │
|
|
1430
|
-
├─────────────┼───────────────────────────────────────────────┤
|
|
1431
|
-
│ High (3) │ Medium(3) High(6) Critical(9) │
|
|
1432
|
-
│ Medium (2) │ Low(2) Medium(4) High(6) │
|
|
1433
|
-
│ Low (1) │ Low(1) Low(2) Medium(3) │
|
|
1434
|
-
└─────────────┴───────────────────────────────────────────────┘
|
|
1435
|
-
```
|
|
1436
|
-
|
|
1437
|
-
#### Risk Factors
|
|
1438
|
-
|
|
1439
|
-
**Likelihood Factors** (Technical Risk):
|
|
1440
|
-
- Complexity of the feature
|
|
1441
|
-
- New/unfamiliar technology
|
|
1442
|
-
- Code quality metrics
|
|
1443
|
-
- Developer experience
|
|
1444
|
-
- Frequency of changes
|
|
1445
|
-
- Integration complexity
|
|
1446
|
-
|
|
1447
|
-
**Impact Factors** (Business Risk):
|
|
1448
|
-
- Revenue impact
|
|
1449
|
-
- User base affected
|
|
1450
|
-
- Regulatory compliance
|
|
1451
|
-
- Brand reputation
|
|
1452
|
-
- Data security/privacy
|
|
1453
|
-
- Operational criticality
|
|
1454
|
-
|
|
1455
|
-
### Risk-Based Test Prioritization
|
|
1456
|
-
|
|
1457
|
-
```
|
|
1458
|
-
Example: E-commerce Application
|
|
1459
|
-
|
|
1460
|
-
┌──────────────────────┬─────────┬─────────┬───────┬──────────┐
|
|
1461
|
-
│ Feature │Likelihood│ Impact │ Risk │ Priority │
|
|
1462
|
-
├──────────────────────┼─────────┼─────────┼───────┼──────────┤
|
|
1463
|
-
│ Payment Processing │ 2 │ 3 │ 6 │ 1 │
|
|
1464
|
-
│ User Authentication │ 2 │ 3 │ 6 │ 2 │
|
|
1465
|
-
│ Shopping Cart │ 2 │ 2 │ 4 │ 3 │
|
|
1466
|
-
│ Product Search │ 1 │ 2 │ 2 │ 4 │
|
|
1467
|
-
│ Product Reviews │ 1 │ 1 │ 1 │ 5 │
|
|
1468
|
-
│ Wishlist │ 1 │ 1 │ 1 │ 6 │
|
|
1469
|
-
└──────────────────────┴─────────┴─────────┴───────┴──────────┘
|
|
1470
|
-
|
|
1471
|
-
Test Effort Allocation:
|
|
1472
|
-
- Critical (Risk 6+): 50% of test effort, most thorough coverage
|
|
1473
|
-
- Medium (Risk 3-5): 30% of test effort, standard coverage
|
|
1474
|
-
- Low (Risk 1-2): 20% of test effort, basic coverage
|
|
1475
|
-
```
|
|
1476
|
-
|
|
1477
|
-
### Risk-Based Test Planning
|
|
1478
|
-
|
|
1479
|
-
```markdown
|
|
1480
|
-
# Risk-Based Test Plan Template
|
|
1481
|
-
|
|
1482
|
-
## 1. Risk Identification
|
|
1483
|
-
List all features/components and their risk factors.
|
|
1484
|
-
|
|
1485
|
-
## 2. Risk Analysis
|
|
1486
|
-
Calculate risk scores using Likelihood × Impact.
|
|
1487
|
-
|
|
1488
|
-
## 3. Test Prioritization
|
|
1489
|
-
| Priority | Features | Test Depth |
|
|
1490
|
-
|----------|----------|------------|
|
|
1491
|
-
| P1 | Payment, Auth | Full coverage, all techniques |
|
|
1492
|
-
| P2 | Cart, Checkout | Standard coverage |
|
|
1493
|
-
| P3 | Search, Browse | Basic happy path |
|
|
1494
|
-
| P4 | Reviews, Wishlist | Minimal, smoke tests |
|
|
1495
|
-
|
|
1496
|
-
## 4. Risk Mitigation
|
|
1497
|
-
- P1 features: 100% automated tests, manual exploratory
|
|
1498
|
-
- P2 features: 80% automated, sample manual tests
|
|
1499
|
-
- P3 features: Key scenarios automated
|
|
1500
|
-
- P4 features: Basic smoke tests only
|
|
1501
|
-
|
|
1502
|
-
## 5. Residual Risk Acceptance
|
|
1503
|
-
Document accepted risks for low-priority features.
|
|
1504
|
-
```
|
|
1505
|
-
|
|
1506
|
-
### Continuous Risk Assessment
|
|
1507
|
-
|
|
1508
|
-
```
|
|
1509
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1510
|
-
│ Continuous Risk Re-evaluation │
|
|
1511
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1512
|
-
│ │
|
|
1513
|
-
│ Sprint Start ──▶ During Sprint ──▶ Sprint End │
|
|
1514
|
-
│ │ │ │ │
|
|
1515
|
-
│ ▼ ▼ ▼ │
|
|
1516
|
-
│ Initial Risk Update based on: Review: │
|
|
1517
|
-
│ Assessment - Defects found - Actual vs Expected │
|
|
1518
|
-
│ - Code changes - Adjust for next │
|
|
1519
|
-
│ - New requirements sprint │
|
|
1520
|
-
│ │
|
|
1521
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1522
|
-
```
|
|
27
|
+
| Abbreviation | Full Term | Description |
|
|
28
|
+
|--------------|-----------|-------------|
|
|
29
|
+
| **UT** | Unit Testing | Testing individual functions/methods in isolation |
|
|
30
|
+
| **IT** | Integration Testing | Testing interactions between components |
|
|
31
|
+
| **ST** | System Testing | Testing the complete integrated system |
|
|
32
|
+
| **AT** | Acceptance Testing | Testing against business acceptance criteria |
|
|
33
|
+
| **E2E** | End-to-End Testing | Testing complete user workflows |
|
|
34
|
+
| **UAT** | User Acceptance Testing | Acceptance testing performed by end users |
|
|
35
|
+
| **SIT** | System Integration Testing | Testing integration of multiple systems |
|
|
1523
36
|
|
|
1524
|
-
|
|
1525
|
-
|
|
1526
|
-
## Specialized Testing
|
|
1527
|
-
|
|
1528
|
-
Modern software development requires specialized testing types beyond traditional categories.
|
|
1529
|
-
|
|
1530
|
-
### Contract Testing
|
|
1531
|
-
|
|
1532
|
-
Verifies that services communicate correctly according to agreed contracts. Essential for microservices architectures.
|
|
1533
|
-
|
|
1534
|
-
```
|
|
1535
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1536
|
-
│ Consumer-Driven Contract Testing │
|
|
1537
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1538
|
-
│ │
|
|
1539
|
-
│ Consumer Provider │
|
|
1540
|
-
│ (Frontend) (API) │
|
|
1541
|
-
│ │ │ │
|
|
1542
|
-
│ │ 1. Define expectations │ │
|
|
1543
|
-
│ │─────────────────────────────▶ │ │
|
|
1544
|
-
│ │ │ │
|
|
1545
|
-
│ │ 2. Generate contract │ │
|
|
1546
|
-
│ │ (Pact file) │ │
|
|
1547
|
-
│ │ │ │
|
|
1548
|
-
│ │ 3. Share contract │ │
|
|
1549
|
-
│ │─────────────────────────────▶ │ │
|
|
1550
|
-
│ │ │ │
|
|
1551
|
-
│ │ 4. Provider verifies │
|
|
1552
|
-
│ │ against contract │
|
|
1553
|
-
│ │ │ │
|
|
1554
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1555
|
-
```
|
|
1556
|
-
|
|
1557
|
-
**Tools**: Pact, Spring Cloud Contract, Postman
|
|
1558
|
-
|
|
1559
|
-
```typescript
|
|
1560
|
-
// Consumer Test (Pact.js Example)
|
|
1561
|
-
describe('User API Contract', () => {
|
|
1562
|
-
it('returns user by ID', async () => {
|
|
1563
|
-
await provider.addInteraction({
|
|
1564
|
-
state: 'user with ID 123 exists',
|
|
1565
|
-
uponReceiving: 'a request for user 123',
|
|
1566
|
-
withRequest: {
|
|
1567
|
-
method: 'GET',
|
|
1568
|
-
path: '/users/123'
|
|
1569
|
-
},
|
|
1570
|
-
willRespondWith: {
|
|
1571
|
-
status: 200,
|
|
1572
|
-
body: {
|
|
1573
|
-
id: '123',
|
|
1574
|
-
name: Matchers.string('John Doe'),
|
|
1575
|
-
email: Matchers.email()
|
|
1576
|
-
}
|
|
1577
|
-
}
|
|
1578
|
-
});
|
|
1579
|
-
|
|
1580
|
-
const user = await userClient.getUser('123');
|
|
1581
|
-
expect(user.id).toBe('123');
|
|
1582
|
-
});
|
|
1583
|
-
});
|
|
1584
|
-
```
|
|
1585
|
-
|
|
1586
|
-
### Mutation Testing
|
|
1587
|
-
|
|
1588
|
-
Evaluates test suite quality by introducing small changes (mutants) to the code and checking if tests detect them.
|
|
1589
|
-
|
|
1590
|
-
```
|
|
1591
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1592
|
-
│ Mutation Testing │
|
|
1593
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1594
|
-
│ │
|
|
1595
|
-
│ Original Code: Mutant (change operator): │
|
|
1596
|
-
│ if (a > b) if (a >= b) ← Boundary mutant │
|
|
1597
|
-
│ if (a > b) if (a < b) ← Negation mutant │
|
|
1598
|
-
│ return a + b; return a - b; ← Arithmetic mutant │
|
|
1599
|
-
│ return true; return false; ← Return value mutant │
|
|
1600
|
-
│ │
|
|
1601
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1602
|
-
│ Mutation Score = (Killed Mutants / Total Mutants) × 100 │
|
|
1603
|
-
│ │
|
|
1604
|
-
│ - Killed: Test failed → Good, test detected the change │
|
|
1605
|
-
│ - Survived: Test passed → Bad, test missed the defect │
|
|
1606
|
-
│ │
|
|
1607
|
-
│ Target: > 80% mutation score for critical code │
|
|
1608
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1609
|
-
```
|
|
1610
|
-
|
|
1611
|
-
**Tools**: Stryker (JS/TS), PITest (Java), mutmut (Python)
|
|
1612
|
-
|
|
1613
|
-
```bash
|
|
1614
|
-
# Run mutation testing with Stryker
|
|
1615
|
-
npx stryker run
|
|
1616
|
-
|
|
1617
|
-
# Example output:
|
|
1618
|
-
# Mutation score: 85.7%
|
|
1619
|
-
# Killed: 180 Survived: 30 No coverage: 10
|
|
1620
|
-
```
|
|
1621
|
-
|
|
1622
|
-
### Chaos Engineering
|
|
1623
|
-
|
|
1624
|
-
Proactively tests system resilience by injecting failures in production or staging environments.
|
|
1625
|
-
|
|
1626
|
-
```
|
|
1627
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1628
|
-
│ Chaos Engineering Principles │
|
|
1629
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1630
|
-
│ │
|
|
1631
|
-
│ 1. Define "steady state" (normal behavior metrics) │
|
|
1632
|
-
│ 2. Hypothesize steady state continues during chaos │
|
|
1633
|
-
│ 3. Introduce real-world events: │
|
|
1634
|
-
│ - Server failures │
|
|
1635
|
-
│ - Network latency/partition │
|
|
1636
|
-
│ - Resource exhaustion │
|
|
1637
|
-
│ - Clock skew │
|
|
1638
|
-
│ 4. Try to disprove the hypothesis │
|
|
1639
|
-
│ 5. Minimize blast radius (start small) │
|
|
1640
|
-
│ │
|
|
1641
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1642
|
-
```
|
|
1643
|
-
|
|
1644
|
-
**Tools**: Chaos Monkey, Gremlin, Litmus, Chaos Toolkit
|
|
1645
|
-
|
|
1646
|
-
```yaml
|
|
1647
|
-
# Chaos Toolkit Experiment Example
|
|
1648
|
-
title: "Service resilience when database is slow"
|
|
1649
|
-
description: "Verify the service degrades gracefully"
|
|
1650
|
-
|
|
1651
|
-
steady-state-hypothesis:
|
|
1652
|
-
title: "Service responds within SLA"
|
|
1653
|
-
probes:
|
|
1654
|
-
- name: "service-responds"
|
|
1655
|
-
type: probe
|
|
1656
|
-
provider:
|
|
1657
|
-
type: http
|
|
1658
|
-
url: "http://service/health"
|
|
1659
|
-
timeout: 3
|
|
1660
|
-
|
|
1661
|
-
method:
|
|
1662
|
-
- name: "inject-db-latency"
|
|
1663
|
-
type: action
|
|
1664
|
-
provider:
|
|
1665
|
-
type: process
|
|
1666
|
-
path: "tc"
|
|
1667
|
-
arguments: "qdisc add dev eth0 root netem delay 500ms"
|
|
1668
|
-
pauses:
|
|
1669
|
-
after: 30
|
|
1670
|
-
|
|
1671
|
-
rollbacks:
|
|
1672
|
-
- name: "remove-latency"
|
|
1673
|
-
type: action
|
|
1674
|
-
provider:
|
|
1675
|
-
type: process
|
|
1676
|
-
path: "tc"
|
|
1677
|
-
arguments: "qdisc del dev eth0 root"
|
|
1678
|
-
```
|
|
1679
|
-
|
|
1680
|
-
### Property-Based Testing
|
|
1681
|
-
|
|
1682
|
-
Generates random test inputs based on properties the code should satisfy.
|
|
1683
|
-
|
|
1684
|
-
```python
|
|
1685
|
-
# Python Example with Hypothesis
|
|
1686
|
-
from hypothesis import given, strategies as st
|
|
1687
|
-
|
|
1688
|
-
# Property: Sorting should be idempotent
|
|
1689
|
-
@given(st.lists(st.integers()))
|
|
1690
|
-
def test_sort_idempotent(xs):
|
|
1691
|
-
sorted_once = sorted(xs)
|
|
1692
|
-
sorted_twice = sorted(sorted_once)
|
|
1693
|
-
assert sorted_once == sorted_twice
|
|
1694
|
-
|
|
1695
|
-
# Property: Sorted list should have same elements
|
|
1696
|
-
@given(st.lists(st.integers()))
|
|
1697
|
-
def test_sort_preserves_elements(xs):
|
|
1698
|
-
sorted_xs = sorted(xs)
|
|
1699
|
-
assert sorted(xs) == sorted(sorted_xs)
|
|
1700
|
-
assert len(xs) == len(sorted_xs)
|
|
1701
|
-
|
|
1702
|
-
# Property: JSON encode/decode roundtrip
|
|
1703
|
-
@given(st.dictionaries(st.text(), st.integers()))
|
|
1704
|
-
def test_json_roundtrip(d):
|
|
1705
|
-
assert json.loads(json.dumps(d)) == d
|
|
1706
|
-
```
|
|
1707
|
-
|
|
1708
|
-
**Tools**: Hypothesis (Python), fast-check (JS/TS), QuickCheck (Haskell), FsCheck (C#)
|
|
1709
|
-
|
|
1710
|
-
### Visual Regression Testing
|
|
1711
|
-
|
|
1712
|
-
Detects unintended visual changes in UI components.
|
|
1713
|
-
|
|
1714
|
-
```
|
|
1715
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1716
|
-
│ Visual Regression Testing │
|
|
1717
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1718
|
-
│ │
|
|
1719
|
-
│ 1. Capture baseline screenshots │
|
|
1720
|
-
│ 2. Run tests, capture new screenshots │
|
|
1721
|
-
│ 3. Compare pixel-by-pixel or perceptually │
|
|
1722
|
-
│ 4. Flag differences for review │
|
|
1723
|
-
│ │
|
|
1724
|
-
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
|
1725
|
-
│ │ Baseline │ → │ Current │ → │ Diff │ │
|
|
1726
|
-
│ │ Screenshot │ │ Screenshot │ │ Report │ │
|
|
1727
|
-
│ └─────────────┘ └─────────────┘ └─────────────┘ │
|
|
1728
|
-
│ │
|
|
1729
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1730
|
-
```
|
|
1731
|
-
|
|
1732
|
-
**Tools**: Percy, Chromatic, BackstopJS, Playwright visual comparisons
|
|
1733
|
-
|
|
1734
|
-
```typescript
|
|
1735
|
-
// Playwright Visual Regression Example
|
|
1736
|
-
test('homepage visual regression', async ({ page }) => {
|
|
1737
|
-
await page.goto('/');
|
|
1738
|
-
await expect(page).toHaveScreenshot('homepage.png', {
|
|
1739
|
-
maxDiffPixels: 100 // Allow minor differences
|
|
1740
|
-
});
|
|
1741
|
-
});
|
|
1742
|
-
|
|
1743
|
-
test('button component visual regression', async ({ page }) => {
|
|
1744
|
-
await page.goto('/components/button');
|
|
1745
|
-
const button = page.locator('[data-testid="primary-button"]');
|
|
1746
|
-
await expect(button).toHaveScreenshot('primary-button.png');
|
|
1747
|
-
});
|
|
1748
|
-
```
|
|
1749
|
-
|
|
1750
|
-
### Accessibility Testing (a11y)
|
|
1751
|
-
|
|
1752
|
-
Ensures applications are usable by people with disabilities.
|
|
1753
|
-
|
|
1754
|
-
```
|
|
1755
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1756
|
-
│ Accessibility Testing Checklist │
|
|
1757
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1758
|
-
│ WCAG 2.1 Guidelines │
|
|
1759
|
-
│ ├── Perceivable │
|
|
1760
|
-
│ │ ├── Alt text for images │
|
|
1761
|
-
│ │ ├── Captions for video │
|
|
1762
|
-
│ │ └── Sufficient color contrast │
|
|
1763
|
-
│ ├── Operable │
|
|
1764
|
-
│ │ ├── Keyboard navigation │
|
|
1765
|
-
│ │ ├── Focus indicators visible │
|
|
1766
|
-
│ │ └── No keyboard traps │
|
|
1767
|
-
│ ├── Understandable │
|
|
1768
|
-
│ │ ├── Clear labels and instructions │
|
|
1769
|
-
│ │ ├── Predictable navigation │
|
|
1770
|
-
│ │ └── Error identification │
|
|
1771
|
-
│ └── Robust │
|
|
1772
|
-
│ ├── Valid HTML │
|
|
1773
|
-
│ └── ARIA attributes correct │
|
|
1774
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1775
|
-
```
|
|
1776
|
-
|
|
1777
|
-
**Tools**: axe-core, Pa11y, WAVE, Lighthouse
|
|
1778
|
-
|
|
1779
|
-
```typescript
|
|
1780
|
-
// Playwright + axe-core Example
|
|
1781
|
-
import AxeBuilder from '@axe-core/playwright';
|
|
1782
|
-
|
|
1783
|
-
test('homepage accessibility', async ({ page }) => {
|
|
1784
|
-
await page.goto('/');
|
|
1785
|
-
|
|
1786
|
-
const results = await new AxeBuilder({ page })
|
|
1787
|
-
.withTags(['wcag2a', 'wcag2aa'])
|
|
1788
|
-
.analyze();
|
|
1789
|
-
|
|
1790
|
-
expect(results.violations).toEqual([]);
|
|
1791
|
-
});
|
|
1792
|
-
```
|
|
1793
|
-
|
|
1794
|
-
### Additional Specialized Testing Types (SWEBOK)
|
|
1795
|
-
|
|
1796
|
-
The following test types are defined in SWEBOK v4.0 as test objectives:
|
|
1797
|
-
|
|
1798
|
-
#### Installation Testing
|
|
1799
|
-
|
|
1800
|
-
Verifies that the software installs correctly in the target environment.
|
|
1801
|
-
|
|
1802
|
-
```
|
|
1803
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1804
|
-
│ Installation Testing Checklist │
|
|
1805
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1806
|
-
│ │
|
|
1807
|
-
│ Pre-Installation │
|
|
1808
|
-
│ ├── Prerequisites verified (OS, runtime, dependencies) │
|
|
1809
|
-
│ ├── Disk space requirements met │
|
|
1810
|
-
│ └── Permissions and access rights confirmed │
|
|
1811
|
-
│ │
|
|
1812
|
-
│ Installation Process │
|
|
1813
|
-
│ ├── Fresh installation succeeds │
|
|
1814
|
-
│ ├── Upgrade from previous version succeeds │
|
|
1815
|
-
│ ├── Custom installation options work │
|
|
1816
|
-
│ └── Silent/unattended installation works │
|
|
1817
|
-
│ │
|
|
1818
|
-
│ Post-Installation │
|
|
1819
|
-
│ ├── Application launches correctly │
|
|
1820
|
-
│ ├── Configuration files created properly │
|
|
1821
|
-
│ ├── Registry/system settings correct │
|
|
1822
|
-
│ └── Uninstallation removes all components │
|
|
1823
|
-
│ │
|
|
1824
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1825
|
-
```
|
|
1826
|
-
|
|
1827
|
-
#### Alpha and Beta Testing
|
|
1828
|
-
|
|
1829
|
-
| Phase | Participants | Environment | Purpose |
|
|
1830
|
-
|-------|--------------|-------------|---------|
|
|
1831
|
-
| **Alpha** | Internal users, developers | Development site | Early validation, major issues |
|
|
1832
|
-
| **Beta** | External users, customers | Customer site | Real-world validation, feedback |
|
|
1833
|
-
| **Open Beta** | General public | Various | Wide exposure, stress testing |
|
|
1834
|
-
|
|
1835
|
-
#### Recovery Testing
|
|
1836
|
-
|
|
1837
|
-
Tests the system's ability to recover from failures.
|
|
1838
|
-
|
|
1839
|
-
```
|
|
1840
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1841
|
-
│ Recovery Testing Scenarios │
|
|
1842
|
-
├─────────────────────────────────────────────────────────────┤
|
|
1843
|
-
│ │
|
|
1844
|
-
│ Hardware Failures │
|
|
1845
|
-
│ ├── Power outage during transaction │
|
|
1846
|
-
│ ├── Disk failure and RAID recovery │
|
|
1847
|
-
│ └── Network disconnection and reconnection │
|
|
1848
|
-
│ │
|
|
1849
|
-
│ Software Failures │
|
|
1850
|
-
│ ├── Application crash recovery │
|
|
1851
|
-
│ ├── Database connection loss and reconnect │
|
|
1852
|
-
│ └── Service restart behavior │
|
|
1853
|
-
│ │
|
|
1854
|
-
│ Data Recovery │
|
|
1855
|
-
│ ├── Backup and restore procedures │
|
|
1856
|
-
│ ├── Transaction rollback │
|
|
1857
|
-
│ └── Point-in-time recovery │
|
|
1858
|
-
│ │
|
|
1859
|
-
│ Metrics: │
|
|
1860
|
-
│ • Recovery Time Objective (RTO): Max acceptable downtime │
|
|
1861
|
-
│ • Recovery Point Objective (RPO): Max acceptable data loss │
|
|
1862
|
-
│ │
|
|
1863
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1864
|
-
```
|
|
37
|
+
> **Note**: "IT" in this document always refers to "Integration Testing", not "Information Technology".
|
|
1865
38
|
|
|
1866
|
-
|
|
1867
|
-
|
|
1868
|
-
Tests the system under different configurations.
|
|
1869
|
-
|
|
1870
|
-
```
|
|
1871
|
-
Configuration Matrix Example:
|
|
1872
|
-
|
|
1873
|
-
┌──────────────────┬─────────┬─────────┬─────────┐
|
|
1874
|
-
│ Configuration │ Config A│ Config B│ Config C│
|
|
1875
|
-
├──────────────────┼─────────┼─────────┼─────────┤
|
|
1876
|
-
│ Memory │ 4 GB │ 8 GB │ 16 GB │
|
|
1877
|
-
│ Database │ SQLite │ MySQL │ PostgreSQL│
|
|
1878
|
-
│ Cache │ None │ Redis │ Memcached│
|
|
1879
|
-
│ Load Balancer │ None │ Nginx │ HAProxy │
|
|
1880
|
-
└──────────────────┴─────────┴─────────┴─────────┘
|
|
1881
|
-
|
|
1882
|
-
Test each supported configuration combination.
|
|
1883
|
-
```
|
|
1884
|
-
|
|
1885
|
-
#### Back-to-Back Testing
|
|
39
|
+
---
|
|
1886
40
|
|
|
1887
|
-
|
|
41
|
+
## Testing Framework Selection
|
|
1888
42
|
|
|
1889
|
-
|
|
1890
|
-
|
|
1891
|
-
|
|
1892
|
-
|
|
1893
|
-
│ │
|
|
1894
|
-
│ Use Cases: │
|
|
1895
|
-
│ ├── Comparing new version against old version │
|
|
1896
|
-
│ ├── Comparing optimized code against reference impl │
|
|
1897
|
-
│ ├── Cross-platform consistency verification │
|
|
1898
|
-
│ └── Validating refactored code behavior │
|
|
1899
|
-
│ │
|
|
1900
|
-
│ Process: │
|
|
1901
|
-
│ ┌─────────┐ ┌─────────────┐ ┌──────────┐ │
|
|
1902
|
-
│ │ Input │────▶│ Version A │────▶│ Output A │──┐ │
|
|
1903
|
-
│ │ Data │ └─────────────┘ └──────────┘ │ │
|
|
1904
|
-
│ │ │ ┌─────────────┐ ┌──────────┐ ▼ │
|
|
1905
|
-
│ │ │────▶│ Version B │────▶│ Output B │─▶Compare │
|
|
1906
|
-
│ └─────────┘ └─────────────┘ └──────────┘ │
|
|
1907
|
-
│ │
|
|
1908
|
-
└─────────────────────────────────────────────────────────────┘
|
|
1909
|
-
```
|
|
43
|
+
| Framework | Levels | Best For |
|
|
44
|
+
|-----------|--------|----------|
|
|
45
|
+
| **ISTQB** | UT → IT/SIT → ST → AT/UAT | Enterprise, compliance, formal QA |
|
|
46
|
+
| **Industry Pyramid** | UT (70%) → IT (20%) → E2E (10%) | Agile, DevOps, CI/CD |
|
|
1910
47
|
|
|
1911
|
-
|
|
48
|
+
---
|
|
1912
49
|
|
|
1913
|
-
|
|
50
|
+
## Testing Pyramid (Default Ratios)
|
|
1914
51
|
|
|
1915
52
|
```
|
|
1916
|
-
|
|
1917
|
-
│
|
|
1918
|
-
|
|
1919
|
-
|
|
1920
|
-
│
|
|
1921
|
-
|
|
1922
|
-
|
|
1923
|
-
│
|
|
1924
|
-
|
|
1925
|
-
│ Task-Based Methods │
|
|
1926
|
-
│ ├── Task completion rate │
|
|
1927
|
-
│ ├── Time on task │
|
|
1928
|
-
│ └── Error rate │
|
|
1929
|
-
│ │
|
|
1930
|
-
│ Survey Methods │
|
|
1931
|
-
│ ├── System Usability Scale (SUS) │
|
|
1932
|
-
│ ├── Net Promoter Score (NPS) │
|
|
1933
|
-
│ └── Post-task questionnaires │
|
|
1934
|
-
│ │
|
|
1935
|
-
│ Key Metrics: │
|
|
1936
|
-
│ • Learnability: Time to complete first task │
|
|
1937
|
-
│ • Efficiency: Time to complete after learning │
|
|
1938
|
-
│ • Memorability: Performance after period of non-use │
|
|
1939
|
-
│ • Satisfaction: User rating scores │
|
|
1940
|
-
│ │
|
|
1941
|
-
└─────────────────────────────────────────────────────────────┘
|
|
53
|
+
┌─────────┐
|
|
54
|
+
│ E2E │ ← 10% (Fewer, slower, expensive)
|
|
55
|
+
─┴─────────┴─
|
|
56
|
+
┌─────────────┐
|
|
57
|
+
│ IT/SIT │ ← 20% (Integration Testing)
|
|
58
|
+
─┴─────────────┴─
|
|
59
|
+
┌─────────────────┐
|
|
60
|
+
│ UT │ ← 70% (Unit Testing - Foundation)
|
|
61
|
+
└─────────────────┘
|
|
1942
62
|
```
|
|
1943
63
|
|
|
1944
|
-
|
|
1945
|
-
|
|
1946
|
-
|
|
|
1947
|
-
|
|
1948
|
-
|
|
|
1949
|
-
| Mutation Testing | Test quality assessment | Critical code paths |
|
|
1950
|
-
| Chaos Engineering | Resilience validation | Production systems |
|
|
1951
|
-
| Property-Based | Edge case discovery | Algorithms, parsers |
|
|
1952
|
-
| Visual Regression | UI consistency | Frontend changes |
|
|
1953
|
-
| Accessibility | Inclusive design | All user-facing apps |
|
|
1954
|
-
| Installation | Deployment verification | Release candidates |
|
|
1955
|
-
| Alpha/Beta | Early user validation | Pre-release phases |
|
|
1956
|
-
| Recovery | Failure recovery | Business-critical systems |
|
|
1957
|
-
| Configuration | Multi-config support | Cross-platform apps |
|
|
1958
|
-
| Back-to-Back | Version comparison | Migrations, refactoring |
|
|
1959
|
-
| Usability | User experience | User-facing applications |
|
|
64
|
+
| Level | Percentage | Execution Time Target |
|
|
65
|
+
|-------|------------|----------------------|
|
|
66
|
+
| Unit Testing (UT) | 70% | < 10 min total |
|
|
67
|
+
| Integration Testing (IT) | 20% | < 30 min total |
|
|
68
|
+
| E2E Testing | 10% | < 2 hours total |
|
|
1960
69
|
|
|
1961
70
|
---
|
|
1962
71
|
|
|
1963
|
-
## Test
|
|
1964
|
-
|
|
1965
|
-
Quantitative measures for evaluating both the software under test and the quality of testing itself.
|
|
1966
|
-
|
|
1967
|
-
**Reference**: SWEBOK v4.0 Section 5.4
|
|
72
|
+
## Test Level Requirements
|
|
1968
73
|
|
|
1969
|
-
###
|
|
74
|
+
### Unit Testing (UT)
|
|
1970
75
|
|
|
1971
|
-
|
|
1972
|
-
|
|
1973
|
-
```
|
|
1974
|
-
Fault Density = Total Faults / Size
|
|
1975
|
-
|
|
1976
|
-
Where Size can be:
|
|
1977
|
-
- KLOC (Thousands of Lines of Code)
|
|
1978
|
-
- Function Points
|
|
1979
|
-
- Story Points
|
|
1980
|
-
|
|
1981
|
-
Industry Benchmarks:
|
|
1982
|
-
┌─────────────────────┬────────────────────────────┐
|
|
1983
|
-
│ Quality Level │ Faults per KLOC │
|
|
1984
|
-
├─────────────────────┼────────────────────────────┤
|
|
1985
|
-
│ Typical │ 1 - 25 │
|
|
1986
|
-
│ Good │ 0.5 - 1 │
|
|
1987
|
-
│ Excellent │ < 0.5 │
|
|
1988
|
-
│ Safety-Critical │ < 0.1 │
|
|
1989
|
-
└─────────────────────┴────────────────────────────┘
|
|
1990
|
-
```
|
|
1991
|
-
|
|
1992
|
-
#### Reliability Growth Models
|
|
1993
|
-
|
|
1994
|
-
Track defect discovery rate over time to predict remaining defects.
|
|
1995
|
-
|
|
1996
|
-
```
|
|
1997
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
1998
|
-
│ Reliability Growth Curve │
|
|
1999
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2000
|
-
│ │
|
|
2001
|
-
│ Defects │
|
|
2002
|
-
│ Found │
|
|
2003
|
-
│ │ ╭──────────────────────────────── │
|
|
2004
|
-
│ │ ╭─╯ │
|
|
2005
|
-
│ │ ╭╯ │
|
|
2006
|
-
│ │ ╭╯ │
|
|
2007
|
-
│ │╭╯ │
|
|
2008
|
-
│ ├╯ │
|
|
2009
|
-
│ └──────────────────────────────────────▶ Time │
|
|
2010
|
-
│ │
|
|
2011
|
-
│ As testing progresses: │
|
|
2012
|
-
│ • Defect discovery rate decreases │
|
|
2013
|
-
│ • Curve approaches asymptote (total defects) │
|
|
2014
|
-
│ • Remaining defects can be estimated │
|
|
2015
|
-
│ │
|
|
2016
|
-
│ Common Models: │
|
|
2017
|
-
│ • Goel-Okumoto: Exponential │
|
|
2018
|
-
│ • Musa: Logarithmic │
|
|
2019
|
-
│ • Jelinski-Moranda: De-eutrophication │
|
|
2020
|
-
│ │
|
|
2021
|
-
└─────────────────────────────────────────────────────────────┘
|
|
2022
|
-
```
|
|
76
|
+
**Characteristics**: Isolated, Fast (< 100ms each), Deterministic
|
|
2023
77
|
|
|
2024
|
-
|
|
78
|
+
#### Scope
|
|
2025
79
|
|
|
2026
|
-
|
|
80
|
+
| Include | Exclude |
|
|
81
|
+
|---------|---------|
|
|
82
|
+
| Single function/method | Database queries |
|
|
83
|
+
| Single class | External API calls |
|
|
84
|
+
| Pure business logic | File I/O operations |
|
|
85
|
+
| Data transformations | Multi-class interactions |
|
|
86
|
+
| Validation rules | Network calls |
|
|
2027
87
|
|
|
2028
|
-
|
|
88
|
+
#### Naming Convention
|
|
2029
89
|
|
|
90
|
+
**File Naming**:
|
|
2030
91
|
```
|
|
2031
|
-
|
|
2032
|
-
|
|
2033
|
-
|
|
2034
|
-
- Similar detection rate for real faults
|
|
2035
|
-
- Remaining faults ≈ Found faults / Detection rate
|
|
2036
|
-
|
|
2037
|
-
Example:
|
|
2038
|
-
- Seeded 20 faults, tests found 16 (80%)
|
|
2039
|
-
- Found 40 real faults during testing
|
|
2040
|
-
- Estimated total real faults ≈ 40 / 0.8 = 50
|
|
2041
|
-
- Estimated remaining ≈ 10 faults
|
|
92
|
+
[ClassName]Tests.[ext] # C#
|
|
93
|
+
[ClassName].test.[ext] # TypeScript/JavaScript
|
|
94
|
+
[class_name]_test.[ext] # Python, Go
|
|
2042
95
|
```
|
|
2043
96
|
|
|
2044
|
-
|
|
97
|
+
**Method Naming** (choose ONE per project):
|
|
2045
98
|
|
|
2046
|
-
|
|
|
2047
|
-
|
|
2048
|
-
|
|
|
2049
|
-
|
|
|
2050
|
-
|
|
|
2051
|
-
| MC/DC | Conditions independently affecting decision | 100% (safety-critical) |
|
|
2052
|
-
| Requirements Coverage | Tested requirements / Total requirements | 100% |
|
|
2053
|
-
| Mutation Score | Killed mutants / Total mutants | ≥ 80% |
|
|
99
|
+
| Style | Best For | Example |
|
|
100
|
+
|-------|----------|---------|
|
|
101
|
+
| `[Method]_[Scenario]_[Result]` | C#, Java | `CalculateTotal_NegativePrice_ThrowsException()` |
|
|
102
|
+
| `should_[behavior]_when_[condition]` | JavaScript/TypeScript | `should_reject_login_when_account_locked()` |
|
|
103
|
+
| `test_[method]_[scenario]_[expected]` | Python (pytest) | `test_validate_email_invalid_format_returns_false()` |
|
|
2054
104
|
|
|
2055
|
-
|
|
105
|
+
#### Coverage Thresholds
|
|
2056
106
|
|
|
2057
|
-
|
|
2058
|
-
|
|
2059
|
-
|
|
2060
|
-
|
|
2061
|
-
|
|
2062
|
-
│ Efficiency Metrics │
|
|
2063
|
-
│ ├── Test Case Productivity = Tests created / Effort │
|
|
2064
|
-
│ ├── Defect Detection Rate = Defects found / Test hours │
|
|
2065
|
-
│ └── Automation Rate = Automated tests / Total tests │
|
|
2066
|
-
│ │
|
|
2067
|
-
│ Effectiveness Metrics │
|
|
2068
|
-
│ ├── Defect Leakage = Prod defects / Total defects │
|
|
2069
|
-
│ ├── Test Effectiveness = Defects found / Defects present │
|
|
2070
|
-
│ └── Phase Containment = Defects found in phase / Injected │
|
|
2071
|
-
│ │
|
|
2072
|
-
│ Progress Metrics │
|
|
2073
|
-
│ ├── Test Execution Rate = Tests run / Tests planned │
|
|
2074
|
-
│ ├── Pass Rate = Tests passed / Tests executed │
|
|
2075
|
-
│ └── Blocking Rate = Blocked tests / Total tests │
|
|
2076
|
-
│ │
|
|
2077
|
-
└─────────────────────────────────────────────────────────────┘
|
|
2078
|
-
```
|
|
107
|
+
| Metric | Minimum | Recommended |
|
|
108
|
+
|--------|---------|-------------|
|
|
109
|
+
| Line Coverage | 70% | 85% |
|
|
110
|
+
| Branch Coverage | 60% | 80% |
|
|
111
|
+
| Function Coverage | 80% | 90% |
|
|
2079
112
|
|
|
2080
113
|
---
|
|
2081
114
|
|
|
2082
|
-
|
|
2083
|
-
|
|
2084
|
-
Systematic tracking and management of defects throughout the testing lifecycle.
|
|
2085
|
-
|
|
2086
|
-
**Reference**: ISTQB CTFL v4.0 Section 5.5, ISO/IEC/IEEE 29119-3
|
|
2087
|
-
|
|
2088
|
-
### Defect Lifecycle
|
|
2089
|
-
|
|
2090
|
-
```
|
|
2091
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
2092
|
-
│ Defect Lifecycle │
|
|
2093
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2094
|
-
│ │
|
|
2095
|
-
│ ┌────────┐ ┌────────┐ ┌──────────┐ │
|
|
2096
|
-
│ │ New │───▶│ Open │───▶│ In Progress│ │
|
|
2097
|
-
│ └────────┘ └────────┘ └──────────┘ │
|
|
2098
|
-
│ │ │ │
|
|
2099
|
-
│ │ ▼ │
|
|
2100
|
-
│ │ ┌──────────┐ │
|
|
2101
|
-
│ │ │ Fixed │ │
|
|
2102
|
-
│ │ └──────────┘ │
|
|
2103
|
-
│ │ │ │
|
|
2104
|
-
│ │ ▼ │
|
|
2105
|
-
│ │ ┌──────────┐ ┌──────────┐ │
|
|
2106
|
-
│ │ │ Verified │───▶│ Closed │ │
|
|
2107
|
-
│ │ └──────────┘ └──────────┘ │
|
|
2108
|
-
│ │ │ │
|
|
2109
|
-
│ │ ▼ │
|
|
2110
|
-
│ │ ┌──────────┐ │
|
|
2111
|
-
│ └───────▶│ Reopened │ │
|
|
2112
|
-
│ └──────────┘ │
|
|
2113
|
-
│ │
|
|
2114
|
-
│ Alternative paths: │
|
|
2115
|
-
│ - New → Rejected (not a bug, duplicate) │
|
|
2116
|
-
│ - Open → Deferred (postponed to later release) │
|
|
2117
|
-
│ - Open → Won't Fix (accepted risk) │
|
|
2118
|
-
│ │
|
|
2119
|
-
└─────────────────────────────────────────────────────────────┘
|
|
2120
|
-
```
|
|
2121
|
-
|
|
2122
|
-
### Defect Report Template
|
|
2123
|
-
|
|
2124
|
-
```markdown
|
|
2125
|
-
# Defect Report
|
|
2126
|
-
|
|
2127
|
-
## Summary
|
|
2128
|
-
[One-line description of the defect]
|
|
2129
|
-
|
|
2130
|
-
## ID: BUG-1234
|
|
2131
|
-
## Priority: P1/P2/P3/P4
|
|
2132
|
-
## Severity: Critical/High/Medium/Low
|
|
2133
|
-
|
|
2134
|
-
## Environment
|
|
2135
|
-
- **Version**: 2.3.1
|
|
2136
|
-
- **OS**: Windows 11 / macOS 14.2
|
|
2137
|
-
- **Browser**: Chrome 120
|
|
2138
|
-
- **Device**: Desktop / Mobile
|
|
2139
|
-
|
|
2140
|
-
## Steps to Reproduce
|
|
2141
|
-
1. Navigate to login page
|
|
2142
|
-
2. Enter valid username "testuser"
|
|
2143
|
-
3. Enter invalid password "wrong123"
|
|
2144
|
-
4. Click "Login" button
|
|
2145
|
-
5. Observe error message
|
|
2146
|
-
|
|
2147
|
-
## Expected Result
|
|
2148
|
-
Error message: "Invalid credentials. Please try again."
|
|
2149
|
-
|
|
2150
|
-
## Actual Result
|
|
2151
|
-
Generic error: "Something went wrong."
|
|
115
|
+
### Integration Testing (IT)
|
|
2152
116
|
|
|
2153
|
-
|
|
2154
|
-
[Attach relevant media]
|
|
117
|
+
**Characteristics**: Component integration, Real dependencies (often containerized), 1-10 seconds each
|
|
2155
118
|
|
|
2156
|
-
|
|
2157
|
-
- Occurs consistently (100% reproducible)
|
|
2158
|
-
- Related to ticket: FEAT-456
|
|
2159
|
-
- Log file: [attached]
|
|
2160
|
-
```
|
|
2161
|
-
|
|
2162
|
-
### Severity vs Priority
|
|
119
|
+
#### When Required
|
|
2163
120
|
|
|
2164
|
-
|
|
2165
|
-
|----------|-------------|----------|
|
|
2166
|
-
| Critical | System crash, data loss, security breach | App won't start, data corruption |
|
|
2167
|
-
| High | Major feature broken, no workaround | Cannot complete checkout |
|
|
2168
|
-
| Medium | Feature impaired but has workaround | Export fails, manual copy works |
|
|
2169
|
-
| Low | Minor issue, cosmetic | Typo, alignment issue |
|
|
121
|
+
**Decision Rule**: If your unit test uses a wildcard matcher (`any()`, `It.IsAny<>`, `Arg.Any<>`) for a query/filter parameter, that functionality MUST have an integration test.
|
|
2170
122
|
|
|
2171
|
-
|
|
|
2172
|
-
|
|
2173
|
-
|
|
|
2174
|
-
|
|
|
2175
|
-
|
|
|
2176
|
-
|
|
|
123
|
+
| Scenario | Reason |
|
|
124
|
+
|----------|--------|
|
|
125
|
+
| Query predicates | Mocks cannot verify filter expressions |
|
|
126
|
+
| Entity relationships | Verify foreign key correctness |
|
|
127
|
+
| Composite keys | In-memory DB may differ from real DB |
|
|
128
|
+
| Field mapping | DTO ↔ Entity transformations |
|
|
129
|
+
| Pagination | Row ordering and counting |
|
|
130
|
+
| Transactions | Rollback behavior |
|
|
2177
131
|
|
|
2178
|
-
|
|
132
|
+
#### Scope
|
|
2179
133
|
|
|
2180
|
-
|
|
2181
|
-
|
|
2182
|
-
|
|
2183
|
-
|
|
2184
|
-
|
|
2185
|
-
|
|
2186
|
-
|
|
2187
|
-
|
|
2188
|
-
│ Defect Detection Efficiency (DDE) = │
|
|
2189
|
-
│ Defects found before release / Total defects × 100 │
|
|
2190
|
-
│ - Target: > 90% │
|
|
2191
|
-
│ │
|
|
2192
|
-
│ Defect Removal Efficiency (DRE) = │
|
|
2193
|
-
│ Defects removed / Defects injected × 100 │
|
|
2194
|
-
│ - Target: > 95% │
|
|
2195
|
-
│ │
|
|
2196
|
-
│ Mean Time to Detect (MTTD) = Avg time from injection │
|
|
2197
|
-
│ Mean Time to Resolve (MTTR) = Avg time from report to fix │
|
|
2198
|
-
│ │
|
|
2199
|
-
│ Defect Leakage Rate = Production defects / Total defects │
|
|
2200
|
-
│ - Target: < 5% │
|
|
2201
|
-
│ │
|
|
2202
|
-
└─────────────────────────────────────────────────────────────┘
|
|
2203
|
-
```
|
|
134
|
+
| Include | Exclude |
|
|
135
|
+
|---------|---------|
|
|
136
|
+
| Database CRUD operations | Full user workflows |
|
|
137
|
+
| Repository + Database | Cross-service communication |
|
|
138
|
+
| Service + Repository | UI interactions |
|
|
139
|
+
| API endpoint + Service layer | |
|
|
140
|
+
| Message queue producers/consumers | |
|
|
141
|
+
| Cache read/write operations | |
|
|
2204
142
|
|
|
2205
|
-
|
|
143
|
+
#### Naming Convention
|
|
2206
144
|
|
|
2207
145
|
```
|
|
2208
|
-
|
|
2209
|
-
|
|
2210
|
-
|
|
2211
|
-
│ Requirements (35-40%) │
|
|
2212
|
-
│ ├── Ambiguous requirements │
|
|
2213
|
-
│ ├── Missing requirements │
|
|
2214
|
-
│ └── Changed requirements │
|
|
2215
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2216
|
-
│ Design (20-25%) │
|
|
2217
|
-
│ ├── Incorrect algorithm │
|
|
2218
|
-
│ ├── Integration issues │
|
|
2219
|
-
│ └── Performance not considered │
|
|
2220
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2221
|
-
│ Code (25-30%) │
|
|
2222
|
-
│ ├── Logic errors │
|
|
2223
|
-
│ ├── Boundary conditions │
|
|
2224
|
-
│ └── Exception handling │
|
|
2225
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2226
|
-
│ Environment (10-15%) │
|
|
2227
|
-
│ ├── Configuration issues │
|
|
2228
|
-
│ ├── Third-party dependencies │
|
|
2229
|
-
│ └── Data issues │
|
|
2230
|
-
└─────────────────────────────────────────────────────────────┘
|
|
146
|
+
[ComponentName]IntegrationTests.[ext]
|
|
147
|
+
[ComponentName].integration.test.[ext]
|
|
148
|
+
[ComponentName].itest.[ext]
|
|
2231
149
|
```
|
|
2232
150
|
|
|
2233
151
|
---
|
|
2234
152
|
|
|
2235
|
-
|
|
2236
|
-
|
|
2237
|
-
Structured approach to planning, monitoring, and controlling testing activities.
|
|
153
|
+
### System Testing (ST)
|
|
2238
154
|
|
|
2239
|
-
**
|
|
155
|
+
**Characteristics**: Complete system, Production-like environment, Requirement-based
|
|
2240
156
|
|
|
2241
|
-
|
|
157
|
+
#### Scope
|
|
2242
158
|
|
|
2243
|
-
|
|
159
|
+
| Include | Exclude |
|
|
160
|
+
|---------|---------|
|
|
161
|
+
| Complete API workflows | UI visual testing |
|
|
162
|
+
| Cross-service transactions | User journey simulations |
|
|
163
|
+
| Data flow through entire system | A/B testing scenarios |
|
|
164
|
+
| Security requirements | |
|
|
165
|
+
| Performance under load | |
|
|
166
|
+
| Error handling & recovery | |
|
|
2244
167
|
|
|
2245
|
-
|
|
2246
|
-
┌─────────────────────────────────────────────────────────────┐
|
|
2247
|
-
│ Test Plan Structure │
|
|
2248
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2249
|
-
│ │
|
|
2250
|
-
│ 1. Test Plan Identification │
|
|
2251
|
-
│ - Project name, version, date │
|
|
2252
|
-
│ │
|
|
2253
|
-
│ 2. Introduction │
|
|
2254
|
-
│ - Purpose, scope, objectives │
|
|
2255
|
-
│ │
|
|
2256
|
-
│ 3. Test Items │
|
|
2257
|
-
│ - Features to be tested │
|
|
2258
|
-
│ - Features not to be tested │
|
|
2259
|
-
│ │
|
|
2260
|
-
│ 4. Test Approach │
|
|
2261
|
-
│ - Test levels and types │
|
|
2262
|
-
│ - Test techniques │
|
|
2263
|
-
│ - Entry/exit criteria │
|
|
2264
|
-
│ │
|
|
2265
|
-
│ 5. Test Environment │
|
|
2266
|
-
│ - Hardware, software, tools │
|
|
2267
|
-
│ - Test data requirements │
|
|
2268
|
-
│ │
|
|
2269
|
-
│ 6. Test Schedule │
|
|
2270
|
-
│ - Milestones, dependencies │
|
|
2271
|
-
│ │
|
|
2272
|
-
│ 7. Roles and Responsibilities │
|
|
2273
|
-
│ - Team structure, RACI matrix │
|
|
2274
|
-
│ │
|
|
2275
|
-
│ 8. Risks and Contingencies │
|
|
2276
|
-
│ - Risk assessment, mitigation │
|
|
2277
|
-
│ │
|
|
2278
|
-
│ 9. Approvals │
|
|
2279
|
-
│ - Sign-off requirements │
|
|
2280
|
-
│ │
|
|
2281
|
-
└─────────────────────────────────────────────────────────────┘
|
|
2282
|
-
```
|
|
2283
|
-
|
|
2284
|
-
#### Entry and Exit Criteria
|
|
2285
|
-
|
|
2286
|
-
| Criteria Type | Examples |
|
|
2287
|
-
|---------------|----------|
|
|
2288
|
-
| **Entry Criteria** (start testing) | Requirements approved, code complete, environment ready, test data available |
|
|
2289
|
-
| **Exit Criteria** (stop testing) | All P1/P2 tests passed, coverage target met, no critical defects open |
|
|
2290
|
-
|
|
2291
|
-
```markdown
|
|
2292
|
-
## Entry Criteria for System Testing
|
|
2293
|
-
- [ ] All integration tests passed
|
|
2294
|
-
- [ ] Test environment deployed and verified
|
|
2295
|
-
- [ ] Test data prepared and loaded
|
|
2296
|
-
- [ ] Test cases reviewed and approved
|
|
2297
|
-
- [ ] All blockers from IT resolved
|
|
2298
|
-
|
|
2299
|
-
## Exit Criteria for System Testing
|
|
2300
|
-
- [ ] 100% of critical test cases executed
|
|
2301
|
-
- [ ] 95% of high priority test cases passed
|
|
2302
|
-
- [ ] No Critical or High severity defects open
|
|
2303
|
-
- [ ] Code coverage > 80%
|
|
2304
|
-
- [ ] Performance benchmarks met
|
|
2305
|
-
```
|
|
168
|
+
#### Types
|
|
2306
169
|
|
|
2307
|
-
|
|
2308
|
-
|
|
2309
|
-
|
|
2310
|
-
|
|
2311
|
-
|
|
2312
|
-
|
|
2313
|
-
|
|
2314
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2315
|
-
│ │
|
|
2316
|
-
│ Execution Metrics: │
|
|
2317
|
-
│ - Tests Planned: 500 │
|
|
2318
|
-
│ - Tests Executed: 350 (70%) │
|
|
2319
|
-
│ - Tests Passed: 320 (91% of executed) │
|
|
2320
|
-
│ - Tests Failed: 30 (9% of executed) │
|
|
2321
|
-
│ - Tests Blocked: 15 │
|
|
2322
|
-
│ │
|
|
2323
|
-
│ Defect Metrics: │
|
|
2324
|
-
│ - Total Defects Found: 45 │
|
|
2325
|
-
│ - Open Defects: 12 (3 Critical, 5 High, 4 Medium) │
|
|
2326
|
-
│ - Fixed Defects: 28 │
|
|
2327
|
-
│ - Verified Defects: 20 │
|
|
2328
|
-
│ │
|
|
2329
|
-
│ Coverage Metrics: │
|
|
2330
|
-
│ - Requirements Coverage: 85% │
|
|
2331
|
-
│ - Code Coverage: 78% │
|
|
2332
|
-
│ - Risk Coverage: 90% │
|
|
2333
|
-
│ │
|
|
2334
|
-
└─────────────────────────────────────────────────────────────┘
|
|
2335
|
-
```
|
|
170
|
+
| Type | Description |
|
|
171
|
+
|------|-------------|
|
|
172
|
+
| Functional | Verify features work as specified |
|
|
173
|
+
| Performance | Load, stress, scalability testing |
|
|
174
|
+
| Security | Penetration, vulnerability scanning |
|
|
175
|
+
| Reliability | Failover, recovery, stability |
|
|
176
|
+
| Compatibility | Cross-platform, browser compatibility |
|
|
2336
177
|
|
|
2337
|
-
####
|
|
178
|
+
#### Naming Convention
|
|
2338
179
|
|
|
2339
180
|
```
|
|
2340
|
-
|
|
2341
|
-
|
|
2342
|
-
|
|
2343
|
-
Test Execution Defect Status
|
|
2344
|
-
█████████████████░░░░ 85% Open: ███░░░░░░░ 12
|
|
2345
|
-
Fixed: ████████░░ 28
|
|
2346
|
-
Passed │████████████████│ 320 Verified: ██████░░░░ 20
|
|
2347
|
-
Failed │██ │ 30
|
|
2348
|
-
Blocked │█ │ 15 Trend: ↓ (improving)
|
|
2349
|
-
|
|
2350
|
-
Coverage Build Health
|
|
2351
|
-
Requirements: ███████████░ 85% Last Build: ✓ Pass
|
|
2352
|
-
Code: ████████░░░░ 78% Unit Tests: 245/245 ✓
|
|
2353
|
-
Risk Items: █████████░░░ 90% Int Tests: 89/92 ⚠
|
|
2354
|
-
Flaky Rate: 2.1%
|
|
181
|
+
[Feature]SystemTests.[ext]
|
|
182
|
+
[Feature].system.test.[ext]
|
|
183
|
+
[Feature]_st.[ext]
|
|
2355
184
|
```
|
|
2356
185
|
|
|
2357
|
-
|
|
2358
|
-
|
|
2359
|
-
#### Test Summary Report
|
|
2360
|
-
|
|
2361
|
-
```markdown
|
|
2362
|
-
# Test Summary Report
|
|
2363
|
-
|
|
2364
|
-
## Project: [Project Name]
|
|
2365
|
-
## Version: [Version Number]
|
|
2366
|
-
## Test Period: [Start Date] - [End Date]
|
|
2367
|
-
|
|
2368
|
-
## Executive Summary
|
|
2369
|
-
[Brief overview of testing activities and outcomes]
|
|
2370
|
-
|
|
2371
|
-
## Test Scope
|
|
2372
|
-
- Features tested: [list]
|
|
2373
|
-
- Features not tested: [list with justification]
|
|
2374
|
-
|
|
2375
|
-
## Test Results Summary
|
|
2376
|
-
|
|
2377
|
-
| Test Level | Planned | Executed | Passed | Failed | Pass Rate |
|
|
2378
|
-
|------------|---------|----------|--------|--------|-----------|
|
|
2379
|
-
| Unit | 500 | 500 | 498 | 2 | 99.6% |
|
|
2380
|
-
| Integration| 150 | 150 | 145 | 5 | 96.7% |
|
|
2381
|
-
| System | 200 | 195 | 188 | 7 | 96.4% |
|
|
2382
|
-
| E2E | 50 | 48 | 46 | 2 | 95.8% |
|
|
2383
|
-
|
|
2384
|
-
## Defect Summary
|
|
2385
|
-
|
|
2386
|
-
| Severity | Found | Fixed | Open | Deferred |
|
|
2387
|
-
|----------|-------|-------|------|----------|
|
|
2388
|
-
| Critical | 2 | 2 | 0 | 0 |
|
|
2389
|
-
| High | 8 | 7 | 1 | 0 |
|
|
2390
|
-
| Medium | 15 | 12 | 2 | 1 |
|
|
2391
|
-
| Low | 20 | 10 | 5 | 5 |
|
|
2392
|
-
|
|
2393
|
-
## Coverage Analysis
|
|
2394
|
-
- Requirements coverage: 95%
|
|
2395
|
-
- Code coverage: 82%
|
|
2396
|
-
- Risk coverage: 100% (all high-risk items tested)
|
|
186
|
+
---
|
|
2397
187
|
|
|
2398
|
-
|
|
2399
|
-
1. [Issue 1]: Impact and mitigation
|
|
2400
|
-
2. [Issue 2]: Impact and mitigation
|
|
188
|
+
### End-to-End Testing (E2E)
|
|
2401
189
|
|
|
2402
|
-
|
|
2403
|
-
1. [Recommendation 1]
|
|
2404
|
-
2. [Recommendation 2]
|
|
190
|
+
**Characteristics**: User perspective, Full stack (UI → API → Database), Slowest (30s+ each)
|
|
2405
191
|
|
|
2406
|
-
|
|
2407
|
-
[Final assessment and release recommendation]
|
|
192
|
+
#### Scope
|
|
2408
193
|
|
|
2409
|
-
|
|
2410
|
-
|
|
2411
|
-
|
|
2412
|
-
|
|
2413
|
-
|
|
194
|
+
| Include | Exclude |
|
|
195
|
+
|---------|---------|
|
|
196
|
+
| Critical user journeys | Every possible user path |
|
|
197
|
+
| Login/Authentication flows | Edge cases (use UT/IT) |
|
|
198
|
+
| Core business transactions | Performance benchmarking |
|
|
199
|
+
| Cross-browser functionality | |
|
|
200
|
+
| Smoke tests for deployments | |
|
|
2414
201
|
|
|
2415
|
-
|
|
202
|
+
#### Naming Convention
|
|
2416
203
|
|
|
2417
204
|
```
|
|
2418
|
-
|
|
2419
|
-
|
|
2420
|
-
|
|
2421
|
-
│ │
|
|
2422
|
-
│ ┌───────────┐ │
|
|
2423
|
-
│ │ Plan │ │
|
|
2424
|
-
│ └─────┬─────┘ │
|
|
2425
|
-
│ │ │
|
|
2426
|
-
│ ▼ │
|
|
2427
|
-
│ ┌───────────┐ ┌───────────┐ │
|
|
2428
|
-
│ │ Act │◀───────▶│ Do │ │
|
|
2429
|
-
│ └───────────┘ └─────┬─────┘ │
|
|
2430
|
-
│ ▲ │ │
|
|
2431
|
-
│ │ ▼ │
|
|
2432
|
-
│ ┌─────┴─────┐ │
|
|
2433
|
-
│ │ Check │ │
|
|
2434
|
-
│ └───────────┘ │
|
|
2435
|
-
│ │
|
|
2436
|
-
│ Retrospective Questions: │
|
|
2437
|
-
│ - What went well? │
|
|
2438
|
-
│ - What could be improved? │
|
|
2439
|
-
│ - What actions will we take? │
|
|
2440
|
-
│ │
|
|
2441
|
-
└─────────────────────────────────────────────────────────────┘
|
|
205
|
+
[UserJourney].e2e.[ext]
|
|
206
|
+
[Feature].e2e.spec.[ext]
|
|
207
|
+
e2e/[feature]/[scenario].[ext]
|
|
2442
208
|
```
|
|
2443
209
|
|
|
2444
210
|
---
|
|
2445
211
|
|
|
2446
212
|
## Test Doubles
|
|
2447
213
|
|
|
2448
|
-
|
|
2449
|
-
|
|
2450
|
-
| Type | Purpose | Example Use |
|
|
214
|
+
| Type | Purpose | When to Use |
|
|
2451
215
|
|------|---------|-------------|
|
|
2452
216
|
| **Stub** | Returns predefined values | Fixed API responses |
|
|
2453
217
|
| **Mock** | Verifies interactions | Verify method called |
|
|
@@ -2455,78 +219,34 @@ Risk Items: █████████░░░ 90% Int Tests: 89/92 ⚠
|
|
|
2455
219
|
| **Spy** | Records calls, delegates to real | Partial mocking |
|
|
2456
220
|
| **Dummy** | Placeholder, never used | Fill required parameters |
|
|
2457
221
|
|
|
2458
|
-
### Usage
|
|
222
|
+
### Usage by Test Level
|
|
2459
223
|
|
|
2460
|
-
|
|
2461
|
-
|
|
2462
|
-
|
|
2463
|
-
|
|
2464
|
-
|
|
2465
|
-
|
|
2466
|
-
│ └── Verify interactions with Mocks │
|
|
2467
|
-
├─────────────────────────────────────────────────────────┤
|
|
2468
|
-
│ Integration Tests (IT) │
|
|
2469
|
-
│ ├── Use Fakes for databases (in-memory, containers) │
|
|
2470
|
-
│ ├── Use Stubs for external APIs │
|
|
2471
|
-
│ └── Minimize mocking - test real integrations │
|
|
2472
|
-
├─────────────────────────────────────────────────────────┤
|
|
2473
|
-
│ System Tests (ST) │
|
|
2474
|
-
│ ├── Use real components whenever possible │
|
|
2475
|
-
│ ├── Use Fakes only for external third-party services │
|
|
2476
|
-
│ └── No mocking within the system boundary │
|
|
2477
|
-
├─────────────────────────────────────────────────────────┤
|
|
2478
|
-
│ E2E Tests │
|
|
2479
|
-
│ ├── Use real everything │
|
|
2480
|
-
│ └── Stub only external payment/email services │
|
|
2481
|
-
└─────────────────────────────────────────────────────────┘
|
|
2482
|
-
```
|
|
224
|
+
| Level | Guidance |
|
|
225
|
+
|-------|----------|
|
|
226
|
+
| **UT** | Use Mocks/Stubs for all external dependencies |
|
|
227
|
+
| **IT** | Use Fakes for databases, Stubs for external APIs |
|
|
228
|
+
| **ST** | Use real components, Fakes only for external third-party |
|
|
229
|
+
| **E2E** | Use real everything; stub only external payment/email |
|
|
2483
230
|
|
|
2484
231
|
---
|
|
2485
232
|
|
|
2486
233
|
## Mock Limitations
|
|
2487
234
|
|
|
2488
|
-
|
|
2489
|
-
|
|
2490
|
-
**Problem**:
|
|
2491
|
-
When mocking repository methods that accept query predicates (e.g., lambda expressions,
|
|
2492
|
-
filter functions), using wildcard matchers like `any()` ignores the actual query logic,
|
|
2493
|
-
allowing incorrect queries to pass unit tests.
|
|
235
|
+
**Problem**: Wildcard matchers (`any()`, `It.IsAny<>`) ignore actual query logic, allowing incorrect queries to pass.
|
|
2494
236
|
|
|
2495
|
-
**
|
|
237
|
+
**Rule**: If mocking a method that accepts a query/filter/predicate parameter, you MUST have a corresponding integration test to verify the query logic.
|
|
2496
238
|
|
|
2497
239
|
```python
|
|
2498
|
-
# Python
|
|
240
|
+
# Example - Python
|
|
2499
241
|
# ❌ This test cannot verify query correctness
|
|
2500
242
|
mock_repo.find.return_value = users
|
|
2501
|
-
# Query could be wrong, test still passes
|
|
2502
243
|
|
|
2503
244
|
# ✓ Add integration test to verify actual query
|
|
2504
245
|
```
|
|
2505
246
|
|
|
2506
|
-
```typescript
|
|
2507
|
-
// TypeScript Example
|
|
2508
|
-
// ❌ Jest mock ignores actual filter
|
|
2509
|
-
jest.spyOn(repo, 'findBy').mockResolvedValue(users);
|
|
2510
|
-
|
|
2511
|
-
// ✓ Verify with integration test
|
|
2512
|
-
```
|
|
2513
|
-
|
|
2514
|
-
```csharp
|
|
2515
|
-
// C# Example
|
|
2516
|
-
// ❌ Moq ignores the actual expression
|
|
2517
|
-
_repo.Setup(r => r.FindAsync(It.IsAny<Expression<Func<User, bool>>>()))
|
|
2518
|
-
.ReturnsAsync(users);
|
|
2519
|
-
|
|
2520
|
-
// ✓ Verify with integration test or use It.Is<> to validate
|
|
2521
|
-
```
|
|
2522
|
-
|
|
2523
|
-
**Rule of Thumb**:
|
|
2524
|
-
If your unit test mocks a method that accepts a query/filter/predicate parameter,
|
|
2525
|
-
you MUST have a corresponding integration test to verify the query logic.
|
|
2526
|
-
|
|
2527
247
|
---
|
|
2528
248
|
|
|
2529
|
-
## Test Data
|
|
249
|
+
## Test Data Requirements
|
|
2530
250
|
|
|
2531
251
|
### Principles
|
|
2532
252
|
|
|
@@ -2535,294 +255,63 @@ you MUST have a corresponding integration test to verify the query logic.
|
|
|
2535
255
|
3. **Determinism**: Tests don't depend on shared state
|
|
2536
256
|
4. **Readability**: Test data clearly shows intent
|
|
2537
257
|
|
|
2538
|
-
###
|
|
2539
|
-
|
|
2540
|
-
```csharp
|
|
2541
|
-
// Builder Pattern for Test Data
|
|
2542
|
-
public class UserBuilder
|
|
2543
|
-
{
|
|
2544
|
-
private string _name = "Default User";
|
|
2545
|
-
private string _email = "default@example.com";
|
|
2546
|
-
private bool _isActive = true;
|
|
2547
|
-
|
|
2548
|
-
public UserBuilder WithName(string name)
|
|
2549
|
-
{
|
|
2550
|
-
_name = name;
|
|
2551
|
-
return this;
|
|
2552
|
-
}
|
|
2553
|
-
|
|
2554
|
-
public UserBuilder WithEmail(string email)
|
|
2555
|
-
{
|
|
2556
|
-
_email = email;
|
|
2557
|
-
return this;
|
|
2558
|
-
}
|
|
2559
|
-
|
|
2560
|
-
public UserBuilder Inactive()
|
|
2561
|
-
{
|
|
2562
|
-
_isActive = false;
|
|
2563
|
-
return this;
|
|
2564
|
-
}
|
|
2565
|
-
|
|
2566
|
-
public User Build() => new User
|
|
2567
|
-
{
|
|
2568
|
-
Name = _name,
|
|
2569
|
-
Email = _email,
|
|
2570
|
-
IsActive = _isActive
|
|
2571
|
-
};
|
|
2572
|
-
}
|
|
2573
|
-
|
|
2574
|
-
// Usage in tests
|
|
2575
|
-
var activeUser = new UserBuilder()
|
|
2576
|
-
.WithName("Active User")
|
|
2577
|
-
.Build();
|
|
2578
|
-
|
|
2579
|
-
var inactiveUser = new UserBuilder()
|
|
2580
|
-
.WithName("Inactive User")
|
|
2581
|
-
.Inactive()
|
|
2582
|
-
.Build();
|
|
2583
|
-
```
|
|
2584
|
-
|
|
2585
|
-
### Distinct Identifiers
|
|
2586
|
-
|
|
2587
|
-
When entities have both a surrogate key (auto-generated ID) and a business identifier
|
|
2588
|
-
(e.g., employee number, department code), test data MUST use different values for each.
|
|
258
|
+
### Distinct Identifiers Rule
|
|
2589
259
|
|
|
2590
|
-
|
|
2591
|
-
If test data uses identical values for both fields, field mapping errors go undetected.
|
|
260
|
+
When entities have both a surrogate key (auto-generated ID) and a business identifier, test data MUST use different values for each.
|
|
2592
261
|
|
|
2593
262
|
```python
|
|
2594
|
-
#
|
|
2595
|
-
# ❌ Wrong: id equals business_code
|
|
263
|
+
# ❌ Wrong: id equals business_code - mapping errors undetected
|
|
2596
264
|
dept = Department(id=1, business_code=1)
|
|
2597
265
|
|
|
2598
266
|
# ✓ Correct: distinct values catch mapping errors
|
|
2599
267
|
dept = Department(id=1, business_code=1001)
|
|
2600
268
|
```
|
|
2601
269
|
|
|
2602
|
-
|
|
2603
|
-
// C# Example
|
|
2604
|
-
// ❌ Wrong: Id equals DeptId - mapping errors go undetected
|
|
2605
|
-
var dept = new Department { Id = 1, DeptId = 1 };
|
|
2606
|
-
|
|
2607
|
-
// ✓ Correct: distinct values catch field mapping bugs
|
|
2608
|
-
var dept = new Department { Id = 1, DeptId = 1001 };
|
|
2609
|
-
```
|
|
2610
|
-
|
|
2611
|
-
**Validation**:
|
|
2612
|
-
```csharp
|
|
2613
|
-
// C#
|
|
2614
|
-
testData.Dept.Id.Should().NotBe(testData.Dept.DeptId,
|
|
2615
|
-
"Test precondition: Id must differ from business identifier");
|
|
2616
|
-
```
|
|
2617
|
-
|
|
2618
|
-
### Composite Keys
|
|
270
|
+
### Composite Keys Rule
|
|
2619
271
|
|
|
2620
272
|
For entities with composite primary keys, ensure each record has a unique key combination.
|
|
2621
273
|
|
|
2622
|
-
```
|
|
2623
|
-
|
|
2624
|
-
|
|
2625
|
-
|
|
2626
|
-
var batch2 = new BatchRecord { Id = 0, SendTime = now }; // Conflict!
|
|
2627
|
-
|
|
2628
|
-
// ✓ Unique combinations
|
|
2629
|
-
var batch1 = new BatchRecord { Id = 0, SendTime = now.AddSeconds(1) };
|
|
2630
|
-
var batch2 = new BatchRecord { Id = 0, SendTime = now.AddSeconds(2) };
|
|
2631
|
-
```
|
|
274
|
+
```python
|
|
275
|
+
# ❌ Key collision
|
|
276
|
+
batch1 = BatchRecord(id=0, send_time=now)
|
|
277
|
+
batch2 = BatchRecord(id=0, send_time=now) # Conflict!
|
|
2632
278
|
|
|
2633
|
-
|
|
2634
|
-
|
|
2635
|
-
|
|
2636
|
-
// C# Helper Example
|
|
2637
|
-
private static int _timeOffset = 0;
|
|
2638
|
-
public static BatchRecord CreateWithUniqueKey(DateTime baseTime)
|
|
2639
|
-
{
|
|
2640
|
-
return new BatchRecord
|
|
2641
|
-
{
|
|
2642
|
-
SendTime = baseTime.AddSeconds(Interlocked.Increment(ref _timeOffset))
|
|
2643
|
-
};
|
|
2644
|
-
}
|
|
279
|
+
# ✓ Unique combinations
|
|
280
|
+
batch1 = BatchRecord(id=0, send_time=now + timedelta(seconds=1))
|
|
281
|
+
batch2 = BatchRecord(id=0, send_time=now + timedelta(seconds=2))
|
|
2645
282
|
```
|
|
2646
283
|
|
|
2647
284
|
---
|
|
2648
285
|
|
|
2649
|
-
## Test Environment
|
|
286
|
+
## Test Environment
|
|
2650
287
|
|
|
2651
|
-
###
|
|
288
|
+
### Language-Specific Tools
|
|
2652
289
|
|
|
2653
|
-
|
|
2654
|
-
|
|
2655
|
-
|
|
2656
|
-
|
|
2657
|
-
|
|
2658
|
-
|
|
2659
|
-
- **CI/CD Parity**: Local environment matches CI environment
|
|
2660
|
-
|
|
2661
|
-
### Language-Specific Virtual Environments
|
|
2662
|
-
|
|
2663
|
-
| Language | Tools | Lock File |
|
|
2664
|
-
|----------|-------|-----------|
|
|
2665
|
-
| Python | venv, virtualenv, conda, poetry | requirements.txt, poetry.lock |
|
|
2666
|
-
| Node.js | nvm, fnm + npm/yarn/pnpm | package-lock.json, yarn.lock |
|
|
2667
|
-
| Ruby | rbenv, rvm, bundler | Gemfile.lock |
|
|
2668
|
-
| Java | SDKMAN, jenv, Maven/Gradle | pom.xml, build.gradle.lock |
|
|
290
|
+
| Language | Version Manager | Lock File |
|
|
291
|
+
|----------|----------------|-----------|
|
|
292
|
+
| Python | venv, virtualenv, poetry | requirements.txt, poetry.lock |
|
|
293
|
+
| Node.js | nvm, fnm | package-lock.json, yarn.lock |
|
|
294
|
+
| Ruby | rbenv, rvm | Gemfile.lock |
|
|
295
|
+
| Java | SDKMAN, jenv | pom.xml, build.gradle.lock |
|
|
2669
296
|
| .NET | dotnet SDK | packages.lock.json |
|
|
2670
297
|
| Go | go mod | go.sum |
|
|
2671
298
|
| Rust | rustup, cargo | Cargo.lock |
|
|
2672
299
|
|
|
2673
|
-
|
|
300
|
+
### Best Practices
|
|
2674
301
|
|
|
2675
302
|
1. **Always use virtual environments** for development and testing
|
|
2676
303
|
2. **Commit lock files** to version control
|
|
2677
304
|
3. **Pin versions** in CI/CD pipelines
|
|
2678
305
|
4. **Document required runtime versions** in README or .tool-versions
|
|
2679
306
|
|
|
2680
|
-
|
|
2681
|
-
|
|
2682
|
-
```bash
|
|
2683
|
-
# Create virtual environment
|
|
2684
|
-
python -m venv .venv
|
|
2685
|
-
|
|
2686
|
-
# Activate
|
|
2687
|
-
source .venv/bin/activate # Linux/macOS
|
|
2688
|
-
.venv\Scripts\activate # Windows
|
|
2689
|
-
|
|
2690
|
-
# Install dependencies
|
|
2691
|
-
pip install -r requirements.txt
|
|
2692
|
-
|
|
2693
|
-
# Run tests
|
|
2694
|
-
pytest tests/
|
|
2695
|
-
```
|
|
2696
|
-
|
|
2697
|
-
#### Example: Node.js with nvm
|
|
2698
|
-
|
|
2699
|
-
```bash
|
|
2700
|
-
# Use project's Node version
|
|
2701
|
-
nvm use
|
|
2702
|
-
|
|
2703
|
-
# Install dependencies
|
|
2704
|
-
npm ci
|
|
2705
|
-
|
|
2706
|
-
# Run tests
|
|
2707
|
-
npm test
|
|
2708
|
-
```
|
|
2709
|
-
|
|
2710
|
-
### Containerized Testing
|
|
2711
|
-
|
|
2712
|
-
Use containers to provide consistent external dependencies (databases, message queues, etc.) for integration and system tests.
|
|
2713
|
-
|
|
2714
|
-
#### When to Use
|
|
2715
|
-
|
|
2716
|
-
| Test Level | Container Usage |
|
|
2717
|
-
|------------|-----------------|
|
|
2718
|
-
| UT (Unit Testing) | ❌ Not needed - use mocks |
|
|
2719
|
-
| IT (Integration Testing) | ✅ Testcontainers for databases, caches |
|
|
2720
|
-
| ST (System Testing) | ✅ Docker Compose for full environment |
|
|
2721
|
-
| E2E (End-to-End) | ✅ Full containerized stack |
|
|
2722
|
-
|
|
2723
|
-
#### Testcontainers
|
|
2724
|
-
|
|
2725
|
-
Testcontainers provides lightweight, disposable containers for testing.
|
|
2726
|
-
|
|
2727
|
-
```csharp
|
|
2728
|
-
// C# Example with Testcontainers
|
|
2729
|
-
public class DatabaseIntegrationTests : IAsyncLifetime
|
|
2730
|
-
{
|
|
2731
|
-
private readonly PostgreSqlContainer _postgres = new PostgreSqlBuilder()
|
|
2732
|
-
.WithImage("postgres:15")
|
|
2733
|
-
.Build();
|
|
2734
|
-
|
|
2735
|
-
public async Task InitializeAsync()
|
|
2736
|
-
{
|
|
2737
|
-
await _postgres.StartAsync();
|
|
2738
|
-
}
|
|
2739
|
-
|
|
2740
|
-
public async Task DisposeAsync()
|
|
2741
|
-
{
|
|
2742
|
-
await _postgres.DisposeAsync();
|
|
2743
|
-
}
|
|
2744
|
-
|
|
2745
|
-
[Fact]
|
|
2746
|
-
public async Task Should_Connect_To_Database()
|
|
2747
|
-
{
|
|
2748
|
-
var connectionString = _postgres.GetConnectionString();
|
|
2749
|
-
// Use connectionString for tests
|
|
2750
|
-
}
|
|
2751
|
-
}
|
|
2752
|
-
```
|
|
2753
|
-
|
|
2754
|
-
```python
|
|
2755
|
-
# Python Example with Testcontainers
|
|
2756
|
-
import pytest
|
|
2757
|
-
from testcontainers.postgres import PostgresContainer
|
|
2758
|
-
|
|
2759
|
-
@pytest.fixture(scope="module")
|
|
2760
|
-
def postgres_container():
|
|
2761
|
-
with PostgresContainer("postgres:15") as postgres:
|
|
2762
|
-
yield postgres
|
|
2763
|
-
|
|
2764
|
-
def test_database_connection(postgres_container):
|
|
2765
|
-
connection_url = postgres_container.get_connection_url()
|
|
2766
|
-
# Use connection_url for tests
|
|
2767
|
-
```
|
|
2768
|
-
|
|
2769
|
-
#### Docker Compose for System Tests
|
|
2770
|
-
|
|
2771
|
-
```yaml
|
|
2772
|
-
# docker-compose.test.yml
|
|
2773
|
-
version: '3.8'
|
|
2774
|
-
services:
|
|
2775
|
-
app:
|
|
2776
|
-
build: .
|
|
2777
|
-
depends_on:
|
|
2778
|
-
- db
|
|
2779
|
-
- redis
|
|
2780
|
-
- rabbitmq
|
|
2781
|
-
environment:
|
|
2782
|
-
- DATABASE_URL=postgres://test:test@db:5432/testdb
|
|
2783
|
-
- REDIS_URL=redis://redis:6379
|
|
2784
|
-
- RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672
|
|
2785
|
-
|
|
2786
|
-
db:
|
|
2787
|
-
image: postgres:15
|
|
2788
|
-
environment:
|
|
2789
|
-
POSTGRES_USER: test
|
|
2790
|
-
POSTGRES_PASSWORD: test
|
|
2791
|
-
POSTGRES_DB: testdb
|
|
2792
|
-
|
|
2793
|
-
redis:
|
|
2794
|
-
image: redis:7-alpine
|
|
2795
|
-
|
|
2796
|
-
rabbitmq:
|
|
2797
|
-
image: rabbitmq:3-management
|
|
2798
|
-
```
|
|
2799
|
-
|
|
2800
|
-
```bash
|
|
2801
|
-
# Run system tests with Docker Compose
|
|
2802
|
-
docker-compose -f docker-compose.test.yml up -d
|
|
2803
|
-
npm run test:system
|
|
2804
|
-
docker-compose -f docker-compose.test.yml down -v
|
|
2805
|
-
```
|
|
2806
|
-
|
|
2807
|
-
### Environment Parity Checklist
|
|
307
|
+
### Container Usage by Test Level
|
|
2808
308
|
|
|
2809
|
-
|
|
2810
|
-
|
|
2811
|
-
|
|
2812
|
-
|
|
2813
|
-
|
|
2814
|
-
|
|
2815
|
-
│ ✅ Lock files committed and used in CI (npm ci, pip -r) │
|
|
2816
|
-
│ ✅ Environment variables documented and consistent │
|
|
2817
|
-
│ ✅ Container images tagged with specific versions │
|
|
2818
|
-
│ ✅ .tool-versions or similar for runtime version mgmt │
|
|
2819
|
-
├─────────────────────────────────────────────────────────────┤
|
|
2820
|
-
│ ❌ Using "latest" tags in production/CI │
|
|
2821
|
-
│ ❌ Different DB versions between dev and CI │
|
|
2822
|
-
│ ❌ Missing lock files in repository │
|
|
2823
|
-
│ ❌ Hardcoded paths or machine-specific configurations │
|
|
2824
|
-
└─────────────────────────────────────────────────────────────┘
|
|
2825
|
-
```
|
|
309
|
+
| Level | Container Usage |
|
|
310
|
+
|-------|-----------------|
|
|
311
|
+
| UT | Not needed - use mocks |
|
|
312
|
+
| IT | Testcontainers for databases, caches |
|
|
313
|
+
| ST | Docker Compose for full environment |
|
|
314
|
+
| E2E | Full containerized stack |
|
|
2826
315
|
|
|
2827
316
|
---
|
|
2828
317
|
|
|
@@ -2830,63 +319,22 @@ docker-compose -f docker-compose.test.yml down -v
|
|
|
2830
319
|
|
|
2831
320
|
### Test Execution Strategy
|
|
2832
321
|
|
|
2833
|
-
|
|
2834
|
-
|
|
2835
|
-
|
|
2836
|
-
|
|
2837
|
-
|
|
2838
|
-
|
|
2839
|
-
- e2e-test # Run on release candidates
|
|
2840
|
-
|
|
2841
|
-
unit-test:
|
|
2842
|
-
stage: unit-test
|
|
2843
|
-
script:
|
|
2844
|
-
- npm run test:unit
|
|
2845
|
-
timeout: 10m
|
|
2846
|
-
rules:
|
|
2847
|
-
- if: $CI_PIPELINE_SOURCE == "push"
|
|
2848
|
-
|
|
2849
|
-
integration-test:
|
|
2850
|
-
stage: integration-test
|
|
2851
|
-
services:
|
|
2852
|
-
- postgres:14
|
|
2853
|
-
- redis:7
|
|
2854
|
-
script:
|
|
2855
|
-
- npm run test:integration
|
|
2856
|
-
timeout: 30m
|
|
2857
|
-
rules:
|
|
2858
|
-
- if: $CI_PIPELINE_SOURCE == "push"
|
|
2859
|
-
|
|
2860
|
-
system-test:
|
|
2861
|
-
stage: system-test
|
|
2862
|
-
environment: staging
|
|
2863
|
-
script:
|
|
2864
|
-
- npm run test:system
|
|
2865
|
-
timeout: 2h
|
|
2866
|
-
rules:
|
|
2867
|
-
- if: $CI_COMMIT_BRANCH == "main"
|
|
2868
|
-
|
|
2869
|
-
e2e-test:
|
|
2870
|
-
stage: e2e-test
|
|
2871
|
-
environment: staging
|
|
2872
|
-
script:
|
|
2873
|
-
- npm run test:e2e
|
|
2874
|
-
timeout: 4h
|
|
2875
|
-
rules:
|
|
2876
|
-
- if: $CI_COMMIT_TAG
|
|
2877
|
-
```
|
|
2878
|
-
|
|
2879
|
-
### Test Reports
|
|
322
|
+
| Stage | When | Timeout |
|
|
323
|
+
|-------|------|---------|
|
|
324
|
+
| Unit Test | Every commit | 10 min |
|
|
325
|
+
| Integration Test | Every commit | 30 min |
|
|
326
|
+
| System Test | PR merge to main | 2 hours |
|
|
327
|
+
| E2E Test | Release candidates | 4 hours |
|
|
2880
328
|
|
|
2881
|
-
Required
|
|
329
|
+
### Required Metrics
|
|
2882
330
|
|
|
2883
331
|
| Metric | UT | IT | ST | E2E |
|
|
2884
332
|
|--------|----|----|----|----|
|
|
2885
|
-
| Pass/Fail Count |
|
|
2886
|
-
| Execution Time |
|
|
2887
|
-
| Coverage % |
|
|
2888
|
-
| Flaky Test Rate |
|
|
2889
|
-
| Screenshots/Videos |
|
|
333
|
+
| Pass/Fail Count | Required | Required | Required | Required |
|
|
334
|
+
| Execution Time | Required | Required | Required | Required |
|
|
335
|
+
| Coverage % | Required | Required | Optional | Not needed |
|
|
336
|
+
| Flaky Test Rate | Required | Required | Required | Required |
|
|
337
|
+
| Screenshots/Videos | Not needed | Not needed | Optional | Required |
|
|
2890
338
|
|
|
2891
339
|
---
|
|
2892
340
|
|
|
@@ -2894,20 +342,10 @@ Required metrics for each test level:
|
|
|
2894
342
|
|
|
2895
343
|
### AAA Pattern
|
|
2896
344
|
|
|
2897
|
-
```
|
|
2898
|
-
|
|
2899
|
-
|
|
2900
|
-
|
|
2901
|
-
// Arrange - Set up test data and environment
|
|
2902
|
-
var input = CreateTestInput();
|
|
2903
|
-
var sut = new SystemUnderTest();
|
|
2904
|
-
|
|
2905
|
-
// Act - Execute the behavior under test
|
|
2906
|
-
var result = sut.Execute(input);
|
|
2907
|
-
|
|
2908
|
-
// Assert - Verify the result
|
|
2909
|
-
Assert.AreEqual(expected, result);
|
|
2910
|
-
}
|
|
345
|
+
```
|
|
346
|
+
// Arrange - Set up test data and environment
|
|
347
|
+
// Act - Execute the behavior under test
|
|
348
|
+
// Assert - Verify the result
|
|
2911
349
|
```
|
|
2912
350
|
|
|
2913
351
|
### FIRST Principles
|
|
@@ -2922,209 +360,89 @@ public void MethodName_Scenario_ExpectedBehavior()
|
|
|
2922
360
|
|
|
2923
361
|
### Anti-Patterns to Avoid
|
|
2924
362
|
|
|
2925
|
-
|
|
2926
|
-
|
|
2927
|
-
|
|
2928
|
-
|
|
2929
|
-
|
|
2930
|
-
|
|
2931
|
-
|
|
2932
|
-
❌ Testing Implementation Details
|
|
2933
|
-
Tests that break when refactoring
|
|
2934
|
-
|
|
2935
|
-
❌ Over-Mocking
|
|
2936
|
-
Mocking so much that nothing real is tested
|
|
2937
|
-
|
|
2938
|
-
❌ Missing Assertions
|
|
2939
|
-
Tests that verify nothing meaningful
|
|
2940
|
-
|
|
2941
|
-
❌ Magic Numbers/Strings
|
|
2942
|
-
Unexplained values in test code
|
|
2943
|
-
|
|
2944
|
-
❌ Identical Test IDs
|
|
2945
|
-
Using same values for surrogate and business keys
|
|
2946
|
-
```
|
|
363
|
+
- Test Interdependence (tests must run in specific order)
|
|
364
|
+
- Flaky Tests (sometimes pass, sometimes fail)
|
|
365
|
+
- Testing Implementation Details (tests break on refactoring)
|
|
366
|
+
- Over-Mocking (nothing real is tested)
|
|
367
|
+
- Missing Assertions (tests verify nothing meaningful)
|
|
368
|
+
- Magic Numbers/Strings (unexplained values)
|
|
369
|
+
- Identical Test IDs (same values for surrogate and business keys)
|
|
2947
370
|
|
|
2948
371
|
---
|
|
2949
372
|
|
|
2950
373
|
## Test Documentation Structure
|
|
2951
374
|
|
|
2952
|
-
Improve test discoverability by maintaining a standardized documentation structure within test directories.
|
|
2953
|
-
|
|
2954
375
|
### tests/README.md Required Sections
|
|
2955
376
|
|
|
2956
|
-
Every `tests/` directory SHOULD include a README.md with
|
|
377
|
+
Every `tests/` directory SHOULD include a README.md with:
|
|
2957
378
|
|
|
2958
379
|
#### 1. Test Overview Table
|
|
2959
380
|
|
|
2960
|
-
List all test types with count, technology stack, and execution environment.
|
|
2961
|
-
|
|
2962
|
-
```markdown
|
|
2963
381
|
| Test Type | Count | Framework | Environment |
|
|
2964
382
|
|-----------|-------|-----------|-------------|
|
|
2965
383
|
| Unit Tests | 150 | Jest | Node.js |
|
|
2966
384
|
| Integration Tests | 45 | Jest | Node.js + TestContainers |
|
|
2967
385
|
| E2E Tests | 12 | Playwright | Browser |
|
|
2968
|
-
```
|
|
2969
386
|
|
|
2970
387
|
#### 2. Current Status Section
|
|
2971
388
|
|
|
2972
|
-
Display latest test execution results with clear pass/fail indicators.
|
|
2973
|
-
|
|
2974
|
-
```markdown
|
|
2975
|
-
## Current Status
|
|
2976
|
-
|
|
2977
389
|
| Metric | Value | Target | Status |
|
|
2978
390
|
|--------|-------|--------|--------|
|
|
2979
|
-
| Pass Rate | 98.5% |
|
|
2980
|
-
| Line Coverage | 82% |
|
|
2981
|
-
| Branch Coverage | 75% |
|
|
2982
|
-
|
|
2983
|
-
**Last Run**: 2026-01-20 14:30 UTC
|
|
2984
|
-
**Report**: [test-report-20260120-143000.md](results/test-report-20260120-143000.md)
|
|
2985
|
-
```
|
|
391
|
+
| Pass Rate | 98.5% | >= 95% | Pass |
|
|
392
|
+
| Line Coverage | 82% | >= 80% | Pass |
|
|
393
|
+
| Branch Coverage | 75% | >= 70% | Pass |
|
|
2986
394
|
|
|
2987
395
|
#### 3. Report Links Section
|
|
2988
396
|
|
|
2989
|
-
Include links to test reports, gap analysis, and coverage summary.
|
|
2990
|
-
|
|
2991
|
-
```markdown
|
|
2992
|
-
## Reports
|
|
2993
|
-
|
|
2994
397
|
| Report Type | Location | Description |
|
|
2995
398
|
|-------------|----------|-------------|
|
|
2996
|
-
| Test Results | `results/` | Timestamped
|
|
2997
|
-
| Coverage | `coverage/` | Code coverage reports
|
|
2998
|
-
| Gap Analysis | `docs/gap-analysis.md` | Missing
|
|
2999
|
-
```
|
|
399
|
+
| Test Results | `results/` | Timestamped execution reports |
|
|
400
|
+
| Coverage | `coverage/` | Code coverage reports |
|
|
401
|
+
| Gap Analysis | `docs/gap-analysis.md` | Missing coverage analysis |
|
|
3000
402
|
|
|
3001
403
|
### Test Report Naming Convention
|
|
3002
404
|
|
|
3003
405
|
| Item | Convention | Example |
|
|
3004
406
|
|------|------------|---------|
|
|
3005
|
-
| Report filename | `test-report-YYYYMMDD-HHMMSS.md` | `test-report-
|
|
3006
|
-
| Report directory | `tests/results/` |
|
|
3007
|
-
| Coverage directory | `tests/coverage/` |
|
|
3008
|
-
| Gap analysis | `tests/docs/gap-analysis.md` | - |
|
|
407
|
+
| Report filename | `test-report-YYYYMMDD-HHMMSS.md` | `test-report-20260129-143000.md` |
|
|
408
|
+
| Report directory | `tests/results/` | |
|
|
409
|
+
| Coverage directory | `tests/coverage/` | |
|
|
3009
410
|
|
|
3010
|
-
### Directory Structure
|
|
411
|
+
### Directory Structure
|
|
3011
412
|
|
|
3012
413
|
```
|
|
3013
414
|
tests/
|
|
3014
415
|
├── README.md # Test overview and status
|
|
3015
416
|
├── results/ # Test execution reports
|
|
3016
|
-
│ ├── test-report-20260120-143000.md
|
|
3017
|
-
│ └── test-report-20260119-090000.md
|
|
3018
417
|
├── coverage/ # Coverage reports
|
|
3019
|
-
│ ├── lcov-report/
|
|
3020
|
-
│ └── coverage-summary.json
|
|
3021
418
|
├── docs/ # Test documentation
|
|
3022
|
-
│ └── gap-analysis.md
|
|
3023
419
|
├── unit/ # Unit tests
|
|
3024
420
|
├── integration/ # Integration tests
|
|
3025
421
|
└── e2e/ # End-to-end tests
|
|
3026
422
|
```
|
|
3027
423
|
|
|
3028
|
-
### Benefits
|
|
3029
|
-
|
|
3030
|
-
| Benefit | Description |
|
|
3031
|
-
|---------|-------------|
|
|
3032
|
-
| **Discoverability** | Developers can quickly find test status without searching |
|
|
3033
|
-
| **Transparency** | Current quality metrics visible at a glance |
|
|
3034
|
-
| **Traceability** | Historical reports enable trend analysis |
|
|
3035
|
-
| **Onboarding** | New team members understand test structure immediately |
|
|
3036
|
-
|
|
3037
424
|
---
|
|
3038
425
|
|
|
3039
|
-
##
|
|
426
|
+
## Coverage Targets Summary
|
|
3040
427
|
|
|
3041
|
-
|
|
3042
|
-
|
|
3043
|
-
|
|
3044
|
-
|
|
3045
|
-
|
|
3046
|
-
|
|
3047
|
-
│ • SWEBOK v4.0 - Software engineering knowledge │
|
|
3048
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3049
|
-
│ Testing Fundamentals (SWEBOK) │
|
|
3050
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3051
|
-
│ Error → Fault → Failure (cause chain) │
|
|
3052
|
-
│ Oracle Problem: How to know expected result? │
|
|
3053
|
-
│ Limitation: Testing proves presence, not absence of bugs │
|
|
3054
|
-
│ Testability: Controllability + Observability + Isolability │
|
|
3055
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3056
|
-
│ Testing Framework Options │
|
|
3057
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3058
|
-
│ ISTQB: UT → IT/SIT → ST → AT/UAT (Enterprise/Compliance) │
|
|
3059
|
-
│ Industry: UT (70%) → IT (20%) → E2E (10%) (Agile/DevOps) │
|
|
3060
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3061
|
-
│ Testing Levels Summary │
|
|
3062
|
-
├──────────┬──────────────────────────────────────────────────┤
|
|
3063
|
-
│ UT │ Single unit, isolated, mocked deps, < 100ms │
|
|
3064
|
-
├──────────┼──────────────────────────────────────────────────┤
|
|
3065
|
-
│ IT/SIT │ Component integration, real DB, 1-10 sec │
|
|
3066
|
-
├──────────┼──────────────────────────────────────────────────┤
|
|
3067
|
-
│ ST │ Full system, requirement-based (ISTQB only) │
|
|
3068
|
-
├──────────┼──────────────────────────────────────────────────┤
|
|
3069
|
-
│ E2E │ User journeys, UI to DB, critical paths only │
|
|
3070
|
-
├──────────┼──────────────────────────────────────────────────┤
|
|
3071
|
-
│ AT/UAT │ Business validation by end users (ISTQB only) │
|
|
3072
|
-
├──────────┴──────────────────────────────────────────────────┤
|
|
3073
|
-
│ Test Types (ISTQB) │
|
|
3074
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3075
|
-
│ Functional: Verify system behavior (black-box) │
|
|
3076
|
-
│ Non-Functional: Performance, security, usability │
|
|
3077
|
-
│ Structural: Code coverage, white-box techniques │
|
|
3078
|
-
│ Change-Related: Confirmation & regression testing │
|
|
3079
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3080
|
-
│ Test Design Techniques │
|
|
3081
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3082
|
-
│ Black-Box: EP, BVA, Decision Table, State Transition │
|
|
3083
|
-
│ Pairwise Testing, Use Case Testing │
|
|
3084
|
-
│ White-Box: Statement, Branch, Condition, MC/DC │
|
|
3085
|
-
│ Data Flow (All-Defs, All-Uses) │
|
|
3086
|
-
│ Experience: Exploratory, Error Guessing, Checklists │
|
|
3087
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3088
|
-
│ Coverage Targets │
|
|
3089
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3090
|
-
│ Line: 70% min / 85% recommended │
|
|
3091
|
-
│ Branch: 60% min / 80% recommended │
|
|
3092
|
-
│ Function: 80% min / 90% recommended │
|
|
3093
|
-
│ Mutation Score: ≥ 80% for critical code │
|
|
3094
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3095
|
-
│ Specialized Testing Types │
|
|
3096
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3097
|
-
│ Contract: API compatibility for microservices │
|
|
3098
|
-
│ Mutation: Test quality via code mutations │
|
|
3099
|
-
│ Chaos: Resilience via failure injection │
|
|
3100
|
-
│ Property-Based: Random inputs based on properties │
|
|
3101
|
-
│ Visual Regression: UI screenshot comparison │
|
|
3102
|
-
│ Accessibility: WCAG compliance validation │
|
|
3103
|
-
│ Installation: Deployment verification │
|
|
3104
|
-
│ Recovery: Failure recovery (RTO/RPO) │
|
|
3105
|
-
│ Configuration: Multi-config testing │
|
|
3106
|
-
│ Usability: User experience (SUS, NPS) │
|
|
3107
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3108
|
-
│ Test-Related Measures │
|
|
3109
|
-
├─────────────────────────────────────────────────────────────┤
|
|
3110
|
-
│ Fault Density = Faults / KLOC (target: < 1) │
|
|
3111
|
-
│ Defect Leakage = Prod defects / Total defects (< 5%) │
|
|
3112
|
-
│ Fault Seeding: Insert known faults to measure detection │
|
|
3113
|
-
│ Reliability Growth: Track defect rate over time │
|
|
3114
|
-
└─────────────────────────────────────────────────────────────┘
|
|
3115
|
-
```
|
|
428
|
+
| Metric | Minimum | Recommended |
|
|
429
|
+
|--------|---------|-------------|
|
|
430
|
+
| Line | 70% | 85% |
|
|
431
|
+
| Branch | 60% | 80% |
|
|
432
|
+
| Function | 80% | 90% |
|
|
433
|
+
| Mutation Score | - | >= 80% (critical code) |
|
|
3116
434
|
|
|
3117
435
|
---
|
|
3118
436
|
|
|
3119
437
|
## Related Standards
|
|
3120
438
|
|
|
439
|
+
- [Testing Theory Knowledge Base](../skills/testing-guide/testing-theory.md) - Educational content, examples, techniques
|
|
3121
440
|
- [Test-Driven Development](test-driven-development.md) - TDD/BDD/ATDD methodology
|
|
3122
|
-
- [Test Completeness Dimensions](test-completeness-dimensions.md) -
|
|
441
|
+
- [Test Completeness Dimensions](test-completeness-dimensions.md) - 8-dimension test coverage
|
|
3123
442
|
- [Spec-Driven Development](spec-driven-development.md) - SDD workflow integration
|
|
3124
|
-
- [Anti-Hallucination Standard](anti-hallucination.md)
|
|
3125
443
|
- [Code Check-in Standards](checkin-standards.md)
|
|
3126
444
|
- [Code Review Checklist](code-review-checklist.md)
|
|
3127
|
-
- [
|
|
445
|
+
- [Deployment Standards](deployment-standards.md) - Test requirements for deployment readiness
|
|
3128
446
|
|
|
3129
447
|
---
|
|
3130
448
|
|
|
@@ -3132,13 +450,14 @@ tests/
|
|
|
3132
450
|
|
|
3133
451
|
| Version | Date | Changes |
|
|
3134
452
|
|---------|------|---------|
|
|
3135
|
-
|
|
|
3136
|
-
| 2.
|
|
3137
|
-
| 2.
|
|
3138
|
-
|
|
|
3139
|
-
| 1.
|
|
3140
|
-
| 1.
|
|
3141
|
-
| 1.1.
|
|
453
|
+
| 3.0.0 | 2026-01-29 | **Major refactor**: Split into Rules (this file) and Theory (testing-theory.md). Reduced from 141KB/3185 lines to ~12KB/350 lines. All educational content moved to skills/testing-guide/testing-theory.md. Rules-only format optimized for AI agent consumption. |
|
|
454
|
+
| 2.2.0 | 2026-01-20 | Added Test Documentation Structure section |
|
|
455
|
+
| 2.1.0 | 2026-01-05 | Added SWEBOK v4.0 reference, Testing Fundamentals, Test-Related Measures |
|
|
456
|
+
| 2.0.0 | 2026-01-05 | Major update aligned with ISTQB CTFL v4.0 and ISO/IEC/IEEE 29119 |
|
|
457
|
+
| 1.3.0 | 2025-12-29 | Add Testing Framework Selection, IT/SIT abbreviation clarification |
|
|
458
|
+
| 1.2.0 | 2025-12-19 | Add Mock Limitations, Integration Test requirements, Test Data patterns |
|
|
459
|
+
| 1.1.1 | 2025-12-11 | Improved System test example with generic domain concepts |
|
|
460
|
+
| 1.1.0 | 2025-12-05 | Add test environment isolation section |
|
|
3142
461
|
| 1.0.0 | 2025-12-05 | Initial testing standards with UT/IT/ST/E2E coverage |
|
|
3143
462
|
|
|
3144
463
|
---
|