universal-dev-standards 5.17.0 → 6.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/uds.js +13 -135
- package/bundled/ai/standards/acceptance-criteria-traceability.ai.yaml +13 -4
- package/bundled/ai/standards/accessibility-standards.ai.yaml +1 -1
- package/bundled/ai/standards/ai-response-navigation.ai.yaml +15 -2
- package/bundled/ai/standards/api-design-standards.ai.yaml +27 -4
- package/bundled/ai/standards/behavior-snapshot.ai.yaml +86 -9
- package/bundled/ai/standards/checkin-standards.ai.yaml +2 -2
- package/bundled/ai/standards/code-review.ai.yaml +2 -2
- package/bundled/ai/standards/context-aware-loading.ai.yaml +3 -3
- package/bundled/ai/standards/data-migration-testing.ai.yaml +79 -3
- package/bundled/ai/standards/deprecation-standards.ai.yaml +10 -2
- package/bundled/ai/standards/developer-memory.ai.yaml +4 -3
- package/bundled/ai/standards/flow-based-testing.ai.yaml +2 -2
- package/bundled/ai/standards/forward-derivation-standards.ai.yaml +26 -2
- package/bundled/ai/standards/full-coverage-testing.ai.yaml +34 -2
- package/bundled/ai/standards/git-worktree.ai.yaml +3 -3
- package/bundled/ai/standards/logging.ai.yaml +97 -3
- package/bundled/ai/standards/mock-boundary.ai.yaml +48 -2
- package/bundled/ai/standards/model-provenance.ai.yaml +297 -0
- package/bundled/ai/standards/model-selection.ai.yaml +11 -2
- package/bundled/ai/standards/observability-standards.ai.yaml +10 -0
- package/bundled/ai/standards/performance-standards.ai.yaml +46 -1
- package/bundled/ai/standards/pii-classification.ai.yaml +15 -1
- package/bundled/ai/standards/pipeline-security-gates.ai.yaml +10 -0
- package/bundled/ai/standards/privacy-standards.ai.yaml +2 -2
- package/bundled/ai/standards/project-context-memory.ai.yaml +2 -2
- package/bundled/ai/standards/refactoring-standards.ai.yaml +68 -3
- package/bundled/ai/standards/resource-cost-boundary.ai.yaml +279 -0
- package/bundled/ai/standards/reverse-engineering-standards.ai.yaml +55 -2
- package/bundled/ai/standards/security-testing.ai.yaml +13 -2
- package/bundled/ai/standards/skill-standard-alignment-check.ai.yaml +37 -2
- package/bundled/ai/standards/user-journey-testing.ai.yaml +107 -0
- package/bundled/ai/standards/verification-oracle.ai.yaml +278 -0
- package/bundled/ai/standards/versioning.ai.yaml +32 -38
- package/bundled/core/acceptance-criteria-traceability.md +15 -5
- package/bundled/core/accessibility-standards.md +8 -4
- package/bundled/core/ai-friendly-architecture.md +1 -1
- package/bundled/core/ai-response-navigation.md +31 -3
- package/bundled/core/anti-sycophancy-prompting.md +1 -1
- package/bundled/core/api-design-standards.md +92 -7
- package/bundled/core/audit-trail.md +119 -0
- package/bundled/core/behavior-snapshot.md +94 -7
- package/bundled/core/browser-compatibility-standards.md +15 -2
- package/bundled/core/checkin-standards.md +9 -2
- package/bundled/core/code-review-checklist.md +10 -2
- package/bundled/core/container-image-standards.md +97 -0
- package/bundled/core/context-aware-loading.md +2 -2
- package/bundled/core/cost-budget-test.md +1 -1
- package/bundled/core/cross-flow-regression.md +3 -2
- package/bundled/core/data-contract.md +104 -0
- package/bundled/core/data-migration-testing.md +90 -0
- package/bundled/core/data-pipeline.md +113 -0
- package/bundled/core/deprecation-standards.md +16 -2
- package/bundled/core/developer-memory.md +13 -8
- package/bundled/core/documentation-writing-standards.md +1 -1
- package/bundled/core/error-code-standards.md +4 -3
- package/bundled/core/flaky-test-management.md +1 -1
- package/bundled/core/flow-based-testing.md +3 -3
- package/bundled/core/forward-derivation-standards.md +33 -5
- package/bundled/core/full-coverage-testing.md +72 -0
- package/bundled/core/git-worktree.md +4 -4
- package/bundled/core/guides/performance-guide.md +1 -1
- package/bundled/core/guides/security-guide.md +1 -1
- package/bundled/core/health-check-standards.md +2 -2
- package/bundled/core/iac-design-principles.md +97 -0
- package/bundled/core/incident-response.md +119 -0
- package/bundled/core/license-compliance.md +2 -0
- package/bundled/core/logging-standards.md +78 -2
- package/bundled/core/mock-boundary.md +54 -2
- package/bundled/core/model-provenance.md +181 -0
- package/bundled/core/model-selection.md +27 -2
- package/bundled/core/packaging-standards.md +1 -0
- package/bundled/core/performance-standards.md +96 -2
- package/bundled/core/pii-classification.md +148 -0
- package/bundled/core/pipeline-security-gates.md +22 -2
- package/bundled/core/postmortem-standards.md +2 -0
- package/bundled/core/prd-standards.md +91 -0
- package/bundled/core/privacy-standards.md +14 -3
- package/bundled/core/product-metrics-standards.md +100 -0
- package/bundled/core/project-context-memory.md +8 -2
- package/bundled/core/prompt-regression.md +1 -1
- package/bundled/core/refactoring-standards.md +44 -3
- package/bundled/core/replay-test.md +1 -1
- package/bundled/core/resource-cost-boundary.md +180 -0
- package/bundled/core/reverse-engineering-standards.md +66 -2
- package/bundled/core/runbook.md +113 -0
- package/bundled/core/schema-evolution.md +105 -0
- package/bundled/core/secret-management-standards.md +110 -0
- package/bundled/core/security-testing.md +26 -2
- package/bundled/core/self-review-protocol.md +2 -2
- package/bundled/core/skill-standard-alignment-check.md +53 -0
- package/bundled/core/slo-sli.md +109 -0
- package/bundled/core/smoke-test.md +1 -1
- package/bundled/core/tech-debt-standards.md +1 -1
- package/bundled/core/test-data-standards.md +2 -2
- package/bundled/core/user-journey-testing.md +102 -0
- package/bundled/core/user-story-mapping.md +96 -0
- package/bundled/core/verification-oracle.md +167 -0
- package/bundled/core/versioning.md +133 -109
- package/bundled/locales/COVERAGE.md +84 -73
- package/bundled/locales/zh-CN/CHANGELOG.md +71 -6
- package/bundled/locales/zh-CN/CLAUDE.md +1 -1
- package/bundled/locales/zh-CN/README.md +22 -9
- package/bundled/locales/zh-CN/SECURITY.md +1 -1
- package/bundled/locales/zh-CN/adoption/DAILY-WORKFLOW-GUIDE.md +2 -2
- package/bundled/locales/zh-CN/core/acceptance-criteria-traceability.md +4 -6
- package/bundled/locales/zh-CN/core/accessibility-standards.md +1 -1
- package/bundled/locales/zh-CN/core/adversarial-test.md +226 -0
- package/bundled/locales/zh-CN/core/agent-behavior-discipline.md +187 -0
- package/bundled/locales/zh-CN/core/ai-response-navigation.md +32 -6
- package/bundled/locales/zh-CN/core/anti-sycophancy-prompting.md +1 -1
- package/bundled/locales/zh-CN/core/api-design-standards.md +1 -1
- package/bundled/locales/zh-CN/core/behavior-snapshot.md +335 -0
- package/bundled/locales/zh-CN/core/browser-compatibility-standards.md +229 -0
- package/bundled/locales/zh-CN/core/cd-deployment-strategies.md +135 -0
- package/bundled/locales/zh-CN/core/chaos-injection-tests.md +130 -0
- package/bundled/locales/zh-CN/core/checkin-standards.md +1 -1
- package/bundled/locales/zh-CN/core/code-review-checklist.md +1 -1
- package/bundled/locales/zh-CN/core/container-security.md +535 -0
- package/bundled/locales/zh-CN/core/context-aware-loading.md +1 -1
- package/bundled/locales/zh-CN/core/contract-testing-standards.md +191 -0
- package/bundled/locales/zh-CN/core/cost-budget-test.md +86 -0
- package/bundled/locales/zh-CN/core/cross-flow-regression.md +199 -0
- package/bundled/locales/zh-CN/core/data-migration-testing.md +217 -0
- package/bundled/locales/zh-CN/core/deployment-standards.md +328 -9
- package/bundled/locales/zh-CN/core/deprecation-standards.md +1 -1
- package/bundled/locales/zh-CN/core/developer-memory.md +2 -2
- package/bundled/locales/zh-CN/core/disaster-recovery-drill.md +87 -0
- package/bundled/locales/zh-CN/core/documentation-structure.md +1 -1
- package/bundled/locales/zh-CN/core/documentation-writing-standards.md +1 -1
- package/bundled/locales/zh-CN/core/error-code-standards.md +2 -2
- package/bundled/locales/zh-CN/core/feature-manifest-standard.md +222 -0
- package/bundled/locales/zh-CN/core/flaky-test-management.md +87 -0
- package/bundled/locales/zh-CN/core/flow-based-testing.md +284 -0
- package/bundled/locales/zh-CN/core/forward-derivation-standards.md +3 -4
- package/bundled/locales/zh-CN/core/full-coverage-testing.md +197 -0
- package/bundled/locales/zh-CN/core/git-worktree.md +1 -1
- package/bundled/locales/zh-CN/core/governance-layer.md +160 -0
- package/bundled/locales/zh-CN/core/guides/performance-guide.md +515 -0
- package/bundled/locales/zh-CN/core/guides/security-guide.md +494 -0
- package/bundled/locales/zh-CN/core/knowledge-graph-memory.md +128 -0
- package/bundled/locales/zh-CN/core/license-compliance.md +129 -0
- package/bundled/locales/zh-CN/core/llm-output-validation.md +192 -0
- package/bundled/locales/zh-CN/core/logging-standards.md +130 -8
- package/bundled/locales/zh-CN/core/mock-boundary.md +109 -0
- package/bundled/locales/zh-CN/core/model-selection.md +28 -5
- package/bundled/locales/zh-CN/core/mutation-testing.md +106 -0
- package/bundled/locales/zh-CN/core/no-cicd-deployment.md +219 -0
- package/bundled/locales/zh-CN/core/packaging-standards.md +76 -5
- package/bundled/locales/zh-CN/core/pipeline-security-gates.md +126 -0
- package/bundled/locales/zh-CN/core/policy-as-code-testing.md +203 -0
- package/bundled/locales/zh-CN/core/privacy-standards.md +1 -1
- package/bundled/locales/zh-CN/core/project-context-memory.md +1 -1
- package/bundled/locales/zh-CN/core/prompt-regression.md +88 -0
- package/bundled/locales/zh-CN/core/property-based-testing.md +87 -0
- package/bundled/locales/zh-CN/core/release-quality-manifest.md +207 -0
- package/bundled/locales/zh-CN/core/release-readiness-gate.md +193 -0
- package/bundled/locales/zh-CN/core/replay-test.md +102 -0
- package/bundled/locales/zh-CN/core/reverse-engineering-standards.md +46 -1
- package/bundled/locales/zh-CN/core/rollback-standards.md +120 -0
- package/bundled/locales/zh-CN/core/sast-advanced.md +309 -0
- package/bundled/locales/zh-CN/core/secure-op.md +328 -0
- package/bundled/locales/zh-CN/core/security-testing.md +96 -0
- package/bundled/locales/zh-CN/core/self-review-protocol.md +167 -0
- package/bundled/locales/zh-CN/core/server-ops-security.md +507 -0
- package/bundled/locales/zh-CN/core/smoke-test.md +79 -0
- package/bundled/locales/zh-CN/core/spec-driven-development.md +39 -3
- package/bundled/locales/zh-CN/core/supply-chain-attestation.md +131 -0
- package/bundled/locales/zh-CN/core/versioning.md +1 -1
- package/bundled/locales/zh-CN/docs/CHEATSHEET.md +31 -6
- package/bundled/locales/zh-CN/docs/FEATURE-REFERENCE.md +61 -34
- package/bundled/locales/zh-CN/docs/MIGRATION-v6.md +88 -0
- package/bundled/locales/zh-CN/docs/USAGE-MODES-COMPARISON.md +2 -2
- package/bundled/locales/zh-CN/docs/USER-MANUAL.md +14 -14
- package/bundled/locales/zh-CN/docs/specs/system/memory-adoption-strategy.md +105 -0
- package/bundled/locales/zh-CN/docs/user/FAQ.md +132 -0
- package/bundled/locales/zh-CN/docs/user/GETTING-STARTED.md +144 -0
- package/bundled/locales/zh-CN/docs/user/GLOSSARY.md +178 -0
- package/bundled/locales/zh-CN/docs/user/README.md +70 -0
- package/bundled/locales/zh-CN/docs/user/TROUBLESHOOTING.md +190 -0
- package/bundled/locales/zh-CN/integrations/github-copilot/COPILOT-CHAT-REFERENCE.md +1 -1
- package/bundled/locales/zh-CN/integrations/github-copilot/README.md +1 -1
- package/bundled/locales/zh-CN/integrations/github-copilot/skills-mapping.md +3 -3
- package/bundled/locales/zh-CN/integrations/opencode/skills-mapping.md +3 -3
- package/bundled/locales/zh-CN/methodologies/guides/sdd-guide.md +836 -0
- package/bundled/locales/zh-CN/options/changelog/auto-generated.md +166 -0
- package/bundled/locales/zh-CN/options/changelog/keep-a-changelog.md +140 -0
- package/bundled/locales/zh-CN/options/code-review/automated-review.md +214 -0
- package/bundled/locales/zh-CN/options/code-review/pair-programming.md +166 -0
- package/bundled/locales/zh-CN/options/code-review/pr-review.md +167 -0
- package/bundled/locales/zh-CN/options/documentation/api-docs.md +191 -0
- package/bundled/locales/zh-CN/options/documentation/markdown-docs.md +150 -0
- package/bundled/locales/zh-CN/options/documentation/wiki-style.md +131 -0
- package/bundled/locales/zh-CN/options/project-structure/kotlin.md +144 -0
- package/bundled/locales/zh-CN/options/project-structure/php.md +168 -0
- package/bundled/locales/zh-CN/options/project-structure/ruby.md +156 -0
- package/bundled/locales/zh-CN/options/project-structure/rust.md +136 -0
- package/bundled/locales/zh-CN/options/project-structure/swift.md +165 -0
- package/bundled/locales/zh-CN/options/testing/contract-testing.md +237 -0
- package/bundled/locales/zh-CN/options/testing/industry-pyramid.md +200 -0
- package/bundled/locales/zh-CN/options/testing/istqb-framework.md +144 -0
- package/bundled/locales/zh-CN/options/testing/performance-testing.md +251 -0
- package/bundled/locales/zh-CN/options/testing/security-testing.md +192 -0
- package/bundled/locales/zh-CN/skills/README.md +89 -126
- package/bundled/locales/zh-CN/skills/ac-coverage/SKILL.md +5 -7
- package/bundled/locales/zh-CN/skills/adr-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/agents/code-architect.md +263 -0
- package/bundled/locales/zh-CN/skills/agents/doc-writer.md +410 -0
- package/bundled/locales/zh-CN/skills/agents/reviewer.md +357 -0
- package/bundled/locales/zh-CN/skills/agents/spec-analyst.md +410 -0
- package/bundled/locales/zh-CN/skills/agents/test-specialist.md +368 -0
- package/bundled/locales/zh-CN/skills/ai-collaboration-standards/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/ai-friendly-architecture/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/atdd-assistant/acceptance-criteria-guide.md +1 -1
- package/bundled/locales/zh-CN/skills/atdd-assistant/atdd-workflow.md +3 -4
- package/bundled/locales/zh-CN/skills/audit-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/bdd-assistant/guide.md +1 -2
- package/bundled/locales/zh-CN/skills/brainstorm-assistant/SKILL.md +109 -13
- package/bundled/locales/zh-CN/skills/brainstorm-assistant/guide.md +30 -8
- package/bundled/locales/zh-CN/skills/checkin-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/ci-cd-assistant/SKILL.md +50 -0
- package/bundled/locales/zh-CN/skills/code-review-assistant/SKILL.md +5 -5
- package/bundled/locales/zh-CN/skills/commands/ac-coverage.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/atdd.md +3 -3
- package/bundled/locales/zh-CN/skills/commands/bdd.md +2 -2
- package/bundled/locales/zh-CN/skills/commands/brainstorm.md +26 -17
- package/bundled/locales/zh-CN/skills/commands/{review.md → code-review.md} +4 -4
- package/bundled/locales/zh-CN/skills/commands/derive-all.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/derive-atdd.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/derive-bdd.md +2 -3
- package/bundled/locales/zh-CN/skills/commands/derive-tdd.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/derive.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/dev-workflow.md +2 -2
- package/bundled/locales/zh-CN/skills/commands/methodology.md +4 -4
- package/bundled/locales/zh-CN/skills/commands/pr.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/tdd.md +1 -1
- package/bundled/locales/zh-CN/skills/commit-standards/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/contract-test-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/dev-methodology/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/dev-methodology/create-methodology.md +456 -0
- package/bundled/locales/zh-CN/skills/dev-methodology/guide.md +6 -6
- package/bundled/locales/zh-CN/skills/dev-methodology/runtime.md +296 -0
- package/bundled/locales/zh-CN/skills/dev-workflow-guide/SKILL.md +5 -5
- package/bundled/locales/zh-CN/skills/docs-generator/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/e2e-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/incident-response-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/journey-test-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/logging-guide/SKILL.md +146 -142
- package/bundled/locales/zh-CN/skills/migration-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/observability-assistant/guide.md +1 -1
- package/bundled/locales/zh-CN/skills/pr-automation-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/project-discovery/guide.md +2 -2
- package/bundled/locales/zh-CN/skills/reverse-engineer/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/runbook-assistant/guide.md +1 -1
- package/bundled/locales/zh-CN/skills/security-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/skill-builder/SKILL.md +3 -3
- package/bundled/locales/zh-CN/skills/slo-assistant/guide.md +1 -1
- package/bundled/locales/zh-CN/skills/spec-derivation/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/spec-derivation/guide.md +8 -9
- package/bundled/locales/zh-CN/skills/spec-driven-dev/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/sweep/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/tdd-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/testing-guide/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/testing-guide/testing-theory.md +2298 -0
- package/bundled/locales/zh-CN/skills/workflows/README.md +451 -0
- package/bundled/locales/zh-TW/CHANGELOG.md +71 -6
- package/bundled/locales/zh-TW/CLAUDE.md +1 -1
- package/bundled/locales/zh-TW/MAINTENANCE.md +25 -2
- package/bundled/locales/zh-TW/README.md +22 -9
- package/bundled/locales/zh-TW/SECURITY.md +1 -1
- package/bundled/locales/zh-TW/adoption/DAILY-WORKFLOW-GUIDE.md +2 -2
- package/bundled/locales/zh-TW/ai/standards/versioning.ai.yaml +8 -48
- package/bundled/locales/zh-TW/core/acceptance-criteria-traceability.md +4 -6
- package/bundled/locales/zh-TW/core/accessibility-standards.md +1 -1
- package/bundled/locales/zh-TW/core/adversarial-test.md +226 -0
- package/bundled/locales/zh-TW/core/agent-behavior-discipline.md +187 -0
- package/bundled/locales/zh-TW/core/ai-response-navigation.md +32 -6
- package/bundled/locales/zh-TW/core/anti-sycophancy-prompting.md +1 -1
- package/bundled/locales/zh-TW/core/api-design-standards.md +91 -9
- package/bundled/locales/zh-TW/core/audit-trail.md +110 -0
- package/bundled/locales/zh-TW/core/behavior-snapshot.md +335 -0
- package/bundled/locales/zh-TW/core/browser-compatibility-standards.md +1 -0
- package/bundled/locales/zh-TW/core/cd-deployment-strategies.md +135 -0
- package/bundled/locales/zh-TW/core/chaos-injection-tests.md +130 -0
- package/bundled/locales/zh-TW/core/checkin-standards.md +1 -1
- package/bundled/locales/zh-TW/core/code-review-checklist.md +1 -1
- package/bundled/locales/zh-TW/core/container-image-standards.md +93 -0
- package/bundled/locales/zh-TW/core/container-security.md +535 -0
- package/bundled/locales/zh-TW/core/cost-budget-test.md +86 -0
- package/bundled/locales/zh-TW/core/cross-flow-regression.md +1 -0
- package/bundled/locales/zh-TW/core/data-contract.md +101 -0
- package/bundled/locales/zh-TW/core/data-migration-testing.md +217 -0
- package/bundled/locales/zh-TW/core/data-pipeline.md +105 -0
- package/bundled/locales/zh-TW/core/deployment-standards.md +363 -25
- package/bundled/locales/zh-TW/core/deprecation-standards.md +17 -4
- package/bundled/locales/zh-TW/core/developer-memory.md +2 -2
- package/bundled/locales/zh-TW/core/disaster-recovery-drill.md +87 -0
- package/bundled/locales/zh-TW/core/documentation-writing-standards.md +1 -1
- package/bundled/locales/zh-TW/core/error-code-standards.md +2 -2
- package/bundled/locales/zh-TW/core/feature-manifest-standard.md +222 -0
- package/bundled/locales/zh-TW/core/flaky-test-management.md +87 -0
- package/bundled/locales/zh-TW/core/flow-based-testing.md +284 -0
- package/bundled/locales/zh-TW/core/forward-derivation-standards.md +3 -4
- package/bundled/locales/zh-TW/core/full-coverage-testing.md +250 -0
- package/bundled/locales/zh-TW/core/git-worktree.md +1 -1
- package/bundled/locales/zh-TW/core/guides/performance-guide.md +515 -0
- package/bundled/locales/zh-TW/core/guides/security-guide.md +494 -0
- package/bundled/locales/zh-TW/core/iac-design-principles.md +90 -0
- package/bundled/locales/zh-TW/core/incident-response.md +111 -0
- package/bundled/locales/zh-TW/core/license-compliance.md +129 -0
- package/bundled/locales/zh-TW/core/llm-output-validation.md +192 -0
- package/bundled/locales/zh-TW/core/logging-standards.md +208 -4
- package/bundled/locales/zh-TW/core/mock-boundary.md +161 -0
- package/bundled/locales/zh-TW/core/model-provenance.md +173 -0
- package/bundled/locales/zh-TW/core/model-selection.md +28 -5
- package/bundled/locales/zh-TW/core/mutation-testing.md +106 -0
- package/bundled/locales/zh-TW/core/no-cicd-deployment.md +219 -0
- package/bundled/locales/zh-TW/core/packaging-standards.md +76 -5
- package/bundled/locales/zh-TW/core/performance-standards.md +84 -5
- package/bundled/locales/zh-TW/core/pii-classification.md +102 -0
- package/bundled/locales/zh-TW/core/pipeline-security-gates.md +126 -0
- package/bundled/locales/zh-TW/core/policy-as-code-testing.md +203 -0
- package/bundled/locales/zh-TW/core/prd-standards.md +88 -0
- package/bundled/locales/zh-TW/core/privacy-standards.md +1 -1
- package/bundled/locales/zh-TW/core/product-metrics-standards.md +96 -0
- package/bundled/locales/zh-TW/core/project-context-memory.md +1 -1
- package/bundled/locales/zh-TW/core/prompt-regression.md +88 -0
- package/bundled/locales/zh-TW/core/property-based-testing.md +87 -0
- package/bundled/locales/zh-TW/core/refactoring-standards.md +43 -6
- package/bundled/locales/zh-TW/core/release-quality-manifest.md +207 -0
- package/bundled/locales/zh-TW/core/replay-test.md +102 -0
- package/bundled/locales/zh-TW/core/resource-cost-boundary.md +175 -0
- package/bundled/locales/zh-TW/core/reverse-engineering-standards.md +64 -5
- package/bundled/locales/zh-TW/core/rollback-standards.md +120 -0
- package/bundled/locales/zh-TW/core/runbook.md +110 -0
- package/bundled/locales/zh-TW/core/sast-advanced.md +309 -0
- package/bundled/locales/zh-TW/core/schema-evolution.md +98 -0
- package/bundled/locales/zh-TW/core/secret-management-standards.md +101 -0
- package/bundled/locales/zh-TW/core/secure-op.md +328 -0
- package/bundled/locales/zh-TW/core/security-testing.md +96 -0
- package/bundled/locales/zh-TW/core/self-review-protocol.md +2 -2
- package/bundled/locales/zh-TW/core/server-ops-security.md +507 -0
- package/bundled/locales/zh-TW/core/slo-sli.md +108 -0
- package/bundled/locales/zh-TW/core/smoke-test.md +79 -0
- package/bundled/locales/zh-TW/core/spec-driven-development.md +19 -11
- package/bundled/locales/zh-TW/core/supply-chain-attestation.md +131 -0
- package/bundled/locales/zh-TW/core/user-journey-testing.md +111 -0
- package/bundled/locales/zh-TW/core/user-story-mapping.md +94 -0
- package/bundled/locales/zh-TW/core/verification-oracle.md +159 -0
- package/bundled/locales/zh-TW/core/versioning.md +112 -112
- package/bundled/locales/zh-TW/docs/CHEATSHEET.md +31 -6
- package/bundled/locales/zh-TW/docs/DEV-WORKFLOW-MAPPING.md +6 -6
- package/bundled/locales/zh-TW/docs/FEATURE-REFERENCE.md +61 -34
- package/bundled/locales/zh-TW/docs/MIGRATION-v6.md +88 -0
- package/bundled/locales/zh-TW/docs/USAGE-MODES-COMPARISON.md +2 -2
- package/bundled/locales/zh-TW/docs/USER-MANUAL.md +14 -14
- package/bundled/locales/zh-TW/docs/specs/system/memory-adoption-strategy.md +105 -0
- package/bundled/locales/zh-TW/docs/user/FAQ.md +132 -0
- package/bundled/locales/zh-TW/docs/user/GETTING-STARTED.md +144 -0
- package/bundled/locales/zh-TW/docs/user/GLOSSARY.md +178 -0
- package/bundled/locales/zh-TW/docs/user/README.md +70 -0
- package/bundled/locales/zh-TW/docs/user/TROUBLESHOOTING.md +190 -0
- package/bundled/locales/zh-TW/integrations/github-copilot/COPILOT-CHAT-REFERENCE.md +1 -1
- package/bundled/locales/zh-TW/integrations/github-copilot/README.md +1 -1
- package/bundled/locales/zh-TW/integrations/github-copilot/skills-mapping.md +3 -3
- package/bundled/locales/zh-TW/integrations/opencode/skills-mapping.md +3 -3
- package/bundled/locales/zh-TW/methodologies/guides/sdd-guide.md +523 -26
- package/bundled/locales/zh-TW/options/changelog/auto-generated.md +166 -0
- package/bundled/locales/zh-TW/options/changelog/keep-a-changelog.md +140 -0
- package/bundled/locales/zh-TW/options/code-review/automated-review.md +214 -0
- package/bundled/locales/zh-TW/options/code-review/pair-programming.md +166 -0
- package/bundled/locales/zh-TW/options/code-review/pr-review.md +167 -0
- package/bundled/locales/zh-TW/options/documentation/api-docs.md +191 -0
- package/bundled/locales/zh-TW/options/documentation/markdown-docs.md +150 -0
- package/bundled/locales/zh-TW/options/documentation/wiki-style.md +131 -0
- package/bundled/locales/zh-TW/options/project-structure/kotlin.md +144 -0
- package/bundled/locales/zh-TW/options/project-structure/php.md +168 -0
- package/bundled/locales/zh-TW/options/project-structure/ruby.md +156 -0
- package/bundled/locales/zh-TW/options/project-structure/rust.md +136 -0
- package/bundled/locales/zh-TW/options/project-structure/swift.md +165 -0
- package/bundled/locales/zh-TW/options/testing/contract-testing.md +237 -0
- package/bundled/locales/zh-TW/options/testing/industry-pyramid.md +200 -0
- package/bundled/locales/zh-TW/options/testing/istqb-framework.md +144 -0
- package/bundled/locales/zh-TW/options/testing/performance-testing.md +251 -0
- package/bundled/locales/zh-TW/options/testing/security-testing.md +192 -0
- package/bundled/locales/zh-TW/skills/README.md +91 -128
- package/bundled/locales/zh-TW/skills/ac-coverage/SKILL.md +4 -6
- package/bundled/locales/zh-TW/skills/adr-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/agents/code-architect.md +263 -0
- package/bundled/locales/zh-TW/skills/agents/doc-writer.md +410 -0
- package/bundled/locales/zh-TW/skills/agents/reviewer.md +357 -0
- package/bundled/locales/zh-TW/skills/agents/spec-analyst.md +410 -0
- package/bundled/locales/zh-TW/skills/agents/test-specialist.md +368 -0
- package/bundled/locales/zh-TW/skills/ai-collaboration-standards/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/ai-friendly-architecture/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/atdd-assistant/SKILL.md +2 -0
- package/bundled/locales/zh-TW/skills/atdd-assistant/acceptance-criteria-guide.md +1 -1
- package/bundled/locales/zh-TW/skills/atdd-assistant/atdd-workflow.md +3 -4
- package/bundled/locales/zh-TW/skills/audit-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/bdd-assistant/SKILL.md +2 -0
- package/bundled/locales/zh-TW/skills/bdd-assistant/guide.md +1 -2
- package/bundled/locales/zh-TW/skills/brainstorm-assistant/SKILL.md +110 -14
- package/bundled/locales/zh-TW/skills/brainstorm-assistant/guide.md +31 -7
- package/bundled/locales/zh-TW/skills/checkin-assistant/SKILL.md +3 -1
- package/bundled/locales/zh-TW/skills/ci-cd-assistant/SKILL.md +50 -0
- package/bundled/locales/zh-TW/skills/code-review-assistant/SKILL.md +7 -5
- package/bundled/locales/zh-TW/skills/commands/ac-coverage.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/atdd.md +3 -3
- package/bundled/locales/zh-TW/skills/commands/bdd.md +2 -2
- package/bundled/locales/zh-TW/skills/commands/brainstorm.md +26 -17
- package/bundled/locales/zh-TW/skills/commands/{review.md → code-review.md} +5 -5
- package/bundled/locales/zh-TW/skills/commands/derive-all.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/derive-atdd.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/derive-bdd.md +2 -3
- package/bundled/locales/zh-TW/skills/commands/derive-tdd.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/derive.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/dev-workflow.md +2 -2
- package/bundled/locales/zh-TW/skills/commands/methodology.md +4 -4
- package/bundled/locales/zh-TW/skills/commands/pr.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/tdd.md +1 -1
- package/bundled/locales/zh-TW/skills/commit-standards/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/contract-test-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/dev-methodology/create-methodology.md +456 -0
- package/bundled/locales/zh-TW/skills/dev-methodology/guide.md +5 -5
- package/bundled/locales/zh-TW/skills/dev-methodology/runtime.md +296 -0
- package/bundled/locales/zh-TW/skills/dev-workflow-guide/SKILL.md +5 -5
- package/bundled/locales/zh-TW/skills/docs-generator/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/e2e-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/incident-response-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/logging-guide/SKILL.md +29 -27
- package/bundled/locales/zh-TW/skills/migration-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/pr-automation-assistant/SKILL.md +3 -1
- package/bundled/locales/zh-TW/skills/project-discovery/guide.md +2 -2
- package/bundled/locales/zh-TW/skills/reverse-engineer/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/security-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/spec-derivation/guide.md +7 -8
- package/bundled/locales/zh-TW/skills/spec-driven-dev/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/sweep/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/tdd-assistant/SKILL.md +3 -1
- package/bundled/locales/zh-TW/skills/testing-guide/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/testing-guide/testing-theory.md +2298 -0
- package/bundled/locales/zh-TW/skills/workflows/README.md +451 -0
- package/bundled/skills/README.md +1 -1
- package/bundled/skills/ac-coverage/SKILL.md +17 -7
- package/bundled/skills/adr-assistant/SKILL.md +1 -1
- package/bundled/skills/agents/code-architect.md +1 -1
- package/bundled/skills/agents/doc-writer.md +1 -1
- package/bundled/skills/agents/reviewer.md +2 -2
- package/bundled/skills/agents/spec-analyst.md +1 -1
- package/bundled/skills/agents/test-specialist.md +1 -1
- package/bundled/skills/ai-collaboration-standards/SKILL.md +2 -2
- package/bundled/skills/ai-friendly-architecture/SKILL.md +2 -2
- package/bundled/skills/ai-instruction-standards/SKILL.md +1 -1
- package/bundled/skills/atdd-assistant/SKILL.md +7 -0
- package/bundled/skills/atdd-assistant/acceptance-criteria-guide.md +1 -1
- package/bundled/skills/atdd-assistant/atdd-workflow.md +6 -8
- package/bundled/skills/audit-assistant/SKILL.md +2 -2
- package/bundled/skills/bdd-assistant/SKILL.md +7 -0
- package/bundled/skills/bdd-assistant/guide.md +1 -2
- package/bundled/skills/brainstorm-assistant/SKILL.md +133 -11
- package/bundled/skills/brainstorm-assistant/guide.md +33 -5
- package/bundled/skills/checkin-assistant/SKILL.md +8 -1
- package/bundled/skills/ci-cd-assistant/SKILL.md +57 -0
- package/bundled/skills/code-review-assistant/SKILL.md +15 -8
- package/bundled/skills/commands/COMMAND-FAMILY-OVERVIEW.md +2 -2
- package/bundled/skills/commands/COMMAND-INDEX.json +3 -3
- package/bundled/skills/commands/README.md +2 -2
- package/bundled/skills/commands/ac-coverage.md +1 -1
- package/bundled/skills/commands/atdd.md +2 -2
- package/bundled/skills/commands/bdd.md +2 -2
- package/bundled/skills/commands/brainstorm.md +25 -14
- package/bundled/skills/commands/{review.md → code-review.md} +6 -6
- package/bundled/skills/commands/derive-all.md +1 -1
- package/bundled/skills/commands/derive-atdd.md +1 -1
- package/bundled/skills/commands/derive-bdd.md +2 -3
- package/bundled/skills/commands/derive-tdd.md +1 -1
- package/bundled/skills/commands/derive.md +1 -1
- package/bundled/skills/commands/dev-workflow.md +1 -1
- package/bundled/skills/commands/journey-test.md +45 -0
- package/bundled/skills/commands/methodology.md +4 -4
- package/bundled/skills/commands/pr.md +1 -1
- package/bundled/skills/commands/skill-builder.md +42 -0
- package/bundled/skills/commands/tdd.md +1 -1
- package/bundled/skills/dev-methodology/create-methodology.md +1 -1
- package/bundled/skills/dev-methodology/guide.md +1 -1
- package/bundled/skills/dev-methodology/runtime.md +1 -1
- package/bundled/skills/dev-workflow-guide/SKILL.md +3 -3
- package/bundled/skills/dev-workflow-guide/workflow-phases.md +3 -3
- package/bundled/skills/docs-generator/SKILL.md +3 -3
- package/bundled/skills/incident-response-assistant/SKILL.md +2 -2
- package/bundled/skills/logging-guide/SKILL.md +38 -2
- package/bundled/skills/migration-assistant/SKILL.md +226 -0
- package/bundled/skills/observability-assistant/SKILL.md +74 -0
- package/bundled/skills/pr-automation-assistant/SKILL.md +5 -1
- package/bundled/skills/project-discovery/guide.md +2 -2
- package/bundled/skills/push/SKILL.md +1 -1
- package/bundled/skills/release-standards/SKILL.md +19 -5
- package/bundled/skills/security-assistant/SKILL.md +1 -1
- package/bundled/skills/skill-builder/SKILL.md +1 -1
- package/bundled/skills/spec-derivation/guide.md +3 -4
- package/bundled/skills/spec-driven-dev/SKILL.md +2 -2
- package/bundled/skills/sweep/SKILL.md +1 -1
- package/bundled/skills/tdd-assistant/SKILL.md +8 -1
- package/bundled/skills/testing-guide/SKILL.md +1 -1
- package/bundled/skills/testing-guide/testing-theory.md +1 -1
- package/bundled/skills/workflows/README.md +2 -2
- package/package.json +3 -3
- package/src/commands/audit.js +30 -9
- package/src/commands/config.js +33 -2
- package/src/commands/hitl.js +14 -1
- package/src/commands/init.js +110 -58
- package/src/commands/quickstart.js +1 -2
- package/src/commands/release.js +51 -6
- package/src/commands/run-intent.js +9 -0
- package/src/commands/spec-split.js +27 -2
- package/src/commands/update.js +22 -6
- package/src/config/ai-agent-paths.js +3 -1
- package/src/i18n/messages.js +3 -102
- package/src/missions/MissionManager.js +44 -12
- package/src/utils/build-manifest.js +35 -3
- package/src/utils/friction-detector.js +71 -22
- package/src/utils/health-checker.js +29 -2
- package/src/utils/transaction.js +111 -0
- package/src/utils/version-promote.js +16 -0
- package/standards-registry.json +58 -146
- package/bundled/ai/standards/agent-communication-protocol.ai.yaml +0 -42
- package/bundled/ai/standards/agent-dispatch.ai.yaml +0 -42
- package/bundled/ai/standards/branch-completion.ai.yaml +0 -44
- package/bundled/ai/standards/change-batching-standards.ai.yaml +0 -44
- package/bundled/ai/standards/execution-history.ai.yaml +0 -42
- package/bundled/ai/standards/pipeline-integration-standards.ai.yaml +0 -42
- package/bundled/ai/standards/workflow-enforcement.ai.yaml +0 -44
- package/bundled/ai/standards/workflow-state-protocol.ai.yaml +0 -43
- package/src/commands/flow.js +0 -260
- package/src/commands/start.js +0 -372
- package/src/commands/sweep.js +0 -151
- package/src/commands/workflow.js +0 -681
|
@@ -0,0 +1,2298 @@
|
|
|
1
|
+
---
|
|
2
|
+
source: skills/testing-guide/testing-theory.md
|
|
3
|
+
source_version: 1.0.0
|
|
4
|
+
translation_version: 1.0.0
|
|
5
|
+
status: current
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Testing Theory Knowledge Base
|
|
9
|
+
|
|
10
|
+
> **語言**: [English](../../../../skills/testing-guide/testing-theory.md) | 繁體中文
|
|
11
|
+
|
|
12
|
+
**版本**: 1.0.0
|
|
13
|
+
**最後更新**: 2026-01-29
|
|
14
|
+
**適用性**: 供測試人員與開發者參考的教學資料
|
|
15
|
+
**範圍**: universal
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Purpose
|
|
20
|
+
|
|
21
|
+
本文件提供軟體測試的理論基礎、教學內容與詳細範例。它作為一份完整的知識庫,用以支援 [Testing Standards](../../core/testing-standards.md) 中所定義的可執行規則。
|
|
22
|
+
|
|
23
|
+
**參考標準**:
|
|
24
|
+
- [ISTQB CTFL v4.0](https://istqb.org/certifications/certified-tester-foundation-level-ctfl-v4-0/)
|
|
25
|
+
- [ISO/IEC/IEEE 29119](https://www.iso.org/standard/81291.html)
|
|
26
|
+
- [SWEBOK v4.0](https://www.computer.org/education/bodies-of-knowledge/software-engineering) - IEEE Computer Society
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Table of Contents
|
|
31
|
+
|
|
32
|
+
1. [測試基礎概念](#testing-fundamentals)
|
|
33
|
+
2. [靜態測試](#static-testing)
|
|
34
|
+
3. [測試設計技術](#test-design-techniques)
|
|
35
|
+
4. [經驗導向測試](#experience-based-testing)
|
|
36
|
+
5. [風險導向測試](#risk-based-testing)
|
|
37
|
+
6. [特殊化測試](#specialized-testing)
|
|
38
|
+
7. [測試相關度量](#test-related-measures)
|
|
39
|
+
8. [缺陷管理](#defect-management)
|
|
40
|
+
9. [測試流程管理](#test-process-management)
|
|
41
|
+
10. [各測試層級的程式碼範例](#code-examples-by-test-level)
|
|
42
|
+
11. [快速參考卡](#quick-reference-card)
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Testing Fundamentals
|
|
47
|
+
|
|
48
|
+
理解軟體測試的理論基礎,對於有效的測試設計與執行至關重要。
|
|
49
|
+
|
|
50
|
+
**參考**: SWEBOK v4.0 第 5 章
|
|
51
|
+
|
|
52
|
+
### Faults vs Failures
|
|
53
|
+
|
|
54
|
+
```
|
|
55
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
56
|
+
│ Faults, Errors, and Failures │
|
|
57
|
+
├─────────────────────────────────────────────────────────────┤
|
|
58
|
+
│ │
|
|
59
|
+
│ Error (Mistake) │
|
|
60
|
+
│ │ │
|
|
61
|
+
│ ▼ introduced during development │
|
|
62
|
+
│ ┌─────────┐ │
|
|
63
|
+
│ │ Fault │ (Defect/Bug in the code) │
|
|
64
|
+
│ └─────────┘ │
|
|
65
|
+
│ │ │
|
|
66
|
+
│ ▼ when executed │
|
|
67
|
+
│ ┌─────────┐ │
|
|
68
|
+
│ │ Failure │ (Observable incorrect behavior) │
|
|
69
|
+
│ └─────────┘ │
|
|
70
|
+
│ │
|
|
71
|
+
│ Key Points: │
|
|
72
|
+
│ • A fault may exist without causing a failure │
|
|
73
|
+
│ • A failure requires a fault to be executed │
|
|
74
|
+
│ • Multiple faults can cause the same failure │
|
|
75
|
+
│ • Testing finds failures; debugging finds faults │
|
|
76
|
+
│ │
|
|
77
|
+
└─────────────────────────────────────────────────────────────┘
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### The Oracle Problem
|
|
81
|
+
|
|
82
|
+
**oracle(判定機制)** 是任何用以判斷所觀察到的程式行為是否正確的機制。oracle problem 探討的是「如何得知預期結果」這項挑戰。
|
|
83
|
+
|
|
84
|
+
```
|
|
85
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
86
|
+
│ Oracle Types │
|
|
87
|
+
├─────────────────────────────────────────────────────────────┤
|
|
88
|
+
│ │
|
|
89
|
+
│ Human Oracle │
|
|
90
|
+
│ └── Manual verification by domain expert │
|
|
91
|
+
│ │
|
|
92
|
+
│ Specification-Based Oracle │
|
|
93
|
+
│ └── Compare against formal requirements │
|
|
94
|
+
│ │
|
|
95
|
+
│ Reference Implementation │
|
|
96
|
+
│ └── Compare against known-correct implementation │
|
|
97
|
+
│ │
|
|
98
|
+
│ Metamorphic Oracle │
|
|
99
|
+
│ └── Verify relationships between outputs │
|
|
100
|
+
│ (e.g., sort(sort(x)) == sort(x)) │
|
|
101
|
+
│ │
|
|
102
|
+
│ Statistical Oracle │
|
|
103
|
+
│ └── Validate against expected distributions │
|
|
104
|
+
│ │
|
|
105
|
+
│ Implicit Oracle │
|
|
106
|
+
│ └── No crash, no exception, no timeout │
|
|
107
|
+
│ │
|
|
108
|
+
└─────────────────────────────────────────────────────────────┘
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
### Theoretical Limitations of Testing
|
|
112
|
+
|
|
113
|
+
> 「程式測試可以用來證明 bug 的存在,但永遠無法證明其不存在。」 — Edsger W. Dijkstra
|
|
114
|
+
|
|
115
|
+
```
|
|
116
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
117
|
+
│ Testing Limitations │
|
|
118
|
+
├─────────────────────────────────────────────────────────────┤
|
|
119
|
+
│ │
|
|
120
|
+
│ Exhaustive Testing is Impossible │
|
|
121
|
+
│ ├── Infinite input domain │
|
|
122
|
+
│ ├── State space explosion │
|
|
123
|
+
│ └── Timing and concurrency variations │
|
|
124
|
+
│ │
|
|
125
|
+
│ Infeasible Paths │
|
|
126
|
+
│ ├── Some code paths cannot be executed by any input │
|
|
127
|
+
│ ├── Dead code may appear reachable in CFG │
|
|
128
|
+
│ └── 100% path coverage may be mathematically impossible │
|
|
129
|
+
│ │
|
|
130
|
+
│ Halting Problem │
|
|
131
|
+
│ └── Cannot algorithmically determine if all tests halt │
|
|
132
|
+
│ │
|
|
133
|
+
│ Implications: │
|
|
134
|
+
│ • Testing is sampling, not proof │
|
|
135
|
+
│ • Risk-based prioritization is essential │
|
|
136
|
+
│ • Combine testing with formal methods where critical │
|
|
137
|
+
│ │
|
|
138
|
+
└─────────────────────────────────────────────────────────────┘
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
### Testability
|
|
142
|
+
|
|
143
|
+
Testability(可測試性)衡量軟體能多容易地被有效測試。
|
|
144
|
+
|
|
145
|
+
| Factor | 說明 | 如何改善 |
|
|
146
|
+
|--------|-------------|----------------|
|
|
147
|
+
| **Controllability** | 為測試設定系統狀態的能力 | 提供測試掛鉤、依賴注入 |
|
|
148
|
+
| **Observability** | 觀察測試結果的能力 | 加入日誌、暴露內部狀態 |
|
|
149
|
+
| **Isolability** | 獨立測試各元件的能力 | 模組化設計、清楚的介面 |
|
|
150
|
+
| **Understandability** | 預期行為的清晰程度 | 良好的文件、清楚的規格 |
|
|
151
|
+
| **Stability** | 變動頻率低 | 在測試階段前凍結功能 |
|
|
152
|
+
| **Simplicity** | 複雜度低 | 降低 cyclomatic complexity |
|
|
153
|
+
|
|
154
|
+
### Test Adequacy Criteria
|
|
155
|
+
|
|
156
|
+
測試充分性準則回答的是:「我們測得夠不夠?」
|
|
157
|
+
|
|
158
|
+
```
|
|
159
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
160
|
+
│ Test Adequacy Criteria │
|
|
161
|
+
├─────────────────────────────────────────────────────────────┤
|
|
162
|
+
│ │
|
|
163
|
+
│ Coverage-Based Criteria │
|
|
164
|
+
│ ├── Statement Coverage: % of statements executed │
|
|
165
|
+
│ ├── Branch Coverage: % of branches taken │
|
|
166
|
+
│ ├── Condition Coverage: % of conditions evaluated │
|
|
167
|
+
│ └── MC/DC: Modified condition/decision coverage │
|
|
168
|
+
│ │
|
|
169
|
+
│ Fault-Based Criteria │
|
|
170
|
+
│ ├── Mutation Score: % of mutants killed │
|
|
171
|
+
│ └── Fault Seeding: % of seeded faults found │
|
|
172
|
+
│ │
|
|
173
|
+
│ Requirements-Based Criteria │
|
|
174
|
+
│ └── Requirements Coverage: % of requirements tested │
|
|
175
|
+
│ │
|
|
176
|
+
│ Risk-Based Criteria │
|
|
177
|
+
│ └── Risk Coverage: % of high-risk items tested │
|
|
178
|
+
│ │
|
|
179
|
+
│ Stopping Rules: │
|
|
180
|
+
│ • Achieved target coverage level │
|
|
181
|
+
│ • Defect discovery rate drops below threshold │
|
|
182
|
+
│ • Budget/time exhausted (with documented risk) │
|
|
183
|
+
│ │
|
|
184
|
+
└─────────────────────────────────────────────────────────────┘
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
## Static Testing
|
|
190
|
+
|
|
191
|
+
靜態測試在不執行軟體的情況下檢查工作產物(程式碼、文件、需求)。它在 SDLC 較早階段找出缺陷,從而補足動態測試。
|
|
192
|
+
|
|
193
|
+
**參考**: ISTQB CTFL v4.0 第 3 章, ISO/IEC/IEEE 29119-4
|
|
194
|
+
|
|
195
|
+
### Types of Static Testing
|
|
196
|
+
|
|
197
|
+
```
|
|
198
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
199
|
+
│ Static Testing Types │
|
|
200
|
+
├─────────────────────────────────────────────────────────────┤
|
|
201
|
+
│ Reviews (Manual) │
|
|
202
|
+
│ ├── Informal Review: Ad-hoc, no formal process │
|
|
203
|
+
│ ├── Walkthrough: Author-led, educational purpose │
|
|
204
|
+
│ ├── Technical Review: Peer-led, find defects │
|
|
205
|
+
│ └── Inspection: Formal, metrics-driven, most rigorous │
|
|
206
|
+
├─────────────────────────────────────────────────────────────┤
|
|
207
|
+
│ Static Analysis (Tool-based) │
|
|
208
|
+
│ ├── Code Analysis: Linters, complexity analyzers │
|
|
209
|
+
│ ├── Security Analysis: SAST tools (SonarQube, Checkmarx) │
|
|
210
|
+
│ ├── Architecture Analysis: Dependency checks │
|
|
211
|
+
│ └── Data Flow Analysis: Variable usage patterns │
|
|
212
|
+
└─────────────────────────────────────────────────────────────┘
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### Review Process (ISTQB)
|
|
216
|
+
|
|
217
|
+
| 階段 | 活動 |
|
|
218
|
+
|-------|------------|
|
|
219
|
+
| Planning | 定義範圍、進入準則、角色 |
|
|
220
|
+
| Initiate Review | 分發工作產物 |
|
|
221
|
+
| Individual Review | 每位審查者獨立檢視 |
|
|
222
|
+
| Issue Communication | 於會議中討論發現 |
|
|
223
|
+
| Fixing & Reporting | 作者修正、收集度量 |
|
|
224
|
+
|
|
225
|
+
### Static Analysis Tools by Language
|
|
226
|
+
|
|
227
|
+
| 語言 | Linting | Security | Complexity |
|
|
228
|
+
|----------|---------|----------|------------|
|
|
229
|
+
| JavaScript/TypeScript | ESLint, Biome | npm audit, Snyk | ESLint complexity rules |
|
|
230
|
+
| Python | Pylint, Ruff, Flake8 | Bandit, Safety | Radon |
|
|
231
|
+
| Java | Checkstyle, PMD | SpotBugs, OWASP DC | JaCoCo |
|
|
232
|
+
| C# | StyleCop, Roslyn | Security Code Scan | NDepend |
|
|
233
|
+
| Go | golangci-lint | gosec | gocyclo |
|
|
234
|
+
|
|
235
|
+
### When to Use Static Testing
|
|
236
|
+
|
|
237
|
+
| 工作產物 | Review Type | 工具 |
|
|
238
|
+
|--------------|-------------|-------|
|
|
239
|
+
| Requirements | Inspection, Walkthrough | - |
|
|
240
|
+
| Design Documents | Technical Review | 架構工具 |
|
|
241
|
+
| Code | Technical Review, Tool Analysis | Linters, SAST |
|
|
242
|
+
| Test Plans | Walkthrough | - |
|
|
243
|
+
| User Documentation | Walkthrough | 拼字/文法檢查器 |
|
|
244
|
+
|
|
245
|
+
### Code Review Checklist
|
|
246
|
+
|
|
247
|
+
```
|
|
248
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
249
|
+
│ Code Review Focus Areas │
|
|
250
|
+
├─────────────────────────────────────────────────────────────┤
|
|
251
|
+
│ Functionality │
|
|
252
|
+
│ ├── Does the code do what it's supposed to do? │
|
|
253
|
+
│ ├── Are edge cases handled? │
|
|
254
|
+
│ └── Is error handling appropriate? │
|
|
255
|
+
├─────────────────────────────────────────────────────────────┤
|
|
256
|
+
│ Maintainability │
|
|
257
|
+
│ ├── Is the code readable and well-organized? │
|
|
258
|
+
│ ├── Are names meaningful? │
|
|
259
|
+
│ └── Is complexity manageable? │
|
|
260
|
+
├─────────────────────────────────────────────────────────────┤
|
|
261
|
+
│ Security │
|
|
262
|
+
│ ├── Input validation present? │
|
|
263
|
+
│ ├── No hardcoded secrets? │
|
|
264
|
+
│ └── SQL injection / XSS prevention? │
|
|
265
|
+
├─────────────────────────────────────────────────────────────┤
|
|
266
|
+
│ Performance │
|
|
267
|
+
│ ├── No obvious inefficiencies? │
|
|
268
|
+
│ ├── Resource cleanup (connections, files)? │
|
|
269
|
+
│ └── Appropriate data structures? │
|
|
270
|
+
└─────────────────────────────────────────────────────────────┘
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
### Integration with CI/CD
|
|
274
|
+
|
|
275
|
+
```yaml
|
|
276
|
+
# Example: Static analysis in CI pipeline
|
|
277
|
+
static-analysis:
|
|
278
|
+
stage: test
|
|
279
|
+
script:
|
|
280
|
+
- npm run lint # Linting
|
|
281
|
+
- npm run lint:security # Security analysis
|
|
282
|
+
- npx sonarqube-scanner # Quality gates
|
|
283
|
+
rules:
|
|
284
|
+
- if: $CI_PIPELINE_SOURCE == "push"
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
---
|
|
288
|
+
|
|
289
|
+
## Test Design Techniques
|
|
290
|
+
|
|
291
|
+
測試設計技術是用以推導與選擇測試案例的方法。ISTQB 與 ISO/IEC/IEEE 29119 將其分類為三大主要途徑。
|
|
292
|
+
|
|
293
|
+
**參考**: ISTQB CTFL v4.0 第 4 章, ISO/IEC/IEEE 29119-4
|
|
294
|
+
|
|
295
|
+
### Technique Categories
|
|
296
|
+
|
|
297
|
+
```
|
|
298
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
299
|
+
│ Test Design Technique Categories │
|
|
300
|
+
├─────────────────────────────────────────────────────────────┤
|
|
301
|
+
│ Specification-Based (Black-Box) │
|
|
302
|
+
│ └── Derive tests from requirements/specifications │
|
|
303
|
+
├─────────────────────────────────────────────────────────────┤
|
|
304
|
+
│ Structure-Based (White-Box) │
|
|
305
|
+
│ └── Derive tests from internal code structure │
|
|
306
|
+
├─────────────────────────────────────────────────────────────┤
|
|
307
|
+
│ Experience-Based │
|
|
308
|
+
│ └── Derive tests from tester knowledge and intuition │
|
|
309
|
+
└─────────────────────────────────────────────────────────────┘
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
### Specification-Based Techniques (Black-Box)
|
|
313
|
+
|
|
314
|
+
#### 1. Equivalence Partitioning (EP)
|
|
315
|
+
|
|
316
|
+
將輸入資料劃分為多個分區,同一分區內的所有值應被系統以相同方式處理。
|
|
317
|
+
|
|
318
|
+
```
|
|
319
|
+
Input: Age (valid range: 18-65)
|
|
320
|
+
|
|
321
|
+
┌─────────────┬─────────────┬─────────────┐
|
|
322
|
+
│ Invalid │ Valid │ Invalid │
|
|
323
|
+
│ < 18 │ 18 - 65 │ > 65 │
|
|
324
|
+
├─────────────┼─────────────┼─────────────┤
|
|
325
|
+
│ Partition 1│ Partition 2│ Partition 3│
|
|
326
|
+
│ Test: 10 │ Test: 30 │ Test: 70 │
|
|
327
|
+
└─────────────┴─────────────┴─────────────┘
|
|
328
|
+
|
|
329
|
+
Coverage: At least 1 test case per partition
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
```python
|
|
333
|
+
# Python Example: Equivalence Partitioning
|
|
334
|
+
def test_age_validation():
|
|
335
|
+
validator = AgeValidator(min_age=18, max_age=65)
|
|
336
|
+
|
|
337
|
+
# Partition 1: Below minimum (invalid)
|
|
338
|
+
assert validator.is_valid(10) == False
|
|
339
|
+
|
|
340
|
+
# Partition 2: Valid range
|
|
341
|
+
assert validator.is_valid(30) == True
|
|
342
|
+
|
|
343
|
+
# Partition 3: Above maximum (invalid)
|
|
344
|
+
assert validator.is_valid(70) == False
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
#### 2. Boundary Value Analysis (BVA)
|
|
348
|
+
|
|
349
|
+
在 equivalence partitions 的邊界處測試,因為缺陷最可能出現在此。
|
|
350
|
+
|
|
351
|
+
```
|
|
352
|
+
Input: Quantity (valid range: 1-100)
|
|
353
|
+
|
|
354
|
+
0 1 2 ... 99 100 101
|
|
355
|
+
│ │ │ │ │ │
|
|
356
|
+
▼ ▼ ▼ ▼ ▼ ▼
|
|
357
|
+
Invalid Valid Valid Valid Valid Invalid
|
|
358
|
+
(boundary)(boundary) (boundary)(boundary)
|
|
359
|
+
|
|
360
|
+
Test Values: 0, 1, 2, 99, 100, 101
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
```typescript
|
|
364
|
+
// TypeScript Example: Boundary Value Analysis
|
|
365
|
+
describe('Quantity Validation', () => {
|
|
366
|
+
const validator = new QuantityValidator(1, 100);
|
|
367
|
+
|
|
368
|
+
// Lower boundary
|
|
369
|
+
it('rejects 0 (below minimum)', () => {
|
|
370
|
+
expect(validator.isValid(0)).toBe(false);
|
|
371
|
+
});
|
|
372
|
+
|
|
373
|
+
it('accepts 1 (minimum boundary)', () => {
|
|
374
|
+
expect(validator.isValid(1)).toBe(true);
|
|
375
|
+
});
|
|
376
|
+
|
|
377
|
+
it('accepts 2 (above minimum)', () => {
|
|
378
|
+
expect(validator.isValid(2)).toBe(true);
|
|
379
|
+
});
|
|
380
|
+
|
|
381
|
+
// Upper boundary
|
|
382
|
+
it('accepts 99 (below maximum)', () => {
|
|
383
|
+
expect(validator.isValid(99)).toBe(true);
|
|
384
|
+
});
|
|
385
|
+
|
|
386
|
+
it('accepts 100 (maximum boundary)', () => {
|
|
387
|
+
expect(validator.isValid(100)).toBe(true);
|
|
388
|
+
});
|
|
389
|
+
|
|
390
|
+
it('rejects 101 (above maximum)', () => {
|
|
391
|
+
expect(validator.isValid(101)).toBe(false);
|
|
392
|
+
});
|
|
393
|
+
});
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
#### 3. Decision Table Testing
|
|
397
|
+
|
|
398
|
+
以多個條件來捕捉複雜的商業規則。
|
|
399
|
+
|
|
400
|
+
```
|
|
401
|
+
Discount Rules:
|
|
402
|
+
- Premium member: 20% discount
|
|
403
|
+
- Order > $100: 10% discount
|
|
404
|
+
- Both conditions: 25% discount (not cumulative)
|
|
405
|
+
|
|
406
|
+
┌─────────────────┬──────┬──────┬──────┬──────┐
|
|
407
|
+
│ Conditions │ TC1 │ TC2 │ TC3 │ TC4 │
|
|
408
|
+
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
409
|
+
│ Premium Member │ N │ Y │ N │ Y │
|
|
410
|
+
│ Order > $100 │ N │ N │ Y │ Y │
|
|
411
|
+
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
412
|
+
│ Actions │ │ │ │ │
|
|
413
|
+
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
414
|
+
│ Discount % │ 0% │ 20% │ 10% │ 25% │
|
|
415
|
+
└─────────────────┴──────┴──────┴──────┴──────┘
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
```csharp
|
|
419
|
+
// C# Example: Decision Table Testing
|
|
420
|
+
[TestClass]
|
|
421
|
+
public class DiscountCalculatorTests
|
|
422
|
+
{
|
|
423
|
+
private DiscountCalculator _calculator;
|
|
424
|
+
|
|
425
|
+
[TestInitialize]
|
|
426
|
+
public void Setup() => _calculator = new DiscountCalculator();
|
|
427
|
+
|
|
428
|
+
[TestMethod]
|
|
429
|
+
public void Calculate_NotPremium_SmallOrder_NoDiscount()
|
|
430
|
+
{
|
|
431
|
+
var result = _calculator.Calculate(isPremium: false, orderAmount: 50);
|
|
432
|
+
Assert.AreEqual(0, result.DiscountPercent);
|
|
433
|
+
}
|
|
434
|
+
|
|
435
|
+
[TestMethod]
|
|
436
|
+
public void Calculate_Premium_SmallOrder_20PercentDiscount()
|
|
437
|
+
{
|
|
438
|
+
var result = _calculator.Calculate(isPremium: true, orderAmount: 50);
|
|
439
|
+
Assert.AreEqual(20, result.DiscountPercent);
|
|
440
|
+
}
|
|
441
|
+
|
|
442
|
+
[TestMethod]
|
|
443
|
+
public void Calculate_NotPremium_LargeOrder_10PercentDiscount()
|
|
444
|
+
{
|
|
445
|
+
var result = _calculator.Calculate(isPremium: false, orderAmount: 150);
|
|
446
|
+
Assert.AreEqual(10, result.DiscountPercent);
|
|
447
|
+
}
|
|
448
|
+
|
|
449
|
+
[TestMethod]
|
|
450
|
+
public void Calculate_Premium_LargeOrder_25PercentDiscount()
|
|
451
|
+
{
|
|
452
|
+
var result = _calculator.Calculate(isPremium: true, orderAmount: 150);
|
|
453
|
+
Assert.AreEqual(25, result.DiscountPercent);
|
|
454
|
+
}
|
|
455
|
+
}
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
#### 4. State Transition Testing
|
|
459
|
+
|
|
460
|
+
依系統狀態與狀態轉移來測試行為變化。
|
|
461
|
+
|
|
462
|
+
```
|
|
463
|
+
Order State Machine:
|
|
464
|
+
|
|
465
|
+
┌─────────┐ place() ┌───────────┐ pay() ┌────────┐
|
|
466
|
+
│ Draft │──────────────▶│ Pending │───────────▶│ Paid │
|
|
467
|
+
└─────────┘ └───────────┘ └────────┘
|
|
468
|
+
│ │ │
|
|
469
|
+
│ cancel() │ cancel() │ ship()
|
|
470
|
+
▼ ▼ ▼
|
|
471
|
+
┌───────────┐ ┌───────────┐ ┌──────────┐
|
|
472
|
+
│ Cancelled │ │ Cancelled │ │ Shipped │
|
|
473
|
+
└───────────┘ └───────────┘ └──────────┘
|
|
474
|
+
|
|
475
|
+
State Transition Table:
|
|
476
|
+
┌───────────┬─────────┬───────────┬───────────┬───────────┐
|
|
477
|
+
│ Current │ place() │ pay() │ ship() │ cancel() │
|
|
478
|
+
├───────────┼─────────┼───────────┼───────────┼───────────┤
|
|
479
|
+
│ Draft │ Pending │ Invalid │ Invalid │ Cancelled │
|
|
480
|
+
│ Pending │ Invalid │ Paid │ Invalid │ Cancelled │
|
|
481
|
+
│ Paid │ Invalid │ Invalid │ Shipped │ Invalid │
|
|
482
|
+
│ Shipped │ Invalid │ Invalid │ Invalid │ Invalid │
|
|
483
|
+
│ Cancelled │ Invalid │ Invalid │ Invalid │ Invalid │
|
|
484
|
+
└───────────┴─────────┴───────────┴───────────┴───────────┘
|
|
485
|
+
```
|
|
486
|
+
|
|
487
|
+
```python
|
|
488
|
+
# Python Example: State Transition Testing
|
|
489
|
+
class TestOrderStateMachine:
|
|
490
|
+
def test_draft_to_pending_on_place(self):
|
|
491
|
+
order = Order(state="draft")
|
|
492
|
+
order.place()
|
|
493
|
+
assert order.state == "pending"
|
|
494
|
+
|
|
495
|
+
def test_pending_to_paid_on_pay(self):
|
|
496
|
+
order = Order(state="pending")
|
|
497
|
+
order.pay()
|
|
498
|
+
assert order.state == "paid"
|
|
499
|
+
|
|
500
|
+
def test_paid_to_shipped_on_ship(self):
|
|
501
|
+
order = Order(state="paid")
|
|
502
|
+
order.ship()
|
|
503
|
+
assert order.state == "shipped"
|
|
504
|
+
|
|
505
|
+
def test_invalid_transition_pay_from_draft(self):
|
|
506
|
+
order = Order(state="draft")
|
|
507
|
+
with pytest.raises(InvalidTransitionError):
|
|
508
|
+
order.pay()
|
|
509
|
+
|
|
510
|
+
def test_invalid_transition_cancel_from_shipped(self):
|
|
511
|
+
order = Order(state="shipped")
|
|
512
|
+
with pytest.raises(InvalidTransitionError):
|
|
513
|
+
order.cancel()
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
#### 5. Use Case Testing
|
|
517
|
+
|
|
518
|
+
測試從頭到尾的完整使用者情境。
|
|
519
|
+
|
|
520
|
+
```
|
|
521
|
+
Use Case: User Login
|
|
522
|
+
|
|
523
|
+
Primary Flow:
|
|
524
|
+
1. User enters username
|
|
525
|
+
2. User enters password
|
|
526
|
+
3. User clicks login
|
|
527
|
+
4. System validates credentials
|
|
528
|
+
5. System redirects to dashboard
|
|
529
|
+
|
|
530
|
+
Alternative Flows:
|
|
531
|
+
A1. Invalid credentials → Show error, remain on login
|
|
532
|
+
A2. Account locked → Show locked message
|
|
533
|
+
A3. Password expired → Redirect to password change
|
|
534
|
+
|
|
535
|
+
Test Cases:
|
|
536
|
+
- TC1: Valid credentials → Dashboard (Primary)
|
|
537
|
+
- TC2: Invalid password → Error message (A1)
|
|
538
|
+
- TC3: Invalid username → Error message (A1)
|
|
539
|
+
- TC4: Locked account → Locked message (A2)
|
|
540
|
+
- TC5: Expired password → Password change page (A3)
|
|
541
|
+
```
|
|
542
|
+
|
|
543
|
+
#### 6. Pairwise Testing (Combinatorial Testing)
|
|
544
|
+
|
|
545
|
+
測試所有可能的輸入參數值兩兩配對,在維持良好覆蓋率的同時減少測試案例數。
|
|
546
|
+
|
|
547
|
+
**參考**: SWEBOK v4.0 - Input Domain-Based Techniques
|
|
548
|
+
|
|
549
|
+
```
|
|
550
|
+
Example: Login Form with 3 parameters, each with 3 values
|
|
551
|
+
|
|
552
|
+
Parameters:
|
|
553
|
+
- Browser: Chrome, Firefox, Safari
|
|
554
|
+
- OS: Windows, macOS, Linux
|
|
555
|
+
- Language: English, Spanish, French
|
|
556
|
+
|
|
557
|
+
Full Combinatorial: 3 × 3 × 3 = 27 test cases
|
|
558
|
+
Pairwise Coverage: 9 test cases (covers all pairs)
|
|
559
|
+
|
|
560
|
+
┌──────┬──────────┬─────────┬──────────┐
|
|
561
|
+
│ TC │ Browser │ OS │ Language │
|
|
562
|
+
├──────┼──────────┼─────────┼──────────┤
|
|
563
|
+
│ 1 │ Chrome │ Windows │ English │
|
|
564
|
+
│ 2 │ Chrome │ macOS │ Spanish │
|
|
565
|
+
│ 3 │ Chrome │ Linux │ French │
|
|
566
|
+
│ 4 │ Firefox │ Windows │ Spanish │
|
|
567
|
+
│ 5 │ Firefox │ macOS │ French │
|
|
568
|
+
│ 6 │ Firefox │ Linux │ English │
|
|
569
|
+
│ 7 │ Safari │ Windows │ French │
|
|
570
|
+
│ 8 │ Safari │ macOS │ English │
|
|
571
|
+
│ 9 │ Safari │ Linux │ Spanish │
|
|
572
|
+
└──────┴──────────┴─────────┴──────────┘
|
|
573
|
+
```
|
|
574
|
+
|
|
575
|
+
**工具**: PICT (Microsoft), AllPairs, Jenny
|
|
576
|
+
|
|
577
|
+
```bash
|
|
578
|
+
# Using PICT (Pairwise Independent Combinatorial Testing)
|
|
579
|
+
# Input file: params.txt
|
|
580
|
+
# Browser: Chrome, Firefox, Safari
|
|
581
|
+
# OS: Windows, macOS, Linux
|
|
582
|
+
# Language: English, Spanish, French
|
|
583
|
+
|
|
584
|
+
pict params.txt > test_cases.txt
|
|
585
|
+
```
|
|
586
|
+
|
|
587
|
+
#### 7. Data Flow Testing
|
|
588
|
+
|
|
589
|
+
藉由追蹤變數的定義(definition)與使用(use),測試資料在程式中的流動。
|
|
590
|
+
|
|
591
|
+
**參考**: SWEBOK v4.0 - Code-Based Techniques
|
|
592
|
+
|
|
593
|
+
```
|
|
594
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
595
|
+
│ Data Flow Terminology │
|
|
596
|
+
├─────────────────────────────────────────────────────────────┤
|
|
597
|
+
│ │
|
|
598
|
+
│ Definition (def): Variable is assigned a value │
|
|
599
|
+
│ Use (use): Variable's value is accessed │
|
|
600
|
+
│ - c-use: Computational use (in calculation) │
|
|
601
|
+
│ - p-use: Predicate use (in condition) │
|
|
602
|
+
│ │
|
|
603
|
+
│ def-use pair: Path from definition to use │
|
|
604
|
+
│ def-clear path: No redefinition between def and use │
|
|
605
|
+
│ │
|
|
606
|
+
│ Example: │
|
|
607
|
+
│ 1: x = 5 ← def(x) │
|
|
608
|
+
│ 2: y = x + 1 ← c-use(x), def(y) │
|
|
609
|
+
│ 3: if (x > 0) ← p-use(x) │
|
|
610
|
+
│ 4: z = y * 2 ← c-use(y), def(z) │
|
|
611
|
+
│ │
|
|
612
|
+
└─────────────────────────────────────────────────────────────┘
|
|
613
|
+
```
|
|
614
|
+
|
|
615
|
+
**覆蓋準則**:
|
|
616
|
+
|
|
617
|
+
| 準則 | 要求 | 強度 |
|
|
618
|
+
|-----------|-------------|----------|
|
|
619
|
+
| All-Defs | 每個 def 至少抵達一個 use | 弱 |
|
|
620
|
+
| All-Uses | 每個 def-use pair 都被執行到 | 中 |
|
|
621
|
+
| All-DU-Paths | 從 def 到 use 的所有 def-clear path | 強 |
|
|
622
|
+
|
|
623
|
+
```python
|
|
624
|
+
# Python Example: Data Flow Testing
|
|
625
|
+
def calculate_discount(price, quantity, is_member):
|
|
626
|
+
# def: total
|
|
627
|
+
total = price * quantity
|
|
628
|
+
|
|
629
|
+
# def: discount
|
|
630
|
+
if is_member: # p-use: is_member
|
|
631
|
+
discount = 0.1 # def: discount (path 1)
|
|
632
|
+
else:
|
|
633
|
+
discount = 0.0 # def: discount (path 2)
|
|
634
|
+
|
|
635
|
+
# c-use: total, discount
|
|
636
|
+
if total > 100: # p-use: total
|
|
637
|
+
discount += 0.05 # c-use: discount, def: discount
|
|
638
|
+
|
|
639
|
+
# c-use: total, discount
|
|
640
|
+
return total * (1 - discount)
|
|
641
|
+
|
|
642
|
+
# Test cases for All-Uses coverage:
|
|
643
|
+
# TC1: is_member=True, total>100 → exercises def@line4, use@line10
|
|
644
|
+
# TC2: is_member=False, total>100 → exercises def@line6, use@line10
|
|
645
|
+
# TC3: is_member=True, total<=100 → exercises def@line4, use@line12
|
|
646
|
+
# TC4: is_member=False, total<=100 → exercises def@line6, use@line12
|
|
647
|
+
```
|
|
648
|
+
|
|
649
|
+
### Structure-Based Techniques (White-Box)
|
|
650
|
+
|
|
651
|
+
#### 1. Statement Coverage
|
|
652
|
+
|
|
653
|
+
確保每一條敘述至少被執行一次。
|
|
654
|
+
|
|
655
|
+
```
|
|
656
|
+
Code:
|
|
657
|
+
1 function categorize(score) {
|
|
658
|
+
2 let result;
|
|
659
|
+
3 if (score >= 90) {
|
|
660
|
+
4 result = 'A';
|
|
661
|
+
5 } else if (score >= 70) {
|
|
662
|
+
6 result = 'B';
|
|
663
|
+
7 } else {
|
|
664
|
+
8 result = 'C';
|
|
665
|
+
9 }
|
|
666
|
+
10 return result;
|
|
667
|
+
11 }
|
|
668
|
+
|
|
669
|
+
100% Statement Coverage requires tests:
|
|
670
|
+
- Test 1: score = 95 → executes lines 1-4, 10
|
|
671
|
+
- Test 2: score = 80 → executes lines 1-3, 5-6, 10
|
|
672
|
+
- Test 3: score = 50 → executes lines 1-3, 5, 7-10
|
|
673
|
+
```
|
|
674
|
+
|
|
675
|
+
#### 2. Branch Coverage
|
|
676
|
+
|
|
677
|
+
確保每一個分支(決策結果)都被執行到。
|
|
678
|
+
|
|
679
|
+
```
|
|
680
|
+
Code:
|
|
681
|
+
if (a > 0 && b > 0) {
|
|
682
|
+
doSomething();
|
|
683
|
+
}
|
|
684
|
+
|
|
685
|
+
Branches:
|
|
686
|
+
- True branch: a > 0 AND b > 0 is true
|
|
687
|
+
- False branch: a > 0 AND b > 0 is false
|
|
688
|
+
|
|
689
|
+
100% Branch Coverage:
|
|
690
|
+
- Test 1: a = 1, b = 1 → True branch
|
|
691
|
+
- Test 2: a = 0, b = 1 → False branch (or a = 1, b = 0)
|
|
692
|
+
```
|
|
693
|
+
|
|
694
|
+
#### 3. Condition Coverage
|
|
695
|
+
|
|
696
|
+
確保決策中的每個條件都被評估為 true 與 false。
|
|
697
|
+
|
|
698
|
+
```
|
|
699
|
+
Code:
|
|
700
|
+
if (a > 0 && b > 0) { ... }
|
|
701
|
+
|
|
702
|
+
Conditions: (a > 0), (b > 0)
|
|
703
|
+
|
|
704
|
+
100% Condition Coverage:
|
|
705
|
+
- Test 1: a = 1, b = 1 → (a > 0) = true, (b > 0) = true
|
|
706
|
+
- Test 2: a = 0, b = 0 → (a > 0) = false, (b > 0) = false
|
|
707
|
+
|
|
708
|
+
Note: Does not guarantee branch coverage!
|
|
709
|
+
```
|
|
710
|
+
|
|
711
|
+
#### 4. MC/DC (Modified Condition/Decision Coverage)
|
|
712
|
+
|
|
713
|
+
每個條件都能獨立影響決策結果。安全關鍵系統會要求此覆蓋。
|
|
714
|
+
|
|
715
|
+
```
|
|
716
|
+
Code:
|
|
717
|
+
if (a && b) { ... }
|
|
718
|
+
|
|
719
|
+
MC/DC Requirements:
|
|
720
|
+
1. Each condition evaluated to true and false
|
|
721
|
+
2. Each condition independently affects outcome
|
|
722
|
+
|
|
723
|
+
Test Cases:
|
|
724
|
+
- Test 1: a = true, b = true → Decision = true
|
|
725
|
+
- Test 2: a = false, b = true → Decision = false (a changed outcome)
|
|
726
|
+
- Test 3: a = true, b = false → Decision = false (b changed outcome)
|
|
727
|
+
```
|
|
728
|
+
|
|
729
|
+
### Technique Selection Guide
|
|
730
|
+
|
|
731
|
+
| 技術 | 最適合 | 測試層級 |
|
|
732
|
+
|-----------|----------|------------|
|
|
733
|
+
| Equivalence Partitioning | 輸入驗證、範圍 | UT, IT |
|
|
734
|
+
| Boundary Value Analysis | 數值上下限、日期 | UT, IT |
|
|
735
|
+
| Decision Table | 複雜商業規則 | UT, IT, ST |
|
|
736
|
+
| State Transition | 工作流程、狀態變化 | IT, ST |
|
|
737
|
+
| Use Case Testing | 使用者情境 | ST, E2E |
|
|
738
|
+
| Pairwise Testing | 多參數組合 | IT, ST |
|
|
739
|
+
| Data Flow Testing | 變數生命週期驗證 | UT |
|
|
740
|
+
| Statement Coverage | 基本程式碼覆蓋 | UT |
|
|
741
|
+
| Branch Coverage | 決策邏輯 | UT |
|
|
742
|
+
| MC/DC | 安全關鍵系統 | UT |
|
|
743
|
+
|
|
744
|
+
---
|
|
745
|
+
|
|
746
|
+
## Experience-Based Testing
|
|
747
|
+
|
|
748
|
+
經驗導向測試運用測試人員的知識、技能與直覺來設計與執行測試。它補足了系統化技術。
|
|
749
|
+
|
|
750
|
+
**參考**: ISTQB CTFL v4.0 第 4.4 節
|
|
751
|
+
|
|
752
|
+
### Types of Experience-Based Testing
|
|
753
|
+
|
|
754
|
+
#### 1. Exploratory Testing
|
|
755
|
+
|
|
756
|
+
同時進行測試設計、執行與學習。測試人員在沒有預先腳本的情況下探索系統。
|
|
757
|
+
|
|
758
|
+
```
|
|
759
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
760
|
+
│ Exploratory Testing Session │
|
|
761
|
+
├─────────────────────────────────────────────────────────────┤
|
|
762
|
+
│ Charter: "Explore the checkout process focusing on │
|
|
763
|
+
│ payment failure scenarios" │
|
|
764
|
+
├─────────────────────────────────────────────────────────────┤
|
|
765
|
+
│ Time-box: 60 minutes │
|
|
766
|
+
├─────────────────────────────────────────────────────────────┤
|
|
767
|
+
│ Notes: │
|
|
768
|
+
│ - Tested expired credit card → Got generic error │
|
|
769
|
+
│ - Tested insufficient funds → Same generic error │
|
|
770
|
+
│ - Found: No specific error messages for different failures │
|
|
771
|
+
│ - Bug: Retry with same card doesn't clear previous error │
|
|
772
|
+
├─────────────────────────────────────────────────────────────┤
|
|
773
|
+
│ Bugs Found: 2 │
|
|
774
|
+
│ Areas for Further Testing: Error message specificity │
|
|
775
|
+
└─────────────────────────────────────────────────────────────┘
|
|
776
|
+
```
|
|
777
|
+
|
|
778
|
+
**Session-Based Test Management (SBTM)**:
|
|
779
|
+
|
|
780
|
+
| 要素 | 說明 |
|
|
781
|
+
|---------|-------------|
|
|
782
|
+
| Charter | 該 session 的任務宣言 |
|
|
783
|
+
| Time-box | 固定時段(通常 60-90 分鐘) |
|
|
784
|
+
| Session Notes | 觀察、疑問、找到的 bug |
|
|
785
|
+
| Debrief | 與團隊一同檢視發現 |
|
|
786
|
+
|
|
787
|
+
#### 2. Error Guessing
|
|
788
|
+
|
|
789
|
+
依據對類似系統或常見錯誤的經驗來預期缺陷。
|
|
790
|
+
|
|
791
|
+
```
|
|
792
|
+
Common Error Categories to Guess:
|
|
793
|
+
|
|
794
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
795
|
+
│ Input Errors │
|
|
796
|
+
│ ├── Empty/null inputs │
|
|
797
|
+
│ ├── Very long strings │
|
|
798
|
+
│ ├── Special characters (', ", <, >, &, etc.) │
|
|
799
|
+
│ ├── Unicode/emoji characters │
|
|
800
|
+
│ └── Negative numbers where positive expected │
|
|
801
|
+
├─────────────────────────────────────────────────────────────┤
|
|
802
|
+
│ Boundary Errors │
|
|
803
|
+
│ ├── Off-by-one errors │
|
|
804
|
+
│ ├── Integer overflow/underflow │
|
|
805
|
+
│ ├── Date boundaries (leap years, month ends) │
|
|
806
|
+
│ └── Array index out of bounds │
|
|
807
|
+
├─────────────────────────────────────────────────────────────┤
|
|
808
|
+
│ State Errors │
|
|
809
|
+
│ ├── Operations on deleted records │
|
|
810
|
+
│ ├── Concurrent modifications │
|
|
811
|
+
│ ├── Session timeout during operation │
|
|
812
|
+
│ └── Browser back button after submission │
|
|
813
|
+
├─────────────────────────────────────────────────────────────┤
|
|
814
|
+
│ Environment Errors │
|
|
815
|
+
│ ├── Network timeout/disconnection │
|
|
816
|
+
│ ├── Low disk space │
|
|
817
|
+
│ ├── Database connection pool exhaustion │
|
|
818
|
+
│ └── Time zone differences │
|
|
819
|
+
└─────────────────────────────────────────────────────────────┘
|
|
820
|
+
```
|
|
821
|
+
|
|
822
|
+
#### 3. Checklist-Based Testing
|
|
823
|
+
|
|
824
|
+
使用依經驗與標準建立的預先檢查清單。
|
|
825
|
+
|
|
826
|
+
```markdown
|
|
827
|
+
# API Endpoint Testing Checklist
|
|
828
|
+
|
|
829
|
+
## Authentication & Authorization
|
|
830
|
+
- [ ] Endpoint rejects unauthenticated requests
|
|
831
|
+
- [ ] Endpoint rejects expired tokens
|
|
832
|
+
- [ ] Endpoint enforces role-based access
|
|
833
|
+
- [ ] Different user roles see appropriate data
|
|
834
|
+
|
|
835
|
+
## Input Validation
|
|
836
|
+
- [ ] Required fields are validated
|
|
837
|
+
- [ ] Data types are enforced
|
|
838
|
+
- [ ] String length limits enforced
|
|
839
|
+
- [ ] Malicious input sanitized (XSS, SQL injection)
|
|
840
|
+
|
|
841
|
+
## Response Handling
|
|
842
|
+
- [ ] Success responses have correct status codes
|
|
843
|
+
- [ ] Error responses have meaningful messages
|
|
844
|
+
- [ ] Response format matches API documentation
|
|
845
|
+
- [ ] Pagination works correctly
|
|
846
|
+
|
|
847
|
+
## Performance
|
|
848
|
+
- [ ] Response time under load is acceptable
|
|
849
|
+
- [ ] No N+1 query problems
|
|
850
|
+
- [ ] Large datasets handled gracefully
|
|
851
|
+
```
|
|
852
|
+
|
|
853
|
+
### When to Use Experience-Based Testing
|
|
854
|
+
|
|
855
|
+
| 情境 | 建議技術 |
|
|
856
|
+
|----------|----------------------|
|
|
857
|
+
| 新功能探索 | Exploratory Testing |
|
|
858
|
+
| 無文件的舊系統 | Exploratory Testing |
|
|
859
|
+
| 已知的問題區域 | Error Guessing |
|
|
860
|
+
| 回歸測試 | Checklist-Based |
|
|
861
|
+
| 時間壓力/快速驗證 | Error Guessing |
|
|
862
|
+
| 合規性驗證 | Checklist-Based |
|
|
863
|
+
|
|
864
|
+
### Combining with Systematic Techniques
|
|
865
|
+
|
|
866
|
+
```
|
|
867
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
868
|
+
│ Optimal Testing Approach │
|
|
869
|
+
├─────────────────────────────────────────────────────────────┤
|
|
870
|
+
│ │
|
|
871
|
+
│ Systematic Techniques Experience-Based │
|
|
872
|
+
│ (70-80% of effort) (20-30% of effort) │
|
|
873
|
+
│ │
|
|
874
|
+
│ ┌──────────────────┐ ┌──────────────────┐ │
|
|
875
|
+
│ │ Equivalence Part.│ │ Exploratory │ │
|
|
876
|
+
│ │ Boundary Value │ + │ Error Guessing │ │
|
|
877
|
+
│ │ Decision Tables │ │ Checklists │ │
|
|
878
|
+
│ └──────────────────┘ └──────────────────┘ │
|
|
879
|
+
│ │
|
|
880
|
+
│ Provides: Provides: │
|
|
881
|
+
│ - Coverage - Edge cases │
|
|
882
|
+
│ - Repeatability - Real-world scenarios │
|
|
883
|
+
│ - Documentation - Quick feedback │
|
|
884
|
+
│ │
|
|
885
|
+
└─────────────────────────────────────────────────────────────┘
|
|
886
|
+
```
|
|
887
|
+
|
|
888
|
+
---
|
|
889
|
+
|
|
890
|
+
## Risk-Based Testing
|
|
891
|
+
|
|
892
|
+
風險導向測試依據潛在失敗的可能性與影響,來排定測試工作的優先順序。
|
|
893
|
+
|
|
894
|
+
**參考**: ISTQB CTFL v4.0 第 5.2 節
|
|
895
|
+
|
|
896
|
+
### Risk Assessment
|
|
897
|
+
|
|
898
|
+
#### Risk Formula
|
|
899
|
+
|
|
900
|
+
```
|
|
901
|
+
Risk Level = Likelihood × Impact
|
|
902
|
+
|
|
903
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
904
|
+
│ Risk Matrix │
|
|
905
|
+
├─────────────┬───────────────────────────────────────────────┤
|
|
906
|
+
│ │ Impact │
|
|
907
|
+
│ Likelihood │ Low (1) Medium (2) High (3) │
|
|
908
|
+
├─────────────┼───────────────────────────────────────────────┤
|
|
909
|
+
│ High (3) │ Medium(3) High(6) Critical(9) │
|
|
910
|
+
│ Medium (2) │ Low(2) Medium(4) High(6) │
|
|
911
|
+
│ Low (1) │ Low(1) Low(2) Medium(3) │
|
|
912
|
+
└─────────────┴───────────────────────────────────────────────┘
|
|
913
|
+
```
|
|
914
|
+
|
|
915
|
+
#### Risk Factors
|
|
916
|
+
|
|
917
|
+
**Likelihood Factors(技術風險)**:
|
|
918
|
+
- 功能的複雜度
|
|
919
|
+
- 新/不熟悉的技術
|
|
920
|
+
- 程式碼品質度量
|
|
921
|
+
- 開發者經驗
|
|
922
|
+
- 變更頻率
|
|
923
|
+
- 整合複雜度
|
|
924
|
+
|
|
925
|
+
**Impact Factors(商業風險)**:
|
|
926
|
+
- 營收影響
|
|
927
|
+
- 受影響的使用者規模
|
|
928
|
+
- 法規合規
|
|
929
|
+
- 品牌聲譽
|
|
930
|
+
- 資料安全/隱私
|
|
931
|
+
- 維運關鍵性
|
|
932
|
+
|
|
933
|
+
### Risk-Based Test Prioritization
|
|
934
|
+
|
|
935
|
+
```
|
|
936
|
+
Example: E-commerce Application
|
|
937
|
+
|
|
938
|
+
┌──────────────────────┬─────────┬─────────┬───────┬──────────┐
|
|
939
|
+
│ Feature │Likelihood│ Impact │ Risk │ Priority │
|
|
940
|
+
├──────────────────────┼─────────┼─────────┼───────┼──────────┤
|
|
941
|
+
│ Payment Processing │ 2 │ 3 │ 6 │ 1 │
|
|
942
|
+
│ User Authentication │ 2 │ 3 │ 6 │ 2 │
|
|
943
|
+
│ Shopping Cart │ 2 │ 2 │ 4 │ 3 │
|
|
944
|
+
│ Product Search │ 1 │ 2 │ 2 │ 4 │
|
|
945
|
+
│ Product Reviews │ 1 │ 1 │ 1 │ 5 │
|
|
946
|
+
│ Wishlist │ 1 │ 1 │ 1 │ 6 │
|
|
947
|
+
└──────────────────────┴─────────┴─────────┴───────┴──────────┘
|
|
948
|
+
|
|
949
|
+
Test Effort Allocation:
|
|
950
|
+
- Critical (Risk 6+): 50% of test effort, most thorough coverage
|
|
951
|
+
- Medium (Risk 3-5): 30% of test effort, standard coverage
|
|
952
|
+
- Low (Risk 1-2): 20% of test effort, basic coverage
|
|
953
|
+
```
|
|
954
|
+
|
|
955
|
+
### Risk-Based Test Planning
|
|
956
|
+
|
|
957
|
+
```markdown
|
|
958
|
+
# Risk-Based Test Plan Template
|
|
959
|
+
|
|
960
|
+
## 1. Risk Identification
|
|
961
|
+
List all features/components and their risk factors.
|
|
962
|
+
|
|
963
|
+
## 2. Risk Analysis
|
|
964
|
+
Calculate risk scores using Likelihood × Impact.
|
|
965
|
+
|
|
966
|
+
## 3. Test Prioritization
|
|
967
|
+
| Priority | Features | Test Depth |
|
|
968
|
+
|----------|----------|------------|
|
|
969
|
+
| P1 | Payment, Auth | Full coverage, all techniques |
|
|
970
|
+
| P2 | Cart, Checkout | Standard coverage |
|
|
971
|
+
| P3 | Search, Browse | Basic happy path |
|
|
972
|
+
| P4 | Reviews, Wishlist | Minimal, smoke tests |
|
|
973
|
+
|
|
974
|
+
## 4. Risk Mitigation
|
|
975
|
+
- P1 features: 100% automated tests, manual exploratory
|
|
976
|
+
- P2 features: 80% automated, sample manual tests
|
|
977
|
+
- P3 features: Key scenarios automated
|
|
978
|
+
- P4 features: Basic smoke tests only
|
|
979
|
+
|
|
980
|
+
## 5. Residual Risk Acceptance
|
|
981
|
+
Document accepted risks for low-priority features.
|
|
982
|
+
```
|
|
983
|
+
|
|
984
|
+
### Continuous Risk Assessment
|
|
985
|
+
|
|
986
|
+
```
|
|
987
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
988
|
+
│ Continuous Risk Re-evaluation │
|
|
989
|
+
├─────────────────────────────────────────────────────────────┤
|
|
990
|
+
│ │
|
|
991
|
+
│ Sprint Start ──▶ During Sprint ──▶ Sprint End │
|
|
992
|
+
│ │ │ │ │
|
|
993
|
+
│ ▼ ▼ ▼ │
|
|
994
|
+
│ Initial Risk Update based on: Review: │
|
|
995
|
+
│ Assessment - Defects found - Actual vs Expected │
|
|
996
|
+
│ - Code changes - Adjust for next │
|
|
997
|
+
│ - New requirements sprint │
|
|
998
|
+
│ │
|
|
999
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1000
|
+
```
|
|
1001
|
+
|
|
1002
|
+
---
|
|
1003
|
+
|
|
1004
|
+
## Specialized Testing
|
|
1005
|
+
|
|
1006
|
+
現代軟體開發需要超越傳統分類的特殊化測試類型。
|
|
1007
|
+
|
|
1008
|
+
### Contract Testing
|
|
1009
|
+
|
|
1010
|
+
驗證各服務是否依照約定的合約正確通訊。對微服務架構而言至關重要。
|
|
1011
|
+
|
|
1012
|
+
```
|
|
1013
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1014
|
+
│ Consumer-Driven Contract Testing │
|
|
1015
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1016
|
+
│ │
|
|
1017
|
+
│ Consumer Provider │
|
|
1018
|
+
│ (Frontend) (API) │
|
|
1019
|
+
│ │ │ │
|
|
1020
|
+
│ │ 1. Define expectations │ │
|
|
1021
|
+
│ │─────────────────────────────▶ │ │
|
|
1022
|
+
│ │ │ │
|
|
1023
|
+
│ │ 2. Generate contract │ │
|
|
1024
|
+
│ │ (Pact file) │ │
|
|
1025
|
+
│ │ │ │
|
|
1026
|
+
│ │ 3. Share contract │ │
|
|
1027
|
+
│ │─────────────────────────────▶ │ │
|
|
1028
|
+
│ │ │ │
|
|
1029
|
+
│ │ 4. Provider verifies │
|
|
1030
|
+
│ │ against contract │
|
|
1031
|
+
│ │ │ │
|
|
1032
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1033
|
+
```
|
|
1034
|
+
|
|
1035
|
+
**工具**: Pact, Spring Cloud Contract, Postman
|
|
1036
|
+
|
|
1037
|
+
```typescript
|
|
1038
|
+
// Consumer Test (Pact.js Example)
|
|
1039
|
+
describe('User API Contract', () => {
|
|
1040
|
+
it('returns user by ID', async () => {
|
|
1041
|
+
await provider.addInteraction({
|
|
1042
|
+
state: 'user with ID 123 exists',
|
|
1043
|
+
uponReceiving: 'a request for user 123',
|
|
1044
|
+
withRequest: {
|
|
1045
|
+
method: 'GET',
|
|
1046
|
+
path: '/users/123'
|
|
1047
|
+
},
|
|
1048
|
+
willRespondWith: {
|
|
1049
|
+
status: 200,
|
|
1050
|
+
body: {
|
|
1051
|
+
id: '123',
|
|
1052
|
+
name: Matchers.string('John Doe'),
|
|
1053
|
+
email: Matchers.email()
|
|
1054
|
+
}
|
|
1055
|
+
}
|
|
1056
|
+
});
|
|
1057
|
+
|
|
1058
|
+
const user = await userClient.getUser('123');
|
|
1059
|
+
expect(user.id).toBe('123');
|
|
1060
|
+
});
|
|
1061
|
+
});
|
|
1062
|
+
```
|
|
1063
|
+
|
|
1064
|
+
### Mutation Testing
|
|
1065
|
+
|
|
1066
|
+
藉由對程式碼引入微小變更(mutants),並檢查測試是否能偵測到,以評估測試套件的品質。
|
|
1067
|
+
|
|
1068
|
+
```
|
|
1069
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1070
|
+
│ Mutation Testing │
|
|
1071
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1072
|
+
│ │
|
|
1073
|
+
│ Original Code: Mutant (change operator): │
|
|
1074
|
+
│ if (a > b) if (a >= b) ← Boundary mutant │
|
|
1075
|
+
│ if (a > b) if (a < b) ← Negation mutant │
|
|
1076
|
+
│ return a + b; return a - b; ← Arithmetic mutant │
|
|
1077
|
+
│ return true; return false; ← Return value mutant │
|
|
1078
|
+
│ │
|
|
1079
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1080
|
+
│ Mutation Score = (Killed Mutants / Total Mutants) × 100 │
|
|
1081
|
+
│ │
|
|
1082
|
+
│ - Killed: Test failed → Good, test detected the change │
|
|
1083
|
+
│ - Survived: Test passed → Bad, test missed the defect │
|
|
1084
|
+
│ │
|
|
1085
|
+
│ Target: > 80% mutation score for critical code │
|
|
1086
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1087
|
+
```
|
|
1088
|
+
|
|
1089
|
+
**工具**: Stryker (JS/TS), PITest (Java), mutmut (Python)
|
|
1090
|
+
|
|
1091
|
+
```bash
|
|
1092
|
+
# Run mutation testing with Stryker
|
|
1093
|
+
npx stryker run
|
|
1094
|
+
|
|
1095
|
+
# Example output:
|
|
1096
|
+
# Mutation score: 85.7%
|
|
1097
|
+
# Killed: 180 Survived: 30 No coverage: 10
|
|
1098
|
+
```
|
|
1099
|
+
|
|
1100
|
+
### Chaos Engineering
|
|
1101
|
+
|
|
1102
|
+
藉由在生產或預備環境中注入故障,主動測試系統的韌性。
|
|
1103
|
+
|
|
1104
|
+
```
|
|
1105
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1106
|
+
│ Chaos Engineering Principles │
|
|
1107
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1108
|
+
│ │
|
|
1109
|
+
│ 1. Define "steady state" (normal behavior metrics) │
|
|
1110
|
+
│ 2. Hypothesize steady state continues during chaos │
|
|
1111
|
+
│ 3. Introduce real-world events: │
|
|
1112
|
+
│ - Server failures │
|
|
1113
|
+
│ - Network latency/partition │
|
|
1114
|
+
│ - Resource exhaustion │
|
|
1115
|
+
│ - Clock skew │
|
|
1116
|
+
│ 4. Try to disprove the hypothesis │
|
|
1117
|
+
│ 5. Minimize blast radius (start small) │
|
|
1118
|
+
│ │
|
|
1119
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1120
|
+
```
|
|
1121
|
+
|
|
1122
|
+
**工具**: Chaos Monkey, Gremlin, Litmus, Chaos Toolkit
|
|
1123
|
+
|
|
1124
|
+
```yaml
|
|
1125
|
+
# Chaos Toolkit Experiment Example
|
|
1126
|
+
title: "Service resilience when database is slow"
|
|
1127
|
+
description: "Verify the service degrades gracefully"
|
|
1128
|
+
|
|
1129
|
+
steady-state-hypothesis:
|
|
1130
|
+
title: "Service responds within SLA"
|
|
1131
|
+
probes:
|
|
1132
|
+
- name: "service-responds"
|
|
1133
|
+
type: probe
|
|
1134
|
+
provider:
|
|
1135
|
+
type: http
|
|
1136
|
+
url: "http://service/health"
|
|
1137
|
+
timeout: 3
|
|
1138
|
+
|
|
1139
|
+
method:
|
|
1140
|
+
- name: "inject-db-latency"
|
|
1141
|
+
type: action
|
|
1142
|
+
provider:
|
|
1143
|
+
type: process
|
|
1144
|
+
path: "tc"
|
|
1145
|
+
arguments: "qdisc add dev eth0 root netem delay 500ms"
|
|
1146
|
+
pauses:
|
|
1147
|
+
after: 30
|
|
1148
|
+
|
|
1149
|
+
rollbacks:
|
|
1150
|
+
- name: "remove-latency"
|
|
1151
|
+
type: action
|
|
1152
|
+
provider:
|
|
1153
|
+
type: process
|
|
1154
|
+
path: "tc"
|
|
1155
|
+
arguments: "qdisc del dev eth0 root"
|
|
1156
|
+
```
|
|
1157
|
+
|
|
1158
|
+
### Property-Based Testing
|
|
1159
|
+
|
|
1160
|
+
根據程式碼應滿足的性質,產生隨機的測試輸入。
|
|
1161
|
+
|
|
1162
|
+
```python
|
|
1163
|
+
# Python Example with Hypothesis
|
|
1164
|
+
from hypothesis import given, strategies as st
|
|
1165
|
+
|
|
1166
|
+
# Property: Sorting should be idempotent
|
|
1167
|
+
@given(st.lists(st.integers()))
|
|
1168
|
+
def test_sort_idempotent(xs):
|
|
1169
|
+
sorted_once = sorted(xs)
|
|
1170
|
+
sorted_twice = sorted(sorted_once)
|
|
1171
|
+
assert sorted_once == sorted_twice
|
|
1172
|
+
|
|
1173
|
+
# Property: Sorted list should have same elements
|
|
1174
|
+
@given(st.lists(st.integers()))
|
|
1175
|
+
def test_sort_preserves_elements(xs):
|
|
1176
|
+
sorted_xs = sorted(xs)
|
|
1177
|
+
assert sorted(xs) == sorted(sorted_xs)
|
|
1178
|
+
assert len(xs) == len(sorted_xs)
|
|
1179
|
+
|
|
1180
|
+
# Property: JSON encode/decode roundtrip
|
|
1181
|
+
@given(st.dictionaries(st.text(), st.integers()))
|
|
1182
|
+
def test_json_roundtrip(d):
|
|
1183
|
+
assert json.loads(json.dumps(d)) == d
|
|
1184
|
+
```
|
|
1185
|
+
|
|
1186
|
+
**工具**: Hypothesis (Python), fast-check (JS/TS), QuickCheck (Haskell), FsCheck (C#)
|
|
1187
|
+
|
|
1188
|
+
### Visual Regression Testing
|
|
1189
|
+
|
|
1190
|
+
偵測 UI 元件中非預期的視覺變化。
|
|
1191
|
+
|
|
1192
|
+
```
|
|
1193
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1194
|
+
│ Visual Regression Testing │
|
|
1195
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1196
|
+
│ │
|
|
1197
|
+
│ 1. Capture baseline screenshots │
|
|
1198
|
+
│ 2. Run tests, capture new screenshots │
|
|
1199
|
+
│ 3. Compare pixel-by-pixel or perceptually │
|
|
1200
|
+
│ 4. Flag differences for review │
|
|
1201
|
+
│ │
|
|
1202
|
+
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
|
1203
|
+
│ │ Baseline │ → │ Current │ → │ Diff │ │
|
|
1204
|
+
│ │ Screenshot │ │ Screenshot │ │ Report │ │
|
|
1205
|
+
│ └─────────────┘ └─────────────┘ └─────────────┘ │
|
|
1206
|
+
│ │
|
|
1207
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1208
|
+
```
|
|
1209
|
+
|
|
1210
|
+
**工具**: Percy, Chromatic, BackstopJS, Playwright visual comparisons
|
|
1211
|
+
|
|
1212
|
+
```typescript
|
|
1213
|
+
// Playwright Visual Regression Example
|
|
1214
|
+
test('homepage visual regression', async ({ page }) => {
|
|
1215
|
+
await page.goto('/');
|
|
1216
|
+
await expect(page).toHaveScreenshot('homepage.png', {
|
|
1217
|
+
maxDiffPixels: 100 // Allow minor differences
|
|
1218
|
+
});
|
|
1219
|
+
});
|
|
1220
|
+
|
|
1221
|
+
test('button component visual regression', async ({ page }) => {
|
|
1222
|
+
await page.goto('/components/button');
|
|
1223
|
+
const button = page.locator('[data-testid="primary-button"]');
|
|
1224
|
+
await expect(button).toHaveScreenshot('primary-button.png');
|
|
1225
|
+
});
|
|
1226
|
+
```
|
|
1227
|
+
|
|
1228
|
+
### Accessibility Testing (a11y)
|
|
1229
|
+
|
|
1230
|
+
確保應用程式可供身心障礙者使用。
|
|
1231
|
+
|
|
1232
|
+
```
|
|
1233
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1234
|
+
│ Accessibility Testing Checklist │
|
|
1235
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1236
|
+
│ WCAG 2.1 Guidelines │
|
|
1237
|
+
│ ├── Perceivable │
|
|
1238
|
+
│ │ ├── Alt text for images │
|
|
1239
|
+
│ │ ├── Captions for video │
|
|
1240
|
+
│ │ └── Sufficient color contrast │
|
|
1241
|
+
│ ├── Operable │
|
|
1242
|
+
│ │ ├── Keyboard navigation │
|
|
1243
|
+
│ │ ├── Focus indicators visible │
|
|
1244
|
+
│ │ └── No keyboard traps │
|
|
1245
|
+
│ ├── Understandable │
|
|
1246
|
+
│ │ ├── Clear labels and instructions │
|
|
1247
|
+
│ │ ├── Predictable navigation │
|
|
1248
|
+
│ │ └── Error identification │
|
|
1249
|
+
│ └── Robust │
|
|
1250
|
+
│ ├── Valid HTML │
|
|
1251
|
+
│ └── ARIA attributes correct │
|
|
1252
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1253
|
+
```
|
|
1254
|
+
|
|
1255
|
+
**工具**: axe-core, Pa11y, WAVE, Lighthouse
|
|
1256
|
+
|
|
1257
|
+
```typescript
|
|
1258
|
+
// Playwright + axe-core Example
|
|
1259
|
+
import AxeBuilder from '@axe-core/playwright';
|
|
1260
|
+
|
|
1261
|
+
test('homepage accessibility', async ({ page }) => {
|
|
1262
|
+
await page.goto('/');
|
|
1263
|
+
|
|
1264
|
+
const results = await new AxeBuilder({ page })
|
|
1265
|
+
.withTags(['wcag2a', 'wcag2aa'])
|
|
1266
|
+
.analyze();
|
|
1267
|
+
|
|
1268
|
+
expect(results.violations).toEqual([]);
|
|
1269
|
+
});
|
|
1270
|
+
```
|
|
1271
|
+
|
|
1272
|
+
### Additional Specialized Testing Types (SWEBOK)
|
|
1273
|
+
|
|
1274
|
+
以下測試類型在 SWEBOK v4.0 中被定義為測試目標:
|
|
1275
|
+
|
|
1276
|
+
#### Installation Testing
|
|
1277
|
+
|
|
1278
|
+
驗證軟體能在目標環境中正確安裝。
|
|
1279
|
+
|
|
1280
|
+
```
|
|
1281
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1282
|
+
│ Installation Testing Checklist │
|
|
1283
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1284
|
+
│ │
|
|
1285
|
+
│ Pre-Installation │
|
|
1286
|
+
│ ├── Prerequisites verified (OS, runtime, dependencies) │
|
|
1287
|
+
│ ├── Disk space requirements met │
|
|
1288
|
+
│ └── Permissions and access rights confirmed │
|
|
1289
|
+
│ │
|
|
1290
|
+
│ Installation Process │
|
|
1291
|
+
│ ├── Fresh installation succeeds │
|
|
1292
|
+
│ ├── Upgrade from previous version succeeds │
|
|
1293
|
+
│ ├── Custom installation options work │
|
|
1294
|
+
│ └── Silent/unattended installation works │
|
|
1295
|
+
│ │
|
|
1296
|
+
│ Post-Installation │
|
|
1297
|
+
│ ├── Application launches correctly │
|
|
1298
|
+
│ ├── Configuration files created properly │
|
|
1299
|
+
│ ├── Registry/system settings correct │
|
|
1300
|
+
│ └── Uninstallation removes all components │
|
|
1301
|
+
│ │
|
|
1302
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1303
|
+
```
|
|
1304
|
+
|
|
1305
|
+
#### Alpha and Beta Testing
|
|
1306
|
+
|
|
1307
|
+
| 階段 | 參與者 | 環境 | 目的 |
|
|
1308
|
+
|-------|--------------|-------------|---------|
|
|
1309
|
+
| **Alpha** | 內部使用者、開發者 | 開發場域 | 早期驗證、重大問題 |
|
|
1310
|
+
| **Beta** | 外部使用者、客戶 | 客戶場域 | 真實世界驗證、回饋 |
|
|
1311
|
+
| **Open Beta** | 一般大眾 | 各種環境 | 廣泛曝光、壓力測試 |
|
|
1312
|
+
|
|
1313
|
+
#### Recovery Testing
|
|
1314
|
+
|
|
1315
|
+
測試系統從故障中復原的能力。
|
|
1316
|
+
|
|
1317
|
+
```
|
|
1318
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1319
|
+
│ Recovery Testing Scenarios │
|
|
1320
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1321
|
+
│ │
|
|
1322
|
+
│ Hardware Failures │
|
|
1323
|
+
│ ├── Power outage during transaction │
|
|
1324
|
+
│ ├── Disk failure and RAID recovery │
|
|
1325
|
+
│ └── Network disconnection and reconnection │
|
|
1326
|
+
│ │
|
|
1327
|
+
│ Software Failures │
|
|
1328
|
+
│ ├── Application crash recovery │
|
|
1329
|
+
│ ├── Database connection loss and reconnect │
|
|
1330
|
+
│ └── Service restart behavior │
|
|
1331
|
+
│ │
|
|
1332
|
+
│ Data Recovery │
|
|
1333
|
+
│ ├── Backup and restore procedures │
|
|
1334
|
+
│ ├── Transaction rollback │
|
|
1335
|
+
│ └── Point-in-time recovery │
|
|
1336
|
+
│ │
|
|
1337
|
+
│ Metrics: │
|
|
1338
|
+
│ • Recovery Time Objective (RTO): Max acceptable downtime │
|
|
1339
|
+
│ • Recovery Point Objective (RPO): Max acceptable data loss │
|
|
1340
|
+
│ │
|
|
1341
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1342
|
+
```
|
|
1343
|
+
|
|
1344
|
+
#### Configuration Testing
|
|
1345
|
+
|
|
1346
|
+
在不同配置下測試系統。
|
|
1347
|
+
|
|
1348
|
+
```
|
|
1349
|
+
Configuration Matrix Example:
|
|
1350
|
+
|
|
1351
|
+
┌──────────────────┬─────────┬─────────┬─────────┐
|
|
1352
|
+
│ Configuration │ Config A│ Config B│ Config C│
|
|
1353
|
+
├──────────────────┼─────────┼─────────┼─────────┤
|
|
1354
|
+
│ Memory │ 4 GB │ 8 GB │ 16 GB │
|
|
1355
|
+
│ Database │ SQLite │ MySQL │ PostgreSQL│
|
|
1356
|
+
│ Cache │ None │ Redis │ Memcached│
|
|
1357
|
+
│ Load Balancer │ None │ Nginx │ HAProxy │
|
|
1358
|
+
└──────────────────┴─────────┴─────────┴─────────┘
|
|
1359
|
+
|
|
1360
|
+
Test each supported configuration combination.
|
|
1361
|
+
```
|
|
1362
|
+
|
|
1363
|
+
#### Back-to-Back Testing
|
|
1364
|
+
|
|
1365
|
+
以相同輸入比較多個實作的輸出。
|
|
1366
|
+
|
|
1367
|
+
```
|
|
1368
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1369
|
+
│ Back-to-Back Testing │
|
|
1370
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1371
|
+
│ │
|
|
1372
|
+
│ Use Cases: │
|
|
1373
|
+
│ ├── Comparing new version against old version │
|
|
1374
|
+
│ ├── Comparing optimized code against reference impl │
|
|
1375
|
+
│ ├── Cross-platform consistency verification │
|
|
1376
|
+
│ └── Validating refactored code behavior │
|
|
1377
|
+
│ │
|
|
1378
|
+
│ Process: │
|
|
1379
|
+
│ ┌─────────┐ ┌─────────────┐ ┌──────────┐ │
|
|
1380
|
+
│ │ Input │────▶│ Version A │────▶│ Output A │──┐ │
|
|
1381
|
+
│ │ Data │ └─────────────┘ └──────────┘ │ │
|
|
1382
|
+
│ │ │ ┌─────────────┐ ┌──────────┐ ▼ │
|
|
1383
|
+
│ │ │────▶│ Version B │────▶│ Output B │─▶Compare │
|
|
1384
|
+
│ └─────────┘ └─────────────┘ └──────────┘ │
|
|
1385
|
+
│ │
|
|
1386
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1387
|
+
```
|
|
1388
|
+
|
|
1389
|
+
#### Usability Testing
|
|
1390
|
+
|
|
1391
|
+
評估使用者學習與使用系統的容易程度。
|
|
1392
|
+
|
|
1393
|
+
```
|
|
1394
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1395
|
+
│ Usability Testing Methods │
|
|
1396
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1397
|
+
│ │
|
|
1398
|
+
│ Observational Methods │
|
|
1399
|
+
│ ├── Think-aloud protocol │
|
|
1400
|
+
│ ├── Eye tracking │
|
|
1401
|
+
│ └── Session recording │
|
|
1402
|
+
│ │
|
|
1403
|
+
│ Task-Based Methods │
|
|
1404
|
+
│ ├── Task completion rate │
|
|
1405
|
+
│ ├── Time on task │
|
|
1406
|
+
│ └── Error rate │
|
|
1407
|
+
│ │
|
|
1408
|
+
│ Survey Methods │
|
|
1409
|
+
│ ├── System Usability Scale (SUS) │
|
|
1410
|
+
│ ├── Net Promoter Score (NPS) │
|
|
1411
|
+
│ └── Post-task questionnaires │
|
|
1412
|
+
│ │
|
|
1413
|
+
│ Key Metrics: │
|
|
1414
|
+
│ • Learnability: Time to complete first task │
|
|
1415
|
+
│ • Efficiency: Time to complete after learning │
|
|
1416
|
+
│ • Memorability: Performance after period of non-use │
|
|
1417
|
+
│ • Satisfaction: User rating scores │
|
|
1418
|
+
│ │
|
|
1419
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1420
|
+
```
|
|
1421
|
+
|
|
1422
|
+
### Specialized Testing Summary
|
|
1423
|
+
|
|
1424
|
+
| 類型 | 目的 | 使用時機 |
|
|
1425
|
+
|------|---------|-------------|
|
|
1426
|
+
| Contract Testing | API 相容性 | 微服務 |
|
|
1427
|
+
| Mutation Testing | 測試品質評估 | 關鍵程式碼路徑 |
|
|
1428
|
+
| Chaos Engineering | 韌性驗證 | 生產系統 |
|
|
1429
|
+
| Property-Based | 邊界案例發掘 | 演算法、parser |
|
|
1430
|
+
| Visual Regression | UI 一致性 | 前端變更 |
|
|
1431
|
+
| Accessibility | 無障礙設計 | 所有面向使用者的應用 |
|
|
1432
|
+
| Installation | 部署驗證 | 候選發行版本 |
|
|
1433
|
+
| Alpha/Beta | 早期使用者驗證 | 發行前階段 |
|
|
1434
|
+
| Recovery | 故障復原 | 業務關鍵系統 |
|
|
1435
|
+
| Configuration | 多配置支援 | 跨平台應用 |
|
|
1436
|
+
| Back-to-Back | 版本比較 | 遷移、重構 |
|
|
1437
|
+
| Usability | 使用者體驗 | 面向使用者的應用 |
|
|
1438
|
+
|
|
1439
|
+
---
|
|
1440
|
+
|
|
1441
|
+
## Test-Related Measures
|
|
1442
|
+
|
|
1443
|
+
用以評估受測軟體本身以及測試本身品質的量化度量。
|
|
1444
|
+
|
|
1445
|
+
**參考**: SWEBOK v4.0 第 5.4 節
|
|
1446
|
+
|
|
1447
|
+
### Program Evaluation Measures
|
|
1448
|
+
|
|
1449
|
+
#### Fault Density
|
|
1450
|
+
|
|
1451
|
+
```
|
|
1452
|
+
Fault Density = Total Faults / Size
|
|
1453
|
+
|
|
1454
|
+
Where Size can be:
|
|
1455
|
+
- KLOC (Thousands of Lines of Code)
|
|
1456
|
+
- Function Points
|
|
1457
|
+
- Story Points
|
|
1458
|
+
|
|
1459
|
+
Industry Benchmarks:
|
|
1460
|
+
┌─────────────────────┬────────────────────────────┐
|
|
1461
|
+
│ Quality Level │ Faults per KLOC │
|
|
1462
|
+
├─────────────────────┼────────────────────────────┤
|
|
1463
|
+
│ Typical │ 1 - 25 │
|
|
1464
|
+
│ Good │ 0.5 - 1 │
|
|
1465
|
+
│ Excellent │ < 0.5 │
|
|
1466
|
+
│ Safety-Critical │ < 0.1 │
|
|
1467
|
+
└─────────────────────┴────────────────────────────┘
|
|
1468
|
+
```
|
|
1469
|
+
|
|
1470
|
+
#### Reliability Growth Models
|
|
1471
|
+
|
|
1472
|
+
追蹤缺陷隨時間的發現率,以預測剩餘缺陷。
|
|
1473
|
+
|
|
1474
|
+
```
|
|
1475
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1476
|
+
│ Reliability Growth Curve │
|
|
1477
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1478
|
+
│ │
|
|
1479
|
+
│ Defects │
|
|
1480
|
+
│ Found │
|
|
1481
|
+
│ │ ╭──────────────────────────────── │
|
|
1482
|
+
│ │ ╭─╯ │
|
|
1483
|
+
│ │ ╭╯ │
|
|
1484
|
+
│ │ ╭╯ │
|
|
1485
|
+
│ │╭╯ │
|
|
1486
|
+
│ ├╯ │
|
|
1487
|
+
│ └──────────────────────────────────────▶ Time │
|
|
1488
|
+
│ │
|
|
1489
|
+
│ As testing progresses: │
|
|
1490
|
+
│ • Defect discovery rate decreases │
|
|
1491
|
+
│ • Curve approaches asymptote (total defects) │
|
|
1492
|
+
│ • Remaining defects can be estimated │
|
|
1493
|
+
│ │
|
|
1494
|
+
│ Common Models: │
|
|
1495
|
+
│ • Goel-Okumoto: Exponential │
|
|
1496
|
+
│ • Musa: Logarithmic │
|
|
1497
|
+
│ • Jelinski-Moranda: De-eutrophication │
|
|
1498
|
+
│ │
|
|
1499
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1500
|
+
```
|
|
1501
|
+
|
|
1502
|
+
### Test Evaluation Measures
|
|
1503
|
+
|
|
1504
|
+
#### Fault Seeding
|
|
1505
|
+
|
|
1506
|
+
植入已知的缺陷以衡量測試效能。
|
|
1507
|
+
|
|
1508
|
+
```
|
|
1509
|
+
Fault Detection Effectiveness = Seeded Faults Found / Total Seeded Faults
|
|
1510
|
+
|
|
1511
|
+
If tests find 80% of seeded faults, estimate:
|
|
1512
|
+
- Similar detection rate for real faults
|
|
1513
|
+
- Remaining faults ≈ Found faults / Detection rate
|
|
1514
|
+
|
|
1515
|
+
Example:
|
|
1516
|
+
- Seeded 20 faults, tests found 16 (80%)
|
|
1517
|
+
- Found 40 real faults during testing
|
|
1518
|
+
- Estimated total real faults ≈ 40 / 0.8 = 50
|
|
1519
|
+
- Estimated remaining ≈ 10 faults
|
|
1520
|
+
```
|
|
1521
|
+
|
|
1522
|
+
#### Coverage Metrics Summary
|
|
1523
|
+
|
|
1524
|
+
| Metric | 公式 | 目標 |
|
|
1525
|
+
|--------|---------|--------|
|
|
1526
|
+
| Statement Coverage | 已執行敘述 / 總敘述 | ≥ 70% |
|
|
1527
|
+
| Branch Coverage | 已執行分支 / 總分支 | ≥ 60% |
|
|
1528
|
+
| Condition Coverage | 評估為 T 與 F 的條件 / 總條件 | ≥ 80% |
|
|
1529
|
+
| MC/DC | 能獨立影響決策的條件 | 100%(安全關鍵) |
|
|
1530
|
+
| Requirements Coverage | 已測試需求 / 總需求 | 100% |
|
|
1531
|
+
| Mutation Score | 被殺死的 mutants / 總 mutants | ≥ 80% |
|
|
1532
|
+
|
|
1533
|
+
### Testing Process Metrics
|
|
1534
|
+
|
|
1535
|
+
```
|
|
1536
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1537
|
+
│ Key Testing Metrics │
|
|
1538
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1539
|
+
│ │
|
|
1540
|
+
│ Efficiency Metrics │
|
|
1541
|
+
│ ├── Test Case Productivity = Tests created / Effort │
|
|
1542
|
+
│ ├── Defect Detection Rate = Defects found / Test hours │
|
|
1543
|
+
│ └── Automation Rate = Automated tests / Total tests │
|
|
1544
|
+
│ │
|
|
1545
|
+
│ Effectiveness Metrics │
|
|
1546
|
+
│ ├── Defect Leakage = Prod defects / Total defects │
|
|
1547
|
+
│ ├── Test Effectiveness = Defects found / Defects present │
|
|
1548
|
+
│ └── Phase Containment = Defects found in phase / Injected │
|
|
1549
|
+
│ │
|
|
1550
|
+
│ Progress Metrics │
|
|
1551
|
+
│ ├── Test Execution Rate = Tests run / Tests planned │
|
|
1552
|
+
│ ├── Pass Rate = Tests passed / Tests executed │
|
|
1553
|
+
│ └── Blocking Rate = Blocked tests / Total tests │
|
|
1554
|
+
│ │
|
|
1555
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1556
|
+
```
|
|
1557
|
+
|
|
1558
|
+
---
|
|
1559
|
+
|
|
1560
|
+
## Defect Management
|
|
1561
|
+
|
|
1562
|
+
在整個測試生命週期中對缺陷進行系統化的追蹤與管理。
|
|
1563
|
+
|
|
1564
|
+
**參考**: ISTQB CTFL v4.0 第 5.5 節, ISO/IEC/IEEE 29119-3
|
|
1565
|
+
|
|
1566
|
+
### Defect Lifecycle
|
|
1567
|
+
|
|
1568
|
+
```
|
|
1569
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1570
|
+
│ Defect Lifecycle │
|
|
1571
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1572
|
+
│ │
|
|
1573
|
+
│ ┌────────┐ ┌────────┐ ┌──────────┐ │
|
|
1574
|
+
│ │ New │───▶│ Open │───▶│ In Progress│ │
|
|
1575
|
+
│ └────────┘ └────────┘ └──────────┘ │
|
|
1576
|
+
│ │ │ │
|
|
1577
|
+
│ │ ▼ │
|
|
1578
|
+
│ │ ┌──────────┐ │
|
|
1579
|
+
│ │ │ Fixed │ │
|
|
1580
|
+
│ │ └──────────┘ │
|
|
1581
|
+
│ │ │ │
|
|
1582
|
+
│ │ ▼ │
|
|
1583
|
+
│ │ ┌──────────┐ ┌──────────┐ │
|
|
1584
|
+
│ │ │ Verified │───▶│ Closed │ │
|
|
1585
|
+
│ │ └──────────┘ └──────────┘ │
|
|
1586
|
+
│ │ │ │
|
|
1587
|
+
│ │ ▼ │
|
|
1588
|
+
│ │ ┌──────────┐ │
|
|
1589
|
+
│ └───────▶│ Reopened │ │
|
|
1590
|
+
│ └──────────┘ │
|
|
1591
|
+
│ │
|
|
1592
|
+
│ Alternative paths: │
|
|
1593
|
+
│ - New → Rejected (not a bug, duplicate) │
|
|
1594
|
+
│ - Open → Deferred (postponed to later release) │
|
|
1595
|
+
│ - Open → Won't Fix (accepted risk) │
|
|
1596
|
+
│ │
|
|
1597
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1598
|
+
```
|
|
1599
|
+
|
|
1600
|
+
### Defect Report Template
|
|
1601
|
+
|
|
1602
|
+
```markdown
|
|
1603
|
+
# Defect Report
|
|
1604
|
+
|
|
1605
|
+
## Summary
|
|
1606
|
+
[One-line description of the defect]
|
|
1607
|
+
|
|
1608
|
+
## ID: BUG-1234
|
|
1609
|
+
## Priority: P1/P2/P3/P4
|
|
1610
|
+
## Severity: Critical/High/Medium/Low
|
|
1611
|
+
|
|
1612
|
+
## Environment
|
|
1613
|
+
- **Version**: 2.3.1
|
|
1614
|
+
- **OS**: Windows 11 / macOS 14.2
|
|
1615
|
+
- **Browser**: Chrome 120
|
|
1616
|
+
- **Device**: Desktop / Mobile
|
|
1617
|
+
|
|
1618
|
+
## Steps to Reproduce
|
|
1619
|
+
1. Navigate to login page
|
|
1620
|
+
2. Enter valid username "testuser"
|
|
1621
|
+
3. Enter invalid password "wrong123"
|
|
1622
|
+
4. Click "Login" button
|
|
1623
|
+
5. Observe error message
|
|
1624
|
+
|
|
1625
|
+
## Expected Result
|
|
1626
|
+
Error message: "Invalid credentials. Please try again."
|
|
1627
|
+
|
|
1628
|
+
## Actual Result
|
|
1629
|
+
Generic error: "Something went wrong."
|
|
1630
|
+
|
|
1631
|
+
## Screenshots/Videos
|
|
1632
|
+
[Attach relevant media]
|
|
1633
|
+
|
|
1634
|
+
## Additional Information
|
|
1635
|
+
- Occurs consistently (100% reproducible)
|
|
1636
|
+
- Related to ticket: FEAT-456
|
|
1637
|
+
- Log file: [attached]
|
|
1638
|
+
```
|
|
1639
|
+
|
|
1640
|
+
### Severity vs Priority
|
|
1641
|
+
|
|
1642
|
+
| Severity | 說明 | 範例 |
|
|
1643
|
+
|----------|-------------|----------|
|
|
1644
|
+
| Critical | 系統當機、資料遺失、安全漏洞 | App 無法啟動、資料毀損 |
|
|
1645
|
+
| High | 主要功能損壞、無替代方案 | 無法完成結帳 |
|
|
1646
|
+
| Medium | 功能受損但有替代方案 | 匯出失敗,可手動複製 |
|
|
1647
|
+
| Low | 小問題、外觀瑕疵 | 錯字、對齊問題 |
|
|
1648
|
+
|
|
1649
|
+
| Priority | 說明 | 回應時間 |
|
|
1650
|
+
|----------|-------------|---------------|
|
|
1651
|
+
| P1 | 立即修正 | 當天 |
|
|
1652
|
+
| P2 | 發行前修正 | 本 sprint 內 |
|
|
1653
|
+
| P3 | 有空時修正 | 下個 sprint |
|
|
1654
|
+
| P4 | 修了更好 | Backlog |
|
|
1655
|
+
|
|
1656
|
+
### Defect Metrics
|
|
1657
|
+
|
|
1658
|
+
```
|
|
1659
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1660
|
+
│ Key Defect Metrics │
|
|
1661
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1662
|
+
│ │
|
|
1663
|
+
│ Defect Density = Total Defects / Size (KLOC or FP) │
|
|
1664
|
+
│ - Industry average: 1-25 defects per KLOC │
|
|
1665
|
+
│ │
|
|
1666
|
+
│ Defect Detection Efficiency (DDE) = │
|
|
1667
|
+
│ Defects found before release / Total defects × 100 │
|
|
1668
|
+
│ - Target: > 90% │
|
|
1669
|
+
│ │
|
|
1670
|
+
│ Defect Removal Efficiency (DRE) = │
|
|
1671
|
+
│ Defects removed / Defects injected × 100 │
|
|
1672
|
+
│ - Target: > 95% │
|
|
1673
|
+
│ │
|
|
1674
|
+
│ Mean Time to Detect (MTTD) = Avg time from injection │
|
|
1675
|
+
│ Mean Time to Resolve (MTTR) = Avg time from report to fix │
|
|
1676
|
+
│ │
|
|
1677
|
+
│ Defect Leakage Rate = Production defects / Total defects │
|
|
1678
|
+
│ - Target: < 5% │
|
|
1679
|
+
│ │
|
|
1680
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1681
|
+
```
|
|
1682
|
+
|
|
1683
|
+
### Root Cause Analysis
|
|
1684
|
+
|
|
1685
|
+
```
|
|
1686
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1687
|
+
│ Common Defect Root Causes │
|
|
1688
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1689
|
+
│ Requirements (35-40%) │
|
|
1690
|
+
│ ├── Ambiguous requirements │
|
|
1691
|
+
│ ├── Missing requirements │
|
|
1692
|
+
│ └── Changed requirements │
|
|
1693
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1694
|
+
│ Design (20-25%) │
|
|
1695
|
+
│ ├── Incorrect algorithm │
|
|
1696
|
+
│ ├── Integration issues │
|
|
1697
|
+
│ └── Performance not considered │
|
|
1698
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1699
|
+
│ Code (25-30%) │
|
|
1700
|
+
│ ├── Logic errors │
|
|
1701
|
+
│ ├── Boundary conditions │
|
|
1702
|
+
│ └── Exception handling │
|
|
1703
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1704
|
+
│ Environment (10-15%) │
|
|
1705
|
+
│ ├── Configuration issues │
|
|
1706
|
+
│ ├── Third-party dependencies │
|
|
1707
|
+
│ └── Data issues │
|
|
1708
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1709
|
+
```
|
|
1710
|
+
|
|
1711
|
+
---
|
|
1712
|
+
|
|
1713
|
+
## Test Process Management
|
|
1714
|
+
|
|
1715
|
+
針對測試活動進行規劃、監控與控制的結構化途徑。
|
|
1716
|
+
|
|
1717
|
+
**參考**: ISTQB CTFL v4.0 第 5 章, ISO/IEC/IEEE 29119-2
|
|
1718
|
+
|
|
1719
|
+
### Test Planning
|
|
1720
|
+
|
|
1721
|
+
#### Test Plan Components (ISO/IEC/IEEE 29119-3)
|
|
1722
|
+
|
|
1723
|
+
```
|
|
1724
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1725
|
+
│ Test Plan Structure │
|
|
1726
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1727
|
+
│ │
|
|
1728
|
+
│ 1. Test Plan Identification │
|
|
1729
|
+
│ - Project name, version, date │
|
|
1730
|
+
│ │
|
|
1731
|
+
│ 2. Introduction │
|
|
1732
|
+
│ - Purpose, scope, objectives │
|
|
1733
|
+
│ │
|
|
1734
|
+
│ 3. Test Items │
|
|
1735
|
+
│ - Features to be tested │
|
|
1736
|
+
│ - Features not to be tested │
|
|
1737
|
+
│ │
|
|
1738
|
+
│ 4. Test Approach │
|
|
1739
|
+
│ - Test levels and types │
|
|
1740
|
+
│ - Test techniques │
|
|
1741
|
+
│ - Entry/exit criteria │
|
|
1742
|
+
│ │
|
|
1743
|
+
│ 5. Test Environment │
|
|
1744
|
+
│ - Hardware, software, tools │
|
|
1745
|
+
│ - Test data requirements │
|
|
1746
|
+
│ │
|
|
1747
|
+
│ 6. Test Schedule │
|
|
1748
|
+
│ - Milestones, dependencies │
|
|
1749
|
+
│ │
|
|
1750
|
+
│ 7. Roles and Responsibilities │
|
|
1751
|
+
│ - Team structure, RACI matrix │
|
|
1752
|
+
│ │
|
|
1753
|
+
│ 8. Risks and Contingencies │
|
|
1754
|
+
│ - Risk assessment, mitigation │
|
|
1755
|
+
│ │
|
|
1756
|
+
│ 9. Approvals │
|
|
1757
|
+
│ - Sign-off requirements │
|
|
1758
|
+
│ │
|
|
1759
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1760
|
+
```
|
|
1761
|
+
|
|
1762
|
+
#### Entry and Exit Criteria
|
|
1763
|
+
|
|
1764
|
+
| 準則類型 | 範例 |
|
|
1765
|
+
|---------------|----------|
|
|
1766
|
+
| **Entry Criteria**(開始測試) | 需求已核准、程式碼完成、環境就緒、測試資料齊備 |
|
|
1767
|
+
| **Exit Criteria**(停止測試) | 所有 P1/P2 測試通過、達成覆蓋目標、無未解的嚴重缺陷 |
|
|
1768
|
+
|
|
1769
|
+
```markdown
|
|
1770
|
+
## Entry Criteria for System Testing
|
|
1771
|
+
- [ ] All integration tests passed
|
|
1772
|
+
- [ ] Test environment deployed and verified
|
|
1773
|
+
- [ ] Test data prepared and loaded
|
|
1774
|
+
- [ ] Test cases reviewed and approved
|
|
1775
|
+
- [ ] All blockers from IT resolved
|
|
1776
|
+
|
|
1777
|
+
## Exit Criteria for System Testing
|
|
1778
|
+
- [ ] 100% of critical test cases executed
|
|
1779
|
+
- [ ] 95% of high priority test cases passed
|
|
1780
|
+
- [ ] No Critical or High severity defects open
|
|
1781
|
+
- [ ] Code coverage > 80%
|
|
1782
|
+
- [ ] Performance benchmarks met
|
|
1783
|
+
```
|
|
1784
|
+
|
|
1785
|
+
### Test Monitoring and Control
|
|
1786
|
+
|
|
1787
|
+
#### Key Progress Metrics
|
|
1788
|
+
|
|
1789
|
+
```
|
|
1790
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1791
|
+
│ Test Progress Metrics │
|
|
1792
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1793
|
+
│ │
|
|
1794
|
+
│ Execution Metrics: │
|
|
1795
|
+
│ - Tests Planned: 500 │
|
|
1796
|
+
│ - Tests Executed: 350 (70%) │
|
|
1797
|
+
│ - Tests Passed: 320 (91% of executed) │
|
|
1798
|
+
│ - Tests Failed: 30 (9% of executed) │
|
|
1799
|
+
│ - Tests Blocked: 15 │
|
|
1800
|
+
│ │
|
|
1801
|
+
│ Defect Metrics: │
|
|
1802
|
+
│ - Total Defects Found: 45 │
|
|
1803
|
+
│ - Open Defects: 12 (3 Critical, 5 High, 4 Medium) │
|
|
1804
|
+
│ - Fixed Defects: 28 │
|
|
1805
|
+
│ - Verified Defects: 20 │
|
|
1806
|
+
│ │
|
|
1807
|
+
│ Coverage Metrics: │
|
|
1808
|
+
│ - Requirements Coverage: 85% │
|
|
1809
|
+
│ - Code Coverage: 78% │
|
|
1810
|
+
│ - Risk Coverage: 90% │
|
|
1811
|
+
│ │
|
|
1812
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1813
|
+
```
|
|
1814
|
+
|
|
1815
|
+
#### Test Dashboard Example
|
|
1816
|
+
|
|
1817
|
+
```
|
|
1818
|
+
Test Progress Dashboard - Sprint 15
|
|
1819
|
+
═══════════════════════════════════════════════════════════════
|
|
1820
|
+
|
|
1821
|
+
Test Execution Defect Status
|
|
1822
|
+
█████████████████░░░░ 85% Open: ███░░░░░░░ 12
|
|
1823
|
+
Fixed: ████████░░ 28
|
|
1824
|
+
Passed │████████████████│ 320 Verified: ██████░░░░ 20
|
|
1825
|
+
Failed │██ │ 30
|
|
1826
|
+
Blocked │█ │ 15 Trend: ↓ (improving)
|
|
1827
|
+
|
|
1828
|
+
Coverage Build Health
|
|
1829
|
+
Requirements: ███████████░ 85% Last Build: ✓ Pass
|
|
1830
|
+
Code: ████████░░░░ 78% Unit Tests: 245/245 ✓
|
|
1831
|
+
Risk Items: █████████░░░ 90% Int Tests: 89/92 ⚠
|
|
1832
|
+
Flaky Rate: 2.1%
|
|
1833
|
+
```
|
|
1834
|
+
|
|
1835
|
+
### Test Completion
|
|
1836
|
+
|
|
1837
|
+
#### Test Summary Report
|
|
1838
|
+
|
|
1839
|
+
```markdown
|
|
1840
|
+
# Test Summary Report
|
|
1841
|
+
|
|
1842
|
+
## Project: [Project Name]
|
|
1843
|
+
## Version: [Version Number]
|
|
1844
|
+
## Test Period: [Start Date] - [End Date]
|
|
1845
|
+
|
|
1846
|
+
## Executive Summary
|
|
1847
|
+
[Brief overview of testing activities and outcomes]
|
|
1848
|
+
|
|
1849
|
+
## Test Scope
|
|
1850
|
+
- Features tested: [list]
|
|
1851
|
+
- Features not tested: [list with justification]
|
|
1852
|
+
|
|
1853
|
+
## Test Results Summary
|
|
1854
|
+
|
|
1855
|
+
| Test Level | Planned | Executed | Passed | Failed | Pass Rate |
|
|
1856
|
+
|------------|---------|----------|--------|--------|-----------|
|
|
1857
|
+
| Unit | 500 | 500 | 498 | 2 | 99.6% |
|
|
1858
|
+
| Integration| 150 | 150 | 145 | 5 | 96.7% |
|
|
1859
|
+
| System | 200 | 195 | 188 | 7 | 96.4% |
|
|
1860
|
+
| E2E | 50 | 48 | 46 | 2 | 95.8% |
|
|
1861
|
+
|
|
1862
|
+
## Defect Summary
|
|
1863
|
+
|
|
1864
|
+
| Severity | Found | Fixed | Open | Deferred |
|
|
1865
|
+
|----------|-------|-------|------|----------|
|
|
1866
|
+
| Critical | 2 | 2 | 0 | 0 |
|
|
1867
|
+
| High | 8 | 7 | 1 | 0 |
|
|
1868
|
+
| Medium | 15 | 12 | 2 | 1 |
|
|
1869
|
+
| Low | 20 | 10 | 5 | 5 |
|
|
1870
|
+
|
|
1871
|
+
## Coverage Analysis
|
|
1872
|
+
- Requirements coverage: 95%
|
|
1873
|
+
- Code coverage: 82%
|
|
1874
|
+
- Risk coverage: 100% (all high-risk items tested)
|
|
1875
|
+
|
|
1876
|
+
## Open Issues and Risks
|
|
1877
|
+
1. [Issue 1]: Impact and mitigation
|
|
1878
|
+
2. [Issue 2]: Impact and mitigation
|
|
1879
|
+
|
|
1880
|
+
## Recommendations
|
|
1881
|
+
1. [Recommendation 1]
|
|
1882
|
+
2. [Recommendation 2]
|
|
1883
|
+
|
|
1884
|
+
## Conclusion
|
|
1885
|
+
[Final assessment and release recommendation]
|
|
1886
|
+
|
|
1887
|
+
## Sign-off
|
|
1888
|
+
- QA Lead: [Name, Date]
|
|
1889
|
+
- Dev Lead: [Name, Date]
|
|
1890
|
+
- Product Owner: [Name, Date]
|
|
1891
|
+
```
|
|
1892
|
+
|
|
1893
|
+
### Test Process Improvement
|
|
1894
|
+
|
|
1895
|
+
```
|
|
1896
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1897
|
+
│ Continuous Improvement Cycle │
|
|
1898
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1899
|
+
│ │
|
|
1900
|
+
│ ┌───────────┐ │
|
|
1901
|
+
│ │ Plan │ │
|
|
1902
|
+
│ └─────┬─────┘ │
|
|
1903
|
+
│ │ │
|
|
1904
|
+
│ ▼ │
|
|
1905
|
+
│ ┌───────────┐ ┌───────────┐ │
|
|
1906
|
+
│ │ Act │◀───────▶│ Do │ │
|
|
1907
|
+
│ └───────────┘ └─────┬─────┘ │
|
|
1908
|
+
│ ▲ │ │
|
|
1909
|
+
│ │ ▼ │
|
|
1910
|
+
│ ┌─────┴─────┐ │
|
|
1911
|
+
│ │ Check │ │
|
|
1912
|
+
│ └───────────┘ │
|
|
1913
|
+
│ │
|
|
1914
|
+
│ Retrospective Questions: │
|
|
1915
|
+
│ - What went well? │
|
|
1916
|
+
│ - What could be improved? │
|
|
1917
|
+
│ - What actions will we take? │
|
|
1918
|
+
│ │
|
|
1919
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1920
|
+
```
|
|
1921
|
+
|
|
1922
|
+
---
|
|
1923
|
+
|
|
1924
|
+
## Code Examples by Test Level
|
|
1925
|
+
|
|
1926
|
+
### Unit Testing Example
|
|
1927
|
+
|
|
1928
|
+
```csharp
|
|
1929
|
+
// C# Example
|
|
1930
|
+
[TestClass]
|
|
1931
|
+
public class UserValidatorTests
|
|
1932
|
+
{
|
|
1933
|
+
private UserValidator _validator;
|
|
1934
|
+
|
|
1935
|
+
[TestInitialize]
|
|
1936
|
+
public void Setup()
|
|
1937
|
+
{
|
|
1938
|
+
_validator = new UserValidator();
|
|
1939
|
+
}
|
|
1940
|
+
|
|
1941
|
+
[TestMethod]
|
|
1942
|
+
public void ValidateEmail_ValidFormat_ReturnsTrue()
|
|
1943
|
+
{
|
|
1944
|
+
// Arrange
|
|
1945
|
+
var email = "user@example.com";
|
|
1946
|
+
|
|
1947
|
+
// Act
|
|
1948
|
+
var result = _validator.ValidateEmail(email);
|
|
1949
|
+
|
|
1950
|
+
// Assert
|
|
1951
|
+
Assert.IsTrue(result);
|
|
1952
|
+
}
|
|
1953
|
+
|
|
1954
|
+
[TestMethod]
|
|
1955
|
+
public void ValidateEmail_InvalidFormat_ReturnsFalse()
|
|
1956
|
+
{
|
|
1957
|
+
// Arrange
|
|
1958
|
+
var email = "invalid-email";
|
|
1959
|
+
|
|
1960
|
+
// Act
|
|
1961
|
+
var result = _validator.ValidateEmail(email);
|
|
1962
|
+
|
|
1963
|
+
// Assert
|
|
1964
|
+
Assert.IsFalse(result);
|
|
1965
|
+
}
|
|
1966
|
+
}
|
|
1967
|
+
```
|
|
1968
|
+
|
|
1969
|
+
```typescript
|
|
1970
|
+
// TypeScript Example
|
|
1971
|
+
describe('UserValidator', () => {
|
|
1972
|
+
let validator: UserValidator;
|
|
1973
|
+
|
|
1974
|
+
beforeEach(() => {
|
|
1975
|
+
validator = new UserValidator();
|
|
1976
|
+
});
|
|
1977
|
+
|
|
1978
|
+
describe('validateEmail', () => {
|
|
1979
|
+
it('should return true for valid email format', () => {
|
|
1980
|
+
const result = validator.validateEmail('user@example.com');
|
|
1981
|
+
expect(result).toBe(true);
|
|
1982
|
+
});
|
|
1983
|
+
|
|
1984
|
+
it('should return false for invalid email format', () => {
|
|
1985
|
+
const result = validator.validateEmail('invalid-email');
|
|
1986
|
+
expect(result).toBe(false);
|
|
1987
|
+
});
|
|
1988
|
+
});
|
|
1989
|
+
});
|
|
1990
|
+
```
|
|
1991
|
+
|
|
1992
|
+
### Integration Testing Example
|
|
1993
|
+
|
|
1994
|
+
```csharp
|
|
1995
|
+
// C# Integration Test Example with Test Database
|
|
1996
|
+
[TestClass]
|
|
1997
|
+
public class UserRepositoryIntegrationTests
|
|
1998
|
+
{
|
|
1999
|
+
private TestDbContext _dbContext;
|
|
2000
|
+
private UserRepository _repository;
|
|
2001
|
+
|
|
2002
|
+
[TestInitialize]
|
|
2003
|
+
public async Task Setup()
|
|
2004
|
+
{
|
|
2005
|
+
// Use test database (e.g., SQLite in-memory or Testcontainers)
|
|
2006
|
+
_dbContext = TestDbContextFactory.Create();
|
|
2007
|
+
_repository = new UserRepository(_dbContext);
|
|
2008
|
+
await _dbContext.Database.EnsureCreatedAsync();
|
|
2009
|
+
}
|
|
2010
|
+
|
|
2011
|
+
[TestCleanup]
|
|
2012
|
+
public async Task Cleanup()
|
|
2013
|
+
{
|
|
2014
|
+
await _dbContext.DisposeAsync();
|
|
2015
|
+
}
|
|
2016
|
+
|
|
2017
|
+
[TestMethod]
|
|
2018
|
+
public async Task CreateUser_WithValidData_PersistsToDatabase()
|
|
2019
|
+
{
|
|
2020
|
+
// Arrange
|
|
2021
|
+
var user = new User { Name = "Test User", Email = "test@example.com" };
|
|
2022
|
+
|
|
2023
|
+
// Act
|
|
2024
|
+
await _repository.CreateAsync(user);
|
|
2025
|
+
var savedUser = await _repository.GetByIdAsync(user.Id);
|
|
2026
|
+
|
|
2027
|
+
// Assert
|
|
2028
|
+
Assert.IsNotNull(savedUser);
|
|
2029
|
+
Assert.AreEqual("Test User", savedUser.Name);
|
|
2030
|
+
}
|
|
2031
|
+
}
|
|
2032
|
+
```
|
|
2033
|
+
|
|
2034
|
+
### System Testing Example
|
|
2035
|
+
|
|
2036
|
+
```csharp
|
|
2037
|
+
// System Test Example: Complete Resource Processing Flow
|
|
2038
|
+
[TestClass]
|
|
2039
|
+
public class ResourceProcessingSystemTests
|
|
2040
|
+
{
|
|
2041
|
+
private HttpClient _client;
|
|
2042
|
+
private TestEnvironment _env;
|
|
2043
|
+
|
|
2044
|
+
[TestInitialize]
|
|
2045
|
+
public async Task Setup()
|
|
2046
|
+
{
|
|
2047
|
+
_env = await TestEnvironment.CreateAsync();
|
|
2048
|
+
_client = _env.CreateAuthenticatedClient();
|
|
2049
|
+
}
|
|
2050
|
+
|
|
2051
|
+
[TestMethod]
|
|
2052
|
+
public async Task ProcessResource_CompleteFlow_CompletedSuccessfully()
|
|
2053
|
+
{
|
|
2054
|
+
// Arrange: Create test data
|
|
2055
|
+
var item = await _env.CreateTestItem(value: 100);
|
|
2056
|
+
var user = await _env.CreateTestUser();
|
|
2057
|
+
|
|
2058
|
+
// Act: Execute complete processing flow
|
|
2059
|
+
// Step 1: Create request
|
|
2060
|
+
var requestResponse = await _client.PostAsync("/api/requests",
|
|
2061
|
+
new { itemId = item.Id, quantity = 2 });
|
|
2062
|
+
Assert.AreEqual(HttpStatusCode.OK, requestResponse.StatusCode);
|
|
2063
|
+
|
|
2064
|
+
// Step 2: Submit processing
|
|
2065
|
+
var processResponse = await _client.PostAsync("/api/processes",
|
|
2066
|
+
new { requestId = requestResponse.RequestId, userId = user.Id });
|
|
2067
|
+
var process = await processResponse.Content.ReadAsAsync<Process>();
|
|
2068
|
+
Assert.AreEqual(HttpStatusCode.Created, processResponse.StatusCode);
|
|
2069
|
+
|
|
2070
|
+
// Step 3: Confirm completion
|
|
2071
|
+
var confirmResponse = await _client.PostAsync($"/api/processes/{process.Id}/confirm",
|
|
2072
|
+
new { confirmationType = "standard", amount = 200 });
|
|
2073
|
+
Assert.AreEqual(HttpStatusCode.OK, confirmResponse.StatusCode);
|
|
2074
|
+
|
|
2075
|
+
// Assert: Verify final state
|
|
2076
|
+
var finalProcess = await _client.GetAsync($"/api/processes/{process.Id}");
|
|
2077
|
+
var result = await finalProcess.Content.ReadAsAsync<Process>();
|
|
2078
|
+
|
|
2079
|
+
Assert.AreEqual(ProcessStatus.Completed, result.Status);
|
|
2080
|
+
Assert.AreEqual(200, result.TotalAmount);
|
|
2081
|
+
Assert.IsNotNull(result.Confirmation);
|
|
2082
|
+
}
|
|
2083
|
+
}
|
|
2084
|
+
```
|
|
2085
|
+
|
|
2086
|
+
### E2E Testing Example
|
|
2087
|
+
|
|
2088
|
+
```typescript
|
|
2089
|
+
// Playwright E2E Test Example
|
|
2090
|
+
import { test, expect } from '@playwright/test';
|
|
2091
|
+
|
|
2092
|
+
test.describe('User Registration Journey', () => {
|
|
2093
|
+
test('should complete registration and login successfully', async ({ page }) => {
|
|
2094
|
+
// Step 1: Navigate to registration page
|
|
2095
|
+
await page.goto('/register');
|
|
2096
|
+
|
|
2097
|
+
// Step 2: Fill registration form
|
|
2098
|
+
await page.fill('[data-testid="email"]', 'newuser@example.com');
|
|
2099
|
+
await page.fill('[data-testid="password"]', 'SecurePass123!');
|
|
2100
|
+
await page.fill('[data-testid="confirm-password"]', 'SecurePass123!');
|
|
2101
|
+
await page.click('[data-testid="register-button"]');
|
|
2102
|
+
|
|
2103
|
+
// Step 3: Verify registration success
|
|
2104
|
+
await expect(page.locator('[data-testid="success-message"]'))
|
|
2105
|
+
.toContainText('Registration successful');
|
|
2106
|
+
|
|
2107
|
+
// Step 4: Login with new account
|
|
2108
|
+
await page.goto('/login');
|
|
2109
|
+
await page.fill('[data-testid="email"]', 'newuser@example.com');
|
|
2110
|
+
await page.fill('[data-testid="password"]', 'SecurePass123!');
|
|
2111
|
+
await page.click('[data-testid="login-button"]');
|
|
2112
|
+
|
|
2113
|
+
// Step 5: Verify login success and dashboard redirect
|
|
2114
|
+
await expect(page).toHaveURL('/dashboard');
|
|
2115
|
+
await expect(page.locator('[data-testid="welcome-message"]'))
|
|
2116
|
+
.toContainText('Welcome, newuser@example.com');
|
|
2117
|
+
});
|
|
2118
|
+
});
|
|
2119
|
+
```
|
|
2120
|
+
|
|
2121
|
+
### Containerized Testing Example
|
|
2122
|
+
|
|
2123
|
+
```csharp
|
|
2124
|
+
// C# Example with Testcontainers
|
|
2125
|
+
public class DatabaseIntegrationTests : IAsyncLifetime
|
|
2126
|
+
{
|
|
2127
|
+
private readonly PostgreSqlContainer _postgres = new PostgreSqlBuilder()
|
|
2128
|
+
.WithImage("postgres:15")
|
|
2129
|
+
.Build();
|
|
2130
|
+
|
|
2131
|
+
public async Task InitializeAsync()
|
|
2132
|
+
{
|
|
2133
|
+
await _postgres.StartAsync();
|
|
2134
|
+
}
|
|
2135
|
+
|
|
2136
|
+
public async Task DisposeAsync()
|
|
2137
|
+
{
|
|
2138
|
+
await _postgres.DisposeAsync();
|
|
2139
|
+
}
|
|
2140
|
+
|
|
2141
|
+
[Fact]
|
|
2142
|
+
public async Task Should_Connect_To_Database()
|
|
2143
|
+
{
|
|
2144
|
+
var connectionString = _postgres.GetConnectionString();
|
|
2145
|
+
// Use connectionString for tests
|
|
2146
|
+
}
|
|
2147
|
+
}
|
|
2148
|
+
```
|
|
2149
|
+
|
|
2150
|
+
```python
|
|
2151
|
+
# Python Example with Testcontainers
|
|
2152
|
+
import pytest
|
|
2153
|
+
from testcontainers.postgres import PostgresContainer
|
|
2154
|
+
|
|
2155
|
+
@pytest.fixture(scope="module")
|
|
2156
|
+
def postgres_container():
|
|
2157
|
+
with PostgresContainer("postgres:15") as postgres:
|
|
2158
|
+
yield postgres
|
|
2159
|
+
|
|
2160
|
+
def test_database_connection(postgres_container):
|
|
2161
|
+
connection_url = postgres_container.get_connection_url()
|
|
2162
|
+
# Use connection_url for tests
|
|
2163
|
+
```
|
|
2164
|
+
|
|
2165
|
+
```yaml
|
|
2166
|
+
# docker-compose.test.yml
|
|
2167
|
+
version: '3.8'
|
|
2168
|
+
services:
|
|
2169
|
+
app:
|
|
2170
|
+
build: .
|
|
2171
|
+
depends_on:
|
|
2172
|
+
- db
|
|
2173
|
+
- redis
|
|
2174
|
+
- rabbitmq
|
|
2175
|
+
environment:
|
|
2176
|
+
- DATABASE_URL=postgres://test:test@db:5432/testdb
|
|
2177
|
+
- REDIS_URL=redis://redis:6379
|
|
2178
|
+
- RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672
|
|
2179
|
+
|
|
2180
|
+
db:
|
|
2181
|
+
image: postgres:15
|
|
2182
|
+
environment:
|
|
2183
|
+
POSTGRES_USER: test
|
|
2184
|
+
POSTGRES_PASSWORD: test
|
|
2185
|
+
POSTGRES_DB: testdb
|
|
2186
|
+
|
|
2187
|
+
redis:
|
|
2188
|
+
image: redis:7-alpine
|
|
2189
|
+
|
|
2190
|
+
rabbitmq:
|
|
2191
|
+
image: rabbitmq:3-management
|
|
2192
|
+
```
|
|
2193
|
+
|
|
2194
|
+
---
|
|
2195
|
+
|
|
2196
|
+
## Quick Reference Card
|
|
2197
|
+
|
|
2198
|
+
```
|
|
2199
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
2200
|
+
│ Reference Standards │
|
|
2201
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2202
|
+
│ • ISTQB CTFL v4.0 - Testing certification │
|
|
2203
|
+
│ • ISO/IEC/IEEE 29119 - Testing standards │
|
|
2204
|
+
│ • SWEBOK v4.0 - Software engineering knowledge │
|
|
2205
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2206
|
+
│ Testing Fundamentals (SWEBOK) │
|
|
2207
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2208
|
+
│ Error → Fault → Failure (cause chain) │
|
|
2209
|
+
│ Oracle Problem: How to know expected result? │
|
|
2210
|
+
│ Limitation: Testing proves presence, not absence of bugs │
|
|
2211
|
+
│ Testability: Controllability + Observability + Isolability │
|
|
2212
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2213
|
+
│ Testing Framework Options │
|
|
2214
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2215
|
+
│ ISTQB: UT → IT/SIT → ST → AT/UAT (Enterprise/Compliance) │
|
|
2216
|
+
│ Industry: UT (70%) → IT (20%) → E2E (10%) (Agile/DevOps) │
|
|
2217
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2218
|
+
│ Testing Levels Summary │
|
|
2219
|
+
├──────────┬──────────────────────────────────────────────────┤
|
|
2220
|
+
│ UT │ Single unit, isolated, mocked deps, < 100ms │
|
|
2221
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2222
|
+
│ IT/SIT │ Component integration, real DB, 1-10 sec │
|
|
2223
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2224
|
+
│ ST │ Full system, requirement-based (ISTQB only) │
|
|
2225
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2226
|
+
│ E2E │ User journeys, UI to DB, critical paths only │
|
|
2227
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2228
|
+
│ AT/UAT │ Business validation by end users (ISTQB only) │
|
|
2229
|
+
├──────────┴──────────────────────────────────────────────────┤
|
|
2230
|
+
│ Test Types (ISTQB) │
|
|
2231
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2232
|
+
│ Functional: Verify system behavior (black-box) │
|
|
2233
|
+
│ Non-Functional: Performance, security, usability │
|
|
2234
|
+
│ Structural: Code coverage, white-box techniques │
|
|
2235
|
+
│ Change-Related: Confirmation & regression testing │
|
|
2236
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2237
|
+
│ Test Design Techniques │
|
|
2238
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2239
|
+
│ Black-Box: EP, BVA, Decision Table, State Transition │
|
|
2240
|
+
│ Pairwise Testing, Use Case Testing │
|
|
2241
|
+
│ White-Box: Statement, Branch, Condition, MC/DC │
|
|
2242
|
+
│ Data Flow (All-Defs, All-Uses) │
|
|
2243
|
+
│ Experience: Exploratory, Error Guessing, Checklists │
|
|
2244
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2245
|
+
│ Coverage Targets │
|
|
2246
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2247
|
+
│ Line: 70% min / 85% recommended │
|
|
2248
|
+
│ Branch: 60% min / 80% recommended │
|
|
2249
|
+
│ Function: 80% min / 90% recommended │
|
|
2250
|
+
│ Mutation Score: ≥ 80% for critical code │
|
|
2251
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2252
|
+
│ Specialized Testing Types │
|
|
2253
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2254
|
+
│ Contract: API compatibility for microservices │
|
|
2255
|
+
│ Mutation: Test quality via code mutations │
|
|
2256
|
+
│ Chaos: Resilience via failure injection │
|
|
2257
|
+
│ Property-Based: Random inputs based on properties │
|
|
2258
|
+
│ Visual Regression: UI screenshot comparison │
|
|
2259
|
+
│ Accessibility: WCAG compliance validation │
|
|
2260
|
+
│ Installation: Deployment verification │
|
|
2261
|
+
│ Recovery: Failure recovery (RTO/RPO) │
|
|
2262
|
+
│ Configuration: Multi-config testing │
|
|
2263
|
+
│ Usability: User experience (SUS, NPS) │
|
|
2264
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2265
|
+
│ Test-Related Measures │
|
|
2266
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2267
|
+
│ Fault Density = Faults / KLOC (target: < 1) │
|
|
2268
|
+
│ Defect Leakage = Prod defects / Total defects (< 5%) │
|
|
2269
|
+
│ Fault Seeding: Insert known faults to measure detection │
|
|
2270
|
+
│ Reliability Growth: Track defect rate over time │
|
|
2271
|
+
└─────────────────────────────────────────────────────────────┘
|
|
2272
|
+
```
|
|
2273
|
+
|
|
2274
|
+
---
|
|
2275
|
+
|
|
2276
|
+
## Related Standards
|
|
2277
|
+
|
|
2278
|
+
- [Testing Standards](../../core/testing-standards.md) - 給 AI agent 的可執行規則
|
|
2279
|
+
- [Testing Pyramid](./testing-pyramid.md) - 詳細的金字塔比例
|
|
2280
|
+
- [Test-Driven Development](../../core/test-driven-development.md) - TDD/BDD/ATDD 方法論
|
|
2281
|
+
- [Test Completeness Dimensions](../../core/test-completeness-dimensions.md) - 8 維度測試覆蓋
|
|
2282
|
+
- [Code Review Checklist](../../core/code-review-checklist.md)
|
|
2283
|
+
|
|
2284
|
+
---
|
|
2285
|
+
|
|
2286
|
+
## Version History
|
|
2287
|
+
|
|
2288
|
+
| 版本 | 日期 | 變更 |
|
|
2289
|
+
|---------|------|---------|
|
|
2290
|
+
| 1.0.0 | 2026-01-29 | 初版建立:從 core/testing-standards.md 抽取教學內容,包含 Testing Fundamentals、Static Testing、Test Design Techniques、Experience-Based Testing、Risk-Based Testing、Specialized Testing、Test-Related Measures、Defect Management、Test Process Management、Code Examples 及 Quick Reference Card。 |
|
|
2291
|
+
|
|
2292
|
+
---
|
|
2293
|
+
|
|
2294
|
+
## License
|
|
2295
|
+
|
|
2296
|
+
本文件以 [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) 授權釋出。
|
|
2297
|
+
|
|
2298
|
+
**來源**: [universal-dev-standards](https://github.com/AsiaOstrich/universal-dev-standards)
|