universal-dev-standards 5.17.0 → 6.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/uds.js +13 -135
- package/bundled/ai/options/commit-message/bilingual.ai.yaml +19 -1
- package/bundled/ai/standards/acceptance-criteria-traceability.ai.yaml +13 -4
- package/bundled/ai/standards/accessibility-standards.ai.yaml +1 -1
- package/bundled/ai/standards/ai-response-navigation.ai.yaml +15 -2
- package/bundled/ai/standards/api-design-standards.ai.yaml +27 -4
- package/bundled/ai/standards/behavior-snapshot.ai.yaml +86 -9
- package/bundled/ai/standards/checkin-standards.ai.yaml +3 -3
- package/bundled/ai/standards/code-review.ai.yaml +2 -2
- package/bundled/ai/standards/context-aware-loading.ai.yaml +3 -3
- package/bundled/ai/standards/data-migration-testing.ai.yaml +79 -3
- package/bundled/ai/standards/deprecation-standards.ai.yaml +10 -2
- package/bundled/ai/standards/developer-memory.ai.yaml +4 -3
- package/bundled/ai/standards/documentation-writing-standards.ai.yaml +1 -1
- package/bundled/ai/standards/flow-based-testing.ai.yaml +2 -2
- package/bundled/ai/standards/forward-derivation-standards.ai.yaml +26 -2
- package/bundled/ai/standards/frontend-design-standards.ai.yaml +33 -3
- package/bundled/ai/standards/full-coverage-testing.ai.yaml +34 -2
- package/bundled/ai/standards/git-worktree.ai.yaml +3 -3
- package/bundled/ai/standards/logging.ai.yaml +97 -3
- package/bundled/ai/standards/mock-boundary.ai.yaml +48 -2
- package/bundled/ai/standards/model-provenance.ai.yaml +297 -0
- package/bundled/ai/standards/model-selection.ai.yaml +11 -2
- package/bundled/ai/standards/observability-standards.ai.yaml +10 -0
- package/bundled/ai/standards/performance-standards.ai.yaml +46 -1
- package/bundled/ai/standards/pii-classification.ai.yaml +15 -1
- package/bundled/ai/standards/pipeline-security-gates.ai.yaml +10 -0
- package/bundled/ai/standards/privacy-standards.ai.yaml +2 -2
- package/bundled/ai/standards/project-context-memory.ai.yaml +2 -2
- package/bundled/ai/standards/push-standards.ai.yaml +14 -3
- package/bundled/ai/standards/refactoring-standards.ai.yaml +68 -3
- package/bundled/ai/standards/requirement-engineering.ai.yaml +42 -3
- package/bundled/ai/standards/resource-cost-boundary.ai.yaml +279 -0
- package/bundled/ai/standards/reverse-engineering-standards.ai.yaml +55 -2
- package/bundled/ai/standards/security-testing.ai.yaml +13 -2
- package/bundled/ai/standards/self-review-protocol.ai.yaml +15 -10
- package/bundled/ai/standards/skill-standard-alignment-check.ai.yaml +37 -2
- package/bundled/ai/standards/user-journey-testing.ai.yaml +107 -0
- package/bundled/ai/standards/verification-evidence.ai.yaml +88 -5
- package/bundled/ai/standards/verification-oracle.ai.yaml +278 -0
- package/bundled/ai/standards/versioning.ai.yaml +32 -38
- package/bundled/core/acceptance-criteria-traceability.md +15 -5
- package/bundled/core/accessibility-standards.md +8 -4
- package/bundled/core/ai-friendly-architecture.md +1 -1
- package/bundled/core/ai-response-navigation.md +31 -3
- package/bundled/core/anti-sycophancy-prompting.md +1 -1
- package/bundled/core/api-design-standards.md +92 -7
- package/bundled/core/audit-trail.md +119 -0
- package/bundled/core/behavior-snapshot.md +94 -7
- package/bundled/core/browser-compatibility-standards.md +15 -2
- package/bundled/core/checkin-standards.md +10 -3
- package/bundled/core/code-review-checklist.md +10 -2
- package/bundled/core/container-image-standards.md +97 -0
- package/bundled/core/context-aware-loading.md +2 -2
- package/bundled/core/cost-budget-test.md +1 -1
- package/bundled/core/cross-flow-regression.md +3 -2
- package/bundled/core/data-contract.md +104 -0
- package/bundled/core/data-migration-testing.md +90 -0
- package/bundled/core/data-pipeline.md +113 -0
- package/bundled/core/deprecation-standards.md +16 -2
- package/bundled/core/developer-memory.md +13 -8
- package/bundled/core/documentation-writing-standards.md +2 -2
- package/bundled/core/error-code-standards.md +4 -3
- package/bundled/core/feature-discovery-standards.md +191 -0
- package/bundled/core/flaky-test-management.md +1 -1
- package/bundled/core/flow-based-testing.md +3 -3
- package/bundled/core/forward-derivation-standards.md +33 -5
- package/bundled/core/frontend-design-standards.md +93 -4
- package/bundled/core/full-coverage-testing.md +72 -0
- package/bundled/core/git-worktree.md +4 -4
- package/bundled/core/guides/performance-guide.md +1 -1
- package/bundled/core/guides/security-guide.md +1 -1
- package/bundled/core/health-check-standards.md +2 -2
- package/bundled/core/iac-design-principles.md +97 -0
- package/bundled/core/incident-response.md +119 -0
- package/bundled/core/license-compliance.md +2 -0
- package/bundled/core/logging-standards.md +78 -2
- package/bundled/core/mock-boundary.md +54 -2
- package/bundled/core/model-provenance.md +181 -0
- package/bundled/core/model-selection.md +27 -2
- package/bundled/core/multi-environment-e2e-testing.md +195 -0
- package/bundled/core/packaging-standards.md +1 -0
- package/bundled/core/performance-standards.md +96 -2
- package/bundled/core/pii-classification.md +148 -0
- package/bundled/core/pipeline-security-gates.md +22 -2
- package/bundled/core/postmortem-standards.md +2 -0
- package/bundled/core/prd-standards.md +91 -0
- package/bundled/core/privacy-standards.md +14 -3
- package/bundled/core/product-metrics-standards.md +100 -0
- package/bundled/core/project-context-memory.md +8 -2
- package/bundled/core/prompt-regression.md +1 -1
- package/bundled/core/push-standards.md +82 -0
- package/bundled/core/refactoring-standards.md +44 -3
- package/bundled/core/replay-test.md +1 -1
- package/bundled/core/requirement-engineering.md +2 -2
- package/bundled/core/resource-cost-boundary.md +180 -0
- package/bundled/core/reverse-engineering-standards.md +66 -2
- package/bundled/core/runbook.md +113 -0
- package/bundled/core/schema-evolution.md +105 -0
- package/bundled/core/secret-management-standards.md +110 -0
- package/bundled/core/security-testing.md +26 -2
- package/bundled/core/self-review-protocol.md +15 -10
- package/bundled/core/skill-standard-alignment-check.md +53 -0
- package/bundled/core/slo-sli.md +109 -0
- package/bundled/core/smoke-test.md +1 -1
- package/bundled/core/tech-debt-standards.md +1 -1
- package/bundled/core/test-data-standards.md +2 -2
- package/bundled/core/user-journey-testing.md +102 -0
- package/bundled/core/user-story-mapping.md +96 -0
- package/bundled/core/verification-evidence.md +170 -10
- package/bundled/core/verification-oracle.md +167 -0
- package/bundled/core/versioning.md +133 -109
- package/bundled/locales/COVERAGE.md +94 -77
- package/bundled/locales/zh-CN/CHANGELOG.md +94 -6
- package/bundled/locales/zh-CN/CLAUDE.md +1 -1
- package/bundled/locales/zh-CN/MAINTENANCE.md +42 -661
- package/bundled/locales/zh-CN/README.md +25 -12
- package/bundled/locales/zh-CN/SECURITY.md +2 -2
- package/bundled/locales/zh-CN/adoption/DAILY-WORKFLOW-GUIDE.md +2 -2
- package/bundled/locales/zh-CN/ai/options/commit-message/bilingual.ai.yaml +6 -1
- package/bundled/locales/zh-CN/core/acceptance-criteria-traceability.md +4 -6
- package/bundled/locales/zh-CN/core/accessibility-standards.md +1 -1
- package/bundled/locales/zh-CN/core/adversarial-test.md +226 -0
- package/bundled/locales/zh-CN/core/agent-behavior-discipline.md +187 -0
- package/bundled/locales/zh-CN/core/ai-response-navigation.md +32 -6
- package/bundled/locales/zh-CN/core/anti-sycophancy-prompting.md +1 -1
- package/bundled/locales/zh-CN/core/api-design-standards.md +1 -1
- package/bundled/locales/zh-CN/core/behavior-snapshot.md +335 -0
- package/bundled/locales/zh-CN/core/browser-compatibility-standards.md +239 -0
- package/bundled/locales/zh-CN/core/cd-deployment-strategies.md +135 -0
- package/bundled/locales/zh-CN/core/chaos-injection-tests.md +130 -0
- package/bundled/locales/zh-CN/core/checkin-standards.md +1 -1
- package/bundled/locales/zh-CN/core/code-review-checklist.md +1 -1
- package/bundled/locales/zh-CN/core/container-security.md +535 -0
- package/bundled/locales/zh-CN/core/context-aware-loading.md +1 -1
- package/bundled/locales/zh-CN/core/contract-testing-standards.md +191 -0
- package/bundled/locales/zh-CN/core/cost-budget-test.md +86 -0
- package/bundled/locales/zh-CN/core/cross-flow-regression.md +200 -0
- package/bundled/locales/zh-CN/core/data-migration-testing.md +217 -0
- package/bundled/locales/zh-CN/core/deployment-standards.md +328 -9
- package/bundled/locales/zh-CN/core/deprecation-standards.md +1 -1
- package/bundled/locales/zh-CN/core/developer-memory.md +2 -2
- package/bundled/locales/zh-CN/core/disaster-recovery-drill.md +87 -0
- package/bundled/locales/zh-CN/core/documentation-structure.md +1 -1
- package/bundled/locales/zh-CN/core/documentation-writing-standards.md +1 -1
- package/bundled/locales/zh-CN/core/error-code-standards.md +2 -2
- package/bundled/locales/zh-CN/core/feature-manifest-standard.md +222 -0
- package/bundled/locales/zh-CN/core/flaky-test-management.md +87 -0
- package/bundled/locales/zh-CN/core/flow-based-testing.md +284 -0
- package/bundled/locales/zh-CN/core/forward-derivation-standards.md +3 -4
- package/bundled/locales/zh-CN/core/frontend-design-standards.md +1 -1
- package/bundled/locales/zh-CN/core/full-coverage-testing.md +269 -0
- package/bundled/locales/zh-CN/core/git-worktree.md +1 -1
- package/bundled/locales/zh-CN/core/governance-layer.md +160 -0
- package/bundled/locales/zh-CN/core/guides/performance-guide.md +515 -0
- package/bundled/locales/zh-CN/core/guides/security-guide.md +494 -0
- package/bundled/locales/zh-CN/core/knowledge-graph-memory.md +128 -0
- package/bundled/locales/zh-CN/core/license-compliance.md +131 -0
- package/bundled/locales/zh-CN/core/llm-output-validation.md +192 -0
- package/bundled/locales/zh-CN/core/logging-standards.md +205 -7
- package/bundled/locales/zh-CN/core/mock-boundary.md +161 -0
- package/bundled/locales/zh-CN/core/model-selection.md +28 -5
- package/bundled/locales/zh-CN/core/mutation-testing.md +106 -0
- package/bundled/locales/zh-CN/core/no-cicd-deployment.md +219 -0
- package/bundled/locales/zh-CN/core/packaging-standards.md +77 -5
- package/bundled/locales/zh-CN/core/pipeline-security-gates.md +146 -0
- package/bundled/locales/zh-CN/core/policy-as-code-testing.md +203 -0
- package/bundled/locales/zh-CN/core/privacy-standards.md +1 -1
- package/bundled/locales/zh-CN/core/project-context-memory.md +1 -1
- package/bundled/locales/zh-CN/core/prompt-regression.md +88 -0
- package/bundled/locales/zh-CN/core/property-based-testing.md +87 -0
- package/bundled/locales/zh-CN/core/release-quality-manifest.md +207 -0
- package/bundled/locales/zh-CN/core/release-readiness-gate.md +193 -0
- package/bundled/locales/zh-CN/core/replay-test.md +102 -0
- package/bundled/locales/zh-CN/core/requirement-engineering.md +1 -1
- package/bundled/locales/zh-CN/core/reverse-engineering-standards.md +46 -1
- package/bundled/locales/zh-CN/core/rollback-standards.md +120 -0
- package/bundled/locales/zh-CN/core/sast-advanced.md +309 -0
- package/bundled/locales/zh-CN/core/secure-op.md +328 -0
- package/bundled/locales/zh-CN/core/security-testing.md +113 -0
- package/bundled/locales/zh-CN/core/self-review-protocol.md +172 -0
- package/bundled/locales/zh-CN/core/server-ops-security.md +507 -0
- package/bundled/locales/zh-CN/core/smoke-test.md +79 -0
- package/bundled/locales/zh-CN/core/spec-driven-development.md +39 -3
- package/bundled/locales/zh-CN/core/supply-chain-attestation.md +131 -0
- package/bundled/locales/zh-CN/core/verification-evidence.md +252 -41
- package/bundled/locales/zh-CN/core/versioning.md +1 -1
- package/bundled/locales/zh-CN/docs/CHEATSHEET.md +36 -7
- package/bundled/locales/zh-CN/docs/FEATURE-REFERENCE.md +71 -53
- package/bundled/locales/zh-CN/docs/MIGRATION-v6.md +88 -0
- package/bundled/locales/zh-CN/docs/USAGE-MODES-COMPARISON.md +2 -2
- package/bundled/locales/zh-CN/docs/USER-MANUAL.md +14 -14
- package/bundled/locales/zh-CN/docs/specs/system/memory-adoption-strategy.md +105 -0
- package/bundled/locales/zh-CN/docs/user/FAQ.md +132 -0
- package/bundled/locales/zh-CN/docs/user/GETTING-STARTED.md +144 -0
- package/bundled/locales/zh-CN/docs/user/GLOSSARY.md +178 -0
- package/bundled/locales/zh-CN/docs/user/README.md +70 -0
- package/bundled/locales/zh-CN/docs/user/TROUBLESHOOTING.md +190 -0
- package/bundled/locales/zh-CN/integrations/github-copilot/COPILOT-CHAT-REFERENCE.md +1 -1
- package/bundled/locales/zh-CN/integrations/github-copilot/README.md +1 -1
- package/bundled/locales/zh-CN/integrations/github-copilot/skills-mapping.md +3 -3
- package/bundled/locales/zh-CN/integrations/opencode/skills-mapping.md +3 -3
- package/bundled/locales/zh-CN/methodologies/guides/sdd-guide.md +836 -0
- package/bundled/locales/zh-CN/options/changelog/auto-generated.md +166 -0
- package/bundled/locales/zh-CN/options/changelog/keep-a-changelog.md +140 -0
- package/bundled/locales/zh-CN/options/code-review/automated-review.md +214 -0
- package/bundled/locales/zh-CN/options/code-review/pair-programming.md +166 -0
- package/bundled/locales/zh-CN/options/code-review/pr-review.md +167 -0
- package/bundled/locales/zh-CN/options/commit-message/bilingual.md +10 -3
- package/bundled/locales/zh-CN/options/documentation/api-docs.md +191 -0
- package/bundled/locales/zh-CN/options/documentation/markdown-docs.md +150 -0
- package/bundled/locales/zh-CN/options/documentation/wiki-style.md +131 -0
- package/bundled/locales/zh-CN/options/project-structure/kotlin.md +144 -0
- package/bundled/locales/zh-CN/options/project-structure/php.md +168 -0
- package/bundled/locales/zh-CN/options/project-structure/ruby.md +156 -0
- package/bundled/locales/zh-CN/options/project-structure/rust.md +136 -0
- package/bundled/locales/zh-CN/options/project-structure/swift.md +165 -0
- package/bundled/locales/zh-CN/options/testing/contract-testing.md +237 -0
- package/bundled/locales/zh-CN/options/testing/industry-pyramid.md +200 -0
- package/bundled/locales/zh-CN/options/testing/istqb-framework.md +144 -0
- package/bundled/locales/zh-CN/options/testing/performance-testing.md +251 -0
- package/bundled/locales/zh-CN/options/testing/security-testing.md +192 -0
- package/bundled/locales/zh-CN/skills/README.md +89 -126
- package/bundled/locales/zh-CN/skills/ac-coverage/SKILL.md +5 -7
- package/bundled/locales/zh-CN/skills/adr-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/agents/code-architect.md +263 -0
- package/bundled/locales/zh-CN/skills/agents/doc-writer.md +410 -0
- package/bundled/locales/zh-CN/skills/agents/reviewer.md +357 -0
- package/bundled/locales/zh-CN/skills/agents/spec-analyst.md +410 -0
- package/bundled/locales/zh-CN/skills/agents/test-specialist.md +368 -0
- package/bundled/locales/zh-CN/skills/ai-collaboration-standards/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/ai-friendly-architecture/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/atdd-assistant/acceptance-criteria-guide.md +1 -1
- package/bundled/locales/zh-CN/skills/atdd-assistant/atdd-workflow.md +3 -4
- package/bundled/locales/zh-CN/skills/audit-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/bdd-assistant/guide.md +1 -2
- package/bundled/locales/zh-CN/skills/brainstorm-assistant/SKILL.md +115 -17
- package/bundled/locales/zh-CN/skills/brainstorm-assistant/guide.md +37 -12
- package/bundled/locales/zh-CN/skills/checkin-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/ci-cd-assistant/SKILL.md +50 -0
- package/bundled/locales/zh-CN/skills/code-review-assistant/SKILL.md +5 -5
- package/bundled/locales/zh-CN/skills/commands/ac-coverage.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/atdd.md +3 -3
- package/bundled/locales/zh-CN/skills/commands/bdd.md +2 -2
- package/bundled/locales/zh-CN/skills/commands/brainstorm.md +26 -17
- package/bundled/locales/zh-CN/skills/commands/{review.md → code-review.md} +4 -4
- package/bundled/locales/zh-CN/skills/commands/derive-all.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/derive-atdd.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/derive-bdd.md +2 -3
- package/bundled/locales/zh-CN/skills/commands/derive-tdd.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/derive.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/dev-workflow.md +2 -2
- package/bundled/locales/zh-CN/skills/commands/methodology.md +4 -4
- package/bundled/locales/zh-CN/skills/commands/pr.md +1 -1
- package/bundled/locales/zh-CN/skills/commands/tdd.md +1 -1
- package/bundled/locales/zh-CN/skills/commit-standards/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/contract-test-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/dev-methodology/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/dev-methodology/create-methodology.md +456 -0
- package/bundled/locales/zh-CN/skills/dev-methodology/guide.md +6 -6
- package/bundled/locales/zh-CN/skills/dev-methodology/runtime.md +296 -0
- package/bundled/locales/zh-CN/skills/dev-workflow-guide/SKILL.md +5 -5
- package/bundled/locales/zh-CN/skills/docs-generator/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/e2e-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/incident-response-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/journey-test-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/logging-guide/SKILL.md +169 -142
- package/bundled/locales/zh-CN/skills/migration-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/observability-assistant/guide.md +1 -1
- package/bundled/locales/zh-CN/skills/pr-automation-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/project-discovery/guide.md +2 -2
- package/bundled/locales/zh-CN/skills/reverse-engineer/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/runbook-assistant/guide.md +1 -1
- package/bundled/locales/zh-CN/skills/security-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/skill-builder/SKILL.md +3 -3
- package/bundled/locales/zh-CN/skills/slo-assistant/guide.md +1 -1
- package/bundled/locales/zh-CN/skills/spec-derivation/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/spec-derivation/guide.md +8 -9
- package/bundled/locales/zh-CN/skills/spec-driven-dev/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/sweep/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/tdd-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-CN/skills/testing-guide/SKILL.md +2 -2
- package/bundled/locales/zh-CN/skills/testing-guide/testing-theory.md +2298 -0
- package/bundled/locales/zh-CN/skills/workflows/README.md +451 -0
- package/bundled/locales/zh-TW/CHANGELOG.md +94 -6
- package/bundled/locales/zh-TW/CLAUDE.md +1 -1
- package/bundled/locales/zh-TW/MAINTENANCE.md +51 -4
- package/bundled/locales/zh-TW/README.md +25 -12
- package/bundled/locales/zh-TW/SECURITY.md +2 -2
- package/bundled/locales/zh-TW/adoption/DAILY-WORKFLOW-GUIDE.md +2 -2
- package/bundled/locales/zh-TW/ai/options/commit-message/bilingual.ai.yaml +6 -1
- package/bundled/locales/zh-TW/ai/standards/versioning.ai.yaml +8 -48
- package/bundled/locales/zh-TW/core/acceptance-criteria-traceability.md +4 -6
- package/bundled/locales/zh-TW/core/accessibility-standards.md +1 -1
- package/bundled/locales/zh-TW/core/adversarial-test.md +226 -0
- package/bundled/locales/zh-TW/core/agent-behavior-discipline.md +187 -0
- package/bundled/locales/zh-TW/core/ai-response-navigation.md +32 -6
- package/bundled/locales/zh-TW/core/anti-sycophancy-prompting.md +1 -1
- package/bundled/locales/zh-TW/core/api-design-standards.md +91 -9
- package/bundled/locales/zh-TW/core/audit-trail.md +110 -0
- package/bundled/locales/zh-TW/core/behavior-snapshot.md +335 -0
- package/bundled/locales/zh-TW/core/browser-compatibility-standards.md +1 -0
- package/bundled/locales/zh-TW/core/cd-deployment-strategies.md +135 -0
- package/bundled/locales/zh-TW/core/chaos-injection-tests.md +130 -0
- package/bundled/locales/zh-TW/core/checkin-standards.md +1 -1
- package/bundled/locales/zh-TW/core/code-review-checklist.md +1 -1
- package/bundled/locales/zh-TW/core/container-image-standards.md +93 -0
- package/bundled/locales/zh-TW/core/container-security.md +535 -0
- package/bundled/locales/zh-TW/core/cost-budget-test.md +86 -0
- package/bundled/locales/zh-TW/core/cross-flow-regression.md +1 -0
- package/bundled/locales/zh-TW/core/data-contract.md +101 -0
- package/bundled/locales/zh-TW/core/data-migration-testing.md +217 -0
- package/bundled/locales/zh-TW/core/data-pipeline.md +105 -0
- package/bundled/locales/zh-TW/core/deployment-standards.md +363 -25
- package/bundled/locales/zh-TW/core/deprecation-standards.md +17 -4
- package/bundled/locales/zh-TW/core/developer-memory.md +2 -2
- package/bundled/locales/zh-TW/core/disaster-recovery-drill.md +87 -0
- package/bundled/locales/zh-TW/core/documentation-writing-standards.md +1 -1
- package/bundled/locales/zh-TW/core/error-code-standards.md +2 -2
- package/bundled/locales/zh-TW/core/feature-manifest-standard.md +222 -0
- package/bundled/locales/zh-TW/core/flaky-test-management.md +87 -0
- package/bundled/locales/zh-TW/core/flow-based-testing.md +284 -0
- package/bundled/locales/zh-TW/core/forward-derivation-standards.md +3 -4
- package/bundled/locales/zh-TW/core/frontend-design-standards.md +1 -1
- package/bundled/locales/zh-TW/core/full-coverage-testing.md +250 -0
- package/bundled/locales/zh-TW/core/git-worktree.md +1 -1
- package/bundled/locales/zh-TW/core/guides/performance-guide.md +515 -0
- package/bundled/locales/zh-TW/core/guides/security-guide.md +494 -0
- package/bundled/locales/zh-TW/core/iac-design-principles.md +90 -0
- package/bundled/locales/zh-TW/core/incident-response.md +111 -0
- package/bundled/locales/zh-TW/core/license-compliance.md +131 -0
- package/bundled/locales/zh-TW/core/llm-output-validation.md +192 -0
- package/bundled/locales/zh-TW/core/logging-standards.md +208 -4
- package/bundled/locales/zh-TW/core/mock-boundary.md +161 -0
- package/bundled/locales/zh-TW/core/model-provenance.md +173 -0
- package/bundled/locales/zh-TW/core/model-selection.md +28 -5
- package/bundled/locales/zh-TW/core/mutation-testing.md +106 -0
- package/bundled/locales/zh-TW/core/no-cicd-deployment.md +219 -0
- package/bundled/locales/zh-TW/core/packaging-standards.md +77 -5
- package/bundled/locales/zh-TW/core/performance-standards.md +84 -5
- package/bundled/locales/zh-TW/core/pii-classification.md +102 -0
- package/bundled/locales/zh-TW/core/pipeline-security-gates.md +146 -0
- package/bundled/locales/zh-TW/core/policy-as-code-testing.md +203 -0
- package/bundled/locales/zh-TW/core/prd-standards.md +88 -0
- package/bundled/locales/zh-TW/core/privacy-standards.md +1 -1
- package/bundled/locales/zh-TW/core/product-metrics-standards.md +96 -0
- package/bundled/locales/zh-TW/core/project-context-memory.md +1 -1
- package/bundled/locales/zh-TW/core/prompt-regression.md +88 -0
- package/bundled/locales/zh-TW/core/property-based-testing.md +87 -0
- package/bundled/locales/zh-TW/core/refactoring-standards.md +43 -6
- package/bundled/locales/zh-TW/core/release-quality-manifest.md +207 -0
- package/bundled/locales/zh-TW/core/replay-test.md +102 -0
- package/bundled/locales/zh-TW/core/requirement-engineering.md +1 -1
- package/bundled/locales/zh-TW/core/resource-cost-boundary.md +175 -0
- package/bundled/locales/zh-TW/core/reverse-engineering-standards.md +64 -5
- package/bundled/locales/zh-TW/core/rollback-standards.md +120 -0
- package/bundled/locales/zh-TW/core/runbook.md +110 -0
- package/bundled/locales/zh-TW/core/sast-advanced.md +309 -0
- package/bundled/locales/zh-TW/core/schema-evolution.md +98 -0
- package/bundled/locales/zh-TW/core/secret-management-standards.md +101 -0
- package/bundled/locales/zh-TW/core/secure-op.md +328 -0
- package/bundled/locales/zh-TW/core/security-testing.md +113 -0
- package/bundled/locales/zh-TW/core/self-review-protocol.md +18 -13
- package/bundled/locales/zh-TW/core/server-ops-security.md +507 -0
- package/bundled/locales/zh-TW/core/slo-sli.md +108 -0
- package/bundled/locales/zh-TW/core/smoke-test.md +79 -0
- package/bundled/locales/zh-TW/core/spec-driven-development.md +19 -11
- package/bundled/locales/zh-TW/core/supply-chain-attestation.md +131 -0
- package/bundled/locales/zh-TW/core/user-journey-testing.md +111 -0
- package/bundled/locales/zh-TW/core/user-story-mapping.md +94 -0
- package/bundled/locales/zh-TW/core/verification-evidence.md +260 -31
- package/bundled/locales/zh-TW/core/verification-oracle.md +159 -0
- package/bundled/locales/zh-TW/core/versioning.md +112 -112
- package/bundled/locales/zh-TW/docs/CHEATSHEET.md +36 -7
- package/bundled/locales/zh-TW/docs/DEV-WORKFLOW-MAPPING.md +6 -6
- package/bundled/locales/zh-TW/docs/FEATURE-REFERENCE.md +71 -53
- package/bundled/locales/zh-TW/docs/MIGRATION-v6.md +88 -0
- package/bundled/locales/zh-TW/docs/USAGE-MODES-COMPARISON.md +2 -2
- package/bundled/locales/zh-TW/docs/USER-MANUAL.md +14 -14
- package/bundled/locales/zh-TW/docs/specs/system/memory-adoption-strategy.md +105 -0
- package/bundled/locales/zh-TW/docs/user/FAQ.md +132 -0
- package/bundled/locales/zh-TW/docs/user/GETTING-STARTED.md +144 -0
- package/bundled/locales/zh-TW/docs/user/GLOSSARY.md +178 -0
- package/bundled/locales/zh-TW/docs/user/README.md +70 -0
- package/bundled/locales/zh-TW/docs/user/TROUBLESHOOTING.md +190 -0
- package/bundled/locales/zh-TW/integrations/github-copilot/COPILOT-CHAT-REFERENCE.md +1 -1
- package/bundled/locales/zh-TW/integrations/github-copilot/README.md +1 -1
- package/bundled/locales/zh-TW/integrations/github-copilot/skills-mapping.md +3 -3
- package/bundled/locales/zh-TW/integrations/opencode/skills-mapping.md +3 -3
- package/bundled/locales/zh-TW/methodologies/guides/sdd-guide.md +523 -26
- package/bundled/locales/zh-TW/options/changelog/auto-generated.md +166 -0
- package/bundled/locales/zh-TW/options/changelog/keep-a-changelog.md +140 -0
- package/bundled/locales/zh-TW/options/code-review/automated-review.md +214 -0
- package/bundled/locales/zh-TW/options/code-review/pair-programming.md +166 -0
- package/bundled/locales/zh-TW/options/code-review/pr-review.md +167 -0
- package/bundled/locales/zh-TW/options/commit-message/bilingual.md +10 -3
- package/bundled/locales/zh-TW/options/documentation/api-docs.md +191 -0
- package/bundled/locales/zh-TW/options/documentation/markdown-docs.md +150 -0
- package/bundled/locales/zh-TW/options/documentation/wiki-style.md +131 -0
- package/bundled/locales/zh-TW/options/project-structure/kotlin.md +144 -0
- package/bundled/locales/zh-TW/options/project-structure/php.md +168 -0
- package/bundled/locales/zh-TW/options/project-structure/ruby.md +156 -0
- package/bundled/locales/zh-TW/options/project-structure/rust.md +136 -0
- package/bundled/locales/zh-TW/options/project-structure/swift.md +165 -0
- package/bundled/locales/zh-TW/options/testing/contract-testing.md +237 -0
- package/bundled/locales/zh-TW/options/testing/industry-pyramid.md +200 -0
- package/bundled/locales/zh-TW/options/testing/istqb-framework.md +144 -0
- package/bundled/locales/zh-TW/options/testing/performance-testing.md +251 -0
- package/bundled/locales/zh-TW/options/testing/security-testing.md +192 -0
- package/bundled/locales/zh-TW/skills/README.md +91 -128
- package/bundled/locales/zh-TW/skills/ac-coverage/SKILL.md +4 -6
- package/bundled/locales/zh-TW/skills/adr-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/agents/code-architect.md +263 -0
- package/bundled/locales/zh-TW/skills/agents/doc-writer.md +410 -0
- package/bundled/locales/zh-TW/skills/agents/reviewer.md +357 -0
- package/bundled/locales/zh-TW/skills/agents/spec-analyst.md +410 -0
- package/bundled/locales/zh-TW/skills/agents/test-specialist.md +368 -0
- package/bundled/locales/zh-TW/skills/ai-collaboration-standards/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/ai-friendly-architecture/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/atdd-assistant/SKILL.md +2 -0
- package/bundled/locales/zh-TW/skills/atdd-assistant/acceptance-criteria-guide.md +1 -1
- package/bundled/locales/zh-TW/skills/atdd-assistant/atdd-workflow.md +3 -4
- package/bundled/locales/zh-TW/skills/audit-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/bdd-assistant/SKILL.md +2 -0
- package/bundled/locales/zh-TW/skills/bdd-assistant/guide.md +1 -2
- package/bundled/locales/zh-TW/skills/brainstorm-assistant/SKILL.md +116 -18
- package/bundled/locales/zh-TW/skills/brainstorm-assistant/guide.md +38 -11
- package/bundled/locales/zh-TW/skills/checkin-assistant/SKILL.md +3 -1
- package/bundled/locales/zh-TW/skills/ci-cd-assistant/SKILL.md +50 -0
- package/bundled/locales/zh-TW/skills/code-review-assistant/SKILL.md +7 -5
- package/bundled/locales/zh-TW/skills/commands/ac-coverage.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/atdd.md +3 -3
- package/bundled/locales/zh-TW/skills/commands/bdd.md +2 -2
- package/bundled/locales/zh-TW/skills/commands/brainstorm.md +26 -17
- package/bundled/locales/zh-TW/skills/commands/{review.md → code-review.md} +5 -5
- package/bundled/locales/zh-TW/skills/commands/derive-all.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/derive-atdd.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/derive-bdd.md +2 -3
- package/bundled/locales/zh-TW/skills/commands/derive-tdd.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/derive.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/dev-workflow.md +2 -2
- package/bundled/locales/zh-TW/skills/commands/methodology.md +4 -4
- package/bundled/locales/zh-TW/skills/commands/pr.md +1 -1
- package/bundled/locales/zh-TW/skills/commands/tdd.md +1 -1
- package/bundled/locales/zh-TW/skills/commit-standards/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/contract-test-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/dev-methodology/create-methodology.md +456 -0
- package/bundled/locales/zh-TW/skills/dev-methodology/guide.md +5 -5
- package/bundled/locales/zh-TW/skills/dev-methodology/runtime.md +296 -0
- package/bundled/locales/zh-TW/skills/dev-workflow-guide/SKILL.md +5 -5
- package/bundled/locales/zh-TW/skills/docs-generator/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/e2e-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/incident-response-assistant/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/logging-guide/SKILL.md +52 -27
- package/bundled/locales/zh-TW/skills/migration-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/pr-automation-assistant/SKILL.md +3 -1
- package/bundled/locales/zh-TW/skills/project-discovery/guide.md +2 -2
- package/bundled/locales/zh-TW/skills/reverse-engineer/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/security-assistant/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/spec-derivation/guide.md +7 -8
- package/bundled/locales/zh-TW/skills/spec-driven-dev/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/sweep/SKILL.md +1 -1
- package/bundled/locales/zh-TW/skills/tdd-assistant/SKILL.md +3 -1
- package/bundled/locales/zh-TW/skills/testing-guide/SKILL.md +2 -2
- package/bundled/locales/zh-TW/skills/testing-guide/testing-theory.md +2298 -0
- package/bundled/locales/zh-TW/skills/workflows/README.md +451 -0
- package/bundled/skills/README.md +1 -1
- package/bundled/skills/ac-coverage/SKILL.md +17 -7
- package/bundled/skills/adr-assistant/SKILL.md +1 -1
- package/bundled/skills/agents/code-architect.md +1 -1
- package/bundled/skills/agents/doc-writer.md +1 -1
- package/bundled/skills/agents/reviewer.md +2 -2
- package/bundled/skills/agents/spec-analyst.md +1 -1
- package/bundled/skills/agents/test-specialist.md +1 -1
- package/bundled/skills/ai-collaboration-standards/SKILL.md +2 -2
- package/bundled/skills/ai-friendly-architecture/SKILL.md +2 -2
- package/bundled/skills/ai-instruction-standards/SKILL.md +1 -1
- package/bundled/skills/atdd-assistant/SKILL.md +7 -0
- package/bundled/skills/atdd-assistant/acceptance-criteria-guide.md +1 -1
- package/bundled/skills/atdd-assistant/atdd-workflow.md +6 -8
- package/bundled/skills/audit-assistant/SKILL.md +2 -2
- package/bundled/skills/bdd-assistant/SKILL.md +7 -0
- package/bundled/skills/bdd-assistant/guide.md +1 -2
- package/bundled/skills/brainstorm-assistant/SKILL.md +139 -15
- package/bundled/skills/brainstorm-assistant/guide.md +39 -8
- package/bundled/skills/checkin-assistant/SKILL.md +8 -1
- package/bundled/skills/ci-cd-assistant/SKILL.md +57 -0
- package/bundled/skills/code-review-assistant/SKILL.md +15 -8
- package/bundled/skills/commands/COMMAND-FAMILY-OVERVIEW.md +2 -2
- package/bundled/skills/commands/COMMAND-INDEX.json +3 -3
- package/bundled/skills/commands/README.md +2 -2
- package/bundled/skills/commands/ac-coverage.md +1 -1
- package/bundled/skills/commands/atdd.md +2 -2
- package/bundled/skills/commands/bdd.md +2 -2
- package/bundled/skills/commands/brainstorm.md +25 -14
- package/bundled/skills/commands/{review.md → code-review.md} +6 -6
- package/bundled/skills/commands/derive-all.md +1 -1
- package/bundled/skills/commands/derive-atdd.md +1 -1
- package/bundled/skills/commands/derive-bdd.md +2 -3
- package/bundled/skills/commands/derive-tdd.md +1 -1
- package/bundled/skills/commands/derive.md +1 -1
- package/bundled/skills/commands/dev-workflow.md +1 -1
- package/bundled/skills/commands/journey-test.md +45 -0
- package/bundled/skills/commands/methodology.md +4 -4
- package/bundled/skills/commands/pr.md +1 -1
- package/bundled/skills/commands/skill-builder.md +42 -0
- package/bundled/skills/commands/tdd.md +1 -1
- package/bundled/skills/dev-methodology/create-methodology.md +1 -1
- package/bundled/skills/dev-methodology/guide.md +1 -1
- package/bundled/skills/dev-methodology/runtime.md +1 -1
- package/bundled/skills/dev-workflow-guide/SKILL.md +3 -3
- package/bundled/skills/dev-workflow-guide/workflow-phases.md +3 -3
- package/bundled/skills/docs-generator/SKILL.md +3 -3
- package/bundled/skills/incident-response-assistant/SKILL.md +2 -2
- package/bundled/skills/logging-guide/SKILL.md +38 -2
- package/bundled/skills/migration-assistant/SKILL.md +226 -0
- package/bundled/skills/observability-assistant/SKILL.md +74 -0
- package/bundled/skills/pr-automation-assistant/SKILL.md +5 -1
- package/bundled/skills/project-discovery/guide.md +2 -2
- package/bundled/skills/push/SKILL.md +1 -1
- package/bundled/skills/release-standards/SKILL.md +19 -5
- package/bundled/skills/security-assistant/SKILL.md +1 -1
- package/bundled/skills/skill-builder/SKILL.md +1 -1
- package/bundled/skills/spec-derivation/guide.md +3 -4
- package/bundled/skills/spec-driven-dev/SKILL.md +13 -2
- package/bundled/skills/sweep/SKILL.md +1 -1
- package/bundled/skills/tdd-assistant/SKILL.md +8 -1
- package/bundled/skills/testing-guide/SKILL.md +1 -1
- package/bundled/skills/testing-guide/testing-theory.md +1 -1
- package/bundled/skills/workflows/README.md +2 -2
- package/package.json +4 -4
- package/src/commands/audit.js +30 -9
- package/src/commands/config.js +33 -2
- package/src/commands/hitl.js +14 -1
- package/src/commands/init.js +168 -75
- package/src/commands/quickstart.js +1 -2
- package/src/commands/release.js +51 -6
- package/src/commands/run-intent.js +9 -0
- package/src/commands/spec-split.js +27 -2
- package/src/commands/update.js +22 -6
- package/src/config/ai-agent-paths.js +3 -1
- package/src/i18n/messages.js +3 -102
- package/src/missions/MissionManager.js +44 -12
- package/src/utils/build-manifest.js +35 -3
- package/src/utils/friction-detector.js +71 -22
- package/src/utils/health-checker.js +29 -2
- package/src/utils/transaction.js +111 -0
- package/src/utils/version-promote.js +16 -0
- package/standards-registry.json +61 -149
- package/bundled/ai/standards/agent-communication-protocol.ai.yaml +0 -42
- package/bundled/ai/standards/agent-dispatch.ai.yaml +0 -42
- package/bundled/ai/standards/branch-completion.ai.yaml +0 -44
- package/bundled/ai/standards/change-batching-standards.ai.yaml +0 -44
- package/bundled/ai/standards/execution-history.ai.yaml +0 -42
- package/bundled/ai/standards/pipeline-integration-standards.ai.yaml +0 -42
- package/bundled/ai/standards/workflow-enforcement.ai.yaml +0 -44
- package/bundled/ai/standards/workflow-state-protocol.ai.yaml +0 -43
- package/src/commands/flow.js +0 -260
- package/src/commands/start.js +0 -372
- package/src/commands/sweep.js +0 -151
- package/src/commands/workflow.js +0 -681
|
@@ -0,0 +1,2298 @@
|
|
|
1
|
+
---
|
|
2
|
+
source: skills/testing-guide/testing-theory.md
|
|
3
|
+
source_version: 1.0.0
|
|
4
|
+
translation_version: 1.0.0
|
|
5
|
+
status: current
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Testing Theory Knowledge Base
|
|
9
|
+
|
|
10
|
+
> **语言**: [English](../../../../skills/testing-guide/testing-theory.md) | 简体中文
|
|
11
|
+
|
|
12
|
+
**版本**: 1.0.0
|
|
13
|
+
**最后更新**: 2026-01-29
|
|
14
|
+
**适用性**: 供测试人员与开发者参考的教学资料
|
|
15
|
+
**范围**: universal
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Purpose
|
|
20
|
+
|
|
21
|
+
本文档提供软件测试的理论基础、教学内容与详细示例。它作为一份完整的知识库,用以支撑 [Testing Standards](../../core/testing-standards.md) 中所定义的可执行规则。
|
|
22
|
+
|
|
23
|
+
**参考标准**:
|
|
24
|
+
- [ISTQB CTFL v4.0](https://istqb.org/certifications/certified-tester-foundation-level-ctfl-v4-0/)
|
|
25
|
+
- [ISO/IEC/IEEE 29119](https://www.iso.org/standard/81291.html)
|
|
26
|
+
- [SWEBOK v4.0](https://www.computer.org/education/bodies-of-knowledge/software-engineering) - IEEE Computer Society
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Table of Contents
|
|
31
|
+
|
|
32
|
+
1. [测试基础概念](#testing-fundamentals)
|
|
33
|
+
2. [静态测试](#static-testing)
|
|
34
|
+
3. [测试设计技术](#test-design-techniques)
|
|
35
|
+
4. [经验导向测试](#experience-based-testing)
|
|
36
|
+
5. [风险导向测试](#risk-based-testing)
|
|
37
|
+
6. [专项测试](#specialized-testing)
|
|
38
|
+
7. [测试相关度量](#test-related-measures)
|
|
39
|
+
8. [缺陷管理](#defect-management)
|
|
40
|
+
9. [测试流程管理](#test-process-management)
|
|
41
|
+
10. [各测试层级的代码示例](#code-examples-by-test-level)
|
|
42
|
+
11. [快速参考卡](#quick-reference-card)
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Testing Fundamentals
|
|
47
|
+
|
|
48
|
+
理解软件测试的理论基础,对于有效的测试设计与执行至关重要。
|
|
49
|
+
|
|
50
|
+
**参考**: SWEBOK v4.0 第 5 章
|
|
51
|
+
|
|
52
|
+
### Faults vs Failures
|
|
53
|
+
|
|
54
|
+
```
|
|
55
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
56
|
+
│ Faults, Errors, and Failures │
|
|
57
|
+
├─────────────────────────────────────────────────────────────┤
|
|
58
|
+
│ │
|
|
59
|
+
│ Error (Mistake) │
|
|
60
|
+
│ │ │
|
|
61
|
+
│ ▼ introduced during development │
|
|
62
|
+
│ ┌─────────┐ │
|
|
63
|
+
│ │ Fault │ (Defect/Bug in the code) │
|
|
64
|
+
│ └─────────┘ │
|
|
65
|
+
│ │ │
|
|
66
|
+
│ ▼ when executed │
|
|
67
|
+
│ ┌─────────┐ │
|
|
68
|
+
│ │ Failure │ (Observable incorrect behavior) │
|
|
69
|
+
│ └─────────┘ │
|
|
70
|
+
│ │
|
|
71
|
+
│ Key Points: │
|
|
72
|
+
│ • A fault may exist without causing a failure │
|
|
73
|
+
│ • A failure requires a fault to be executed │
|
|
74
|
+
│ • Multiple faults can cause the same failure │
|
|
75
|
+
│ • Testing finds failures; debugging finds faults │
|
|
76
|
+
│ │
|
|
77
|
+
└─────────────────────────────────────────────────────────────┘
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### The Oracle Problem
|
|
81
|
+
|
|
82
|
+
**oracle(判定机制)** 是任何用以判断所观察到的程序行为是否正确的机制。oracle problem 探讨的是「如何得知预期结果」这一挑战。
|
|
83
|
+
|
|
84
|
+
```
|
|
85
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
86
|
+
│ Oracle Types │
|
|
87
|
+
├─────────────────────────────────────────────────────────────┤
|
|
88
|
+
│ │
|
|
89
|
+
│ Human Oracle │
|
|
90
|
+
│ └── Manual verification by domain expert │
|
|
91
|
+
│ │
|
|
92
|
+
│ Specification-Based Oracle │
|
|
93
|
+
│ └── Compare against formal requirements │
|
|
94
|
+
│ │
|
|
95
|
+
│ Reference Implementation │
|
|
96
|
+
│ └── Compare against known-correct implementation │
|
|
97
|
+
│ │
|
|
98
|
+
│ Metamorphic Oracle │
|
|
99
|
+
│ └── Verify relationships between outputs │
|
|
100
|
+
│ (e.g., sort(sort(x)) == sort(x)) │
|
|
101
|
+
│ │
|
|
102
|
+
│ Statistical Oracle │
|
|
103
|
+
│ └── Validate against expected distributions │
|
|
104
|
+
│ │
|
|
105
|
+
│ Implicit Oracle │
|
|
106
|
+
│ └── No crash, no exception, no timeout │
|
|
107
|
+
│ │
|
|
108
|
+
└─────────────────────────────────────────────────────────────┘
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
### Theoretical Limitations of Testing
|
|
112
|
+
|
|
113
|
+
> 「程序测试可以用来证明 bug 的存在,但永远无法证明其不存在。」 — Edsger W. Dijkstra
|
|
114
|
+
|
|
115
|
+
```
|
|
116
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
117
|
+
│ Testing Limitations │
|
|
118
|
+
├─────────────────────────────────────────────────────────────┤
|
|
119
|
+
│ │
|
|
120
|
+
│ Exhaustive Testing is Impossible │
|
|
121
|
+
│ ├── Infinite input domain │
|
|
122
|
+
│ ├── State space explosion │
|
|
123
|
+
│ └── Timing and concurrency variations │
|
|
124
|
+
│ │
|
|
125
|
+
│ Infeasible Paths │
|
|
126
|
+
│ ├── Some code paths cannot be executed by any input │
|
|
127
|
+
│ ├── Dead code may appear reachable in CFG │
|
|
128
|
+
│ └── 100% path coverage may be mathematically impossible │
|
|
129
|
+
│ │
|
|
130
|
+
│ Halting Problem │
|
|
131
|
+
│ └── Cannot algorithmically determine if all tests halt │
|
|
132
|
+
│ │
|
|
133
|
+
│ Implications: │
|
|
134
|
+
│ • Testing is sampling, not proof │
|
|
135
|
+
│ • Risk-based prioritization is essential │
|
|
136
|
+
│ • Combine testing with formal methods where critical │
|
|
137
|
+
│ │
|
|
138
|
+
└─────────────────────────────────────────────────────────────┘
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
### Testability
|
|
142
|
+
|
|
143
|
+
Testability(可测试性)衡量软件能多容易地被有效测试。
|
|
144
|
+
|
|
145
|
+
| Factor | 说明 | 如何改善 |
|
|
146
|
+
|--------|-------------|----------------|
|
|
147
|
+
| **Controllability** | 为测试设定系统状态的能力 | 提供测试钩子、依赖注入 |
|
|
148
|
+
| **Observability** | 观察测试结果的能力 | 加入日志、暴露内部状态 |
|
|
149
|
+
| **Isolability** | 独立测试各组件的能力 | 模块化设计、清晰的接口 |
|
|
150
|
+
| **Understandability** | 预期行为的清晰程度 | 良好的文档、清晰的规格 |
|
|
151
|
+
| **Stability** | 变动频率低 | 在测试阶段前冻结功能 |
|
|
152
|
+
| **Simplicity** | 复杂度低 | 降低 cyclomatic complexity |
|
|
153
|
+
|
|
154
|
+
### Test Adequacy Criteria
|
|
155
|
+
|
|
156
|
+
测试充分性准则回答的是:「我们测得够不够?」
|
|
157
|
+
|
|
158
|
+
```
|
|
159
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
160
|
+
│ Test Adequacy Criteria │
|
|
161
|
+
├─────────────────────────────────────────────────────────────┤
|
|
162
|
+
│ │
|
|
163
|
+
│ Coverage-Based Criteria │
|
|
164
|
+
│ ├── Statement Coverage: % of statements executed │
|
|
165
|
+
│ ├── Branch Coverage: % of branches taken │
|
|
166
|
+
│ ├── Condition Coverage: % of conditions evaluated │
|
|
167
|
+
│ └── MC/DC: Modified condition/decision coverage │
|
|
168
|
+
│ │
|
|
169
|
+
│ Fault-Based Criteria │
|
|
170
|
+
│ ├── Mutation Score: % of mutants killed │
|
|
171
|
+
│ └── Fault Seeding: % of seeded faults found │
|
|
172
|
+
│ │
|
|
173
|
+
│ Requirements-Based Criteria │
|
|
174
|
+
│ └── Requirements Coverage: % of requirements tested │
|
|
175
|
+
│ │
|
|
176
|
+
│ Risk-Based Criteria │
|
|
177
|
+
│ └── Risk Coverage: % of high-risk items tested │
|
|
178
|
+
│ │
|
|
179
|
+
│ Stopping Rules: │
|
|
180
|
+
│ • Achieved target coverage level │
|
|
181
|
+
│ • Defect discovery rate drops below threshold │
|
|
182
|
+
│ • Budget/time exhausted (with documented risk) │
|
|
183
|
+
│ │
|
|
184
|
+
└─────────────────────────────────────────────────────────────┘
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
## Static Testing
|
|
190
|
+
|
|
191
|
+
静态测试在不执行软件的情况下检查工作产物(代码、文档、需求)。它在 SDLC 较早阶段找出缺陷,从而补足动态测试。
|
|
192
|
+
|
|
193
|
+
**参考**: ISTQB CTFL v4.0 第 3 章, ISO/IEC/IEEE 29119-4
|
|
194
|
+
|
|
195
|
+
### Types of Static Testing
|
|
196
|
+
|
|
197
|
+
```
|
|
198
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
199
|
+
│ Static Testing Types │
|
|
200
|
+
├─────────────────────────────────────────────────────────────┤
|
|
201
|
+
│ Reviews (Manual) │
|
|
202
|
+
│ ├── Informal Review: Ad-hoc, no formal process │
|
|
203
|
+
│ ├── Walkthrough: Author-led, educational purpose │
|
|
204
|
+
│ ├── Technical Review: Peer-led, find defects │
|
|
205
|
+
│ └── Inspection: Formal, metrics-driven, most rigorous │
|
|
206
|
+
├─────────────────────────────────────────────────────────────┤
|
|
207
|
+
│ Static Analysis (Tool-based) │
|
|
208
|
+
│ ├── Code Analysis: Linters, complexity analyzers │
|
|
209
|
+
│ ├── Security Analysis: SAST tools (SonarQube, Checkmarx) │
|
|
210
|
+
│ ├── Architecture Analysis: Dependency checks │
|
|
211
|
+
│ └── Data Flow Analysis: Variable usage patterns │
|
|
212
|
+
└─────────────────────────────────────────────────────────────┘
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### Review Process (ISTQB)
|
|
216
|
+
|
|
217
|
+
| 阶段 | 活动 |
|
|
218
|
+
|-------|------------|
|
|
219
|
+
| Planning | 定义范围、进入准则、角色 |
|
|
220
|
+
| Initiate Review | 分发工作产物 |
|
|
221
|
+
| Individual Review | 每位评审者独立检视 |
|
|
222
|
+
| Issue Communication | 在会议中讨论发现 |
|
|
223
|
+
| Fixing & Reporting | 作者修正、收集度量 |
|
|
224
|
+
|
|
225
|
+
### Static Analysis Tools by Language
|
|
226
|
+
|
|
227
|
+
| 语言 | Linting | Security | Complexity |
|
|
228
|
+
|----------|---------|----------|------------|
|
|
229
|
+
| JavaScript/TypeScript | ESLint, Biome | npm audit, Snyk | ESLint complexity rules |
|
|
230
|
+
| Python | Pylint, Ruff, Flake8 | Bandit, Safety | Radon |
|
|
231
|
+
| Java | Checkstyle, PMD | SpotBugs, OWASP DC | JaCoCo |
|
|
232
|
+
| C# | StyleCop, Roslyn | Security Code Scan | NDepend |
|
|
233
|
+
| Go | golangci-lint | gosec | gocyclo |
|
|
234
|
+
|
|
235
|
+
### When to Use Static Testing
|
|
236
|
+
|
|
237
|
+
| 工作产物 | Review Type | 工具 |
|
|
238
|
+
|--------------|-------------|-------|
|
|
239
|
+
| Requirements | Inspection, Walkthrough | - |
|
|
240
|
+
| Design Documents | Technical Review | 架构工具 |
|
|
241
|
+
| Code | Technical Review, Tool Analysis | Linters, SAST |
|
|
242
|
+
| Test Plans | Walkthrough | - |
|
|
243
|
+
| User Documentation | Walkthrough | 拼写/语法检查器 |
|
|
244
|
+
|
|
245
|
+
### Code Review Checklist
|
|
246
|
+
|
|
247
|
+
```
|
|
248
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
249
|
+
│ Code Review Focus Areas │
|
|
250
|
+
├─────────────────────────────────────────────────────────────┤
|
|
251
|
+
│ Functionality │
|
|
252
|
+
│ ├── Does the code do what it's supposed to do? │
|
|
253
|
+
│ ├── Are edge cases handled? │
|
|
254
|
+
│ └── Is error handling appropriate? │
|
|
255
|
+
├─────────────────────────────────────────────────────────────┤
|
|
256
|
+
│ Maintainability │
|
|
257
|
+
│ ├── Is the code readable and well-organized? │
|
|
258
|
+
│ ├── Are names meaningful? │
|
|
259
|
+
│ └── Is complexity manageable? │
|
|
260
|
+
├─────────────────────────────────────────────────────────────┤
|
|
261
|
+
│ Security │
|
|
262
|
+
│ ├── Input validation present? │
|
|
263
|
+
│ ├── No hardcoded secrets? │
|
|
264
|
+
│ └── SQL injection / XSS prevention? │
|
|
265
|
+
├─────────────────────────────────────────────────────────────┤
|
|
266
|
+
│ Performance │
|
|
267
|
+
│ ├── No obvious inefficiencies? │
|
|
268
|
+
│ ├── Resource cleanup (connections, files)? │
|
|
269
|
+
│ └── Appropriate data structures? │
|
|
270
|
+
└─────────────────────────────────────────────────────────────┘
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
### Integration with CI/CD
|
|
274
|
+
|
|
275
|
+
```yaml
|
|
276
|
+
# Example: Static analysis in CI pipeline
|
|
277
|
+
static-analysis:
|
|
278
|
+
stage: test
|
|
279
|
+
script:
|
|
280
|
+
- npm run lint # Linting
|
|
281
|
+
- npm run lint:security # Security analysis
|
|
282
|
+
- npx sonarqube-scanner # Quality gates
|
|
283
|
+
rules:
|
|
284
|
+
- if: $CI_PIPELINE_SOURCE == "push"
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
---
|
|
288
|
+
|
|
289
|
+
## Test Design Techniques
|
|
290
|
+
|
|
291
|
+
测试设计技术是用以推导与选择测试用例的方法。ISTQB 与 ISO/IEC/IEEE 29119 将其分类为三大主要途径。
|
|
292
|
+
|
|
293
|
+
**参考**: ISTQB CTFL v4.0 第 4 章, ISO/IEC/IEEE 29119-4
|
|
294
|
+
|
|
295
|
+
### Technique Categories
|
|
296
|
+
|
|
297
|
+
```
|
|
298
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
299
|
+
│ Test Design Technique Categories │
|
|
300
|
+
├─────────────────────────────────────────────────────────────┤
|
|
301
|
+
│ Specification-Based (Black-Box) │
|
|
302
|
+
│ └── Derive tests from requirements/specifications │
|
|
303
|
+
├─────────────────────────────────────────────────────────────┤
|
|
304
|
+
│ Structure-Based (White-Box) │
|
|
305
|
+
│ └── Derive tests from internal code structure │
|
|
306
|
+
├─────────────────────────────────────────────────────────────┤
|
|
307
|
+
│ Experience-Based │
|
|
308
|
+
│ └── Derive tests from tester knowledge and intuition │
|
|
309
|
+
└─────────────────────────────────────────────────────────────┘
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
### Specification-Based Techniques (Black-Box)
|
|
313
|
+
|
|
314
|
+
#### 1. Equivalence Partitioning (EP)
|
|
315
|
+
|
|
316
|
+
将输入数据划分为多个分区,同一分区内的所有值应被系统以相同方式处理。
|
|
317
|
+
|
|
318
|
+
```
|
|
319
|
+
Input: Age (valid range: 18-65)
|
|
320
|
+
|
|
321
|
+
┌─────────────┬─────────────┬─────────────┐
|
|
322
|
+
│ Invalid │ Valid │ Invalid │
|
|
323
|
+
│ < 18 │ 18 - 65 │ > 65 │
|
|
324
|
+
├─────────────┼─────────────┼─────────────┤
|
|
325
|
+
│ Partition 1│ Partition 2│ Partition 3│
|
|
326
|
+
│ Test: 10 │ Test: 30 │ Test: 70 │
|
|
327
|
+
└─────────────┴─────────────┴─────────────┘
|
|
328
|
+
|
|
329
|
+
Coverage: At least 1 test case per partition
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
```python
|
|
333
|
+
# Python Example: Equivalence Partitioning
|
|
334
|
+
def test_age_validation():
|
|
335
|
+
validator = AgeValidator(min_age=18, max_age=65)
|
|
336
|
+
|
|
337
|
+
# Partition 1: Below minimum (invalid)
|
|
338
|
+
assert validator.is_valid(10) == False
|
|
339
|
+
|
|
340
|
+
# Partition 2: Valid range
|
|
341
|
+
assert validator.is_valid(30) == True
|
|
342
|
+
|
|
343
|
+
# Partition 3: Above maximum (invalid)
|
|
344
|
+
assert validator.is_valid(70) == False
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
#### 2. Boundary Value Analysis (BVA)
|
|
348
|
+
|
|
349
|
+
在 equivalence partitions 的边界处测试,因为缺陷最可能出现于此。
|
|
350
|
+
|
|
351
|
+
```
|
|
352
|
+
Input: Quantity (valid range: 1-100)
|
|
353
|
+
|
|
354
|
+
0 1 2 ... 99 100 101
|
|
355
|
+
│ │ │ │ │ │
|
|
356
|
+
▼ ▼ ▼ ▼ ▼ ▼
|
|
357
|
+
Invalid Valid Valid Valid Valid Invalid
|
|
358
|
+
(boundary)(boundary) (boundary)(boundary)
|
|
359
|
+
|
|
360
|
+
Test Values: 0, 1, 2, 99, 100, 101
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
```typescript
|
|
364
|
+
// TypeScript Example: Boundary Value Analysis
|
|
365
|
+
describe('Quantity Validation', () => {
|
|
366
|
+
const validator = new QuantityValidator(1, 100);
|
|
367
|
+
|
|
368
|
+
// Lower boundary
|
|
369
|
+
it('rejects 0 (below minimum)', () => {
|
|
370
|
+
expect(validator.isValid(0)).toBe(false);
|
|
371
|
+
});
|
|
372
|
+
|
|
373
|
+
it('accepts 1 (minimum boundary)', () => {
|
|
374
|
+
expect(validator.isValid(1)).toBe(true);
|
|
375
|
+
});
|
|
376
|
+
|
|
377
|
+
it('accepts 2 (above minimum)', () => {
|
|
378
|
+
expect(validator.isValid(2)).toBe(true);
|
|
379
|
+
});
|
|
380
|
+
|
|
381
|
+
// Upper boundary
|
|
382
|
+
it('accepts 99 (below maximum)', () => {
|
|
383
|
+
expect(validator.isValid(99)).toBe(true);
|
|
384
|
+
});
|
|
385
|
+
|
|
386
|
+
it('accepts 100 (maximum boundary)', () => {
|
|
387
|
+
expect(validator.isValid(100)).toBe(true);
|
|
388
|
+
});
|
|
389
|
+
|
|
390
|
+
it('rejects 101 (above maximum)', () => {
|
|
391
|
+
expect(validator.isValid(101)).toBe(false);
|
|
392
|
+
});
|
|
393
|
+
});
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
#### 3. Decision Table Testing
|
|
397
|
+
|
|
398
|
+
以多个条件来捕捉复杂的业务规则。
|
|
399
|
+
|
|
400
|
+
```
|
|
401
|
+
Discount Rules:
|
|
402
|
+
- Premium member: 20% discount
|
|
403
|
+
- Order > $100: 10% discount
|
|
404
|
+
- Both conditions: 25% discount (not cumulative)
|
|
405
|
+
|
|
406
|
+
┌─────────────────┬──────┬──────┬──────┬──────┐
|
|
407
|
+
│ Conditions │ TC1 │ TC2 │ TC3 │ TC4 │
|
|
408
|
+
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
409
|
+
│ Premium Member │ N │ Y │ N │ Y │
|
|
410
|
+
│ Order > $100 │ N │ N │ Y │ Y │
|
|
411
|
+
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
412
|
+
│ Actions │ │ │ │ │
|
|
413
|
+
├─────────────────┼──────┼──────┼──────┼──────┤
|
|
414
|
+
│ Discount % │ 0% │ 20% │ 10% │ 25% │
|
|
415
|
+
└─────────────────┴──────┴──────┴──────┴──────┘
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
```csharp
|
|
419
|
+
// C# Example: Decision Table Testing
|
|
420
|
+
[TestClass]
|
|
421
|
+
public class DiscountCalculatorTests
|
|
422
|
+
{
|
|
423
|
+
private DiscountCalculator _calculator;
|
|
424
|
+
|
|
425
|
+
[TestInitialize]
|
|
426
|
+
public void Setup() => _calculator = new DiscountCalculator();
|
|
427
|
+
|
|
428
|
+
[TestMethod]
|
|
429
|
+
public void Calculate_NotPremium_SmallOrder_NoDiscount()
|
|
430
|
+
{
|
|
431
|
+
var result = _calculator.Calculate(isPremium: false, orderAmount: 50);
|
|
432
|
+
Assert.AreEqual(0, result.DiscountPercent);
|
|
433
|
+
}
|
|
434
|
+
|
|
435
|
+
[TestMethod]
|
|
436
|
+
public void Calculate_Premium_SmallOrder_20PercentDiscount()
|
|
437
|
+
{
|
|
438
|
+
var result = _calculator.Calculate(isPremium: true, orderAmount: 50);
|
|
439
|
+
Assert.AreEqual(20, result.DiscountPercent);
|
|
440
|
+
}
|
|
441
|
+
|
|
442
|
+
[TestMethod]
|
|
443
|
+
public void Calculate_NotPremium_LargeOrder_10PercentDiscount()
|
|
444
|
+
{
|
|
445
|
+
var result = _calculator.Calculate(isPremium: false, orderAmount: 150);
|
|
446
|
+
Assert.AreEqual(10, result.DiscountPercent);
|
|
447
|
+
}
|
|
448
|
+
|
|
449
|
+
[TestMethod]
|
|
450
|
+
public void Calculate_Premium_LargeOrder_25PercentDiscount()
|
|
451
|
+
{
|
|
452
|
+
var result = _calculator.Calculate(isPremium: true, orderAmount: 150);
|
|
453
|
+
Assert.AreEqual(25, result.DiscountPercent);
|
|
454
|
+
}
|
|
455
|
+
}
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
#### 4. State Transition Testing
|
|
459
|
+
|
|
460
|
+
依系统状态与状态转移来测试行为变化。
|
|
461
|
+
|
|
462
|
+
```
|
|
463
|
+
Order State Machine:
|
|
464
|
+
|
|
465
|
+
┌─────────┐ place() ┌───────────┐ pay() ┌────────┐
|
|
466
|
+
│ Draft │──────────────▶│ Pending │───────────▶│ Paid │
|
|
467
|
+
└─────────┘ └───────────┘ └────────┘
|
|
468
|
+
│ │ │
|
|
469
|
+
│ cancel() │ cancel() │ ship()
|
|
470
|
+
▼ ▼ ▼
|
|
471
|
+
┌───────────┐ ┌───────────┐ ┌──────────┐
|
|
472
|
+
│ Cancelled │ │ Cancelled │ │ Shipped │
|
|
473
|
+
└───────────┘ └───────────┘ └──────────┘
|
|
474
|
+
|
|
475
|
+
State Transition Table:
|
|
476
|
+
┌───────────┬─────────┬───────────┬───────────┬───────────┐
|
|
477
|
+
│ Current │ place() │ pay() │ ship() │ cancel() │
|
|
478
|
+
├───────────┼─────────┼───────────┼───────────┼───────────┤
|
|
479
|
+
│ Draft │ Pending │ Invalid │ Invalid │ Cancelled │
|
|
480
|
+
│ Pending │ Invalid │ Paid │ Invalid │ Cancelled │
|
|
481
|
+
│ Paid │ Invalid │ Invalid │ Shipped │ Invalid │
|
|
482
|
+
│ Shipped │ Invalid │ Invalid │ Invalid │ Invalid │
|
|
483
|
+
│ Cancelled │ Invalid │ Invalid │ Invalid │ Invalid │
|
|
484
|
+
└───────────┴─────────┴───────────┴───────────┴───────────┘
|
|
485
|
+
```
|
|
486
|
+
|
|
487
|
+
```python
|
|
488
|
+
# Python Example: State Transition Testing
|
|
489
|
+
class TestOrderStateMachine:
|
|
490
|
+
def test_draft_to_pending_on_place(self):
|
|
491
|
+
order = Order(state="draft")
|
|
492
|
+
order.place()
|
|
493
|
+
assert order.state == "pending"
|
|
494
|
+
|
|
495
|
+
def test_pending_to_paid_on_pay(self):
|
|
496
|
+
order = Order(state="pending")
|
|
497
|
+
order.pay()
|
|
498
|
+
assert order.state == "paid"
|
|
499
|
+
|
|
500
|
+
def test_paid_to_shipped_on_ship(self):
|
|
501
|
+
order = Order(state="paid")
|
|
502
|
+
order.ship()
|
|
503
|
+
assert order.state == "shipped"
|
|
504
|
+
|
|
505
|
+
def test_invalid_transition_pay_from_draft(self):
|
|
506
|
+
order = Order(state="draft")
|
|
507
|
+
with pytest.raises(InvalidTransitionError):
|
|
508
|
+
order.pay()
|
|
509
|
+
|
|
510
|
+
def test_invalid_transition_cancel_from_shipped(self):
|
|
511
|
+
order = Order(state="shipped")
|
|
512
|
+
with pytest.raises(InvalidTransitionError):
|
|
513
|
+
order.cancel()
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
#### 5. Use Case Testing
|
|
517
|
+
|
|
518
|
+
测试从头到尾的完整用户场景。
|
|
519
|
+
|
|
520
|
+
```
|
|
521
|
+
Use Case: User Login
|
|
522
|
+
|
|
523
|
+
Primary Flow:
|
|
524
|
+
1. User enters username
|
|
525
|
+
2. User enters password
|
|
526
|
+
3. User clicks login
|
|
527
|
+
4. System validates credentials
|
|
528
|
+
5. System redirects to dashboard
|
|
529
|
+
|
|
530
|
+
Alternative Flows:
|
|
531
|
+
A1. Invalid credentials → Show error, remain on login
|
|
532
|
+
A2. Account locked → Show locked message
|
|
533
|
+
A3. Password expired → Redirect to password change
|
|
534
|
+
|
|
535
|
+
Test Cases:
|
|
536
|
+
- TC1: Valid credentials → Dashboard (Primary)
|
|
537
|
+
- TC2: Invalid password → Error message (A1)
|
|
538
|
+
- TC3: Invalid username → Error message (A1)
|
|
539
|
+
- TC4: Locked account → Locked message (A2)
|
|
540
|
+
- TC5: Expired password → Password change page (A3)
|
|
541
|
+
```
|
|
542
|
+
|
|
543
|
+
#### 6. Pairwise Testing (Combinatorial Testing)
|
|
544
|
+
|
|
545
|
+
测试所有可能的输入参数值两两配对,在维持良好覆盖率的同时减少测试用例数。
|
|
546
|
+
|
|
547
|
+
**参考**: SWEBOK v4.0 - Input Domain-Based Techniques
|
|
548
|
+
|
|
549
|
+
```
|
|
550
|
+
Example: Login Form with 3 parameters, each with 3 values
|
|
551
|
+
|
|
552
|
+
Parameters:
|
|
553
|
+
- Browser: Chrome, Firefox, Safari
|
|
554
|
+
- OS: Windows, macOS, Linux
|
|
555
|
+
- Language: English, Spanish, French
|
|
556
|
+
|
|
557
|
+
Full Combinatorial: 3 × 3 × 3 = 27 test cases
|
|
558
|
+
Pairwise Coverage: 9 test cases (covers all pairs)
|
|
559
|
+
|
|
560
|
+
┌──────┬──────────┬─────────┬──────────┐
|
|
561
|
+
│ TC │ Browser │ OS │ Language │
|
|
562
|
+
├──────┼──────────┼─────────┼──────────┤
|
|
563
|
+
│ 1 │ Chrome │ Windows │ English │
|
|
564
|
+
│ 2 │ Chrome │ macOS │ Spanish │
|
|
565
|
+
│ 3 │ Chrome │ Linux │ French │
|
|
566
|
+
│ 4 │ Firefox │ Windows │ Spanish │
|
|
567
|
+
│ 5 │ Firefox │ macOS │ French │
|
|
568
|
+
│ 6 │ Firefox │ Linux │ English │
|
|
569
|
+
│ 7 │ Safari │ Windows │ French │
|
|
570
|
+
│ 8 │ Safari │ macOS │ English │
|
|
571
|
+
│ 9 │ Safari │ Linux │ Spanish │
|
|
572
|
+
└──────┴──────────┴─────────┴──────────┘
|
|
573
|
+
```
|
|
574
|
+
|
|
575
|
+
**工具**: PICT (Microsoft), AllPairs, Jenny
|
|
576
|
+
|
|
577
|
+
```bash
|
|
578
|
+
# Using PICT (Pairwise Independent Combinatorial Testing)
|
|
579
|
+
# Input file: params.txt
|
|
580
|
+
# Browser: Chrome, Firefox, Safari
|
|
581
|
+
# OS: Windows, macOS, Linux
|
|
582
|
+
# Language: English, Spanish, French
|
|
583
|
+
|
|
584
|
+
pict params.txt > test_cases.txt
|
|
585
|
+
```
|
|
586
|
+
|
|
587
|
+
#### 7. Data Flow Testing
|
|
588
|
+
|
|
589
|
+
借由追踪变量的定义(definition)与使用(use),测试数据在程序中的流动。
|
|
590
|
+
|
|
591
|
+
**参考**: SWEBOK v4.0 - Code-Based Techniques
|
|
592
|
+
|
|
593
|
+
```
|
|
594
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
595
|
+
│ Data Flow Terminology │
|
|
596
|
+
├─────────────────────────────────────────────────────────────┤
|
|
597
|
+
│ │
|
|
598
|
+
│ Definition (def): Variable is assigned a value │
|
|
599
|
+
│ Use (use): Variable's value is accessed │
|
|
600
|
+
│ - c-use: Computational use (in calculation) │
|
|
601
|
+
│ - p-use: Predicate use (in condition) │
|
|
602
|
+
│ │
|
|
603
|
+
│ def-use pair: Path from definition to use │
|
|
604
|
+
│ def-clear path: No redefinition between def and use │
|
|
605
|
+
│ │
|
|
606
|
+
│ Example: │
|
|
607
|
+
│ 1: x = 5 ← def(x) │
|
|
608
|
+
│ 2: y = x + 1 ← c-use(x), def(y) │
|
|
609
|
+
│ 3: if (x > 0) ← p-use(x) │
|
|
610
|
+
│ 4: z = y * 2 ← c-use(y), def(z) │
|
|
611
|
+
│ │
|
|
612
|
+
└─────────────────────────────────────────────────────────────┘
|
|
613
|
+
```
|
|
614
|
+
|
|
615
|
+
**覆盖准则**:
|
|
616
|
+
|
|
617
|
+
| 准则 | 要求 | 强度 |
|
|
618
|
+
|-----------|-------------|----------|
|
|
619
|
+
| All-Defs | 每个 def 至少抵达一个 use | 弱 |
|
|
620
|
+
| All-Uses | 每个 def-use pair 都被执行到 | 中 |
|
|
621
|
+
| All-DU-Paths | 从 def 到 use 的所有 def-clear path | 强 |
|
|
622
|
+
|
|
623
|
+
```python
|
|
624
|
+
# Python Example: Data Flow Testing
|
|
625
|
+
def calculate_discount(price, quantity, is_member):
|
|
626
|
+
# def: total
|
|
627
|
+
total = price * quantity
|
|
628
|
+
|
|
629
|
+
# def: discount
|
|
630
|
+
if is_member: # p-use: is_member
|
|
631
|
+
discount = 0.1 # def: discount (path 1)
|
|
632
|
+
else:
|
|
633
|
+
discount = 0.0 # def: discount (path 2)
|
|
634
|
+
|
|
635
|
+
# c-use: total, discount
|
|
636
|
+
if total > 100: # p-use: total
|
|
637
|
+
discount += 0.05 # c-use: discount, def: discount
|
|
638
|
+
|
|
639
|
+
# c-use: total, discount
|
|
640
|
+
return total * (1 - discount)
|
|
641
|
+
|
|
642
|
+
# Test cases for All-Uses coverage:
|
|
643
|
+
# TC1: is_member=True, total>100 → exercises def@line4, use@line10
|
|
644
|
+
# TC2: is_member=False, total>100 → exercises def@line6, use@line10
|
|
645
|
+
# TC3: is_member=True, total<=100 → exercises def@line4, use@line12
|
|
646
|
+
# TC4: is_member=False, total<=100 → exercises def@line6, use@line12
|
|
647
|
+
```
|
|
648
|
+
|
|
649
|
+
### Structure-Based Techniques (White-Box)
|
|
650
|
+
|
|
651
|
+
#### 1. Statement Coverage
|
|
652
|
+
|
|
653
|
+
确保每一条语句至少被执行一次。
|
|
654
|
+
|
|
655
|
+
```
|
|
656
|
+
Code:
|
|
657
|
+
1 function categorize(score) {
|
|
658
|
+
2 let result;
|
|
659
|
+
3 if (score >= 90) {
|
|
660
|
+
4 result = 'A';
|
|
661
|
+
5 } else if (score >= 70) {
|
|
662
|
+
6 result = 'B';
|
|
663
|
+
7 } else {
|
|
664
|
+
8 result = 'C';
|
|
665
|
+
9 }
|
|
666
|
+
10 return result;
|
|
667
|
+
11 }
|
|
668
|
+
|
|
669
|
+
100% Statement Coverage requires tests:
|
|
670
|
+
- Test 1: score = 95 → executes lines 1-4, 10
|
|
671
|
+
- Test 2: score = 80 → executes lines 1-3, 5-6, 10
|
|
672
|
+
- Test 3: score = 50 → executes lines 1-3, 5, 7-10
|
|
673
|
+
```
|
|
674
|
+
|
|
675
|
+
#### 2. Branch Coverage
|
|
676
|
+
|
|
677
|
+
确保每一个分支(决策结果)都被执行到。
|
|
678
|
+
|
|
679
|
+
```
|
|
680
|
+
Code:
|
|
681
|
+
if (a > 0 && b > 0) {
|
|
682
|
+
doSomething();
|
|
683
|
+
}
|
|
684
|
+
|
|
685
|
+
Branches:
|
|
686
|
+
- True branch: a > 0 AND b > 0 is true
|
|
687
|
+
- False branch: a > 0 AND b > 0 is false
|
|
688
|
+
|
|
689
|
+
100% Branch Coverage:
|
|
690
|
+
- Test 1: a = 1, b = 1 → True branch
|
|
691
|
+
- Test 2: a = 0, b = 1 → False branch (or a = 1, b = 0)
|
|
692
|
+
```
|
|
693
|
+
|
|
694
|
+
#### 3. Condition Coverage
|
|
695
|
+
|
|
696
|
+
确保决策中的每个条件都被评估为 true 与 false。
|
|
697
|
+
|
|
698
|
+
```
|
|
699
|
+
Code:
|
|
700
|
+
if (a > 0 && b > 0) { ... }
|
|
701
|
+
|
|
702
|
+
Conditions: (a > 0), (b > 0)
|
|
703
|
+
|
|
704
|
+
100% Condition Coverage:
|
|
705
|
+
- Test 1: a = 1, b = 1 → (a > 0) = true, (b > 0) = true
|
|
706
|
+
- Test 2: a = 0, b = 0 → (a > 0) = false, (b > 0) = false
|
|
707
|
+
|
|
708
|
+
Note: Does not guarantee branch coverage!
|
|
709
|
+
```
|
|
710
|
+
|
|
711
|
+
#### 4. MC/DC (Modified Condition/Decision Coverage)
|
|
712
|
+
|
|
713
|
+
每个条件都能独立影响决策结果。安全关键系统会要求此覆盖。
|
|
714
|
+
|
|
715
|
+
```
|
|
716
|
+
Code:
|
|
717
|
+
if (a && b) { ... }
|
|
718
|
+
|
|
719
|
+
MC/DC Requirements:
|
|
720
|
+
1. Each condition evaluated to true and false
|
|
721
|
+
2. Each condition independently affects outcome
|
|
722
|
+
|
|
723
|
+
Test Cases:
|
|
724
|
+
- Test 1: a = true, b = true → Decision = true
|
|
725
|
+
- Test 2: a = false, b = true → Decision = false (a changed outcome)
|
|
726
|
+
- Test 3: a = true, b = false → Decision = false (b changed outcome)
|
|
727
|
+
```
|
|
728
|
+
|
|
729
|
+
### Technique Selection Guide
|
|
730
|
+
|
|
731
|
+
| 技术 | 最适合 | 测试层级 |
|
|
732
|
+
|-----------|----------|------------|
|
|
733
|
+
| Equivalence Partitioning | 输入验证、范围 | UT, IT |
|
|
734
|
+
| Boundary Value Analysis | 数值上下限、日期 | UT, IT |
|
|
735
|
+
| Decision Table | 复杂业务规则 | UT, IT, ST |
|
|
736
|
+
| State Transition | 工作流程、状态变化 | IT, ST |
|
|
737
|
+
| Use Case Testing | 用户场景 | ST, E2E |
|
|
738
|
+
| Pairwise Testing | 多参数组合 | IT, ST |
|
|
739
|
+
| Data Flow Testing | 变量生命周期验证 | UT |
|
|
740
|
+
| Statement Coverage | 基本代码覆盖 | UT |
|
|
741
|
+
| Branch Coverage | 决策逻辑 | UT |
|
|
742
|
+
| MC/DC | 安全关键系统 | UT |
|
|
743
|
+
|
|
744
|
+
---
|
|
745
|
+
|
|
746
|
+
## Experience-Based Testing
|
|
747
|
+
|
|
748
|
+
经验导向测试运用测试人员的知识、技能与直觉来设计与执行测试。它补足了系统化技术。
|
|
749
|
+
|
|
750
|
+
**参考**: ISTQB CTFL v4.0 第 4.4 节
|
|
751
|
+
|
|
752
|
+
### Types of Experience-Based Testing
|
|
753
|
+
|
|
754
|
+
#### 1. Exploratory Testing
|
|
755
|
+
|
|
756
|
+
同时进行测试设计、执行与学习。测试人员在没有预先脚本的情况下探索系统。
|
|
757
|
+
|
|
758
|
+
```
|
|
759
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
760
|
+
│ Exploratory Testing Session │
|
|
761
|
+
├─────────────────────────────────────────────────────────────┤
|
|
762
|
+
│ Charter: "Explore the checkout process focusing on │
|
|
763
|
+
│ payment failure scenarios" │
|
|
764
|
+
├─────────────────────────────────────────────────────────────┤
|
|
765
|
+
│ Time-box: 60 minutes │
|
|
766
|
+
├─────────────────────────────────────────────────────────────┤
|
|
767
|
+
│ Notes: │
|
|
768
|
+
│ - Tested expired credit card → Got generic error │
|
|
769
|
+
│ - Tested insufficient funds → Same generic error │
|
|
770
|
+
│ - Found: No specific error messages for different failures │
|
|
771
|
+
│ - Bug: Retry with same card doesn't clear previous error │
|
|
772
|
+
├─────────────────────────────────────────────────────────────┤
|
|
773
|
+
│ Bugs Found: 2 │
|
|
774
|
+
│ Areas for Further Testing: Error message specificity │
|
|
775
|
+
└─────────────────────────────────────────────────────────────┘
|
|
776
|
+
```
|
|
777
|
+
|
|
778
|
+
**Session-Based Test Management (SBTM)**:
|
|
779
|
+
|
|
780
|
+
| 要素 | 说明 |
|
|
781
|
+
|---------|-------------|
|
|
782
|
+
| Charter | 该 session 的任务宣言 |
|
|
783
|
+
| Time-box | 固定时段(通常 60-90 分钟) |
|
|
784
|
+
| Session Notes | 观察、疑问、找到的 bug |
|
|
785
|
+
| Debrief | 与团队一同检视发现 |
|
|
786
|
+
|
|
787
|
+
#### 2. Error Guessing
|
|
788
|
+
|
|
789
|
+
依据对类似系统或常见错误的经验来预期缺陷。
|
|
790
|
+
|
|
791
|
+
```
|
|
792
|
+
Common Error Categories to Guess:
|
|
793
|
+
|
|
794
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
795
|
+
│ Input Errors │
|
|
796
|
+
│ ├── Empty/null inputs │
|
|
797
|
+
│ ├── Very long strings │
|
|
798
|
+
│ ├── Special characters (', ", <, >, &, etc.) │
|
|
799
|
+
│ ├── Unicode/emoji characters │
|
|
800
|
+
│ └── Negative numbers where positive expected │
|
|
801
|
+
├─────────────────────────────────────────────────────────────┤
|
|
802
|
+
│ Boundary Errors │
|
|
803
|
+
│ ├── Off-by-one errors │
|
|
804
|
+
│ ├── Integer overflow/underflow │
|
|
805
|
+
│ ├── Date boundaries (leap years, month ends) │
|
|
806
|
+
│ └── Array index out of bounds │
|
|
807
|
+
├─────────────────────────────────────────────────────────────┤
|
|
808
|
+
│ State Errors │
|
|
809
|
+
│ ├── Operations on deleted records │
|
|
810
|
+
│ ├── Concurrent modifications │
|
|
811
|
+
│ ├── Session timeout during operation │
|
|
812
|
+
│ └── Browser back button after submission │
|
|
813
|
+
├─────────────────────────────────────────────────────────────┤
|
|
814
|
+
│ Environment Errors │
|
|
815
|
+
│ ├── Network timeout/disconnection │
|
|
816
|
+
│ ├── Low disk space │
|
|
817
|
+
│ ├── Database connection pool exhaustion │
|
|
818
|
+
│ └── Time zone differences │
|
|
819
|
+
└─────────────────────────────────────────────────────────────┘
|
|
820
|
+
```
|
|
821
|
+
|
|
822
|
+
#### 3. Checklist-Based Testing
|
|
823
|
+
|
|
824
|
+
使用依经验与标准建立的预先检查清单。
|
|
825
|
+
|
|
826
|
+
```markdown
|
|
827
|
+
# API Endpoint Testing Checklist
|
|
828
|
+
|
|
829
|
+
## Authentication & Authorization
|
|
830
|
+
- [ ] Endpoint rejects unauthenticated requests
|
|
831
|
+
- [ ] Endpoint rejects expired tokens
|
|
832
|
+
- [ ] Endpoint enforces role-based access
|
|
833
|
+
- [ ] Different user roles see appropriate data
|
|
834
|
+
|
|
835
|
+
## Input Validation
|
|
836
|
+
- [ ] Required fields are validated
|
|
837
|
+
- [ ] Data types are enforced
|
|
838
|
+
- [ ] String length limits enforced
|
|
839
|
+
- [ ] Malicious input sanitized (XSS, SQL injection)
|
|
840
|
+
|
|
841
|
+
## Response Handling
|
|
842
|
+
- [ ] Success responses have correct status codes
|
|
843
|
+
- [ ] Error responses have meaningful messages
|
|
844
|
+
- [ ] Response format matches API documentation
|
|
845
|
+
- [ ] Pagination works correctly
|
|
846
|
+
|
|
847
|
+
## Performance
|
|
848
|
+
- [ ] Response time under load is acceptable
|
|
849
|
+
- [ ] No N+1 query problems
|
|
850
|
+
- [ ] Large datasets handled gracefully
|
|
851
|
+
```
|
|
852
|
+
|
|
853
|
+
### When to Use Experience-Based Testing
|
|
854
|
+
|
|
855
|
+
| 场景 | 建议技术 |
|
|
856
|
+
|----------|----------------------|
|
|
857
|
+
| 新功能探索 | Exploratory Testing |
|
|
858
|
+
| 无文档的遗留系统 | Exploratory Testing |
|
|
859
|
+
| 已知的问题区域 | Error Guessing |
|
|
860
|
+
| 回归测试 | Checklist-Based |
|
|
861
|
+
| 时间压力/快速验证 | Error Guessing |
|
|
862
|
+
| 合规性验证 | Checklist-Based |
|
|
863
|
+
|
|
864
|
+
### Combining with Systematic Techniques
|
|
865
|
+
|
|
866
|
+
```
|
|
867
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
868
|
+
│ Optimal Testing Approach │
|
|
869
|
+
├─────────────────────────────────────────────────────────────┤
|
|
870
|
+
│ │
|
|
871
|
+
│ Systematic Techniques Experience-Based │
|
|
872
|
+
│ (70-80% of effort) (20-30% of effort) │
|
|
873
|
+
│ │
|
|
874
|
+
│ ┌──────────────────┐ ┌──────────────────┐ │
|
|
875
|
+
│ │ Equivalence Part.│ │ Exploratory │ │
|
|
876
|
+
│ │ Boundary Value │ + │ Error Guessing │ │
|
|
877
|
+
│ │ Decision Tables │ │ Checklists │ │
|
|
878
|
+
│ └──────────────────┘ └──────────────────┘ │
|
|
879
|
+
│ │
|
|
880
|
+
│ Provides: Provides: │
|
|
881
|
+
│ - Coverage - Edge cases │
|
|
882
|
+
│ - Repeatability - Real-world scenarios │
|
|
883
|
+
│ - Documentation - Quick feedback │
|
|
884
|
+
│ │
|
|
885
|
+
└─────────────────────────────────────────────────────────────┘
|
|
886
|
+
```
|
|
887
|
+
|
|
888
|
+
---
|
|
889
|
+
|
|
890
|
+
## Risk-Based Testing
|
|
891
|
+
|
|
892
|
+
风险导向测试依据潜在失败的可能性与影响,来排定测试工作的优先顺序。
|
|
893
|
+
|
|
894
|
+
**参考**: ISTQB CTFL v4.0 第 5.2 节
|
|
895
|
+
|
|
896
|
+
### Risk Assessment
|
|
897
|
+
|
|
898
|
+
#### Risk Formula
|
|
899
|
+
|
|
900
|
+
```
|
|
901
|
+
Risk Level = Likelihood × Impact
|
|
902
|
+
|
|
903
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
904
|
+
│ Risk Matrix │
|
|
905
|
+
├─────────────┬───────────────────────────────────────────────┤
|
|
906
|
+
│ │ Impact │
|
|
907
|
+
│ Likelihood │ Low (1) Medium (2) High (3) │
|
|
908
|
+
├─────────────┼───────────────────────────────────────────────┤
|
|
909
|
+
│ High (3) │ Medium(3) High(6) Critical(9) │
|
|
910
|
+
│ Medium (2) │ Low(2) Medium(4) High(6) │
|
|
911
|
+
│ Low (1) │ Low(1) Low(2) Medium(3) │
|
|
912
|
+
└─────────────┴───────────────────────────────────────────────┘
|
|
913
|
+
```
|
|
914
|
+
|
|
915
|
+
#### Risk Factors
|
|
916
|
+
|
|
917
|
+
**Likelihood Factors(技术风险)**:
|
|
918
|
+
- 功能的复杂度
|
|
919
|
+
- 新/不熟悉的技术
|
|
920
|
+
- 代码质量度量
|
|
921
|
+
- 开发者经验
|
|
922
|
+
- 变更频率
|
|
923
|
+
- 集成复杂度
|
|
924
|
+
|
|
925
|
+
**Impact Factors(业务风险)**:
|
|
926
|
+
- 营收影响
|
|
927
|
+
- 受影响的用户规模
|
|
928
|
+
- 法规合规
|
|
929
|
+
- 品牌声誉
|
|
930
|
+
- 数据安全/隐私
|
|
931
|
+
- 运维关键性
|
|
932
|
+
|
|
933
|
+
### Risk-Based Test Prioritization
|
|
934
|
+
|
|
935
|
+
```
|
|
936
|
+
Example: E-commerce Application
|
|
937
|
+
|
|
938
|
+
┌──────────────────────┬─────────┬─────────┬───────┬──────────┐
|
|
939
|
+
│ Feature │Likelihood│ Impact │ Risk │ Priority │
|
|
940
|
+
├──────────────────────┼─────────┼─────────┼───────┼──────────┤
|
|
941
|
+
│ Payment Processing │ 2 │ 3 │ 6 │ 1 │
|
|
942
|
+
│ User Authentication │ 2 │ 3 │ 6 │ 2 │
|
|
943
|
+
│ Shopping Cart │ 2 │ 2 │ 4 │ 3 │
|
|
944
|
+
│ Product Search │ 1 │ 2 │ 2 │ 4 │
|
|
945
|
+
│ Product Reviews │ 1 │ 1 │ 1 │ 5 │
|
|
946
|
+
│ Wishlist │ 1 │ 1 │ 1 │ 6 │
|
|
947
|
+
└──────────────────────┴─────────┴─────────┴───────┴──────────┘
|
|
948
|
+
|
|
949
|
+
Test Effort Allocation:
|
|
950
|
+
- Critical (Risk 6+): 50% of test effort, most thorough coverage
|
|
951
|
+
- Medium (Risk 3-5): 30% of test effort, standard coverage
|
|
952
|
+
- Low (Risk 1-2): 20% of test effort, basic coverage
|
|
953
|
+
```
|
|
954
|
+
|
|
955
|
+
### Risk-Based Test Planning
|
|
956
|
+
|
|
957
|
+
```markdown
|
|
958
|
+
# Risk-Based Test Plan Template
|
|
959
|
+
|
|
960
|
+
## 1. Risk Identification
|
|
961
|
+
List all features/components and their risk factors.
|
|
962
|
+
|
|
963
|
+
## 2. Risk Analysis
|
|
964
|
+
Calculate risk scores using Likelihood × Impact.
|
|
965
|
+
|
|
966
|
+
## 3. Test Prioritization
|
|
967
|
+
| Priority | Features | Test Depth |
|
|
968
|
+
|----------|----------|------------|
|
|
969
|
+
| P1 | Payment, Auth | Full coverage, all techniques |
|
|
970
|
+
| P2 | Cart, Checkout | Standard coverage |
|
|
971
|
+
| P3 | Search, Browse | Basic happy path |
|
|
972
|
+
| P4 | Reviews, Wishlist | Minimal, smoke tests |
|
|
973
|
+
|
|
974
|
+
## 4. Risk Mitigation
|
|
975
|
+
- P1 features: 100% automated tests, manual exploratory
|
|
976
|
+
- P2 features: 80% automated, sample manual tests
|
|
977
|
+
- P3 features: Key scenarios automated
|
|
978
|
+
- P4 features: Basic smoke tests only
|
|
979
|
+
|
|
980
|
+
## 5. Residual Risk Acceptance
|
|
981
|
+
Document accepted risks for low-priority features.
|
|
982
|
+
```
|
|
983
|
+
|
|
984
|
+
### Continuous Risk Assessment
|
|
985
|
+
|
|
986
|
+
```
|
|
987
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
988
|
+
│ Continuous Risk Re-evaluation │
|
|
989
|
+
├─────────────────────────────────────────────────────────────┤
|
|
990
|
+
│ │
|
|
991
|
+
│ Sprint Start ──▶ During Sprint ──▶ Sprint End │
|
|
992
|
+
│ │ │ │ │
|
|
993
|
+
│ ▼ ▼ ▼ │
|
|
994
|
+
│ Initial Risk Update based on: Review: │
|
|
995
|
+
│ Assessment - Defects found - Actual vs Expected │
|
|
996
|
+
│ - Code changes - Adjust for next │
|
|
997
|
+
│ - New requirements sprint │
|
|
998
|
+
│ │
|
|
999
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1000
|
+
```
|
|
1001
|
+
|
|
1002
|
+
---
|
|
1003
|
+
|
|
1004
|
+
## Specialized Testing
|
|
1005
|
+
|
|
1006
|
+
现代软件开发需要超越传统分类的专项测试类型。
|
|
1007
|
+
|
|
1008
|
+
### Contract Testing
|
|
1009
|
+
|
|
1010
|
+
验证各服务是否依照约定的合约正确通讯。对微服务架构而言至关重要。
|
|
1011
|
+
|
|
1012
|
+
```
|
|
1013
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1014
|
+
│ Consumer-Driven Contract Testing │
|
|
1015
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1016
|
+
│ │
|
|
1017
|
+
│ Consumer Provider │
|
|
1018
|
+
│ (Frontend) (API) │
|
|
1019
|
+
│ │ │ │
|
|
1020
|
+
│ │ 1. Define expectations │ │
|
|
1021
|
+
│ │─────────────────────────────▶ │ │
|
|
1022
|
+
│ │ │ │
|
|
1023
|
+
│ │ 2. Generate contract │ │
|
|
1024
|
+
│ │ (Pact file) │ │
|
|
1025
|
+
│ │ │ │
|
|
1026
|
+
│ │ 3. Share contract │ │
|
|
1027
|
+
│ │─────────────────────────────▶ │ │
|
|
1028
|
+
│ │ │ │
|
|
1029
|
+
│ │ 4. Provider verifies │
|
|
1030
|
+
│ │ against contract │
|
|
1031
|
+
│ │ │ │
|
|
1032
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1033
|
+
```
|
|
1034
|
+
|
|
1035
|
+
**工具**: Pact, Spring Cloud Contract, Postman
|
|
1036
|
+
|
|
1037
|
+
```typescript
|
|
1038
|
+
// Consumer Test (Pact.js Example)
|
|
1039
|
+
describe('User API Contract', () => {
|
|
1040
|
+
it('returns user by ID', async () => {
|
|
1041
|
+
await provider.addInteraction({
|
|
1042
|
+
state: 'user with ID 123 exists',
|
|
1043
|
+
uponReceiving: 'a request for user 123',
|
|
1044
|
+
withRequest: {
|
|
1045
|
+
method: 'GET',
|
|
1046
|
+
path: '/users/123'
|
|
1047
|
+
},
|
|
1048
|
+
willRespondWith: {
|
|
1049
|
+
status: 200,
|
|
1050
|
+
body: {
|
|
1051
|
+
id: '123',
|
|
1052
|
+
name: Matchers.string('John Doe'),
|
|
1053
|
+
email: Matchers.email()
|
|
1054
|
+
}
|
|
1055
|
+
}
|
|
1056
|
+
});
|
|
1057
|
+
|
|
1058
|
+
const user = await userClient.getUser('123');
|
|
1059
|
+
expect(user.id).toBe('123');
|
|
1060
|
+
});
|
|
1061
|
+
});
|
|
1062
|
+
```
|
|
1063
|
+
|
|
1064
|
+
### Mutation Testing
|
|
1065
|
+
|
|
1066
|
+
借由对代码引入微小变更(mutants),并检查测试是否能侦测到,以评估测试套件的质量。
|
|
1067
|
+
|
|
1068
|
+
```
|
|
1069
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1070
|
+
│ Mutation Testing │
|
|
1071
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1072
|
+
│ │
|
|
1073
|
+
│ Original Code: Mutant (change operator): │
|
|
1074
|
+
│ if (a > b) if (a >= b) ← Boundary mutant │
|
|
1075
|
+
│ if (a > b) if (a < b) ← Negation mutant │
|
|
1076
|
+
│ return a + b; return a - b; ← Arithmetic mutant │
|
|
1077
|
+
│ return true; return false; ← Return value mutant │
|
|
1078
|
+
│ │
|
|
1079
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1080
|
+
│ Mutation Score = (Killed Mutants / Total Mutants) × 100 │
|
|
1081
|
+
│ │
|
|
1082
|
+
│ - Killed: Test failed → Good, test detected the change │
|
|
1083
|
+
│ - Survived: Test passed → Bad, test missed the defect │
|
|
1084
|
+
│ │
|
|
1085
|
+
│ Target: > 80% mutation score for critical code │
|
|
1086
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1087
|
+
```
|
|
1088
|
+
|
|
1089
|
+
**工具**: Stryker (JS/TS), PITest (Java), mutmut (Python)
|
|
1090
|
+
|
|
1091
|
+
```bash
|
|
1092
|
+
# Run mutation testing with Stryker
|
|
1093
|
+
npx stryker run
|
|
1094
|
+
|
|
1095
|
+
# Example output:
|
|
1096
|
+
# Mutation score: 85.7%
|
|
1097
|
+
# Killed: 180 Survived: 30 No coverage: 10
|
|
1098
|
+
```
|
|
1099
|
+
|
|
1100
|
+
### Chaos Engineering
|
|
1101
|
+
|
|
1102
|
+
借由在生产或预备环境中注入故障,主动测试系统的韧性。
|
|
1103
|
+
|
|
1104
|
+
```
|
|
1105
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1106
|
+
│ Chaos Engineering Principles │
|
|
1107
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1108
|
+
│ │
|
|
1109
|
+
│ 1. Define "steady state" (normal behavior metrics) │
|
|
1110
|
+
│ 2. Hypothesize steady state continues during chaos │
|
|
1111
|
+
│ 3. Introduce real-world events: │
|
|
1112
|
+
│ - Server failures │
|
|
1113
|
+
│ - Network latency/partition │
|
|
1114
|
+
│ - Resource exhaustion │
|
|
1115
|
+
│ - Clock skew │
|
|
1116
|
+
│ 4. Try to disprove the hypothesis │
|
|
1117
|
+
│ 5. Minimize blast radius (start small) │
|
|
1118
|
+
│ │
|
|
1119
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1120
|
+
```
|
|
1121
|
+
|
|
1122
|
+
**工具**: Chaos Monkey, Gremlin, Litmus, Chaos Toolkit
|
|
1123
|
+
|
|
1124
|
+
```yaml
|
|
1125
|
+
# Chaos Toolkit Experiment Example
|
|
1126
|
+
title: "Service resilience when database is slow"
|
|
1127
|
+
description: "Verify the service degrades gracefully"
|
|
1128
|
+
|
|
1129
|
+
steady-state-hypothesis:
|
|
1130
|
+
title: "Service responds within SLA"
|
|
1131
|
+
probes:
|
|
1132
|
+
- name: "service-responds"
|
|
1133
|
+
type: probe
|
|
1134
|
+
provider:
|
|
1135
|
+
type: http
|
|
1136
|
+
url: "http://service/health"
|
|
1137
|
+
timeout: 3
|
|
1138
|
+
|
|
1139
|
+
method:
|
|
1140
|
+
- name: "inject-db-latency"
|
|
1141
|
+
type: action
|
|
1142
|
+
provider:
|
|
1143
|
+
type: process
|
|
1144
|
+
path: "tc"
|
|
1145
|
+
arguments: "qdisc add dev eth0 root netem delay 500ms"
|
|
1146
|
+
pauses:
|
|
1147
|
+
after: 30
|
|
1148
|
+
|
|
1149
|
+
rollbacks:
|
|
1150
|
+
- name: "remove-latency"
|
|
1151
|
+
type: action
|
|
1152
|
+
provider:
|
|
1153
|
+
type: process
|
|
1154
|
+
path: "tc"
|
|
1155
|
+
arguments: "qdisc del dev eth0 root"
|
|
1156
|
+
```
|
|
1157
|
+
|
|
1158
|
+
### Property-Based Testing
|
|
1159
|
+
|
|
1160
|
+
根据代码应满足的性质,生成随机的测试输入。
|
|
1161
|
+
|
|
1162
|
+
```python
|
|
1163
|
+
# Python Example with Hypothesis
|
|
1164
|
+
from hypothesis import given, strategies as st
|
|
1165
|
+
|
|
1166
|
+
# Property: Sorting should be idempotent
|
|
1167
|
+
@given(st.lists(st.integers()))
|
|
1168
|
+
def test_sort_idempotent(xs):
|
|
1169
|
+
sorted_once = sorted(xs)
|
|
1170
|
+
sorted_twice = sorted(sorted_once)
|
|
1171
|
+
assert sorted_once == sorted_twice
|
|
1172
|
+
|
|
1173
|
+
# Property: Sorted list should have same elements
|
|
1174
|
+
@given(st.lists(st.integers()))
|
|
1175
|
+
def test_sort_preserves_elements(xs):
|
|
1176
|
+
sorted_xs = sorted(xs)
|
|
1177
|
+
assert sorted(xs) == sorted(sorted_xs)
|
|
1178
|
+
assert len(xs) == len(sorted_xs)
|
|
1179
|
+
|
|
1180
|
+
# Property: JSON encode/decode roundtrip
|
|
1181
|
+
@given(st.dictionaries(st.text(), st.integers()))
|
|
1182
|
+
def test_json_roundtrip(d):
|
|
1183
|
+
assert json.loads(json.dumps(d)) == d
|
|
1184
|
+
```
|
|
1185
|
+
|
|
1186
|
+
**工具**: Hypothesis (Python), fast-check (JS/TS), QuickCheck (Haskell), FsCheck (C#)
|
|
1187
|
+
|
|
1188
|
+
### Visual Regression Testing
|
|
1189
|
+
|
|
1190
|
+
侦测 UI 组件中非预期的视觉变化。
|
|
1191
|
+
|
|
1192
|
+
```
|
|
1193
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1194
|
+
│ Visual Regression Testing │
|
|
1195
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1196
|
+
│ │
|
|
1197
|
+
│ 1. Capture baseline screenshots │
|
|
1198
|
+
│ 2. Run tests, capture new screenshots │
|
|
1199
|
+
│ 3. Compare pixel-by-pixel or perceptually │
|
|
1200
|
+
│ 4. Flag differences for review │
|
|
1201
|
+
│ │
|
|
1202
|
+
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
|
1203
|
+
│ │ Baseline │ → │ Current │ → │ Diff │ │
|
|
1204
|
+
│ │ Screenshot │ │ Screenshot │ │ Report │ │
|
|
1205
|
+
│ └─────────────┘ └─────────────┘ └─────────────┘ │
|
|
1206
|
+
│ │
|
|
1207
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1208
|
+
```
|
|
1209
|
+
|
|
1210
|
+
**工具**: Percy, Chromatic, BackstopJS, Playwright visual comparisons
|
|
1211
|
+
|
|
1212
|
+
```typescript
|
|
1213
|
+
// Playwright Visual Regression Example
|
|
1214
|
+
test('homepage visual regression', async ({ page }) => {
|
|
1215
|
+
await page.goto('/');
|
|
1216
|
+
await expect(page).toHaveScreenshot('homepage.png', {
|
|
1217
|
+
maxDiffPixels: 100 // Allow minor differences
|
|
1218
|
+
});
|
|
1219
|
+
});
|
|
1220
|
+
|
|
1221
|
+
test('button component visual regression', async ({ page }) => {
|
|
1222
|
+
await page.goto('/components/button');
|
|
1223
|
+
const button = page.locator('[data-testid="primary-button"]');
|
|
1224
|
+
await expect(button).toHaveScreenshot('primary-button.png');
|
|
1225
|
+
});
|
|
1226
|
+
```
|
|
1227
|
+
|
|
1228
|
+
### Accessibility Testing (a11y)
|
|
1229
|
+
|
|
1230
|
+
确保应用程序可供身心障碍者使用。
|
|
1231
|
+
|
|
1232
|
+
```
|
|
1233
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1234
|
+
│ Accessibility Testing Checklist │
|
|
1235
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1236
|
+
│ WCAG 2.1 Guidelines │
|
|
1237
|
+
│ ├── Perceivable │
|
|
1238
|
+
│ │ ├── Alt text for images │
|
|
1239
|
+
│ │ ├── Captions for video │
|
|
1240
|
+
│ │ └── Sufficient color contrast │
|
|
1241
|
+
│ ├── Operable │
|
|
1242
|
+
│ │ ├── Keyboard navigation │
|
|
1243
|
+
│ │ ├── Focus indicators visible │
|
|
1244
|
+
│ │ └── No keyboard traps │
|
|
1245
|
+
│ ├── Understandable │
|
|
1246
|
+
│ │ ├── Clear labels and instructions │
|
|
1247
|
+
│ │ ├── Predictable navigation │
|
|
1248
|
+
│ │ └── Error identification │
|
|
1249
|
+
│ └── Robust │
|
|
1250
|
+
│ ├── Valid HTML │
|
|
1251
|
+
│ └── ARIA attributes correct │
|
|
1252
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1253
|
+
```
|
|
1254
|
+
|
|
1255
|
+
**工具**: axe-core, Pa11y, WAVE, Lighthouse
|
|
1256
|
+
|
|
1257
|
+
```typescript
|
|
1258
|
+
// Playwright + axe-core Example
|
|
1259
|
+
import AxeBuilder from '@axe-core/playwright';
|
|
1260
|
+
|
|
1261
|
+
test('homepage accessibility', async ({ page }) => {
|
|
1262
|
+
await page.goto('/');
|
|
1263
|
+
|
|
1264
|
+
const results = await new AxeBuilder({ page })
|
|
1265
|
+
.withTags(['wcag2a', 'wcag2aa'])
|
|
1266
|
+
.analyze();
|
|
1267
|
+
|
|
1268
|
+
expect(results.violations).toEqual([]);
|
|
1269
|
+
});
|
|
1270
|
+
```
|
|
1271
|
+
|
|
1272
|
+
### Additional Specialized Testing Types (SWEBOK)
|
|
1273
|
+
|
|
1274
|
+
以下测试类型在 SWEBOK v4.0 中被定义为测试目标:
|
|
1275
|
+
|
|
1276
|
+
#### Installation Testing
|
|
1277
|
+
|
|
1278
|
+
验证软件能在目标环境中正确安装。
|
|
1279
|
+
|
|
1280
|
+
```
|
|
1281
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1282
|
+
│ Installation Testing Checklist │
|
|
1283
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1284
|
+
│ │
|
|
1285
|
+
│ Pre-Installation │
|
|
1286
|
+
│ ├── Prerequisites verified (OS, runtime, dependencies) │
|
|
1287
|
+
│ ├── Disk space requirements met │
|
|
1288
|
+
│ └── Permissions and access rights confirmed │
|
|
1289
|
+
│ │
|
|
1290
|
+
│ Installation Process │
|
|
1291
|
+
│ ├── Fresh installation succeeds │
|
|
1292
|
+
│ ├── Upgrade from previous version succeeds │
|
|
1293
|
+
│ ├── Custom installation options work │
|
|
1294
|
+
│ └── Silent/unattended installation works │
|
|
1295
|
+
│ │
|
|
1296
|
+
│ Post-Installation │
|
|
1297
|
+
│ ├── Application launches correctly │
|
|
1298
|
+
│ ├── Configuration files created properly │
|
|
1299
|
+
│ ├── Registry/system settings correct │
|
|
1300
|
+
│ └── Uninstallation removes all components │
|
|
1301
|
+
│ │
|
|
1302
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1303
|
+
```
|
|
1304
|
+
|
|
1305
|
+
#### Alpha and Beta Testing
|
|
1306
|
+
|
|
1307
|
+
| 阶段 | 参与者 | 环境 | 目的 |
|
|
1308
|
+
|-------|--------------|-------------|---------|
|
|
1309
|
+
| **Alpha** | 内部用户、开发者 | 开发场地 | 早期验证、重大问题 |
|
|
1310
|
+
| **Beta** | 外部用户、客户 | 客户场地 | 真实世界验证、反馈 |
|
|
1311
|
+
| **Open Beta** | 一般公众 | 各种环境 | 广泛曝光、压力测试 |
|
|
1312
|
+
|
|
1313
|
+
#### Recovery Testing
|
|
1314
|
+
|
|
1315
|
+
测试系统从故障中恢复的能力。
|
|
1316
|
+
|
|
1317
|
+
```
|
|
1318
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1319
|
+
│ Recovery Testing Scenarios │
|
|
1320
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1321
|
+
│ │
|
|
1322
|
+
│ Hardware Failures │
|
|
1323
|
+
│ ├── Power outage during transaction │
|
|
1324
|
+
│ ├── Disk failure and RAID recovery │
|
|
1325
|
+
│ └── Network disconnection and reconnection │
|
|
1326
|
+
│ │
|
|
1327
|
+
│ Software Failures │
|
|
1328
|
+
│ ├── Application crash recovery │
|
|
1329
|
+
│ ├── Database connection loss and reconnect │
|
|
1330
|
+
│ └── Service restart behavior │
|
|
1331
|
+
│ │
|
|
1332
|
+
│ Data Recovery │
|
|
1333
|
+
│ ├── Backup and restore procedures │
|
|
1334
|
+
│ ├── Transaction rollback │
|
|
1335
|
+
│ └── Point-in-time recovery │
|
|
1336
|
+
│ │
|
|
1337
|
+
│ Metrics: │
|
|
1338
|
+
│ • Recovery Time Objective (RTO): Max acceptable downtime │
|
|
1339
|
+
│ • Recovery Point Objective (RPO): Max acceptable data loss │
|
|
1340
|
+
│ │
|
|
1341
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1342
|
+
```
|
|
1343
|
+
|
|
1344
|
+
#### Configuration Testing
|
|
1345
|
+
|
|
1346
|
+
在不同配置下测试系统。
|
|
1347
|
+
|
|
1348
|
+
```
|
|
1349
|
+
Configuration Matrix Example:
|
|
1350
|
+
|
|
1351
|
+
┌──────────────────┬─────────┬─────────┬─────────┐
|
|
1352
|
+
│ Configuration │ Config A│ Config B│ Config C│
|
|
1353
|
+
├──────────────────┼─────────┼─────────┼─────────┤
|
|
1354
|
+
│ Memory │ 4 GB │ 8 GB │ 16 GB │
|
|
1355
|
+
│ Database │ SQLite │ MySQL │ PostgreSQL│
|
|
1356
|
+
│ Cache │ None │ Redis │ Memcached│
|
|
1357
|
+
│ Load Balancer │ None │ Nginx │ HAProxy │
|
|
1358
|
+
└──────────────────┴─────────┴─────────┴─────────┘
|
|
1359
|
+
|
|
1360
|
+
Test each supported configuration combination.
|
|
1361
|
+
```
|
|
1362
|
+
|
|
1363
|
+
#### Back-to-Back Testing
|
|
1364
|
+
|
|
1365
|
+
以相同输入比较多个实现的输出。
|
|
1366
|
+
|
|
1367
|
+
```
|
|
1368
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1369
|
+
│ Back-to-Back Testing │
|
|
1370
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1371
|
+
│ │
|
|
1372
|
+
│ Use Cases: │
|
|
1373
|
+
│ ├── Comparing new version against old version │
|
|
1374
|
+
│ ├── Comparing optimized code against reference impl │
|
|
1375
|
+
│ ├── Cross-platform consistency verification │
|
|
1376
|
+
│ └── Validating refactored code behavior │
|
|
1377
|
+
│ │
|
|
1378
|
+
│ Process: │
|
|
1379
|
+
│ ┌─────────┐ ┌─────────────┐ ┌──────────┐ │
|
|
1380
|
+
│ │ Input │────▶│ Version A │────▶│ Output A │──┐ │
|
|
1381
|
+
│ │ Data │ └─────────────┘ └──────────┘ │ │
|
|
1382
|
+
│ │ │ ┌─────────────┐ ┌──────────┐ ▼ │
|
|
1383
|
+
│ │ │────▶│ Version B │────▶│ Output B │─▶Compare │
|
|
1384
|
+
│ └─────────┘ └─────────────┘ └──────────┘ │
|
|
1385
|
+
│ │
|
|
1386
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1387
|
+
```
|
|
1388
|
+
|
|
1389
|
+
#### Usability Testing
|
|
1390
|
+
|
|
1391
|
+
评估用户学习与使用系统的容易程度。
|
|
1392
|
+
|
|
1393
|
+
```
|
|
1394
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1395
|
+
│ Usability Testing Methods │
|
|
1396
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1397
|
+
│ │
|
|
1398
|
+
│ Observational Methods │
|
|
1399
|
+
│ ├── Think-aloud protocol │
|
|
1400
|
+
│ ├── Eye tracking │
|
|
1401
|
+
│ └── Session recording │
|
|
1402
|
+
│ │
|
|
1403
|
+
│ Task-Based Methods │
|
|
1404
|
+
│ ├── Task completion rate │
|
|
1405
|
+
│ ├── Time on task │
|
|
1406
|
+
│ └── Error rate │
|
|
1407
|
+
│ │
|
|
1408
|
+
│ Survey Methods │
|
|
1409
|
+
│ ├── System Usability Scale (SUS) │
|
|
1410
|
+
│ ├── Net Promoter Score (NPS) │
|
|
1411
|
+
│ └── Post-task questionnaires │
|
|
1412
|
+
│ │
|
|
1413
|
+
│ Key Metrics: │
|
|
1414
|
+
│ • Learnability: Time to complete first task │
|
|
1415
|
+
│ • Efficiency: Time to complete after learning │
|
|
1416
|
+
│ • Memorability: Performance after period of non-use │
|
|
1417
|
+
│ • Satisfaction: User rating scores │
|
|
1418
|
+
│ │
|
|
1419
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1420
|
+
```
|
|
1421
|
+
|
|
1422
|
+
### Specialized Testing Summary
|
|
1423
|
+
|
|
1424
|
+
| 类型 | 目的 | 使用时机 |
|
|
1425
|
+
|------|---------|-------------|
|
|
1426
|
+
| Contract Testing | API 兼容性 | 微服务 |
|
|
1427
|
+
| Mutation Testing | 测试质量评估 | 关键代码路径 |
|
|
1428
|
+
| Chaos Engineering | 韧性验证 | 生产系统 |
|
|
1429
|
+
| Property-Based | 边界用例发掘 | 算法、parser |
|
|
1430
|
+
| Visual Regression | UI 一致性 | 前端变更 |
|
|
1431
|
+
| Accessibility | 无障碍设计 | 所有面向用户的应用 |
|
|
1432
|
+
| Installation | 部署验证 | 候选发布版本 |
|
|
1433
|
+
| Alpha/Beta | 早期用户验证 | 发布前阶段 |
|
|
1434
|
+
| Recovery | 故障恢复 | 业务关键系统 |
|
|
1435
|
+
| Configuration | 多配置支持 | 跨平台应用 |
|
|
1436
|
+
| Back-to-Back | 版本比较 | 迁移、重构 |
|
|
1437
|
+
| Usability | 用户体验 | 面向用户的应用 |
|
|
1438
|
+
|
|
1439
|
+
---
|
|
1440
|
+
|
|
1441
|
+
## Test-Related Measures
|
|
1442
|
+
|
|
1443
|
+
用以评估受测软件本身以及测试本身质量的量化度量。
|
|
1444
|
+
|
|
1445
|
+
**参考**: SWEBOK v4.0 第 5.4 节
|
|
1446
|
+
|
|
1447
|
+
### Program Evaluation Measures
|
|
1448
|
+
|
|
1449
|
+
#### Fault Density
|
|
1450
|
+
|
|
1451
|
+
```
|
|
1452
|
+
Fault Density = Total Faults / Size
|
|
1453
|
+
|
|
1454
|
+
Where Size can be:
|
|
1455
|
+
- KLOC (Thousands of Lines of Code)
|
|
1456
|
+
- Function Points
|
|
1457
|
+
- Story Points
|
|
1458
|
+
|
|
1459
|
+
Industry Benchmarks:
|
|
1460
|
+
┌─────────────────────┬────────────────────────────┐
|
|
1461
|
+
│ Quality Level │ Faults per KLOC │
|
|
1462
|
+
├─────────────────────┼────────────────────────────┤
|
|
1463
|
+
│ Typical │ 1 - 25 │
|
|
1464
|
+
│ Good │ 0.5 - 1 │
|
|
1465
|
+
│ Excellent │ < 0.5 │
|
|
1466
|
+
│ Safety-Critical │ < 0.1 │
|
|
1467
|
+
└─────────────────────┴────────────────────────────┘
|
|
1468
|
+
```
|
|
1469
|
+
|
|
1470
|
+
#### Reliability Growth Models
|
|
1471
|
+
|
|
1472
|
+
追踪缺陷随时间的发现率,以预测剩余缺陷。
|
|
1473
|
+
|
|
1474
|
+
```
|
|
1475
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1476
|
+
│ Reliability Growth Curve │
|
|
1477
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1478
|
+
│ │
|
|
1479
|
+
│ Defects │
|
|
1480
|
+
│ Found │
|
|
1481
|
+
│ │ ╭──────────────────────────────── │
|
|
1482
|
+
│ │ ╭─╯ │
|
|
1483
|
+
│ │ ╭╯ │
|
|
1484
|
+
│ │ ╭╯ │
|
|
1485
|
+
│ │╭╯ │
|
|
1486
|
+
│ ├╯ │
|
|
1487
|
+
│ └──────────────────────────────────────▶ Time │
|
|
1488
|
+
│ │
|
|
1489
|
+
│ As testing progresses: │
|
|
1490
|
+
│ • Defect discovery rate decreases │
|
|
1491
|
+
│ • Curve approaches asymptote (total defects) │
|
|
1492
|
+
│ • Remaining defects can be estimated │
|
|
1493
|
+
│ │
|
|
1494
|
+
│ Common Models: │
|
|
1495
|
+
│ • Goel-Okumoto: Exponential │
|
|
1496
|
+
│ • Musa: Logarithmic │
|
|
1497
|
+
│ • Jelinski-Moranda: De-eutrophication │
|
|
1498
|
+
│ │
|
|
1499
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1500
|
+
```
|
|
1501
|
+
|
|
1502
|
+
### Test Evaluation Measures
|
|
1503
|
+
|
|
1504
|
+
#### Fault Seeding
|
|
1505
|
+
|
|
1506
|
+
植入已知的缺陷以衡量测试效能。
|
|
1507
|
+
|
|
1508
|
+
```
|
|
1509
|
+
Fault Detection Effectiveness = Seeded Faults Found / Total Seeded Faults
|
|
1510
|
+
|
|
1511
|
+
If tests find 80% of seeded faults, estimate:
|
|
1512
|
+
- Similar detection rate for real faults
|
|
1513
|
+
- Remaining faults ≈ Found faults / Detection rate
|
|
1514
|
+
|
|
1515
|
+
Example:
|
|
1516
|
+
- Seeded 20 faults, tests found 16 (80%)
|
|
1517
|
+
- Found 40 real faults during testing
|
|
1518
|
+
- Estimated total real faults ≈ 40 / 0.8 = 50
|
|
1519
|
+
- Estimated remaining ≈ 10 faults
|
|
1520
|
+
```
|
|
1521
|
+
|
|
1522
|
+
#### Coverage Metrics Summary
|
|
1523
|
+
|
|
1524
|
+
| Metric | 公式 | 目标 |
|
|
1525
|
+
|--------|---------|--------|
|
|
1526
|
+
| Statement Coverage | 已执行语句 / 总语句 | ≥ 70% |
|
|
1527
|
+
| Branch Coverage | 已执行分支 / 总分支 | ≥ 60% |
|
|
1528
|
+
| Condition Coverage | 评估为 T 与 F 的条件 / 总条件 | ≥ 80% |
|
|
1529
|
+
| MC/DC | 能独立影响决策的条件 | 100%(安全关键) |
|
|
1530
|
+
| Requirements Coverage | 已测试需求 / 总需求 | 100% |
|
|
1531
|
+
| Mutation Score | 被杀死的 mutants / 总 mutants | ≥ 80% |
|
|
1532
|
+
|
|
1533
|
+
### Testing Process Metrics
|
|
1534
|
+
|
|
1535
|
+
```
|
|
1536
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1537
|
+
│ Key Testing Metrics │
|
|
1538
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1539
|
+
│ │
|
|
1540
|
+
│ Efficiency Metrics │
|
|
1541
|
+
│ ├── Test Case Productivity = Tests created / Effort │
|
|
1542
|
+
│ ├── Defect Detection Rate = Defects found / Test hours │
|
|
1543
|
+
│ └── Automation Rate = Automated tests / Total tests │
|
|
1544
|
+
│ │
|
|
1545
|
+
│ Effectiveness Metrics │
|
|
1546
|
+
│ ├── Defect Leakage = Prod defects / Total defects │
|
|
1547
|
+
│ ├── Test Effectiveness = Defects found / Defects present │
|
|
1548
|
+
│ └── Phase Containment = Defects found in phase / Injected │
|
|
1549
|
+
│ │
|
|
1550
|
+
│ Progress Metrics │
|
|
1551
|
+
│ ├── Test Execution Rate = Tests run / Tests planned │
|
|
1552
|
+
│ ├── Pass Rate = Tests passed / Tests executed │
|
|
1553
|
+
│ └── Blocking Rate = Blocked tests / Total tests │
|
|
1554
|
+
│ │
|
|
1555
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1556
|
+
```
|
|
1557
|
+
|
|
1558
|
+
---
|
|
1559
|
+
|
|
1560
|
+
## Defect Management
|
|
1561
|
+
|
|
1562
|
+
在整个测试生命周期中对缺陷进行系统化的追踪与管理。
|
|
1563
|
+
|
|
1564
|
+
**参考**: ISTQB CTFL v4.0 第 5.5 节, ISO/IEC/IEEE 29119-3
|
|
1565
|
+
|
|
1566
|
+
### Defect Lifecycle
|
|
1567
|
+
|
|
1568
|
+
```
|
|
1569
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1570
|
+
│ Defect Lifecycle │
|
|
1571
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1572
|
+
│ │
|
|
1573
|
+
│ ┌────────┐ ┌────────┐ ┌──────────┐ │
|
|
1574
|
+
│ │ New │───▶│ Open │───▶│ In Progress│ │
|
|
1575
|
+
│ └────────┘ └────────┘ └──────────┘ │
|
|
1576
|
+
│ │ │ │
|
|
1577
|
+
│ │ ▼ │
|
|
1578
|
+
│ │ ┌──────────┐ │
|
|
1579
|
+
│ │ │ Fixed │ │
|
|
1580
|
+
│ │ └──────────┘ │
|
|
1581
|
+
│ │ │ │
|
|
1582
|
+
│ │ ▼ │
|
|
1583
|
+
│ │ ┌──────────┐ ┌──────────┐ │
|
|
1584
|
+
│ │ │ Verified │───▶│ Closed │ │
|
|
1585
|
+
│ │ └──────────┘ └──────────┘ │
|
|
1586
|
+
│ │ │ │
|
|
1587
|
+
│ │ ▼ │
|
|
1588
|
+
│ │ ┌──────────┐ │
|
|
1589
|
+
│ └───────▶│ Reopened │ │
|
|
1590
|
+
│ └──────────┘ │
|
|
1591
|
+
│ │
|
|
1592
|
+
│ Alternative paths: │
|
|
1593
|
+
│ - New → Rejected (not a bug, duplicate) │
|
|
1594
|
+
│ - Open → Deferred (postponed to later release) │
|
|
1595
|
+
│ - Open → Won't Fix (accepted risk) │
|
|
1596
|
+
│ │
|
|
1597
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1598
|
+
```
|
|
1599
|
+
|
|
1600
|
+
### Defect Report Template
|
|
1601
|
+
|
|
1602
|
+
```markdown
|
|
1603
|
+
# Defect Report
|
|
1604
|
+
|
|
1605
|
+
## Summary
|
|
1606
|
+
[One-line description of the defect]
|
|
1607
|
+
|
|
1608
|
+
## ID: BUG-1234
|
|
1609
|
+
## Priority: P1/P2/P3/P4
|
|
1610
|
+
## Severity: Critical/High/Medium/Low
|
|
1611
|
+
|
|
1612
|
+
## Environment
|
|
1613
|
+
- **Version**: 2.3.1
|
|
1614
|
+
- **OS**: Windows 11 / macOS 14.2
|
|
1615
|
+
- **Browser**: Chrome 120
|
|
1616
|
+
- **Device**: Desktop / Mobile
|
|
1617
|
+
|
|
1618
|
+
## Steps to Reproduce
|
|
1619
|
+
1. Navigate to login page
|
|
1620
|
+
2. Enter valid username "testuser"
|
|
1621
|
+
3. Enter invalid password "wrong123"
|
|
1622
|
+
4. Click "Login" button
|
|
1623
|
+
5. Observe error message
|
|
1624
|
+
|
|
1625
|
+
## Expected Result
|
|
1626
|
+
Error message: "Invalid credentials. Please try again."
|
|
1627
|
+
|
|
1628
|
+
## Actual Result
|
|
1629
|
+
Generic error: "Something went wrong."
|
|
1630
|
+
|
|
1631
|
+
## Screenshots/Videos
|
|
1632
|
+
[Attach relevant media]
|
|
1633
|
+
|
|
1634
|
+
## Additional Information
|
|
1635
|
+
- Occurs consistently (100% reproducible)
|
|
1636
|
+
- Related to ticket: FEAT-456
|
|
1637
|
+
- Log file: [attached]
|
|
1638
|
+
```
|
|
1639
|
+
|
|
1640
|
+
### Severity vs Priority
|
|
1641
|
+
|
|
1642
|
+
| Severity | 说明 | 示例 |
|
|
1643
|
+
|----------|-------------|----------|
|
|
1644
|
+
| Critical | 系统崩溃、数据丢失、安全漏洞 | App 无法启动、数据损坏 |
|
|
1645
|
+
| High | 主要功能损坏、无替代方案 | 无法完成结账 |
|
|
1646
|
+
| Medium | 功能受损但有替代方案 | 导出失败,可手动复制 |
|
|
1647
|
+
| Low | 小问题、外观瑕疵 | 错字、对齐问题 |
|
|
1648
|
+
|
|
1649
|
+
| Priority | 说明 | 响应时间 |
|
|
1650
|
+
|----------|-------------|---------------|
|
|
1651
|
+
| P1 | 立即修正 | 当天 |
|
|
1652
|
+
| P2 | 发布前修正 | 本 sprint 内 |
|
|
1653
|
+
| P3 | 有空时修正 | 下个 sprint |
|
|
1654
|
+
| P4 | 修了更好 | Backlog |
|
|
1655
|
+
|
|
1656
|
+
### Defect Metrics
|
|
1657
|
+
|
|
1658
|
+
```
|
|
1659
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1660
|
+
│ Key Defect Metrics │
|
|
1661
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1662
|
+
│ │
|
|
1663
|
+
│ Defect Density = Total Defects / Size (KLOC or FP) │
|
|
1664
|
+
│ - Industry average: 1-25 defects per KLOC │
|
|
1665
|
+
│ │
|
|
1666
|
+
│ Defect Detection Efficiency (DDE) = │
|
|
1667
|
+
│ Defects found before release / Total defects × 100 │
|
|
1668
|
+
│ - Target: > 90% │
|
|
1669
|
+
│ │
|
|
1670
|
+
│ Defect Removal Efficiency (DRE) = │
|
|
1671
|
+
│ Defects removed / Defects injected × 100 │
|
|
1672
|
+
│ - Target: > 95% │
|
|
1673
|
+
│ │
|
|
1674
|
+
│ Mean Time to Detect (MTTD) = Avg time from injection │
|
|
1675
|
+
│ Mean Time to Resolve (MTTR) = Avg time from report to fix │
|
|
1676
|
+
│ │
|
|
1677
|
+
│ Defect Leakage Rate = Production defects / Total defects │
|
|
1678
|
+
│ - Target: < 5% │
|
|
1679
|
+
│ │
|
|
1680
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1681
|
+
```
|
|
1682
|
+
|
|
1683
|
+
### Root Cause Analysis
|
|
1684
|
+
|
|
1685
|
+
```
|
|
1686
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1687
|
+
│ Common Defect Root Causes │
|
|
1688
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1689
|
+
│ Requirements (35-40%) │
|
|
1690
|
+
│ ├── Ambiguous requirements │
|
|
1691
|
+
│ ├── Missing requirements │
|
|
1692
|
+
│ └── Changed requirements │
|
|
1693
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1694
|
+
│ Design (20-25%) │
|
|
1695
|
+
│ ├── Incorrect algorithm │
|
|
1696
|
+
│ ├── Integration issues │
|
|
1697
|
+
│ └── Performance not considered │
|
|
1698
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1699
|
+
│ Code (25-30%) │
|
|
1700
|
+
│ ├── Logic errors │
|
|
1701
|
+
│ ├── Boundary conditions │
|
|
1702
|
+
│ └── Exception handling │
|
|
1703
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1704
|
+
│ Environment (10-15%) │
|
|
1705
|
+
│ ├── Configuration issues │
|
|
1706
|
+
│ ├── Third-party dependencies │
|
|
1707
|
+
│ └── Data issues │
|
|
1708
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1709
|
+
```
|
|
1710
|
+
|
|
1711
|
+
---
|
|
1712
|
+
|
|
1713
|
+
## Test Process Management
|
|
1714
|
+
|
|
1715
|
+
针对测试活动进行规划、监控与控制的结构化途径。
|
|
1716
|
+
|
|
1717
|
+
**参考**: ISTQB CTFL v4.0 第 5 章, ISO/IEC/IEEE 29119-2
|
|
1718
|
+
|
|
1719
|
+
### Test Planning
|
|
1720
|
+
|
|
1721
|
+
#### Test Plan Components (ISO/IEC/IEEE 29119-3)
|
|
1722
|
+
|
|
1723
|
+
```
|
|
1724
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1725
|
+
│ Test Plan Structure │
|
|
1726
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1727
|
+
│ │
|
|
1728
|
+
│ 1. Test Plan Identification │
|
|
1729
|
+
│ - Project name, version, date │
|
|
1730
|
+
│ │
|
|
1731
|
+
│ 2. Introduction │
|
|
1732
|
+
│ - Purpose, scope, objectives │
|
|
1733
|
+
│ │
|
|
1734
|
+
│ 3. Test Items │
|
|
1735
|
+
│ - Features to be tested │
|
|
1736
|
+
│ - Features not to be tested │
|
|
1737
|
+
│ │
|
|
1738
|
+
│ 4. Test Approach │
|
|
1739
|
+
│ - Test levels and types │
|
|
1740
|
+
│ - Test techniques │
|
|
1741
|
+
│ - Entry/exit criteria │
|
|
1742
|
+
│ │
|
|
1743
|
+
│ 5. Test Environment │
|
|
1744
|
+
│ - Hardware, software, tools │
|
|
1745
|
+
│ - Test data requirements │
|
|
1746
|
+
│ │
|
|
1747
|
+
│ 6. Test Schedule │
|
|
1748
|
+
│ - Milestones, dependencies │
|
|
1749
|
+
│ │
|
|
1750
|
+
│ 7. Roles and Responsibilities │
|
|
1751
|
+
│ - Team structure, RACI matrix │
|
|
1752
|
+
│ │
|
|
1753
|
+
│ 8. Risks and Contingencies │
|
|
1754
|
+
│ - Risk assessment, mitigation │
|
|
1755
|
+
│ │
|
|
1756
|
+
│ 9. Approvals │
|
|
1757
|
+
│ - Sign-off requirements │
|
|
1758
|
+
│ │
|
|
1759
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1760
|
+
```
|
|
1761
|
+
|
|
1762
|
+
#### Entry and Exit Criteria
|
|
1763
|
+
|
|
1764
|
+
| 准则类型 | 示例 |
|
|
1765
|
+
|---------------|----------|
|
|
1766
|
+
| **Entry Criteria**(开始测试) | 需求已批准、代码完成、环境就绪、测试数据齐备 |
|
|
1767
|
+
| **Exit Criteria**(停止测试) | 所有 P1/P2 测试通过、达成覆盖目标、无未解的严重缺陷 |
|
|
1768
|
+
|
|
1769
|
+
```markdown
|
|
1770
|
+
## Entry Criteria for System Testing
|
|
1771
|
+
- [ ] All integration tests passed
|
|
1772
|
+
- [ ] Test environment deployed and verified
|
|
1773
|
+
- [ ] Test data prepared and loaded
|
|
1774
|
+
- [ ] Test cases reviewed and approved
|
|
1775
|
+
- [ ] All blockers from IT resolved
|
|
1776
|
+
|
|
1777
|
+
## Exit Criteria for System Testing
|
|
1778
|
+
- [ ] 100% of critical test cases executed
|
|
1779
|
+
- [ ] 95% of high priority test cases passed
|
|
1780
|
+
- [ ] No Critical or High severity defects open
|
|
1781
|
+
- [ ] Code coverage > 80%
|
|
1782
|
+
- [ ] Performance benchmarks met
|
|
1783
|
+
```
|
|
1784
|
+
|
|
1785
|
+
### Test Monitoring and Control
|
|
1786
|
+
|
|
1787
|
+
#### Key Progress Metrics
|
|
1788
|
+
|
|
1789
|
+
```
|
|
1790
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1791
|
+
│ Test Progress Metrics │
|
|
1792
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1793
|
+
│ │
|
|
1794
|
+
│ Execution Metrics: │
|
|
1795
|
+
│ - Tests Planned: 500 │
|
|
1796
|
+
│ - Tests Executed: 350 (70%) │
|
|
1797
|
+
│ - Tests Passed: 320 (91% of executed) │
|
|
1798
|
+
│ - Tests Failed: 30 (9% of executed) │
|
|
1799
|
+
│ - Tests Blocked: 15 │
|
|
1800
|
+
│ │
|
|
1801
|
+
│ Defect Metrics: │
|
|
1802
|
+
│ - Total Defects Found: 45 │
|
|
1803
|
+
│ - Open Defects: 12 (3 Critical, 5 High, 4 Medium) │
|
|
1804
|
+
│ - Fixed Defects: 28 │
|
|
1805
|
+
│ - Verified Defects: 20 │
|
|
1806
|
+
│ │
|
|
1807
|
+
│ Coverage Metrics: │
|
|
1808
|
+
│ - Requirements Coverage: 85% │
|
|
1809
|
+
│ - Code Coverage: 78% │
|
|
1810
|
+
│ - Risk Coverage: 90% │
|
|
1811
|
+
│ │
|
|
1812
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1813
|
+
```
|
|
1814
|
+
|
|
1815
|
+
#### Test Dashboard Example
|
|
1816
|
+
|
|
1817
|
+
```
|
|
1818
|
+
Test Progress Dashboard - Sprint 15
|
|
1819
|
+
═══════════════════════════════════════════════════════════════
|
|
1820
|
+
|
|
1821
|
+
Test Execution Defect Status
|
|
1822
|
+
█████████████████░░░░ 85% Open: ███░░░░░░░ 12
|
|
1823
|
+
Fixed: ████████░░ 28
|
|
1824
|
+
Passed │████████████████│ 320 Verified: ██████░░░░ 20
|
|
1825
|
+
Failed │██ │ 30
|
|
1826
|
+
Blocked │█ │ 15 Trend: ↓ (improving)
|
|
1827
|
+
|
|
1828
|
+
Coverage Build Health
|
|
1829
|
+
Requirements: ███████████░ 85% Last Build: ✓ Pass
|
|
1830
|
+
Code: ████████░░░░ 78% Unit Tests: 245/245 ✓
|
|
1831
|
+
Risk Items: █████████░░░ 90% Int Tests: 89/92 ⚠
|
|
1832
|
+
Flaky Rate: 2.1%
|
|
1833
|
+
```
|
|
1834
|
+
|
|
1835
|
+
### Test Completion
|
|
1836
|
+
|
|
1837
|
+
#### Test Summary Report
|
|
1838
|
+
|
|
1839
|
+
```markdown
|
|
1840
|
+
# Test Summary Report
|
|
1841
|
+
|
|
1842
|
+
## Project: [Project Name]
|
|
1843
|
+
## Version: [Version Number]
|
|
1844
|
+
## Test Period: [Start Date] - [End Date]
|
|
1845
|
+
|
|
1846
|
+
## Executive Summary
|
|
1847
|
+
[Brief overview of testing activities and outcomes]
|
|
1848
|
+
|
|
1849
|
+
## Test Scope
|
|
1850
|
+
- Features tested: [list]
|
|
1851
|
+
- Features not tested: [list with justification]
|
|
1852
|
+
|
|
1853
|
+
## Test Results Summary
|
|
1854
|
+
|
|
1855
|
+
| Test Level | Planned | Executed | Passed | Failed | Pass Rate |
|
|
1856
|
+
|------------|---------|----------|--------|--------|-----------|
|
|
1857
|
+
| Unit | 500 | 500 | 498 | 2 | 99.6% |
|
|
1858
|
+
| Integration| 150 | 150 | 145 | 5 | 96.7% |
|
|
1859
|
+
| System | 200 | 195 | 188 | 7 | 96.4% |
|
|
1860
|
+
| E2E | 50 | 48 | 46 | 2 | 95.8% |
|
|
1861
|
+
|
|
1862
|
+
## Defect Summary
|
|
1863
|
+
|
|
1864
|
+
| Severity | Found | Fixed | Open | Deferred |
|
|
1865
|
+
|----------|-------|-------|------|----------|
|
|
1866
|
+
| Critical | 2 | 2 | 0 | 0 |
|
|
1867
|
+
| High | 8 | 7 | 1 | 0 |
|
|
1868
|
+
| Medium | 15 | 12 | 2 | 1 |
|
|
1869
|
+
| Low | 20 | 10 | 5 | 5 |
|
|
1870
|
+
|
|
1871
|
+
## Coverage Analysis
|
|
1872
|
+
- Requirements coverage: 95%
|
|
1873
|
+
- Code coverage: 82%
|
|
1874
|
+
- Risk coverage: 100% (all high-risk items tested)
|
|
1875
|
+
|
|
1876
|
+
## Open Issues and Risks
|
|
1877
|
+
1. [Issue 1]: Impact and mitigation
|
|
1878
|
+
2. [Issue 2]: Impact and mitigation
|
|
1879
|
+
|
|
1880
|
+
## Recommendations
|
|
1881
|
+
1. [Recommendation 1]
|
|
1882
|
+
2. [Recommendation 2]
|
|
1883
|
+
|
|
1884
|
+
## Conclusion
|
|
1885
|
+
[Final assessment and release recommendation]
|
|
1886
|
+
|
|
1887
|
+
## Sign-off
|
|
1888
|
+
- QA Lead: [Name, Date]
|
|
1889
|
+
- Dev Lead: [Name, Date]
|
|
1890
|
+
- Product Owner: [Name, Date]
|
|
1891
|
+
```
|
|
1892
|
+
|
|
1893
|
+
### Test Process Improvement
|
|
1894
|
+
|
|
1895
|
+
```
|
|
1896
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
1897
|
+
│ Continuous Improvement Cycle │
|
|
1898
|
+
├─────────────────────────────────────────────────────────────┤
|
|
1899
|
+
│ │
|
|
1900
|
+
│ ┌───────────┐ │
|
|
1901
|
+
│ │ Plan │ │
|
|
1902
|
+
│ └─────┬─────┘ │
|
|
1903
|
+
│ │ │
|
|
1904
|
+
│ ▼ │
|
|
1905
|
+
│ ┌───────────┐ ┌───────────┐ │
|
|
1906
|
+
│ │ Act │◀───────▶│ Do │ │
|
|
1907
|
+
│ └───────────┘ └─────┬─────┘ │
|
|
1908
|
+
│ ▲ │ │
|
|
1909
|
+
│ │ ▼ │
|
|
1910
|
+
│ ┌─────┴─────┐ │
|
|
1911
|
+
│ │ Check │ │
|
|
1912
|
+
│ └───────────┘ │
|
|
1913
|
+
│ │
|
|
1914
|
+
│ Retrospective Questions: │
|
|
1915
|
+
│ - What went well? │
|
|
1916
|
+
│ - What could be improved? │
|
|
1917
|
+
│ - What actions will we take? │
|
|
1918
|
+
│ │
|
|
1919
|
+
└─────────────────────────────────────────────────────────────┘
|
|
1920
|
+
```
|
|
1921
|
+
|
|
1922
|
+
---
|
|
1923
|
+
|
|
1924
|
+
## Code Examples by Test Level
|
|
1925
|
+
|
|
1926
|
+
### Unit Testing Example
|
|
1927
|
+
|
|
1928
|
+
```csharp
|
|
1929
|
+
// C# Example
|
|
1930
|
+
[TestClass]
|
|
1931
|
+
public class UserValidatorTests
|
|
1932
|
+
{
|
|
1933
|
+
private UserValidator _validator;
|
|
1934
|
+
|
|
1935
|
+
[TestInitialize]
|
|
1936
|
+
public void Setup()
|
|
1937
|
+
{
|
|
1938
|
+
_validator = new UserValidator();
|
|
1939
|
+
}
|
|
1940
|
+
|
|
1941
|
+
[TestMethod]
|
|
1942
|
+
public void ValidateEmail_ValidFormat_ReturnsTrue()
|
|
1943
|
+
{
|
|
1944
|
+
// Arrange
|
|
1945
|
+
var email = "user@example.com";
|
|
1946
|
+
|
|
1947
|
+
// Act
|
|
1948
|
+
var result = _validator.ValidateEmail(email);
|
|
1949
|
+
|
|
1950
|
+
// Assert
|
|
1951
|
+
Assert.IsTrue(result);
|
|
1952
|
+
}
|
|
1953
|
+
|
|
1954
|
+
[TestMethod]
|
|
1955
|
+
public void ValidateEmail_InvalidFormat_ReturnsFalse()
|
|
1956
|
+
{
|
|
1957
|
+
// Arrange
|
|
1958
|
+
var email = "invalid-email";
|
|
1959
|
+
|
|
1960
|
+
// Act
|
|
1961
|
+
var result = _validator.ValidateEmail(email);
|
|
1962
|
+
|
|
1963
|
+
// Assert
|
|
1964
|
+
Assert.IsFalse(result);
|
|
1965
|
+
}
|
|
1966
|
+
}
|
|
1967
|
+
```
|
|
1968
|
+
|
|
1969
|
+
```typescript
|
|
1970
|
+
// TypeScript Example
|
|
1971
|
+
describe('UserValidator', () => {
|
|
1972
|
+
let validator: UserValidator;
|
|
1973
|
+
|
|
1974
|
+
beforeEach(() => {
|
|
1975
|
+
validator = new UserValidator();
|
|
1976
|
+
});
|
|
1977
|
+
|
|
1978
|
+
describe('validateEmail', () => {
|
|
1979
|
+
it('should return true for valid email format', () => {
|
|
1980
|
+
const result = validator.validateEmail('user@example.com');
|
|
1981
|
+
expect(result).toBe(true);
|
|
1982
|
+
});
|
|
1983
|
+
|
|
1984
|
+
it('should return false for invalid email format', () => {
|
|
1985
|
+
const result = validator.validateEmail('invalid-email');
|
|
1986
|
+
expect(result).toBe(false);
|
|
1987
|
+
});
|
|
1988
|
+
});
|
|
1989
|
+
});
|
|
1990
|
+
```
|
|
1991
|
+
|
|
1992
|
+
### Integration Testing Example
|
|
1993
|
+
|
|
1994
|
+
```csharp
|
|
1995
|
+
// C# Integration Test Example with Test Database
|
|
1996
|
+
[TestClass]
|
|
1997
|
+
public class UserRepositoryIntegrationTests
|
|
1998
|
+
{
|
|
1999
|
+
private TestDbContext _dbContext;
|
|
2000
|
+
private UserRepository _repository;
|
|
2001
|
+
|
|
2002
|
+
[TestInitialize]
|
|
2003
|
+
public async Task Setup()
|
|
2004
|
+
{
|
|
2005
|
+
// Use test database (e.g., SQLite in-memory or Testcontainers)
|
|
2006
|
+
_dbContext = TestDbContextFactory.Create();
|
|
2007
|
+
_repository = new UserRepository(_dbContext);
|
|
2008
|
+
await _dbContext.Database.EnsureCreatedAsync();
|
|
2009
|
+
}
|
|
2010
|
+
|
|
2011
|
+
[TestCleanup]
|
|
2012
|
+
public async Task Cleanup()
|
|
2013
|
+
{
|
|
2014
|
+
await _dbContext.DisposeAsync();
|
|
2015
|
+
}
|
|
2016
|
+
|
|
2017
|
+
[TestMethod]
|
|
2018
|
+
public async Task CreateUser_WithValidData_PersistsToDatabase()
|
|
2019
|
+
{
|
|
2020
|
+
// Arrange
|
|
2021
|
+
var user = new User { Name = "Test User", Email = "test@example.com" };
|
|
2022
|
+
|
|
2023
|
+
// Act
|
|
2024
|
+
await _repository.CreateAsync(user);
|
|
2025
|
+
var savedUser = await _repository.GetByIdAsync(user.Id);
|
|
2026
|
+
|
|
2027
|
+
// Assert
|
|
2028
|
+
Assert.IsNotNull(savedUser);
|
|
2029
|
+
Assert.AreEqual("Test User", savedUser.Name);
|
|
2030
|
+
}
|
|
2031
|
+
}
|
|
2032
|
+
```
|
|
2033
|
+
|
|
2034
|
+
### System Testing Example
|
|
2035
|
+
|
|
2036
|
+
```csharp
|
|
2037
|
+
// System Test Example: Complete Resource Processing Flow
|
|
2038
|
+
[TestClass]
|
|
2039
|
+
public class ResourceProcessingSystemTests
|
|
2040
|
+
{
|
|
2041
|
+
private HttpClient _client;
|
|
2042
|
+
private TestEnvironment _env;
|
|
2043
|
+
|
|
2044
|
+
[TestInitialize]
|
|
2045
|
+
public async Task Setup()
|
|
2046
|
+
{
|
|
2047
|
+
_env = await TestEnvironment.CreateAsync();
|
|
2048
|
+
_client = _env.CreateAuthenticatedClient();
|
|
2049
|
+
}
|
|
2050
|
+
|
|
2051
|
+
[TestMethod]
|
|
2052
|
+
public async Task ProcessResource_CompleteFlow_CompletedSuccessfully()
|
|
2053
|
+
{
|
|
2054
|
+
// Arrange: Create test data
|
|
2055
|
+
var item = await _env.CreateTestItem(value: 100);
|
|
2056
|
+
var user = await _env.CreateTestUser();
|
|
2057
|
+
|
|
2058
|
+
// Act: Execute complete processing flow
|
|
2059
|
+
// Step 1: Create request
|
|
2060
|
+
var requestResponse = await _client.PostAsync("/api/requests",
|
|
2061
|
+
new { itemId = item.Id, quantity = 2 });
|
|
2062
|
+
Assert.AreEqual(HttpStatusCode.OK, requestResponse.StatusCode);
|
|
2063
|
+
|
|
2064
|
+
// Step 2: Submit processing
|
|
2065
|
+
var processResponse = await _client.PostAsync("/api/processes",
|
|
2066
|
+
new { requestId = requestResponse.RequestId, userId = user.Id });
|
|
2067
|
+
var process = await processResponse.Content.ReadAsAsync<Process>();
|
|
2068
|
+
Assert.AreEqual(HttpStatusCode.Created, processResponse.StatusCode);
|
|
2069
|
+
|
|
2070
|
+
// Step 3: Confirm completion
|
|
2071
|
+
var confirmResponse = await _client.PostAsync($"/api/processes/{process.Id}/confirm",
|
|
2072
|
+
new { confirmationType = "standard", amount = 200 });
|
|
2073
|
+
Assert.AreEqual(HttpStatusCode.OK, confirmResponse.StatusCode);
|
|
2074
|
+
|
|
2075
|
+
// Assert: Verify final state
|
|
2076
|
+
var finalProcess = await _client.GetAsync($"/api/processes/{process.Id}");
|
|
2077
|
+
var result = await finalProcess.Content.ReadAsAsync<Process>();
|
|
2078
|
+
|
|
2079
|
+
Assert.AreEqual(ProcessStatus.Completed, result.Status);
|
|
2080
|
+
Assert.AreEqual(200, result.TotalAmount);
|
|
2081
|
+
Assert.IsNotNull(result.Confirmation);
|
|
2082
|
+
}
|
|
2083
|
+
}
|
|
2084
|
+
```
|
|
2085
|
+
|
|
2086
|
+
### E2E Testing Example
|
|
2087
|
+
|
|
2088
|
+
```typescript
|
|
2089
|
+
// Playwright E2E Test Example
|
|
2090
|
+
import { test, expect } from '@playwright/test';
|
|
2091
|
+
|
|
2092
|
+
test.describe('User Registration Journey', () => {
|
|
2093
|
+
test('should complete registration and login successfully', async ({ page }) => {
|
|
2094
|
+
// Step 1: Navigate to registration page
|
|
2095
|
+
await page.goto('/register');
|
|
2096
|
+
|
|
2097
|
+
// Step 2: Fill registration form
|
|
2098
|
+
await page.fill('[data-testid="email"]', 'newuser@example.com');
|
|
2099
|
+
await page.fill('[data-testid="password"]', 'SecurePass123!');
|
|
2100
|
+
await page.fill('[data-testid="confirm-password"]', 'SecurePass123!');
|
|
2101
|
+
await page.click('[data-testid="register-button"]');
|
|
2102
|
+
|
|
2103
|
+
// Step 3: Verify registration success
|
|
2104
|
+
await expect(page.locator('[data-testid="success-message"]'))
|
|
2105
|
+
.toContainText('Registration successful');
|
|
2106
|
+
|
|
2107
|
+
// Step 4: Login with new account
|
|
2108
|
+
await page.goto('/login');
|
|
2109
|
+
await page.fill('[data-testid="email"]', 'newuser@example.com');
|
|
2110
|
+
await page.fill('[data-testid="password"]', 'SecurePass123!');
|
|
2111
|
+
await page.click('[data-testid="login-button"]');
|
|
2112
|
+
|
|
2113
|
+
// Step 5: Verify login success and dashboard redirect
|
|
2114
|
+
await expect(page).toHaveURL('/dashboard');
|
|
2115
|
+
await expect(page.locator('[data-testid="welcome-message"]'))
|
|
2116
|
+
.toContainText('Welcome, newuser@example.com');
|
|
2117
|
+
});
|
|
2118
|
+
});
|
|
2119
|
+
```
|
|
2120
|
+
|
|
2121
|
+
### Containerized Testing Example
|
|
2122
|
+
|
|
2123
|
+
```csharp
|
|
2124
|
+
// C# Example with Testcontainers
|
|
2125
|
+
public class DatabaseIntegrationTests : IAsyncLifetime
|
|
2126
|
+
{
|
|
2127
|
+
private readonly PostgreSqlContainer _postgres = new PostgreSqlBuilder()
|
|
2128
|
+
.WithImage("postgres:15")
|
|
2129
|
+
.Build();
|
|
2130
|
+
|
|
2131
|
+
public async Task InitializeAsync()
|
|
2132
|
+
{
|
|
2133
|
+
await _postgres.StartAsync();
|
|
2134
|
+
}
|
|
2135
|
+
|
|
2136
|
+
public async Task DisposeAsync()
|
|
2137
|
+
{
|
|
2138
|
+
await _postgres.DisposeAsync();
|
|
2139
|
+
}
|
|
2140
|
+
|
|
2141
|
+
[Fact]
|
|
2142
|
+
public async Task Should_Connect_To_Database()
|
|
2143
|
+
{
|
|
2144
|
+
var connectionString = _postgres.GetConnectionString();
|
|
2145
|
+
// Use connectionString for tests
|
|
2146
|
+
}
|
|
2147
|
+
}
|
|
2148
|
+
```
|
|
2149
|
+
|
|
2150
|
+
```python
|
|
2151
|
+
# Python Example with Testcontainers
|
|
2152
|
+
import pytest
|
|
2153
|
+
from testcontainers.postgres import PostgresContainer
|
|
2154
|
+
|
|
2155
|
+
@pytest.fixture(scope="module")
|
|
2156
|
+
def postgres_container():
|
|
2157
|
+
with PostgresContainer("postgres:15") as postgres:
|
|
2158
|
+
yield postgres
|
|
2159
|
+
|
|
2160
|
+
def test_database_connection(postgres_container):
|
|
2161
|
+
connection_url = postgres_container.get_connection_url()
|
|
2162
|
+
# Use connection_url for tests
|
|
2163
|
+
```
|
|
2164
|
+
|
|
2165
|
+
```yaml
|
|
2166
|
+
# docker-compose.test.yml
|
|
2167
|
+
version: '3.8'
|
|
2168
|
+
services:
|
|
2169
|
+
app:
|
|
2170
|
+
build: .
|
|
2171
|
+
depends_on:
|
|
2172
|
+
- db
|
|
2173
|
+
- redis
|
|
2174
|
+
- rabbitmq
|
|
2175
|
+
environment:
|
|
2176
|
+
- DATABASE_URL=postgres://test:test@db:5432/testdb
|
|
2177
|
+
- REDIS_URL=redis://redis:6379
|
|
2178
|
+
- RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672
|
|
2179
|
+
|
|
2180
|
+
db:
|
|
2181
|
+
image: postgres:15
|
|
2182
|
+
environment:
|
|
2183
|
+
POSTGRES_USER: test
|
|
2184
|
+
POSTGRES_PASSWORD: test
|
|
2185
|
+
POSTGRES_DB: testdb
|
|
2186
|
+
|
|
2187
|
+
redis:
|
|
2188
|
+
image: redis:7-alpine
|
|
2189
|
+
|
|
2190
|
+
rabbitmq:
|
|
2191
|
+
image: rabbitmq:3-management
|
|
2192
|
+
```
|
|
2193
|
+
|
|
2194
|
+
---
|
|
2195
|
+
|
|
2196
|
+
## Quick Reference Card
|
|
2197
|
+
|
|
2198
|
+
```
|
|
2199
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
2200
|
+
│ Reference Standards │
|
|
2201
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2202
|
+
│ • ISTQB CTFL v4.0 - Testing certification │
|
|
2203
|
+
│ • ISO/IEC/IEEE 29119 - Testing standards │
|
|
2204
|
+
│ • SWEBOK v4.0 - Software engineering knowledge │
|
|
2205
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2206
|
+
│ Testing Fundamentals (SWEBOK) │
|
|
2207
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2208
|
+
│ Error → Fault → Failure (cause chain) │
|
|
2209
|
+
│ Oracle Problem: How to know expected result? │
|
|
2210
|
+
│ Limitation: Testing proves presence, not absence of bugs │
|
|
2211
|
+
│ Testability: Controllability + Observability + Isolability │
|
|
2212
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2213
|
+
│ Testing Framework Options │
|
|
2214
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2215
|
+
│ ISTQB: UT → IT/SIT → ST → AT/UAT (Enterprise/Compliance) │
|
|
2216
|
+
│ Industry: UT (70%) → IT (20%) → E2E (10%) (Agile/DevOps) │
|
|
2217
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2218
|
+
│ Testing Levels Summary │
|
|
2219
|
+
├──────────┬──────────────────────────────────────────────────┤
|
|
2220
|
+
│ UT │ Single unit, isolated, mocked deps, < 100ms │
|
|
2221
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2222
|
+
│ IT/SIT │ Component integration, real DB, 1-10 sec │
|
|
2223
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2224
|
+
│ ST │ Full system, requirement-based (ISTQB only) │
|
|
2225
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2226
|
+
│ E2E │ User journeys, UI to DB, critical paths only │
|
|
2227
|
+
├──────────┼──────────────────────────────────────────────────┤
|
|
2228
|
+
│ AT/UAT │ Business validation by end users (ISTQB only) │
|
|
2229
|
+
├──────────┴──────────────────────────────────────────────────┤
|
|
2230
|
+
│ Test Types (ISTQB) │
|
|
2231
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2232
|
+
│ Functional: Verify system behavior (black-box) │
|
|
2233
|
+
│ Non-Functional: Performance, security, usability │
|
|
2234
|
+
│ Structural: Code coverage, white-box techniques │
|
|
2235
|
+
│ Change-Related: Confirmation & regression testing │
|
|
2236
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2237
|
+
│ Test Design Techniques │
|
|
2238
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2239
|
+
│ Black-Box: EP, BVA, Decision Table, State Transition │
|
|
2240
|
+
│ Pairwise Testing, Use Case Testing │
|
|
2241
|
+
│ White-Box: Statement, Branch, Condition, MC/DC │
|
|
2242
|
+
│ Data Flow (All-Defs, All-Uses) │
|
|
2243
|
+
│ Experience: Exploratory, Error Guessing, Checklists │
|
|
2244
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2245
|
+
│ Coverage Targets │
|
|
2246
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2247
|
+
│ Line: 70% min / 85% recommended │
|
|
2248
|
+
│ Branch: 60% min / 80% recommended │
|
|
2249
|
+
│ Function: 80% min / 90% recommended │
|
|
2250
|
+
│ Mutation Score: ≥ 80% for critical code │
|
|
2251
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2252
|
+
│ Specialized Testing Types │
|
|
2253
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2254
|
+
│ Contract: API compatibility for microservices │
|
|
2255
|
+
│ Mutation: Test quality via code mutations │
|
|
2256
|
+
│ Chaos: Resilience via failure injection │
|
|
2257
|
+
│ Property-Based: Random inputs based on properties │
|
|
2258
|
+
│ Visual Regression: UI screenshot comparison │
|
|
2259
|
+
│ Accessibility: WCAG compliance validation │
|
|
2260
|
+
│ Installation: Deployment verification │
|
|
2261
|
+
│ Recovery: Failure recovery (RTO/RPO) │
|
|
2262
|
+
│ Configuration: Multi-config testing │
|
|
2263
|
+
│ Usability: User experience (SUS, NPS) │
|
|
2264
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2265
|
+
│ Test-Related Measures │
|
|
2266
|
+
├─────────────────────────────────────────────────────────────┤
|
|
2267
|
+
│ Fault Density = Faults / KLOC (target: < 1) │
|
|
2268
|
+
│ Defect Leakage = Prod defects / Total defects (< 5%) │
|
|
2269
|
+
│ Fault Seeding: Insert known faults to measure detection │
|
|
2270
|
+
│ Reliability Growth: Track defect rate over time │
|
|
2271
|
+
└─────────────────────────────────────────────────────────────┘
|
|
2272
|
+
```
|
|
2273
|
+
|
|
2274
|
+
---
|
|
2275
|
+
|
|
2276
|
+
## Related Standards
|
|
2277
|
+
|
|
2278
|
+
- [Testing Standards](../../core/testing-standards.md) - 给 AI agent 的可执行规则
|
|
2279
|
+
- [Testing Pyramid](./testing-pyramid.md) - 详细的金字塔比例
|
|
2280
|
+
- [Test-Driven Development](../../core/test-driven-development.md) - TDD/BDD/ATDD 方法论
|
|
2281
|
+
- [Test Completeness Dimensions](../../core/test-completeness-dimensions.md) - 8 维度测试覆盖
|
|
2282
|
+
- [Code Review Checklist](../../core/code-review-checklist.md)
|
|
2283
|
+
|
|
2284
|
+
---
|
|
2285
|
+
|
|
2286
|
+
## Version History
|
|
2287
|
+
|
|
2288
|
+
| 版本 | 日期 | 变更 |
|
|
2289
|
+
|---------|------|---------|
|
|
2290
|
+
| 1.0.0 | 2026-01-29 | 初版建立:从 core/testing-standards.md 抽取教学内容,包含 Testing Fundamentals、Static Testing、Test Design Techniques、Experience-Based Testing、Risk-Based Testing、Specialized Testing、Test-Related Measures、Defect Management、Test Process Management、Code Examples 及 Quick Reference Card。 |
|
|
2291
|
+
|
|
2292
|
+
---
|
|
2293
|
+
|
|
2294
|
+
## License
|
|
2295
|
+
|
|
2296
|
+
本文档以 [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) 授权发布。
|
|
2297
|
+
|
|
2298
|
+
**来源**: [universal-dev-standards](https://github.com/AsiaOstrich/universal-dev-standards)
|