universal-dev-standards 5.5.0 → 5.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/bundled/ai/options/testing/integration-testing.ai.yaml +2 -2
  2. package/bundled/ai/options/testing/unit-testing.ai.yaml +2 -2
  3. package/bundled/ai/standards/browser-compatibility-standards.ai.yaml +63 -0
  4. package/bundled/ai/standards/contract-testing-standards.ai.yaml +62 -0
  5. package/bundled/ai/standards/cross-flow-regression.ai.yaml +61 -0
  6. package/bundled/ai/standards/full-coverage-testing.ai.yaml +192 -0
  7. package/bundled/ai/standards/release-readiness-gate.ai.yaml +77 -0
  8. package/bundled/ai/standards/testing.ai.yaml +20 -13
  9. package/bundled/core/accessibility-standards.md +58 -0
  10. package/bundled/core/branch-completion.md +4 -0
  11. package/bundled/core/browser-compatibility-standards.md +220 -0
  12. package/bundled/core/checkin-standards.md +1 -0
  13. package/bundled/core/contract-testing-standards.md +182 -0
  14. package/bundled/core/cross-flow-regression.md +190 -0
  15. package/bundled/core/flow-based-testing.md +135 -2
  16. package/bundled/core/full-coverage-testing.md +183 -0
  17. package/bundled/core/performance-standards.md +65 -0
  18. package/bundled/core/release-quality-manifest.md +56 -10
  19. package/bundled/core/release-readiness-gate.md +184 -0
  20. package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
  21. package/bundled/locales/zh-CN/README.md +1 -1
  22. package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
  23. package/bundled/locales/zh-TW/README.md +1 -1
  24. package/bundled/locales/zh-TW/core/browser-compatibility-standards.md +11 -0
  25. package/bundled/locales/zh-TW/core/contract-testing-standards.md +11 -0
  26. package/bundled/locales/zh-TW/core/cross-flow-regression.md +11 -0
  27. package/bundled/locales/zh-TW/core/release-readiness-gate.md +11 -0
  28. package/package.json +6 -6
  29. package/src/commands/check.js +43 -0
  30. package/src/commands/init.js +2 -1
  31. package/src/commands/update.js +10 -0
  32. package/standards-registry.json +60 -4
@@ -17,7 +17,7 @@ characteristics:
17
17
  scope: Multiple components working together
18
18
  speed: "<1 second per test"
19
19
  isolation: Partial (may use real dependencies)
20
- pyramid_percentage: 20%
20
+ coverage_policy: "Behavior-completeness ratchet (XSPEC-178) — no fixed % floor. See full-coverage-testing.ai.yaml."
21
21
 
22
22
  rules:
23
23
  - id: test-integration-points
@@ -86,4 +86,4 @@ tools:
86
86
  - MockServer
87
87
  - nock (Node.js)
88
88
 
89
- coverage_target: "Critical paths and integration points"
89
+ coverage_target: "All critical paths AND integration points — not just happy path. Ratchet CI enforced. (XSPEC-178)"
@@ -11,7 +11,7 @@ characteristics:
11
11
  scope: Single function or method
12
12
  speed: "<100ms per test"
13
13
  isolation: Complete (no external dependencies)
14
- pyramid_percentage: 70%
14
+ coverage_policy: "Behavior-completeness ratchet (XSPEC-178) — no fixed % floor. See full-coverage-testing.ai.yaml."
15
15
 
16
16
  rules:
17
17
  - id: test-one-thing
@@ -71,7 +71,7 @@ when_not_to_write:
71
71
  - Testing trivial getters/setters
72
72
  - Testing third-party libraries
73
73
 
74
- coverage_target: "80% of business logic"
74
+ coverage_target: "Behavior-completeness: happy/edge/error path per public function. Ratchet CI — coverage can only increase. (XSPEC-178 — deprecated fixed 80% threshold)"
75
75
 
76
76
  tools:
77
77
  javascript: [Jest, Vitest, Mocha]
@@ -0,0 +1,63 @@
1
+ # Browser Compatibility Standards - AI Optimized
2
+ # Source: core/browser-compatibility-standards.md
3
+
4
+ id: browser-compatibility-standards
5
+ meta:
6
+ version: "1.0.0"
7
+ updated: "2026-05-05"
8
+ source: core/browser-compatibility-standards.md
9
+ description: Browser/device support matrix, Playwright automation, and release gate for frontend projects; N/A for CLI/backend
10
+
11
+ requirements:
12
+ REQ-1:
13
+ id: REQ-BC-001
14
+ title: Explicit Support Matrix
15
+ rule: >
16
+ Frontend projects MUST declare an explicit browser support matrix in
17
+ .browserslistrc and in Playwright project config. At minimum define Tier-1
18
+ (full support, release blocking) and Tier-2 (partial, WARN) browsers.
19
+ Tier-1 default: Chrome/Firefox/Safari/Edge latest + latest-1; iOS Safari
20
+ latest + latest-1; Android Chrome latest.
21
+ rationale: >
22
+ Implicit "supports all browsers" is untestable; an explicit matrix defines
23
+ scope for both engineering and QA.
24
+
25
+ REQ-2:
26
+ id: REQ-BC-002
27
+ title: Viewport Coverage
28
+ rule: >
29
+ Automated tests MUST cover at minimum three viewport widths: 360px (mobile),
30
+ 768px (tablet), 1280px (desktop). No critical flow must be broken at any
31
+ Tier-1 browser × viewport combination.
32
+ rationale: >
33
+ Missing mobile viewport tests produce broken UX for the majority of
34
+ mobile users who represent 50%+ of web traffic globally.
35
+
36
+ REQ-3:
37
+ id: REQ-BC-003
38
+ title: Real iOS Device Testing
39
+ rule: >
40
+ For release candidates, Safari must be tested on real iOS devices (not
41
+ simulators only). WebKit on iOS Simulator diverges from real device WebKit
42
+ in rendering and API behaviour.
43
+ rationale: >
44
+ Simulator-only iOS testing misses a significant class of WebKit rendering
45
+ bugs that only manifest on real hardware.
46
+
47
+ REQ-4:
48
+ id: REQ-BC-004
49
+ title: Release Gate
50
+ rule: >
51
+ Tier-1 browser matrix pass rate MUST be 100% (no test failure = release block).
52
+ Tier-2 must be ≥ 95% (WARN if below). This is Dimension 9 (Tier-3) in
53
+ release-readiness-gate.md. Mark N/A for CLI, backend API, and mobile native projects.
54
+ rationale: >
55
+ 100% Tier-1 requirement is appropriate because unsupported behavior in
56
+ a declared Tier-1 browser is a support commitment violation.
57
+
58
+ quick_reference:
59
+ tier1_default: "Chrome, Firefox, Safari, Edge (latest + latest-1); iOS Safari; Android Chrome latest"
60
+ viewport_minimum: "360px, 768px, 1280px"
61
+ automation: "Playwright matrix config; BrowserStack/Sauce/LambdaTest for cross-OS"
62
+ na_when: "CLI-only, backend API, mobile native (non-web)"
63
+ release_readiness_dimension: "Dim 9, Tier-3"
@@ -0,0 +1,62 @@
1
+ # Contract Testing Standards - AI Optimized
2
+ # Source: core/contract-testing-standards.md
3
+
4
+ id: contract-testing-standards
5
+ meta:
6
+ version: "1.0.0"
7
+ updated: "2026-05-05"
8
+ source: core/contract-testing-standards.md
9
+ description: Consumer-driven contract testing standard with release gate integration; applies to projects with API consumers
10
+
11
+ requirements:
12
+ REQ-1:
13
+ id: REQ-CT-001
14
+ title: Consumer-Driven Flow
15
+ rule: >
16
+ For projects with API consumers, contracts MUST be published by consumers
17
+ to a Pact Broker (or equivalent). Providers MUST verify all active consumer
18
+ contracts in CI. The `can-i-deploy` command MUST return exit 0 before
19
+ any provider deployment.
20
+ rationale: >
21
+ Provider-only contract tests miss the most common class of contract bug:
22
+ the provider ships a change it considers non-breaking, but consumers assumed
23
+ different response structure.
24
+
25
+ REQ-2:
26
+ id: REQ-CT-002
27
+ title: Schema Matchers Not Literals
28
+ rule: >
29
+ Consumer contracts MUST use schema matchers (like(), eachLike(), integer(),
30
+ string(), email()) rather than exact literal values. Only assert fields
31
+ the consumer actually uses.
32
+ rationale: >
33
+ Literal value matching causes brittle contracts that break on realistic data
34
+ variation without indicating any real compatibility issue.
35
+
36
+ REQ-3:
37
+ id: REQ-CT-003
38
+ title: Backward Compatibility Window
39
+ rule: >
40
+ Minor releases MUST maintain N-1 consumer contract compatibility.
41
+ Major releases require a deprecation period: notify consumers, keep old
42
+ contract active until all consumers migrate, then archive.
43
+ rationale: >
44
+ Consumers cannot always upgrade in lockstep; the N-1 window allows them
45
+ one release cycle to migrate without service disruption.
46
+
47
+ REQ-4:
48
+ id: REQ-CT-004
49
+ title: Release Gate
50
+ rule: >
51
+ All active consumer contracts MUST be green before deploying a provider
52
+ to production. This is Dimension 4 (Tier-3) in release-readiness-gate.md.
53
+ Mark N/A when the project has no API consumers.
54
+ rationale: >
55
+ A red consumer contract means a deployed provider will break that consumer
56
+ in production — this is a hard release blocker, not a warning.
57
+
58
+ quick_reference:
59
+ applies_when: "project has API consumers (service-to-service, frontend-to-backend, public API)"
60
+ gate: "can-i-deploy exit 0 before provider deploy; N-1 compat required"
61
+ broker_tag_convention: "main, production, <feature-branch>"
62
+ release_readiness_dimension: "Dim 4, Tier-3"
@@ -0,0 +1,61 @@
1
+ # Cross-Flow Regression Standards - AI Optimized
2
+ # Source: core/cross-flow-regression.md
3
+
4
+ id: cross-flow-regression
5
+ meta:
6
+ version: "1.0.0"
7
+ updated: "2026-05-05"
8
+ source: core/cross-flow-regression.md
9
+ description: Cross-flow regression testing complementing per-flow Multi-Gate; verifies inter-flow state correctness and CUJ pass rates
10
+
11
+ requirements:
12
+ REQ-1:
13
+ id: REQ-CFR-001
14
+ title: CUJ Suite Required per Release
15
+ rule: >
16
+ Every release candidate MUST run the full Critical User Journey (CUJ) regression
17
+ suite. CUJs are end-to-end sequences spanning ≥ 2 flows that share state or are
18
+ commonly executed sequentially. CUJ pass rate MUST be ≥ 95%.
19
+ rationale: >
20
+ Per-flow Multi-Gate testing proves intra-flow correctness; cross-flow tests catch
21
+ state contamination bugs that only manifest when flow outputs become the next
22
+ flow's inputs.
23
+
24
+ REQ-2:
25
+ id: REQ-CFR-002
26
+ title: Retrigger on Flow Spec Change
27
+ rule: >
28
+ When any flow's §2.4 Decision Points or Terminal States are modified, the full
29
+ CUJ regression suite MUST re-run as a pre-merge check — not just tests for the
30
+ changed flow. This prevents silent regressions in downstream flows.
31
+ rationale: >
32
+ A change to one flow's terminal states can produce different intermediate state
33
+ that breaks a previously passing downstream flow.
34
+
35
+ REQ-3:
36
+ id: REQ-CFR-003
37
+ title: Sequential State Threading
38
+ rule: >
39
+ CUJ tests MUST use sequential state threading: a shared `ctx` object accumulates
40
+ state across flows within a single test describe block. State MUST NOT be reset
41
+ between steps of the same CUJ. Each CUJ MUST have its own isolated ctx.
42
+ rationale: >
43
+ Resetting state between steps masks cross-flow contamination; independent ctx
44
+ per CUJ prevents CUJ-to-CUJ interference from masking bugs.
45
+
46
+ REQ-4:
47
+ id: REQ-CFR-004
48
+ title: Release Gate
49
+ rule: >
50
+ CUJ pass rate ≥ 95% = PASS. 90–95% = WARN (document specific failed CUJs).
51
+ < 90% or any business-critical flow combo failure = FAIL (release blocked).
52
+ This is Dimension 6 (Tier-2) in release-readiness-gate.md.
53
+ rationale: >
54
+ 90% threshold accounts for environment-specific flakiness while ensuring
55
+ the vast majority of user journeys are verified end-to-end.
56
+
57
+ quick_reference:
58
+ boundary_with_flow_based_testing: "flow-based-testing = intra-flow; cross-flow-regression = inter-flow"
59
+ cuj_trigger: "every release candidate + any flow §2.4 change"
60
+ pass_rate_target: "≥ 95% overall; 100% for business-critical combos"
61
+ release_readiness_dimension: "Dim 6, Tier-2"
@@ -0,0 +1,192 @@
1
+ # Full Coverage Testing Standards - AI Optimized
2
+ # XSPEC-178: Replaces pyramid threshold model with behavior-completeness paradigm
3
+ # Source: core/full-coverage-testing.md
4
+
5
+ standard:
6
+ id: full-coverage-testing
7
+ name: Full Coverage Testing Standards
8
+ description: Behavior-completeness full coverage paradigm replacing pyramid thresholds. Enforces anti-fake-test rules, STUB marker protocol, ratchet CI, and @ac traceability.
9
+
10
+ meta:
11
+ version: "1.0.0"
12
+ updated: "2026-05-06"
13
+ source: core/full-coverage-testing.md
14
+ replaces: "testing pyramid thresholds (UT≥80%/IT≥70%/E2E happy-path-only)"
15
+ xspec: "XSPEC-178"
16
+ description: AI-era full coverage paradigm — cost of writing tests equals cost of writing code, so there is no reason to set lower thresholds for any test layer.
17
+
18
+ rationale: |
19
+ Traditional pyramid thresholds (UT≥80%, IT≥70%) assumed tests were expensive to write.
20
+ AI code generation eliminates this cost differential — code and tests are produced at the
21
+ same speed. Therefore: maximize coverage everywhere, with behavior-completeness as the
22
+ measure, not a percentage floor.
23
+
24
+ coverage_model:
25
+ type: behavior_completeness
26
+ description: Every public function must have tests for all three behavioral paths
27
+ required_paths:
28
+ - id: happy_path
29
+ description: Normal input produces correct output
30
+ example: "calculateDiscount(100, 0.1) → 90"
31
+ - id: edge_case
32
+ description: Boundary values do not cause unexpected errors
33
+ example: "calculateDiscount(0, 1.0) → 0 without throwing"
34
+ - id: error_path
35
+ description: Invalid input raises clear error or returns error state
36
+ example: "calculateDiscount(-1, 2.0) → throws ArgumentError"
37
+
38
+ ratchet_policy:
39
+ enabled: true
40
+ description: Coverage can only increase, never decrease. PR that regresses coverage is blocked.
41
+ mechanism:
42
+ - Store baseline in .coverage-baseline.json on main branch
43
+ - Every PR compares current coverage against baseline
44
+ - Regression = PR blocked, not merged
45
+ - Improvement = new baseline set on merge
46
+ note: No fixed floor threshold. The current coverage IS the threshold.
47
+
48
+ rules:
49
+ # ── Behavior completeness ──────────────────────────────────────
50
+ - id: three-path-coverage
51
+ trigger: writing tests for any public function
52
+ instruction: |
53
+ Write at least three tests per public function:
54
+ 1. happy_path — normal inputs, expected output
55
+ 2. edge_case — boundary values (zero, max, empty, null)
56
+ 3. error_path — invalid inputs trigger explicit error or error state
57
+ priority: required
58
+
59
+ - id: ac-traceability
60
+ trigger: writing any test
61
+ instruction: |
62
+ Tag each test with the Acceptance Criteria it covers using JSDoc @ac tag.
63
+ Format: /** @ac AC-ID */ above the test function.
64
+ If no AC maps to this test, use: /** @ac UNTRACED */
65
+ priority: recommended
66
+ example: |
67
+ /**
68
+ * @ac AC-US03-2
69
+ */
70
+ it('should block PR when coverage regresses', () => { ... })
71
+
72
+ # ── Anti-fake test rules ───────────────────────────────────────
73
+ - id: no-tautology-assertions
74
+ trigger: writing any test assertion
75
+ instruction: |
76
+ FORBIDDEN: Tautology assertions that always pass regardless of behavior.
77
+ These add false coverage without verifying anything.
78
+ priority: required
79
+ forbidden_patterns:
80
+ - "expect(true).toBe(true)"
81
+ - "expect(false).toBe(false)"
82
+ - "expect(result).toBeDefined() // without specific value"
83
+ - "expect(result).not.toBeNull() // without specific value"
84
+ required_instead: "expect(result).toBe(<specific expected value>)"
85
+
86
+ - id: no-mock-business-logic
87
+ trigger: deciding what to mock
88
+ instruction: |
89
+ FORBIDDEN: Mocking core business logic or your own service functions.
90
+ Mocking your own code means the business logic is never actually executed.
91
+ priority: required
92
+ allowed_to_mock:
93
+ - External HTTP APIs (payment gateways, OAuth providers)
94
+ - Hardware interfaces (sensors, GPIO, Docker daemon)
95
+ - Third-party SDKs with no test mode
96
+ - File system (use tmpdir, not mock)
97
+ forbidden_to_mock:
98
+ - Core business calculation functions
99
+ - Your own service layer methods
100
+ - Database queries (use in-memory SQLite instead)
101
+ - Your own utility functions
102
+
103
+ - id: mock-must-have-reason
104
+ trigger: writing any mock/stub/spy
105
+ instruction: |
106
+ Every jest.mock(), vi.mock(), jest.spyOn(), or sinon.stub() must be preceded
107
+ by a comment explaining WHY this dependency must be mocked.
108
+ Format: // MOCK: <reason — what external dependency and why it cannot be real>
109
+ priority: required
110
+ example: |
111
+ // MOCK: External Stripe payment API — no sandbox available in CI
112
+ jest.mock('./payment-gateway', () => ({ charge: jest.fn().mockResolvedValue({ id: 'ch_test' }) }))
113
+
114
+ # ── STUB marker protocol ───────────────────────────────────────
115
+ - id: stub-marker-required
116
+ trigger: writing any temporary/placeholder implementation
117
+ instruction: |
118
+ ALL temporary implementations, placeholder functions, and fake returns
119
+ MUST be marked with the standard STUB marker.
120
+ Format: // WARNING: STUB — Remove before UAT
121
+ This marker is scanned by pre-push hooks and deploy.sh.
122
+ STUB markers block pushes to main and deployments to UAT/production.
123
+ priority: required
124
+ example: |
125
+ // WARNING: STUB — Remove before UAT
126
+ async function validatePayment(card: Card): Promise<boolean> {
127
+ return true; // Always approve — replace with real Stripe call
128
+ }
129
+
130
+ - id: coverage-exempt-format
131
+ trigger: dealing with genuinely untestable external dependencies
132
+ instruction: |
133
+ If a dependency truly cannot be tested (hardware, live external API with no sandbox),
134
+ declare an explicit exemption with a mandatory reason.
135
+ Format: // COVERAGE_EXEMPT: <specific reason why real test is impossible>
136
+ This exemption is respected by STUB scanners and will not trigger blocking.
137
+ The reason MUST be non-empty and specific.
138
+ priority: required
139
+ example: |
140
+ // COVERAGE_EXEMPT: Hardware temperature sensor — no simulation available in CI
141
+ async function readTemperature(): Promise<number> {
142
+ return hardwareSensor.read();
143
+ }
144
+
145
+ - id: no-silent-stub
146
+ trigger: reviewing code before commit
147
+ instruction: |
148
+ Stubbed/placeholder code without // WARNING: STUB is a violation.
149
+ Common patterns to watch for: functions that always return hardcoded values,
150
+ empty function bodies that should have logic, TODO comments without STUB marker.
151
+ These will eventually reach production undetected.
152
+ priority: required
153
+
154
+ deployment_gates:
155
+ pre_push_to_main:
156
+ action: block
157
+ trigger: "// WARNING: STUB" marker found in src/
158
+ message: "[STUB-BLOCK] STUB markers detected. Push to main rejected."
159
+ deploy_to_uat:
160
+ action: block
161
+ trigger: "// WARNING: STUB" marker found in src/
162
+ message: "[DEPLOY-BLOCK] STUB markers detected. UAT deployment aborted."
163
+ deploy_to_production:
164
+ action: block
165
+ trigger: "// WARNING: STUB" marker found in src/
166
+ message: "[CRITICAL] Production deployment with STUB markers is strictly prohibited."
167
+ deploy_to_staging:
168
+ action: warn
169
+ trigger: "// WARNING: STUB" marker found in src/
170
+ message: "[STUB-WARN] Deploying with STUB markers to staging. NOT permitted in UAT/production."
171
+ feature_branch_push:
172
+ action: warn
173
+ trigger: "// WARNING: STUB" marker found in src/
174
+ message: "[STUB-WARN] STUB markers found. Must remove before merging to main."
175
+
176
+ migration_from_pyramid:
177
+ deprecated:
178
+ - "UT ≥ 80% coverage threshold"
179
+ - "IT ≥ 70% coverage threshold"
180
+ - "E2E happy-path-only requirement"
181
+ replaced_by:
182
+ - "Behavior-completeness: happy/edge/error per public function"
183
+ - "Ratchet CI: coverage can only increase"
184
+ - "Anti-fake rules: no tautology, no business-logic mocks"
185
+ - "STUB protocol: deployment gates on all environments"
186
+
187
+ physical_spec:
188
+ type: custom_script
189
+ validator:
190
+ command: >
191
+ test -f scripts/check-stubs.sh && test -f scripts/check-anti-fake-tests.sh
192
+ rule: "xspec178_enforcement_scripts_present"
@@ -0,0 +1,77 @@
1
+ # Release Readiness Gate Standards - AI Optimized
2
+ # Source: core/release-readiness-gate.md
3
+
4
+ id: release-readiness-gate
5
+ meta:
6
+ version: "1.0.0"
7
+ updated: "2026-05-05"
8
+ source: core/release-readiness-gate.md
9
+ description: Single aggregated release gate covering 16 quality dimensions with tiered sign-off template and RQM integration
10
+
11
+ requirements:
12
+ REQ-1:
13
+ id: REQ-RRG-001
14
+ title: 16-Dimension Coverage
15
+ rule: >
16
+ Every production release MUST evaluate all 16 quality dimensions defined in
17
+ core/release-readiness-gate.md. Tier-1 dimensions block release if FAIL.
18
+ Tier-2 dimensions require documented rationale if WARN. Tier-3 dimensions
19
+ require rationale if N/A.
20
+ rationale: >
21
+ Without explicit multi-dimension coverage, teams pass individual gate checks
22
+ but ship with unverified quality dimensions, creating systematic blind spots.
23
+
24
+ REQ-2:
25
+ id: REQ-RRG-002
26
+ title: Release Readiness Sign-off
27
+ rule: >
28
+ A Release Readiness Sign-off document MUST be created from the template in
29
+ core/release-readiness-gate.md for every release tag. It must be stored at
30
+ .release-readiness/<version>.md. The Overall Decision field must be explicitly
31
+ set to GO or NO-GO by a named release owner.
32
+ rationale: >
33
+ Anonymous or implicit GO decisions remove accountability; the sign-off creates
34
+ a named, dated, auditable record of the go/no-go decision and its evidence.
35
+
36
+ REQ-3:
37
+ id: REQ-RRG-003
38
+ title: Tier-1 Hard Block
39
+ rule: >
40
+ ANY Tier-1 dimension at FAIL status MUST block production deployment.
41
+ Tier-1 dimensions are: Security (Dim 2), DB Migration (Dim 5), Operational
42
+ Readiness (Dim 7), Rollback/DR (Dim 13), Production Smoke (Dim 14).
43
+ rationale: >
44
+ Tier-1 dimensions represent existential risks: security vulnerabilities,
45
+ broken rollback, misconfigured monitoring. No business justification
46
+ overrides a Tier-1 FAIL.
47
+
48
+ REQ-4:
49
+ id: REQ-RRG-004
50
+ title: RQM Alignment
51
+ rule: >
52
+ The machine-readable Release Quality Manifest (release-quality-manifest.md)
53
+ MUST include entries for all automated dimensions (a11y_critical, contract_drift,
54
+ cross_flow_cuj_pass_rate, browser_tier1_pass_rate, capacity_headroom_cpu_pct,
55
+ smoke_pass_rate, flow_gate_report). The RQM overall field must be PASS or WARN
56
+ (never FAIL) before deployment.
57
+ rationale: >
58
+ Human sign-off and machine manifest are complementary; the manifest enables
59
+ automated enforcement while the sign-off provides human accountability.
60
+
61
+ REQ-5:
62
+ id: REQ-RRG-005
63
+ title: Incremental Collection
64
+ rule: >
65
+ Release Readiness Sign-off evidence MUST be collected incrementally throughout
66
+ the release cycle (Gate 0 at PRD, Gate 3 pre-UAT, Gate 4 post-UAT). Creating
67
+ the sign-off on the day of deployment is an anti-pattern.
68
+ rationale: >
69
+ Last-minute sign-offs are rubber stamps; evidence collected late cannot
70
+ be acted upon without delaying the release.
71
+
72
+ quick_reference:
73
+ tier_1_dimensions: "Security, DB Migration, Operational Readiness, Rollback/DR, Production Smoke"
74
+ tier_2_dimensions: "Performance, a11y, Cross-flow Regression, i18n, Docs, Feature Flags, Multi-Gate Flow"
75
+ tier_3_dimensions: "Contract Testing, Browser Compat, Capacity, Compliance/Privacy"
76
+ sign_off_location: ".release-readiness/<version>.md"
77
+ rqm_integration: "flow_gate_report.json → release-quality-manifest.yaml field flow_gate_report"
@@ -4,19 +4,26 @@
4
4
  standard:
5
5
  id: testing
6
6
  name: Testing Standards
7
- description: Testing pyramid and recommended coverage ratios with framework options
7
+ description: Testing structure, FIRST principles, AAA pattern, and framework options. For coverage policy, see full-coverage-testing (XSPEC-178).
8
8
  guidelines:
9
- - "Follow 70/20/7/3 testing pyramid (UT/IT/ST/E2E)"
10
9
  - "Use FIRST principles for unit tests"
11
10
  - "Structure tests with Arrange-Act-Assert"
12
11
  - "Use meaningful test names"
12
+ - "See full-coverage-testing.ai.yaml for coverage policy (XSPEC-178)"
13
13
 
14
14
  meta:
15
- version: "2.0.0"
16
- updated: "2026-02-04"
15
+ version: "2.1.0"
16
+ updated: "2026-05-06"
17
17
  source: core/testing-standards.md
18
18
  guide: skills/testing-guide/testing-theory.md
19
- description: Testing pyramid and recommended coverage ratios with framework options
19
+ description: Testing structure and principles. Coverage policy moved to full-coverage-testing (XSPEC-178).
20
+
21
+ deprecated_rules:
22
+ - id: pyramid-thresholds
23
+ description: "70/20/7/3 pyramid ratio and UT≥80%/IT≥70%/E2E happy-path-only thresholds"
24
+ reason: "AI-era paradigm shift: test generation cost equals code generation cost, so no layer should have a lower threshold. Replaced by behavior-completeness ratchet model."
25
+ since: "5.5.0"
26
+ replaced_by: "full-coverage-testing.ai.yaml (XSPEC-178)"
20
27
 
21
28
  framework_options:
22
29
  decision_point: Select testing framework based on project type
@@ -29,25 +36,25 @@ standard:
29
36
  best_for: Agile development, CI/CD optimization, rapid iteration
30
37
 
31
38
  pyramid:
32
- ratio: "70/20/7/3"
33
- note: "Recommended ratio based on Mike Cohn's test pyramid (empirical, not mandatory). ISTQB defines 4 levels: UT/IT/ST/E2E."
39
+ ratio: "DEPRECATED — see full-coverage-testing.ai.yaml"
40
+ note: "The 70/20/7/3 ratio is deprecated as of v5.5.0 (XSPEC-178). Coverage is now governed by behavior-completeness ratchet, not per-layer percentages. Layer names remain valid as organizational categories, not threshold buckets."
34
41
  levels:
35
42
  - name: Unit Tests (UT)
36
- percentage: 70
37
43
  scope: Single function/method
38
44
  speed: Very fast (<100ms)
45
+ coverage_policy: "See full-coverage-testing.ai.yaml — ratchet model, no fixed % floor"
39
46
  - name: Integration Tests (IT)
40
- percentage: 20
41
47
  scope: Multiple components interacting
42
48
  speed: Fast (<1s)
49
+ coverage_policy: "See full-coverage-testing.ai.yaml — ratchet model, no fixed % floor"
43
50
  - name: System Tests (ST)
44
- percentage: 7
45
51
  scope: Complete subsystem with stubbed external dependencies
46
52
  speed: Medium (<10s)
53
+ coverage_policy: "See full-coverage-testing.ai.yaml — ratchet model, no fixed % floor"
47
54
  - name: E2E Tests
48
- percentage: 3
49
55
  scope: Full user flows across entire system
50
56
  speed: Slow (>10s)
57
+ coverage_policy: "All AC must have E2E coverage — not just happy path"
51
58
 
52
59
  options:
53
60
  testing_framework:
@@ -81,9 +88,9 @@ standard:
81
88
  best_for: Microservices, API integrations, distributed systems
82
89
 
83
90
  rules:
84
- - id: follow-pyramid
91
+ - id: follow-full-coverage
85
92
  trigger: planning test strategy
86
- instruction: Follow 70/20/7/3 recommended ratio for UT/IT/ST/E2E (Mike Cohn empirical, not mandatory)
93
+ instruction: Follow behavior-completeness model every public function needs happy/edge/error path tests. Ratchet CI ensures coverage never decreases. See full-coverage-testing.ai.yaml (XSPEC-178).
87
94
  priority: required
88
95
  - id: first-principles
89
96
  trigger: writing tests
@@ -817,12 +817,70 @@ Before Release:
817
817
 
818
818
  ---
819
819
 
820
+ ## Release-Blocking Threshold
821
+
822
+ This section defines the **minimum a11y gate** that must pass before production release (Dimension 3 in `release-readiness-gate.md`, Tier-2).
823
+
824
+ ### Automated Gate (CI — axe-core / Pa11y)
825
+
826
+ | Severity | Hard Minimum | Warn Band |
827
+ |----------|-------------|-----------|
828
+ | critical | 0 (release blocker) | — |
829
+ | serious | ≤ project-configured threshold (default: 0) | +1–2 → WARN |
830
+ | moderate | advisory, not blocking | — |
831
+ | minor | advisory, not blocking | — |
832
+
833
+ Run axe-core against every critical user-flow screen before release:
834
+
835
+ ```bash
836
+ npx axe-cli <url> --tags wcag2a,wcag2aa --exit # non-zero exit if critical/serious
837
+ ```
838
+
839
+ CI must fail the build if `critical > 0` or `serious > configured-threshold`.
840
+
841
+ ### Conformance Level by Project Type
842
+
843
+ | Project Type | Minimum Conformance | Notes |
844
+ |-------------|-------------------|-------|
845
+ | General web / SaaS | WCAG 2.1 Level AA | Default |
846
+ | Financial, Healthcare, Government | WCAG 2.1 Level AA + Section 508 | Legally mandated |
847
+ | Aspirational / premium brand | WCAG 2.2 Level AA | Recommended |
848
+
849
+ ### Keyboard Navigation Gate (Manual, Pre-UAT)
850
+
851
+ All critical user flows (as defined in requirement-template §2.4) must be fully operable by keyboard alone:
852
+ - Tab / Shift+Tab: navigate all interactive elements
853
+ - Enter / Space: activate controls
854
+ - Escape: dismiss modals / overlays
855
+ - Arrow keys: navigate menus, tabs, tree views
856
+
857
+ Gate: 100% of critical flow steps reachable by keyboard; no focus trap.
858
+
859
+ ### Screen Reader Spot Check (Manual, Pre-UAT)
860
+
861
+ Minimum: 1 critical user flow tested on each of:
862
+ - **Windows**: NVDA + Chrome
863
+ - **macOS**: VoiceOver + Safari
864
+
865
+ Pass criterion: all steps completable without visual reference; all form errors announced; all dynamic content changes announced.
866
+
867
+ ### Release Sign-off Evidence
868
+
869
+ The a11y gate contributes to the Release Readiness Sign-off as:
870
+
871
+ ```
872
+ | 3 | Accessibility | PASS | axe report: 0 critical / 0 serious; keyboard pass; SR spot-check VoiceOver OK | QA Lead |
873
+ ```
874
+
875
+ ---
876
+
820
877
  ## Related Standards
821
878
 
822
879
  - [Documentation Writing Standards](documentation-writing-standards.md) - Accessible documentation
823
880
  - [Code Review Checklist](code-review-checklist.md) - Accessibility review
824
881
  - [Testing Standards](testing-standards.md) - Accessibility testing
825
882
  - [Security Standards](security-standards.md) - Authentication accessibility
883
+ - [Release Readiness Gate](release-readiness-gate.md) - Dimension 3: a11y release gate
826
884
 
827
885
  ---
828
886
 
@@ -156,3 +156,7 @@ If the branch has an associated worktree, the worktree must be cleaned up as par
156
156
  - **Superpowers**: [finishing-a-development-branch](https://github.com/obra/superpowers) (MIT)
157
157
  - **Git Flow**: Branch lifecycle management
158
158
  - **Trunk-Based Development**: Short-lived branches pattern
159
+
160
+ ---
161
+
162
+ > **Branch completion ≠ Release readiness.** A completed branch means all its own quality gates passed. It does not mean the product is ready to ship. See `release-readiness-gate.md` for the 16-dimension release gate that must pass before production deployment.