universal-dev-standards 5.5.0 → 5.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/ai/options/testing/integration-testing.ai.yaml +2 -2
- package/bundled/ai/options/testing/unit-testing.ai.yaml +2 -2
- package/bundled/ai/standards/browser-compatibility-standards.ai.yaml +63 -0
- package/bundled/ai/standards/contract-testing-standards.ai.yaml +62 -0
- package/bundled/ai/standards/cross-flow-regression.ai.yaml +61 -0
- package/bundled/ai/standards/full-coverage-testing.ai.yaml +192 -0
- package/bundled/ai/standards/release-readiness-gate.ai.yaml +77 -0
- package/bundled/ai/standards/testing.ai.yaml +20 -13
- package/bundled/core/accessibility-standards.md +58 -0
- package/bundled/core/branch-completion.md +4 -0
- package/bundled/core/browser-compatibility-standards.md +220 -0
- package/bundled/core/checkin-standards.md +1 -0
- package/bundled/core/contract-testing-standards.md +182 -0
- package/bundled/core/cross-flow-regression.md +190 -0
- package/bundled/core/flow-based-testing.md +135 -2
- package/bundled/core/full-coverage-testing.md +183 -0
- package/bundled/core/performance-standards.md +65 -0
- package/bundled/core/release-quality-manifest.md +56 -10
- package/bundled/core/release-readiness-gate.md +184 -0
- package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
- package/bundled/locales/zh-CN/README.md +1 -1
- package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
- package/bundled/locales/zh-TW/README.md +1 -1
- package/bundled/locales/zh-TW/core/browser-compatibility-standards.md +11 -0
- package/bundled/locales/zh-TW/core/contract-testing-standards.md +11 -0
- package/bundled/locales/zh-TW/core/cross-flow-regression.md +11 -0
- package/bundled/locales/zh-TW/core/release-readiness-gate.md +11 -0
- package/package.json +6 -6
- package/src/commands/check.js +43 -0
- package/src/commands/init.js +2 -1
- package/src/commands/update.js +10 -0
- package/standards-registry.json +60 -4
|
@@ -17,7 +17,7 @@ characteristics:
|
|
|
17
17
|
scope: Multiple components working together
|
|
18
18
|
speed: "<1 second per test"
|
|
19
19
|
isolation: Partial (may use real dependencies)
|
|
20
|
-
|
|
20
|
+
coverage_policy: "Behavior-completeness ratchet (XSPEC-178) — no fixed % floor. See full-coverage-testing.ai.yaml."
|
|
21
21
|
|
|
22
22
|
rules:
|
|
23
23
|
- id: test-integration-points
|
|
@@ -86,4 +86,4 @@ tools:
|
|
|
86
86
|
- MockServer
|
|
87
87
|
- nock (Node.js)
|
|
88
88
|
|
|
89
|
-
coverage_target: "
|
|
89
|
+
coverage_target: "All critical paths AND integration points — not just happy path. Ratchet CI enforced. (XSPEC-178)"
|
|
@@ -11,7 +11,7 @@ characteristics:
|
|
|
11
11
|
scope: Single function or method
|
|
12
12
|
speed: "<100ms per test"
|
|
13
13
|
isolation: Complete (no external dependencies)
|
|
14
|
-
|
|
14
|
+
coverage_policy: "Behavior-completeness ratchet (XSPEC-178) — no fixed % floor. See full-coverage-testing.ai.yaml."
|
|
15
15
|
|
|
16
16
|
rules:
|
|
17
17
|
- id: test-one-thing
|
|
@@ -71,7 +71,7 @@ when_not_to_write:
|
|
|
71
71
|
- Testing trivial getters/setters
|
|
72
72
|
- Testing third-party libraries
|
|
73
73
|
|
|
74
|
-
coverage_target: "80%
|
|
74
|
+
coverage_target: "Behavior-completeness: happy/edge/error path per public function. Ratchet CI — coverage can only increase. (XSPEC-178 — deprecated fixed 80% threshold)"
|
|
75
75
|
|
|
76
76
|
tools:
|
|
77
77
|
javascript: [Jest, Vitest, Mocha]
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Browser Compatibility Standards - AI Optimized
|
|
2
|
+
# Source: core/browser-compatibility-standards.md
|
|
3
|
+
|
|
4
|
+
id: browser-compatibility-standards
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-05"
|
|
8
|
+
source: core/browser-compatibility-standards.md
|
|
9
|
+
description: Browser/device support matrix, Playwright automation, and release gate for frontend projects; N/A for CLI/backend
|
|
10
|
+
|
|
11
|
+
requirements:
|
|
12
|
+
REQ-1:
|
|
13
|
+
id: REQ-BC-001
|
|
14
|
+
title: Explicit Support Matrix
|
|
15
|
+
rule: >
|
|
16
|
+
Frontend projects MUST declare an explicit browser support matrix in
|
|
17
|
+
.browserslistrc and in Playwright project config. At minimum define Tier-1
|
|
18
|
+
(full support, release blocking) and Tier-2 (partial, WARN) browsers.
|
|
19
|
+
Tier-1 default: Chrome/Firefox/Safari/Edge latest + latest-1; iOS Safari
|
|
20
|
+
latest + latest-1; Android Chrome latest.
|
|
21
|
+
rationale: >
|
|
22
|
+
Implicit "supports all browsers" is untestable; an explicit matrix defines
|
|
23
|
+
scope for both engineering and QA.
|
|
24
|
+
|
|
25
|
+
REQ-2:
|
|
26
|
+
id: REQ-BC-002
|
|
27
|
+
title: Viewport Coverage
|
|
28
|
+
rule: >
|
|
29
|
+
Automated tests MUST cover at minimum three viewport widths: 360px (mobile),
|
|
30
|
+
768px (tablet), 1280px (desktop). No critical flow must be broken at any
|
|
31
|
+
Tier-1 browser × viewport combination.
|
|
32
|
+
rationale: >
|
|
33
|
+
Missing mobile viewport tests produce broken UX for the majority of
|
|
34
|
+
mobile users who represent 50%+ of web traffic globally.
|
|
35
|
+
|
|
36
|
+
REQ-3:
|
|
37
|
+
id: REQ-BC-003
|
|
38
|
+
title: Real iOS Device Testing
|
|
39
|
+
rule: >
|
|
40
|
+
For release candidates, Safari must be tested on real iOS devices (not
|
|
41
|
+
simulators only). WebKit on iOS Simulator diverges from real device WebKit
|
|
42
|
+
in rendering and API behaviour.
|
|
43
|
+
rationale: >
|
|
44
|
+
Simulator-only iOS testing misses a significant class of WebKit rendering
|
|
45
|
+
bugs that only manifest on real hardware.
|
|
46
|
+
|
|
47
|
+
REQ-4:
|
|
48
|
+
id: REQ-BC-004
|
|
49
|
+
title: Release Gate
|
|
50
|
+
rule: >
|
|
51
|
+
Tier-1 browser matrix pass rate MUST be 100% (no test failure = release block).
|
|
52
|
+
Tier-2 must be ≥ 95% (WARN if below). This is Dimension 9 (Tier-3) in
|
|
53
|
+
release-readiness-gate.md. Mark N/A for CLI, backend API, and mobile native projects.
|
|
54
|
+
rationale: >
|
|
55
|
+
100% Tier-1 requirement is appropriate because unsupported behavior in
|
|
56
|
+
a declared Tier-1 browser is a support commitment violation.
|
|
57
|
+
|
|
58
|
+
quick_reference:
|
|
59
|
+
tier1_default: "Chrome, Firefox, Safari, Edge (latest + latest-1); iOS Safari; Android Chrome latest"
|
|
60
|
+
viewport_minimum: "360px, 768px, 1280px"
|
|
61
|
+
automation: "Playwright matrix config; BrowserStack/Sauce/LambdaTest for cross-OS"
|
|
62
|
+
na_when: "CLI-only, backend API, mobile native (non-web)"
|
|
63
|
+
release_readiness_dimension: "Dim 9, Tier-3"
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# Contract Testing Standards - AI Optimized
|
|
2
|
+
# Source: core/contract-testing-standards.md
|
|
3
|
+
|
|
4
|
+
id: contract-testing-standards
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-05"
|
|
8
|
+
source: core/contract-testing-standards.md
|
|
9
|
+
description: Consumer-driven contract testing standard with release gate integration; applies to projects with API consumers
|
|
10
|
+
|
|
11
|
+
requirements:
|
|
12
|
+
REQ-1:
|
|
13
|
+
id: REQ-CT-001
|
|
14
|
+
title: Consumer-Driven Flow
|
|
15
|
+
rule: >
|
|
16
|
+
For projects with API consumers, contracts MUST be published by consumers
|
|
17
|
+
to a Pact Broker (or equivalent). Providers MUST verify all active consumer
|
|
18
|
+
contracts in CI. The `can-i-deploy` command MUST return exit 0 before
|
|
19
|
+
any provider deployment.
|
|
20
|
+
rationale: >
|
|
21
|
+
Provider-only contract tests miss the most common class of contract bug:
|
|
22
|
+
the provider ships a change it considers non-breaking, but consumers assumed
|
|
23
|
+
different response structure.
|
|
24
|
+
|
|
25
|
+
REQ-2:
|
|
26
|
+
id: REQ-CT-002
|
|
27
|
+
title: Schema Matchers Not Literals
|
|
28
|
+
rule: >
|
|
29
|
+
Consumer contracts MUST use schema matchers (like(), eachLike(), integer(),
|
|
30
|
+
string(), email()) rather than exact literal values. Only assert fields
|
|
31
|
+
the consumer actually uses.
|
|
32
|
+
rationale: >
|
|
33
|
+
Literal value matching causes brittle contracts that break on realistic data
|
|
34
|
+
variation without indicating any real compatibility issue.
|
|
35
|
+
|
|
36
|
+
REQ-3:
|
|
37
|
+
id: REQ-CT-003
|
|
38
|
+
title: Backward Compatibility Window
|
|
39
|
+
rule: >
|
|
40
|
+
Minor releases MUST maintain N-1 consumer contract compatibility.
|
|
41
|
+
Major releases require a deprecation period: notify consumers, keep old
|
|
42
|
+
contract active until all consumers migrate, then archive.
|
|
43
|
+
rationale: >
|
|
44
|
+
Consumers cannot always upgrade in lockstep; the N-1 window allows them
|
|
45
|
+
one release cycle to migrate without service disruption.
|
|
46
|
+
|
|
47
|
+
REQ-4:
|
|
48
|
+
id: REQ-CT-004
|
|
49
|
+
title: Release Gate
|
|
50
|
+
rule: >
|
|
51
|
+
All active consumer contracts MUST be green before deploying a provider
|
|
52
|
+
to production. This is Dimension 4 (Tier-3) in release-readiness-gate.md.
|
|
53
|
+
Mark N/A when the project has no API consumers.
|
|
54
|
+
rationale: >
|
|
55
|
+
A red consumer contract means a deployed provider will break that consumer
|
|
56
|
+
in production — this is a hard release blocker, not a warning.
|
|
57
|
+
|
|
58
|
+
quick_reference:
|
|
59
|
+
applies_when: "project has API consumers (service-to-service, frontend-to-backend, public API)"
|
|
60
|
+
gate: "can-i-deploy exit 0 before provider deploy; N-1 compat required"
|
|
61
|
+
broker_tag_convention: "main, production, <feature-branch>"
|
|
62
|
+
release_readiness_dimension: "Dim 4, Tier-3"
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# Cross-Flow Regression Standards - AI Optimized
|
|
2
|
+
# Source: core/cross-flow-regression.md
|
|
3
|
+
|
|
4
|
+
id: cross-flow-regression
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-05"
|
|
8
|
+
source: core/cross-flow-regression.md
|
|
9
|
+
description: Cross-flow regression testing complementing per-flow Multi-Gate; verifies inter-flow state correctness and CUJ pass rates
|
|
10
|
+
|
|
11
|
+
requirements:
|
|
12
|
+
REQ-1:
|
|
13
|
+
id: REQ-CFR-001
|
|
14
|
+
title: CUJ Suite Required per Release
|
|
15
|
+
rule: >
|
|
16
|
+
Every release candidate MUST run the full Critical User Journey (CUJ) regression
|
|
17
|
+
suite. CUJs are end-to-end sequences spanning ≥ 2 flows that share state or are
|
|
18
|
+
commonly executed sequentially. CUJ pass rate MUST be ≥ 95%.
|
|
19
|
+
rationale: >
|
|
20
|
+
Per-flow Multi-Gate testing proves intra-flow correctness; cross-flow tests catch
|
|
21
|
+
state contamination bugs that only manifest when flow outputs become the next
|
|
22
|
+
flow's inputs.
|
|
23
|
+
|
|
24
|
+
REQ-2:
|
|
25
|
+
id: REQ-CFR-002
|
|
26
|
+
title: Retrigger on Flow Spec Change
|
|
27
|
+
rule: >
|
|
28
|
+
When any flow's §2.4 Decision Points or Terminal States are modified, the full
|
|
29
|
+
CUJ regression suite MUST re-run as a pre-merge check — not just tests for the
|
|
30
|
+
changed flow. This prevents silent regressions in downstream flows.
|
|
31
|
+
rationale: >
|
|
32
|
+
A change to one flow's terminal states can produce different intermediate state
|
|
33
|
+
that breaks a previously passing downstream flow.
|
|
34
|
+
|
|
35
|
+
REQ-3:
|
|
36
|
+
id: REQ-CFR-003
|
|
37
|
+
title: Sequential State Threading
|
|
38
|
+
rule: >
|
|
39
|
+
CUJ tests MUST use sequential state threading: a shared `ctx` object accumulates
|
|
40
|
+
state across flows within a single test describe block. State MUST NOT be reset
|
|
41
|
+
between steps of the same CUJ. Each CUJ MUST have its own isolated ctx.
|
|
42
|
+
rationale: >
|
|
43
|
+
Resetting state between steps masks cross-flow contamination; independent ctx
|
|
44
|
+
per CUJ prevents CUJ-to-CUJ interference from masking bugs.
|
|
45
|
+
|
|
46
|
+
REQ-4:
|
|
47
|
+
id: REQ-CFR-004
|
|
48
|
+
title: Release Gate
|
|
49
|
+
rule: >
|
|
50
|
+
CUJ pass rate ≥ 95% = PASS. 90–95% = WARN (document specific failed CUJs).
|
|
51
|
+
< 90% or any business-critical flow combo failure = FAIL (release blocked).
|
|
52
|
+
This is Dimension 6 (Tier-2) in release-readiness-gate.md.
|
|
53
|
+
rationale: >
|
|
54
|
+
90% threshold accounts for environment-specific flakiness while ensuring
|
|
55
|
+
the vast majority of user journeys are verified end-to-end.
|
|
56
|
+
|
|
57
|
+
quick_reference:
|
|
58
|
+
boundary_with_flow_based_testing: "flow-based-testing = intra-flow; cross-flow-regression = inter-flow"
|
|
59
|
+
cuj_trigger: "every release candidate + any flow §2.4 change"
|
|
60
|
+
pass_rate_target: "≥ 95% overall; 100% for business-critical combos"
|
|
61
|
+
release_readiness_dimension: "Dim 6, Tier-2"
|
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
# Full Coverage Testing Standards - AI Optimized
|
|
2
|
+
# XSPEC-178: Replaces pyramid threshold model with behavior-completeness paradigm
|
|
3
|
+
# Source: core/full-coverage-testing.md
|
|
4
|
+
|
|
5
|
+
standard:
|
|
6
|
+
id: full-coverage-testing
|
|
7
|
+
name: Full Coverage Testing Standards
|
|
8
|
+
description: Behavior-completeness full coverage paradigm replacing pyramid thresholds. Enforces anti-fake-test rules, STUB marker protocol, ratchet CI, and @ac traceability.
|
|
9
|
+
|
|
10
|
+
meta:
|
|
11
|
+
version: "1.0.0"
|
|
12
|
+
updated: "2026-05-06"
|
|
13
|
+
source: core/full-coverage-testing.md
|
|
14
|
+
replaces: "testing pyramid thresholds (UT≥80%/IT≥70%/E2E happy-path-only)"
|
|
15
|
+
xspec: "XSPEC-178"
|
|
16
|
+
description: AI-era full coverage paradigm — cost of writing tests equals cost of writing code, so there is no reason to set lower thresholds for any test layer.
|
|
17
|
+
|
|
18
|
+
rationale: |
|
|
19
|
+
Traditional pyramid thresholds (UT≥80%, IT≥70%) assumed tests were expensive to write.
|
|
20
|
+
AI code generation eliminates this cost differential — code and tests are produced at the
|
|
21
|
+
same speed. Therefore: maximize coverage everywhere, with behavior-completeness as the
|
|
22
|
+
measure, not a percentage floor.
|
|
23
|
+
|
|
24
|
+
coverage_model:
|
|
25
|
+
type: behavior_completeness
|
|
26
|
+
description: Every public function must have tests for all three behavioral paths
|
|
27
|
+
required_paths:
|
|
28
|
+
- id: happy_path
|
|
29
|
+
description: Normal input produces correct output
|
|
30
|
+
example: "calculateDiscount(100, 0.1) → 90"
|
|
31
|
+
- id: edge_case
|
|
32
|
+
description: Boundary values do not cause unexpected errors
|
|
33
|
+
example: "calculateDiscount(0, 1.0) → 0 without throwing"
|
|
34
|
+
- id: error_path
|
|
35
|
+
description: Invalid input raises clear error or returns error state
|
|
36
|
+
example: "calculateDiscount(-1, 2.0) → throws ArgumentError"
|
|
37
|
+
|
|
38
|
+
ratchet_policy:
|
|
39
|
+
enabled: true
|
|
40
|
+
description: Coverage can only increase, never decrease. PR that regresses coverage is blocked.
|
|
41
|
+
mechanism:
|
|
42
|
+
- Store baseline in .coverage-baseline.json on main branch
|
|
43
|
+
- Every PR compares current coverage against baseline
|
|
44
|
+
- Regression = PR blocked, not merged
|
|
45
|
+
- Improvement = new baseline set on merge
|
|
46
|
+
note: No fixed floor threshold. The current coverage IS the threshold.
|
|
47
|
+
|
|
48
|
+
rules:
|
|
49
|
+
# ── Behavior completeness ──────────────────────────────────────
|
|
50
|
+
- id: three-path-coverage
|
|
51
|
+
trigger: writing tests for any public function
|
|
52
|
+
instruction: |
|
|
53
|
+
Write at least three tests per public function:
|
|
54
|
+
1. happy_path — normal inputs, expected output
|
|
55
|
+
2. edge_case — boundary values (zero, max, empty, null)
|
|
56
|
+
3. error_path — invalid inputs trigger explicit error or error state
|
|
57
|
+
priority: required
|
|
58
|
+
|
|
59
|
+
- id: ac-traceability
|
|
60
|
+
trigger: writing any test
|
|
61
|
+
instruction: |
|
|
62
|
+
Tag each test with the Acceptance Criteria it covers using JSDoc @ac tag.
|
|
63
|
+
Format: /** @ac AC-ID */ above the test function.
|
|
64
|
+
If no AC maps to this test, use: /** @ac UNTRACED */
|
|
65
|
+
priority: recommended
|
|
66
|
+
example: |
|
|
67
|
+
/**
|
|
68
|
+
* @ac AC-US03-2
|
|
69
|
+
*/
|
|
70
|
+
it('should block PR when coverage regresses', () => { ... })
|
|
71
|
+
|
|
72
|
+
# ── Anti-fake test rules ───────────────────────────────────────
|
|
73
|
+
- id: no-tautology-assertions
|
|
74
|
+
trigger: writing any test assertion
|
|
75
|
+
instruction: |
|
|
76
|
+
FORBIDDEN: Tautology assertions that always pass regardless of behavior.
|
|
77
|
+
These add false coverage without verifying anything.
|
|
78
|
+
priority: required
|
|
79
|
+
forbidden_patterns:
|
|
80
|
+
- "expect(true).toBe(true)"
|
|
81
|
+
- "expect(false).toBe(false)"
|
|
82
|
+
- "expect(result).toBeDefined() // without specific value"
|
|
83
|
+
- "expect(result).not.toBeNull() // without specific value"
|
|
84
|
+
required_instead: "expect(result).toBe(<specific expected value>)"
|
|
85
|
+
|
|
86
|
+
- id: no-mock-business-logic
|
|
87
|
+
trigger: deciding what to mock
|
|
88
|
+
instruction: |
|
|
89
|
+
FORBIDDEN: Mocking core business logic or your own service functions.
|
|
90
|
+
Mocking your own code means the business logic is never actually executed.
|
|
91
|
+
priority: required
|
|
92
|
+
allowed_to_mock:
|
|
93
|
+
- External HTTP APIs (payment gateways, OAuth providers)
|
|
94
|
+
- Hardware interfaces (sensors, GPIO, Docker daemon)
|
|
95
|
+
- Third-party SDKs with no test mode
|
|
96
|
+
- File system (use tmpdir, not mock)
|
|
97
|
+
forbidden_to_mock:
|
|
98
|
+
- Core business calculation functions
|
|
99
|
+
- Your own service layer methods
|
|
100
|
+
- Database queries (use in-memory SQLite instead)
|
|
101
|
+
- Your own utility functions
|
|
102
|
+
|
|
103
|
+
- id: mock-must-have-reason
|
|
104
|
+
trigger: writing any mock/stub/spy
|
|
105
|
+
instruction: |
|
|
106
|
+
Every jest.mock(), vi.mock(), jest.spyOn(), or sinon.stub() must be preceded
|
|
107
|
+
by a comment explaining WHY this dependency must be mocked.
|
|
108
|
+
Format: // MOCK: <reason — what external dependency and why it cannot be real>
|
|
109
|
+
priority: required
|
|
110
|
+
example: |
|
|
111
|
+
// MOCK: External Stripe payment API — no sandbox available in CI
|
|
112
|
+
jest.mock('./payment-gateway', () => ({ charge: jest.fn().mockResolvedValue({ id: 'ch_test' }) }))
|
|
113
|
+
|
|
114
|
+
# ── STUB marker protocol ───────────────────────────────────────
|
|
115
|
+
- id: stub-marker-required
|
|
116
|
+
trigger: writing any temporary/placeholder implementation
|
|
117
|
+
instruction: |
|
|
118
|
+
ALL temporary implementations, placeholder functions, and fake returns
|
|
119
|
+
MUST be marked with the standard STUB marker.
|
|
120
|
+
Format: // WARNING: STUB — Remove before UAT
|
|
121
|
+
This marker is scanned by pre-push hooks and deploy.sh.
|
|
122
|
+
STUB markers block pushes to main and deployments to UAT/production.
|
|
123
|
+
priority: required
|
|
124
|
+
example: |
|
|
125
|
+
// WARNING: STUB — Remove before UAT
|
|
126
|
+
async function validatePayment(card: Card): Promise<boolean> {
|
|
127
|
+
return true; // Always approve — replace with real Stripe call
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
- id: coverage-exempt-format
|
|
131
|
+
trigger: dealing with genuinely untestable external dependencies
|
|
132
|
+
instruction: |
|
|
133
|
+
If a dependency truly cannot be tested (hardware, live external API with no sandbox),
|
|
134
|
+
declare an explicit exemption with a mandatory reason.
|
|
135
|
+
Format: // COVERAGE_EXEMPT: <specific reason why real test is impossible>
|
|
136
|
+
This exemption is respected by STUB scanners and will not trigger blocking.
|
|
137
|
+
The reason MUST be non-empty and specific.
|
|
138
|
+
priority: required
|
|
139
|
+
example: |
|
|
140
|
+
// COVERAGE_EXEMPT: Hardware temperature sensor — no simulation available in CI
|
|
141
|
+
async function readTemperature(): Promise<number> {
|
|
142
|
+
return hardwareSensor.read();
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
- id: no-silent-stub
|
|
146
|
+
trigger: reviewing code before commit
|
|
147
|
+
instruction: |
|
|
148
|
+
Stubbed/placeholder code without // WARNING: STUB is a violation.
|
|
149
|
+
Common patterns to watch for: functions that always return hardcoded values,
|
|
150
|
+
empty function bodies that should have logic, TODO comments without STUB marker.
|
|
151
|
+
These will eventually reach production undetected.
|
|
152
|
+
priority: required
|
|
153
|
+
|
|
154
|
+
deployment_gates:
|
|
155
|
+
pre_push_to_main:
|
|
156
|
+
action: block
|
|
157
|
+
trigger: "// WARNING: STUB" marker found in src/
|
|
158
|
+
message: "[STUB-BLOCK] STUB markers detected. Push to main rejected."
|
|
159
|
+
deploy_to_uat:
|
|
160
|
+
action: block
|
|
161
|
+
trigger: "// WARNING: STUB" marker found in src/
|
|
162
|
+
message: "[DEPLOY-BLOCK] STUB markers detected. UAT deployment aborted."
|
|
163
|
+
deploy_to_production:
|
|
164
|
+
action: block
|
|
165
|
+
trigger: "// WARNING: STUB" marker found in src/
|
|
166
|
+
message: "[CRITICAL] Production deployment with STUB markers is strictly prohibited."
|
|
167
|
+
deploy_to_staging:
|
|
168
|
+
action: warn
|
|
169
|
+
trigger: "// WARNING: STUB" marker found in src/
|
|
170
|
+
message: "[STUB-WARN] Deploying with STUB markers to staging. NOT permitted in UAT/production."
|
|
171
|
+
feature_branch_push:
|
|
172
|
+
action: warn
|
|
173
|
+
trigger: "// WARNING: STUB" marker found in src/
|
|
174
|
+
message: "[STUB-WARN] STUB markers found. Must remove before merging to main."
|
|
175
|
+
|
|
176
|
+
migration_from_pyramid:
|
|
177
|
+
deprecated:
|
|
178
|
+
- "UT ≥ 80% coverage threshold"
|
|
179
|
+
- "IT ≥ 70% coverage threshold"
|
|
180
|
+
- "E2E happy-path-only requirement"
|
|
181
|
+
replaced_by:
|
|
182
|
+
- "Behavior-completeness: happy/edge/error per public function"
|
|
183
|
+
- "Ratchet CI: coverage can only increase"
|
|
184
|
+
- "Anti-fake rules: no tautology, no business-logic mocks"
|
|
185
|
+
- "STUB protocol: deployment gates on all environments"
|
|
186
|
+
|
|
187
|
+
physical_spec:
|
|
188
|
+
type: custom_script
|
|
189
|
+
validator:
|
|
190
|
+
command: >
|
|
191
|
+
test -f scripts/check-stubs.sh && test -f scripts/check-anti-fake-tests.sh
|
|
192
|
+
rule: "xspec178_enforcement_scripts_present"
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Release Readiness Gate Standards - AI Optimized
|
|
2
|
+
# Source: core/release-readiness-gate.md
|
|
3
|
+
|
|
4
|
+
id: release-readiness-gate
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-05"
|
|
8
|
+
source: core/release-readiness-gate.md
|
|
9
|
+
description: Single aggregated release gate covering 16 quality dimensions with tiered sign-off template and RQM integration
|
|
10
|
+
|
|
11
|
+
requirements:
|
|
12
|
+
REQ-1:
|
|
13
|
+
id: REQ-RRG-001
|
|
14
|
+
title: 16-Dimension Coverage
|
|
15
|
+
rule: >
|
|
16
|
+
Every production release MUST evaluate all 16 quality dimensions defined in
|
|
17
|
+
core/release-readiness-gate.md. Tier-1 dimensions block release if FAIL.
|
|
18
|
+
Tier-2 dimensions require documented rationale if WARN. Tier-3 dimensions
|
|
19
|
+
require rationale if N/A.
|
|
20
|
+
rationale: >
|
|
21
|
+
Without explicit multi-dimension coverage, teams pass individual gate checks
|
|
22
|
+
but ship with unverified quality dimensions, creating systematic blind spots.
|
|
23
|
+
|
|
24
|
+
REQ-2:
|
|
25
|
+
id: REQ-RRG-002
|
|
26
|
+
title: Release Readiness Sign-off
|
|
27
|
+
rule: >
|
|
28
|
+
A Release Readiness Sign-off document MUST be created from the template in
|
|
29
|
+
core/release-readiness-gate.md for every release tag. It must be stored at
|
|
30
|
+
.release-readiness/<version>.md. The Overall Decision field must be explicitly
|
|
31
|
+
set to GO or NO-GO by a named release owner.
|
|
32
|
+
rationale: >
|
|
33
|
+
Anonymous or implicit GO decisions remove accountability; the sign-off creates
|
|
34
|
+
a named, dated, auditable record of the go/no-go decision and its evidence.
|
|
35
|
+
|
|
36
|
+
REQ-3:
|
|
37
|
+
id: REQ-RRG-003
|
|
38
|
+
title: Tier-1 Hard Block
|
|
39
|
+
rule: >
|
|
40
|
+
ANY Tier-1 dimension at FAIL status MUST block production deployment.
|
|
41
|
+
Tier-1 dimensions are: Security (Dim 2), DB Migration (Dim 5), Operational
|
|
42
|
+
Readiness (Dim 7), Rollback/DR (Dim 13), Production Smoke (Dim 14).
|
|
43
|
+
rationale: >
|
|
44
|
+
Tier-1 dimensions represent existential risks: security vulnerabilities,
|
|
45
|
+
broken rollback, misconfigured monitoring. No business justification
|
|
46
|
+
overrides a Tier-1 FAIL.
|
|
47
|
+
|
|
48
|
+
REQ-4:
|
|
49
|
+
id: REQ-RRG-004
|
|
50
|
+
title: RQM Alignment
|
|
51
|
+
rule: >
|
|
52
|
+
The machine-readable Release Quality Manifest (release-quality-manifest.md)
|
|
53
|
+
MUST include entries for all automated dimensions (a11y_critical, contract_drift,
|
|
54
|
+
cross_flow_cuj_pass_rate, browser_tier1_pass_rate, capacity_headroom_cpu_pct,
|
|
55
|
+
smoke_pass_rate, flow_gate_report). The RQM overall field must be PASS or WARN
|
|
56
|
+
(never FAIL) before deployment.
|
|
57
|
+
rationale: >
|
|
58
|
+
Human sign-off and machine manifest are complementary; the manifest enables
|
|
59
|
+
automated enforcement while the sign-off provides human accountability.
|
|
60
|
+
|
|
61
|
+
REQ-5:
|
|
62
|
+
id: REQ-RRG-005
|
|
63
|
+
title: Incremental Collection
|
|
64
|
+
rule: >
|
|
65
|
+
Release Readiness Sign-off evidence MUST be collected incrementally throughout
|
|
66
|
+
the release cycle (Gate 0 at PRD, Gate 3 pre-UAT, Gate 4 post-UAT). Creating
|
|
67
|
+
the sign-off on the day of deployment is an anti-pattern.
|
|
68
|
+
rationale: >
|
|
69
|
+
Last-minute sign-offs are rubber stamps; evidence collected late cannot
|
|
70
|
+
be acted upon without delaying the release.
|
|
71
|
+
|
|
72
|
+
quick_reference:
|
|
73
|
+
tier_1_dimensions: "Security, DB Migration, Operational Readiness, Rollback/DR, Production Smoke"
|
|
74
|
+
tier_2_dimensions: "Performance, a11y, Cross-flow Regression, i18n, Docs, Feature Flags, Multi-Gate Flow"
|
|
75
|
+
tier_3_dimensions: "Contract Testing, Browser Compat, Capacity, Compliance/Privacy"
|
|
76
|
+
sign_off_location: ".release-readiness/<version>.md"
|
|
77
|
+
rqm_integration: "flow_gate_report.json → release-quality-manifest.yaml field flow_gate_report"
|
|
@@ -4,19 +4,26 @@
|
|
|
4
4
|
standard:
|
|
5
5
|
id: testing
|
|
6
6
|
name: Testing Standards
|
|
7
|
-
description: Testing
|
|
7
|
+
description: Testing structure, FIRST principles, AAA pattern, and framework options. For coverage policy, see full-coverage-testing (XSPEC-178).
|
|
8
8
|
guidelines:
|
|
9
|
-
- "Follow 70/20/7/3 testing pyramid (UT/IT/ST/E2E)"
|
|
10
9
|
- "Use FIRST principles for unit tests"
|
|
11
10
|
- "Structure tests with Arrange-Act-Assert"
|
|
12
11
|
- "Use meaningful test names"
|
|
12
|
+
- "See full-coverage-testing.ai.yaml for coverage policy (XSPEC-178)"
|
|
13
13
|
|
|
14
14
|
meta:
|
|
15
|
-
version: "2.
|
|
16
|
-
updated: "2026-
|
|
15
|
+
version: "2.1.0"
|
|
16
|
+
updated: "2026-05-06"
|
|
17
17
|
source: core/testing-standards.md
|
|
18
18
|
guide: skills/testing-guide/testing-theory.md
|
|
19
|
-
description: Testing
|
|
19
|
+
description: Testing structure and principles. Coverage policy moved to full-coverage-testing (XSPEC-178).
|
|
20
|
+
|
|
21
|
+
deprecated_rules:
|
|
22
|
+
- id: pyramid-thresholds
|
|
23
|
+
description: "70/20/7/3 pyramid ratio and UT≥80%/IT≥70%/E2E happy-path-only thresholds"
|
|
24
|
+
reason: "AI-era paradigm shift: test generation cost equals code generation cost, so no layer should have a lower threshold. Replaced by behavior-completeness ratchet model."
|
|
25
|
+
since: "5.5.0"
|
|
26
|
+
replaced_by: "full-coverage-testing.ai.yaml (XSPEC-178)"
|
|
20
27
|
|
|
21
28
|
framework_options:
|
|
22
29
|
decision_point: Select testing framework based on project type
|
|
@@ -29,25 +36,25 @@ standard:
|
|
|
29
36
|
best_for: Agile development, CI/CD optimization, rapid iteration
|
|
30
37
|
|
|
31
38
|
pyramid:
|
|
32
|
-
ratio: "
|
|
33
|
-
note: "
|
|
39
|
+
ratio: "DEPRECATED — see full-coverage-testing.ai.yaml"
|
|
40
|
+
note: "The 70/20/7/3 ratio is deprecated as of v5.5.0 (XSPEC-178). Coverage is now governed by behavior-completeness ratchet, not per-layer percentages. Layer names remain valid as organizational categories, not threshold buckets."
|
|
34
41
|
levels:
|
|
35
42
|
- name: Unit Tests (UT)
|
|
36
|
-
percentage: 70
|
|
37
43
|
scope: Single function/method
|
|
38
44
|
speed: Very fast (<100ms)
|
|
45
|
+
coverage_policy: "See full-coverage-testing.ai.yaml — ratchet model, no fixed % floor"
|
|
39
46
|
- name: Integration Tests (IT)
|
|
40
|
-
percentage: 20
|
|
41
47
|
scope: Multiple components interacting
|
|
42
48
|
speed: Fast (<1s)
|
|
49
|
+
coverage_policy: "See full-coverage-testing.ai.yaml — ratchet model, no fixed % floor"
|
|
43
50
|
- name: System Tests (ST)
|
|
44
|
-
percentage: 7
|
|
45
51
|
scope: Complete subsystem with stubbed external dependencies
|
|
46
52
|
speed: Medium (<10s)
|
|
53
|
+
coverage_policy: "See full-coverage-testing.ai.yaml — ratchet model, no fixed % floor"
|
|
47
54
|
- name: E2E Tests
|
|
48
|
-
percentage: 3
|
|
49
55
|
scope: Full user flows across entire system
|
|
50
56
|
speed: Slow (>10s)
|
|
57
|
+
coverage_policy: "All AC must have E2E coverage — not just happy path"
|
|
51
58
|
|
|
52
59
|
options:
|
|
53
60
|
testing_framework:
|
|
@@ -81,9 +88,9 @@ standard:
|
|
|
81
88
|
best_for: Microservices, API integrations, distributed systems
|
|
82
89
|
|
|
83
90
|
rules:
|
|
84
|
-
- id: follow-
|
|
91
|
+
- id: follow-full-coverage
|
|
85
92
|
trigger: planning test strategy
|
|
86
|
-
instruction: Follow
|
|
93
|
+
instruction: Follow behavior-completeness model — every public function needs happy/edge/error path tests. Ratchet CI ensures coverage never decreases. See full-coverage-testing.ai.yaml (XSPEC-178).
|
|
87
94
|
priority: required
|
|
88
95
|
- id: first-principles
|
|
89
96
|
trigger: writing tests
|
|
@@ -817,12 +817,70 @@ Before Release:
|
|
|
817
817
|
|
|
818
818
|
---
|
|
819
819
|
|
|
820
|
+
## Release-Blocking Threshold
|
|
821
|
+
|
|
822
|
+
This section defines the **minimum a11y gate** that must pass before production release (Dimension 3 in `release-readiness-gate.md`, Tier-2).
|
|
823
|
+
|
|
824
|
+
### Automated Gate (CI — axe-core / Pa11y)
|
|
825
|
+
|
|
826
|
+
| Severity | Hard Minimum | Warn Band |
|
|
827
|
+
|----------|-------------|-----------|
|
|
828
|
+
| critical | 0 (release blocker) | — |
|
|
829
|
+
| serious | ≤ project-configured threshold (default: 0) | +1–2 → WARN |
|
|
830
|
+
| moderate | advisory, not blocking | — |
|
|
831
|
+
| minor | advisory, not blocking | — |
|
|
832
|
+
|
|
833
|
+
Run axe-core against every critical user-flow screen before release:
|
|
834
|
+
|
|
835
|
+
```bash
|
|
836
|
+
npx axe-cli <url> --tags wcag2a,wcag2aa --exit # non-zero exit if critical/serious
|
|
837
|
+
```
|
|
838
|
+
|
|
839
|
+
CI must fail the build if `critical > 0` or `serious > configured-threshold`.
|
|
840
|
+
|
|
841
|
+
### Conformance Level by Project Type
|
|
842
|
+
|
|
843
|
+
| Project Type | Minimum Conformance | Notes |
|
|
844
|
+
|-------------|-------------------|-------|
|
|
845
|
+
| General web / SaaS | WCAG 2.1 Level AA | Default |
|
|
846
|
+
| Financial, Healthcare, Government | WCAG 2.1 Level AA + Section 508 | Legally mandated |
|
|
847
|
+
| Aspirational / premium brand | WCAG 2.2 Level AA | Recommended |
|
|
848
|
+
|
|
849
|
+
### Keyboard Navigation Gate (Manual, Pre-UAT)
|
|
850
|
+
|
|
851
|
+
All critical user flows (as defined in requirement-template §2.4) must be fully operable by keyboard alone:
|
|
852
|
+
- Tab / Shift+Tab: navigate all interactive elements
|
|
853
|
+
- Enter / Space: activate controls
|
|
854
|
+
- Escape: dismiss modals / overlays
|
|
855
|
+
- Arrow keys: navigate menus, tabs, tree views
|
|
856
|
+
|
|
857
|
+
Gate: 100% of critical flow steps reachable by keyboard; no focus trap.
|
|
858
|
+
|
|
859
|
+
### Screen Reader Spot Check (Manual, Pre-UAT)
|
|
860
|
+
|
|
861
|
+
Minimum: 1 critical user flow tested on each of:
|
|
862
|
+
- **Windows**: NVDA + Chrome
|
|
863
|
+
- **macOS**: VoiceOver + Safari
|
|
864
|
+
|
|
865
|
+
Pass criterion: all steps completable without visual reference; all form errors announced; all dynamic content changes announced.
|
|
866
|
+
|
|
867
|
+
### Release Sign-off Evidence
|
|
868
|
+
|
|
869
|
+
The a11y gate contributes to the Release Readiness Sign-off as:
|
|
870
|
+
|
|
871
|
+
```
|
|
872
|
+
| 3 | Accessibility | PASS | axe report: 0 critical / 0 serious; keyboard pass; SR spot-check VoiceOver OK | QA Lead |
|
|
873
|
+
```
|
|
874
|
+
|
|
875
|
+
---
|
|
876
|
+
|
|
820
877
|
## Related Standards
|
|
821
878
|
|
|
822
879
|
- [Documentation Writing Standards](documentation-writing-standards.md) - Accessible documentation
|
|
823
880
|
- [Code Review Checklist](code-review-checklist.md) - Accessibility review
|
|
824
881
|
- [Testing Standards](testing-standards.md) - Accessibility testing
|
|
825
882
|
- [Security Standards](security-standards.md) - Authentication accessibility
|
|
883
|
+
- [Release Readiness Gate](release-readiness-gate.md) - Dimension 3: a11y release gate
|
|
826
884
|
|
|
827
885
|
---
|
|
828
886
|
|
|
@@ -156,3 +156,7 @@ If the branch has an associated worktree, the worktree must be cleaned up as par
|
|
|
156
156
|
- **Superpowers**: [finishing-a-development-branch](https://github.com/obra/superpowers) (MIT)
|
|
157
157
|
- **Git Flow**: Branch lifecycle management
|
|
158
158
|
- **Trunk-Based Development**: Short-lived branches pattern
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
> **Branch completion ≠ Release readiness.** A completed branch means all its own quality gates passed. It does not mean the product is ready to ship. See `release-readiness-gate.md` for the 16-dimension release gate that must pass before production deployment.
|