universal-dev-standards 5.5.0 → 5.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/ai/options/testing/integration-testing.ai.yaml +2 -2
- package/bundled/ai/options/testing/unit-testing.ai.yaml +2 -2
- package/bundled/ai/standards/browser-compatibility-standards.ai.yaml +63 -0
- package/bundled/ai/standards/contract-testing-standards.ai.yaml +62 -0
- package/bundled/ai/standards/cross-flow-regression.ai.yaml +61 -0
- package/bundled/ai/standards/full-coverage-testing.ai.yaml +192 -0
- package/bundled/ai/standards/release-readiness-gate.ai.yaml +77 -0
- package/bundled/ai/standards/testing.ai.yaml +20 -13
- package/bundled/core/accessibility-standards.md +58 -0
- package/bundled/core/branch-completion.md +4 -0
- package/bundled/core/browser-compatibility-standards.md +220 -0
- package/bundled/core/checkin-standards.md +1 -0
- package/bundled/core/contract-testing-standards.md +182 -0
- package/bundled/core/cross-flow-regression.md +190 -0
- package/bundled/core/flow-based-testing.md +135 -2
- package/bundled/core/full-coverage-testing.md +183 -0
- package/bundled/core/performance-standards.md +65 -0
- package/bundled/core/release-quality-manifest.md +56 -10
- package/bundled/core/release-readiness-gate.md +184 -0
- package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
- package/bundled/locales/zh-CN/README.md +1 -1
- package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
- package/bundled/locales/zh-TW/README.md +1 -1
- package/bundled/locales/zh-TW/core/browser-compatibility-standards.md +11 -0
- package/bundled/locales/zh-TW/core/contract-testing-standards.md +11 -0
- package/bundled/locales/zh-TW/core/cross-flow-regression.md +11 -0
- package/bundled/locales/zh-TW/core/release-readiness-gate.md +11 -0
- package/package.json +6 -6
- package/src/commands/check.js +43 -0
- package/src/commands/init.js +2 -1
- package/src/commands/update.js +10 -0
- package/standards-registry.json +60 -4
|
@@ -0,0 +1,220 @@
|
|
|
1
|
+
# Browser Compatibility Standards
|
|
2
|
+
|
|
3
|
+
> **Language**: English | [繁體中文](../locales/zh-TW/core/browser-compatibility-standards.md)
|
|
4
|
+
|
|
5
|
+
**Version**: 1.0.0
|
|
6
|
+
**Last Updated**: 2026-05-05
|
|
7
|
+
**Applicability**: Frontend projects (web apps, progressive web apps, web components)
|
|
8
|
+
**Scope**: universal
|
|
9
|
+
**Industry Standards**: Browserslist, W3C WebDriver, WebDriver BiDi
|
|
10
|
+
**References**: [caniuse.com](https://caniuse.com/), [Playwright browser support matrix](https://playwright.dev/docs/browsers)
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Purpose
|
|
15
|
+
|
|
16
|
+
This standard defines supported browser and device matrices, testing automation strategies, and the release gate for browser compatibility (Dimension 9 in `release-readiness-gate.md`, Tier-3).
|
|
17
|
+
|
|
18
|
+
Browser compatibility issues are among the most user-visible defects, yet they are systematically under-tested because teams assume "it works in Chrome." Without an explicit supported matrix and automated verification, regressions slip to production and affect large user segments.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Support Tier Definitions
|
|
23
|
+
|
|
24
|
+
| Tier | Definition | Release Gate |
|
|
25
|
+
|------|-----------|--------------|
|
|
26
|
+
| **Tier-1** (Supported) | Full feature parity + automated test coverage | 100% pass — blocks release if any test fails |
|
|
27
|
+
| **Tier-2** (Partial support) | Best-effort; major flows must work | ≥ 95% pass — WARN if below, FAIL if < 90% |
|
|
28
|
+
| **Tier-3** (Best effort) | Not officially supported; defects logged but not blocking | Advisory only |
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Default Browser Matrix
|
|
33
|
+
|
|
34
|
+
Teams MUST declare their supported matrix explicitly. The defaults below cover the majority of web traffic (2025–2026 data):
|
|
35
|
+
|
|
36
|
+
### Tier-1 (Default)
|
|
37
|
+
|
|
38
|
+
| Browser | Versions | Platform |
|
|
39
|
+
|---------|----------|---------|
|
|
40
|
+
| Chrome | latest, latest-1 | Windows, macOS, Linux, Android |
|
|
41
|
+
| Safari | latest, latest-1 | macOS, iOS |
|
|
42
|
+
| Firefox | latest | Windows, macOS, Linux |
|
|
43
|
+
| Edge | latest | Windows, macOS |
|
|
44
|
+
|
|
45
|
+
### Tier-2 (Default)
|
|
46
|
+
|
|
47
|
+
| Browser | Versions | Platform |
|
|
48
|
+
|---------|----------|---------|
|
|
49
|
+
| Chrome | latest-2, latest-3 | Desktop |
|
|
50
|
+
| Safari | latest-2 | macOS, iOS |
|
|
51
|
+
| Samsung Internet | latest | Android |
|
|
52
|
+
| Opera | latest | Desktop |
|
|
53
|
+
|
|
54
|
+
### Tier-3 (Default)
|
|
55
|
+
|
|
56
|
+
| Browser | Notes |
|
|
57
|
+
|---------|-------|
|
|
58
|
+
| IE 11 | EOL; only if contractually required |
|
|
59
|
+
| Chrome < latest-3 | Tracked as known limitation |
|
|
60
|
+
|
|
61
|
+
### Device / Viewport Matrix
|
|
62
|
+
|
|
63
|
+
| Category | Min Width | Representative Device |
|
|
64
|
+
|----------|-----------|-----------------------|
|
|
65
|
+
| Mobile (small) | 360px | Android (small) |
|
|
66
|
+
| Mobile (standard) | 390px | iPhone 14 |
|
|
67
|
+
| Tablet (portrait) | 768px | iPad |
|
|
68
|
+
| Tablet (landscape) | 1024px | iPad landscape |
|
|
69
|
+
| Desktop (small) | 1280px | Laptop 13" |
|
|
70
|
+
| Desktop (standard) | 1440px | Laptop 15" / External monitor |
|
|
71
|
+
| Desktop (wide) | 1920px | Full HD monitor |
|
|
72
|
+
|
|
73
|
+
Minimum: test at **360px, 768px, 1280px** (mobile / tablet / desktop breakpoints).
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Automated Testing
|
|
78
|
+
|
|
79
|
+
### Playwright Matrix Configuration
|
|
80
|
+
|
|
81
|
+
```typescript
|
|
82
|
+
// playwright.config.ts
|
|
83
|
+
import { defineConfig, devices } from "@playwright/test";
|
|
84
|
+
|
|
85
|
+
export default defineConfig({
|
|
86
|
+
projects: [
|
|
87
|
+
// Tier-1 browsers
|
|
88
|
+
{ name: "chromium", use: { ...devices["Desktop Chrome"] } },
|
|
89
|
+
{ name: "firefox", use: { ...devices["Desktop Firefox"] } },
|
|
90
|
+
{ name: "webkit", use: { ...devices["Desktop Safari"] } },
|
|
91
|
+
{ name: "edge", use: { ...devices["Desktop Edge"] } },
|
|
92
|
+
// Mobile Tier-1
|
|
93
|
+
{ name: "mobile-chrome", use: { ...devices["Pixel 7"] } },
|
|
94
|
+
{ name: "mobile-safari", use: { ...devices["iPhone 14"] } },
|
|
95
|
+
// Viewport coverage
|
|
96
|
+
{ name: "tablet", use: { ...devices["iPad Pro 11"] } },
|
|
97
|
+
],
|
|
98
|
+
// Tier-1 failure = build failure
|
|
99
|
+
reporter: [["html"], ["junit", { outputFile: "results/browser-compat.xml" }]],
|
|
100
|
+
});
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### CI Execution Strategy
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
# Run full Tier-1 matrix on release candidate
|
|
107
|
+
npx playwright test --project=chromium,firefox,webkit,edge,mobile-chrome,mobile-safari,tablet
|
|
108
|
+
|
|
109
|
+
# Run Tier-1 only on every PR (fast feedback)
|
|
110
|
+
npx playwright test --project=chromium,firefox,webkit
|
|
111
|
+
|
|
112
|
+
# Run Tier-2 on release candidate (results feed into sign-off as WARN/PASS)
|
|
113
|
+
npx playwright test --project=samsung,opera
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
### Cloud Browser Testing
|
|
117
|
+
|
|
118
|
+
For Tier-1 cross-OS testing (e.g., Safari on Windows-hosted CI), use cloud services:
|
|
119
|
+
|
|
120
|
+
| Service | Use Case |
|
|
121
|
+
|---------|---------|
|
|
122
|
+
| BrowserStack Automate | Commercial projects; widest OS+browser matrix |
|
|
123
|
+
| Sauce Labs | Enterprises with existing contract |
|
|
124
|
+
| LambdaTest | Open-source / cost-sensitive projects |
|
|
125
|
+
|
|
126
|
+
**Minimum cloud testing**: Safari latest + latest-1 on real iOS devices (Simulator is insufficient for WebKit bugs).
|
|
127
|
+
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## Visual Regression (Optional but Recommended)
|
|
131
|
+
|
|
132
|
+
Pixel-diff testing detects layout regressions across browsers:
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
# Using Playwright visual comparisons
|
|
136
|
+
npx playwright test --update-snapshots # update baseline
|
|
137
|
+
npx playwright test # compare against baseline; fail if diff > threshold
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Threshold recommendation: < 0.5% pixel diff for layout components; < 2% for complex interactive components.
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
## Release Gate Criteria
|
|
145
|
+
|
|
146
|
+
This is **Dimension 9** in `release-readiness-gate.md` (Tier-3: required for frontend/web projects; `N/A` for CLI/backend-only).
|
|
147
|
+
|
|
148
|
+
| Gate | Pass | Warn | Fail |
|
|
149
|
+
|------|------|------|------|
|
|
150
|
+
| Tier-1 browser matrix | 100% tests pass | — | Any test fails |
|
|
151
|
+
| Tier-2 browser matrix | ≥ 95% pass | 90–95% | < 90% |
|
|
152
|
+
| Viewport coverage (360/768/1280) | No layout breaks on any Tier-1 browser | — | Any critical flow unusable |
|
|
153
|
+
|
|
154
|
+
### Evidence for Sign-off
|
|
155
|
+
|
|
156
|
+
```
|
|
157
|
+
| 9 | Browser / Device Compat | PASS | Playwright: 6 browsers × 7 viewports, 100% Tier-1; Tier-2: 97%; [junit report link] | QA Lead |
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
### `N/A` Criteria
|
|
161
|
+
|
|
162
|
+
Mark as `N/A` when:
|
|
163
|
+
- Project is CLI-only, pure backend API, or mobile native (not web)
|
|
164
|
+
- Document rationale: `"N/A — backend API service, no browser UI"`
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## Browserslist Configuration
|
|
169
|
+
|
|
170
|
+
Commit a `.browserslistrc` to the repo root to ensure build tools (Babel, PostCSS, Autoprefixer) target the same browsers:
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
# .browserslistrc
|
|
174
|
+
# Tier-1: production targets
|
|
175
|
+
last 2 Chrome versions
|
|
176
|
+
last 2 Firefox versions
|
|
177
|
+
last 2 Safari versions
|
|
178
|
+
last 2 Edge versions
|
|
179
|
+
last 2 iOS versions
|
|
180
|
+
last 2 ChromeAndroid versions
|
|
181
|
+
|
|
182
|
+
# Tier-2: for reference (not in build targets by default)
|
|
183
|
+
# last 4 Chrome versions
|
|
184
|
+
# Samsung Internet >= 14
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
## Anti-Patterns
|
|
190
|
+
|
|
191
|
+
- **Testing only Chrome** — Chrome represents ~65% of desktop traffic; the remaining 35% is Safari/Firefox/Edge users who will find your bugs
|
|
192
|
+
- **Using browser simulators for iOS Safari testing** — WebKit on Simulator diverges from real device WebKit; always test on real iOS for release candidates
|
|
193
|
+
- **Not specifying a matrix** — implicit assumption of "all browsers" is impossible to test; an explicit Tier-1 matrix is better than implicit coverage of none
|
|
194
|
+
- **Treating Tier-3 browser failures as blockers** — Tier-3 is best-effort; logging the issue is appropriate, not blocking the release
|
|
195
|
+
- **Skipping mobile viewport testing** — mobile-first is standard; missing 360px tests will produce broken UX for the majority of mobile users
|
|
196
|
+
|
|
197
|
+
---
|
|
198
|
+
|
|
199
|
+
## Relationship to Other Standards
|
|
200
|
+
|
|
201
|
+
- **`accessibility-standards.md`** — keyboard nav and screen reader tests run across Tier-1 browsers
|
|
202
|
+
- **`e2e-testing.md`** — Playwright matrix config extends E2E tests to multi-browser
|
|
203
|
+
- **`release-readiness-gate.md`** — Dimension 9 (Tier-3)
|
|
204
|
+
- **`performance-standards.md`** — Core Web Vitals targets apply per Tier-1 browser
|
|
205
|
+
|
|
206
|
+
---
|
|
207
|
+
|
|
208
|
+
## Version History
|
|
209
|
+
|
|
210
|
+
| Version | Date | Changes |
|
|
211
|
+
|---------|------|---------|
|
|
212
|
+
| 1.0.0 | 2026-05-05 | Initial release: Tier-1/2/3 matrix, Playwright config, cloud testing, release gate criteria |
|
|
213
|
+
|
|
214
|
+
---
|
|
215
|
+
|
|
216
|
+
## License
|
|
217
|
+
|
|
218
|
+
This standard is released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
|
|
219
|
+
|
|
220
|
+
**Source**: [universal-dev-standards](https://github.com/AsiaOstrich/universal-dev-standards)
|
|
@@ -1121,6 +1121,7 @@ auto_fix:
|
|
|
1121
1121
|
- [Acceptance Criteria Traceability](acceptance-criteria-traceability.md) - AC coverage verification
|
|
1122
1122
|
- [Change Batching Standards](change-batching-standards.md) - Batch threshold triggers
|
|
1123
1123
|
- [Pipeline Integration Standards](pipeline-integration-standards.md) - Automated check-in
|
|
1124
|
+
- [Release Readiness Gate](release-readiness-gate.md) - **Check-in ≠ release readiness**: passing per-commit gates is necessary but not sufficient for production; see the 16-dimension release gate for what "done" means at release time
|
|
1124
1125
|
|
|
1125
1126
|
---
|
|
1126
1127
|
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
# Contract Testing Standards
|
|
2
|
+
|
|
3
|
+
> **Language**: English | [繁體中文](../locales/zh-TW/core/contract-testing-standards.md)
|
|
4
|
+
|
|
5
|
+
**Version**: 1.0.0
|
|
6
|
+
**Last Updated**: 2026-05-05
|
|
7
|
+
**Applicability**: Projects with API consumers (service-to-service, frontend-to-backend, public API)
|
|
8
|
+
**Scope**: universal
|
|
9
|
+
**Industry Standards**: Consumer-Driven Contract Testing (CDCT), Pact Specification v3
|
|
10
|
+
**References**: [pact.io](https://docs.pact.io/), [Spring Cloud Contract](https://spring.io/projects/spring-cloud-contract)
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Purpose
|
|
15
|
+
|
|
16
|
+
Contract testing verifies that a provider (API server) and its consumers (clients) agree on the exact interface — request format, response schema, and status codes — without requiring both sides to be deployed simultaneously.
|
|
17
|
+
|
|
18
|
+
Without contract testing:
|
|
19
|
+
- Provider changes silently break consumers in production
|
|
20
|
+
- Integration tests between services require full environments
|
|
21
|
+
- API versioning decisions are made without evidence of actual usage
|
|
22
|
+
|
|
23
|
+
This standard formalizes consumer-driven contract testing as a **release gate** (Dimension 4 in `release-readiness-gate.md`, Tier-3).
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## Consumer-Driven Contract Flow
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
Consumer writes interaction expectations
|
|
31
|
+
↓
|
|
32
|
+
Consumer publishes contract to Pact Broker
|
|
33
|
+
↓
|
|
34
|
+
Provider CI fetches consumer contracts
|
|
35
|
+
↓
|
|
36
|
+
Provider verifies it can fulfill each interaction
|
|
37
|
+
↓
|
|
38
|
+
Pact Broker records: can-i-deploy result
|
|
39
|
+
↓
|
|
40
|
+
Release gate: ALL consumer contracts GREEN before provider deploy
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Contract Scope
|
|
46
|
+
|
|
47
|
+
A contract covers:
|
|
48
|
+
|
|
49
|
+
| Element | Must Specify | Notes |
|
|
50
|
+
|---------|-------------|-------|
|
|
51
|
+
| Request method | Yes | GET / POST / PUT / PATCH / DELETE |
|
|
52
|
+
| Request path | Yes | Including path params |
|
|
53
|
+
| Request headers | Required only | Do not over-specify optional headers |
|
|
54
|
+
| Request body schema | Yes (for write operations) | Schema-level, not literal values |
|
|
55
|
+
| Response status | Yes | All expected status codes |
|
|
56
|
+
| Response body schema | Yes | Schema-level; use matchers not literals |
|
|
57
|
+
| Response headers | Required only | e.g., `Content-Type` |
|
|
58
|
+
|
|
59
|
+
**Under-specification is preferred over over-specification.** Only assert what the consumer actually uses.
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## Backward Compatibility Window
|
|
64
|
+
|
|
65
|
+
| Release Type | Compatibility Requirement |
|
|
66
|
+
|-------------|--------------------------|
|
|
67
|
+
| Patch | 100% backward compatible; no contract changes expected |
|
|
68
|
+
| Minor | N-1 consumer contract version must still pass |
|
|
69
|
+
| Major | Deprecation period required; consumers notified; old contract archived |
|
|
70
|
+
|
|
71
|
+
**Breaking change policy**: A provider MAY NOT deploy if any active consumer contract is red. Breaking changes require:
|
|
72
|
+
1. New provider version with additive-only changes
|
|
73
|
+
2. Consumer migration to new version
|
|
74
|
+
3. Old contract explicitly deprecated and archived
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## Release Gate Criteria
|
|
79
|
+
|
|
80
|
+
| Criterion | Hard Minimum | Warn Band |
|
|
81
|
+
|-----------|-------------|-----------|
|
|
82
|
+
| All active consumer contracts | 100% green | — (binary: all or nothing) |
|
|
83
|
+
| N-1 backward compatibility | 100% green | — |
|
|
84
|
+
| Deprecated contract cleanup | Old contracts archived within 30 days | > 30 days = WARN |
|
|
85
|
+
|
|
86
|
+
The `can-i-deploy` command in the Pact Broker encapsulates this gate:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
pact-broker can-i-deploy \
|
|
90
|
+
--pacticipant <provider-service> \
|
|
91
|
+
--version <version> \
|
|
92
|
+
--to-environment production
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Exit code 0 = PASS; non-zero = FAIL (blocks release).
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Implementation Guidelines
|
|
100
|
+
|
|
101
|
+
### Consumer Side
|
|
102
|
+
|
|
103
|
+
```typescript
|
|
104
|
+
// Pact consumer test (TypeScript example)
|
|
105
|
+
const interaction = {
|
|
106
|
+
state: "user 42 exists",
|
|
107
|
+
uponReceiving: "a request for user 42",
|
|
108
|
+
withRequest: {
|
|
109
|
+
method: "GET",
|
|
110
|
+
path: "/users/42",
|
|
111
|
+
headers: { Accept: "application/json" },
|
|
112
|
+
},
|
|
113
|
+
willRespondWith: {
|
|
114
|
+
status: 200,
|
|
115
|
+
headers: { "Content-Type": "application/json" },
|
|
116
|
+
body: like({ // schema matcher, not literal
|
|
117
|
+
id: integer(),
|
|
118
|
+
name: string(),
|
|
119
|
+
email: email(),
|
|
120
|
+
}),
|
|
121
|
+
},
|
|
122
|
+
};
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
### Provider Side
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
# Provider verification in CI
|
|
129
|
+
PACT_BROKER_BASE_URL=https://pact-broker.internal \
|
|
130
|
+
PACT_PROVIDER_VERSION=$GIT_SHA \
|
|
131
|
+
npm run test:pact:provider
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### Pact Broker Tags
|
|
135
|
+
|
|
136
|
+
| Tag | Meaning |
|
|
137
|
+
|-----|---------|
|
|
138
|
+
| `main` | Latest from main branch |
|
|
139
|
+
| `production` | Currently deployed to production |
|
|
140
|
+
| `<feature-branch>` | Feature branch contracts (transient) |
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
## Anti-Patterns
|
|
145
|
+
|
|
146
|
+
- **Testing the full API surface** — test only what consumers actually consume; over-specification causes unnecessary contract breaks
|
|
147
|
+
- **Literal value matching** — use schema matchers (`like()`, `eachLike()`, `integer()`) not exact values; contracts should tolerate realistic data variation
|
|
148
|
+
- **Skipping provider verification** — publishing a consumer contract without provider verification means the contract has no enforcement value
|
|
149
|
+
- **Not running `can-i-deploy`** — checking individual contract status is insufficient; `can-i-deploy` evaluates the entire deployment matrix
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## Relationship to Other Standards
|
|
154
|
+
|
|
155
|
+
- **`api-design-standards.md`** — contract testing enforces backwards-compat guarantees stated in API design
|
|
156
|
+
- **`release-readiness-gate.md`** — Dimension 4 (Tier-3: applies when API consumers exist)
|
|
157
|
+
- **`integration-testing.md`** — contract tests complement but do not replace integration tests
|
|
158
|
+
- **`versioning.md`** — semantic versioning interacts with backward compatibility window above
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
## See Also
|
|
163
|
+
|
|
164
|
+
- [Pact Documentation](https://docs.pact.io/)
|
|
165
|
+
- [Can I Deploy](https://docs.pact.io/pact_broker/can_i_deploy)
|
|
166
|
+
- [Consumer-Driven Contracts](https://martinfowler.com/articles/consumerDrivenContracts.html) — Martin Fowler
|
|
167
|
+
|
|
168
|
+
---
|
|
169
|
+
|
|
170
|
+
## Version History
|
|
171
|
+
|
|
172
|
+
| Version | Date | Changes |
|
|
173
|
+
|---------|------|---------|
|
|
174
|
+
| 1.0.0 | 2026-05-05 | Initial release: consumer-driven contract flow, backward compat window, release gate criteria |
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## License
|
|
179
|
+
|
|
180
|
+
This standard is released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
|
|
181
|
+
|
|
182
|
+
**Source**: [universal-dev-standards](https://github.com/AsiaOstrich/universal-dev-standards)
|
|
@@ -0,0 +1,190 @@
|
|
|
1
|
+
# Cross-Flow Regression
|
|
2
|
+
|
|
3
|
+
> **Language**: English | [繁體中文](../locales/zh-TW/core/cross-flow-regression.md)
|
|
4
|
+
|
|
5
|
+
**Version**: 1.0.0
|
|
6
|
+
**Last Updated**: 2026-05-05
|
|
7
|
+
**Applicability**: All software projects with multiple user flows or business processes
|
|
8
|
+
**Scope**: universal
|
|
9
|
+
**Industry Standards**: ISTQB Advanced Test Analyst (Regression Test Strategy)
|
|
10
|
+
**References**: `core/flow-based-testing.md`, `core/testing-standards.md`
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Purpose
|
|
15
|
+
|
|
16
|
+
This standard defines cross-flow regression testing — verifying that changes to one flow do not break other flows, and that **combinations of flows** behave correctly when executed in sequence.
|
|
17
|
+
|
|
18
|
+
### Boundary with `flow-based-testing.md`
|
|
19
|
+
|
|
20
|
+
| Standard | Scope | What It Catches |
|
|
21
|
+
|----------|-------|----------------|
|
|
22
|
+
| `flow-based-testing.md` (Multi-Gate Model) | Single flow: Decision Points, Terminal States, Decision Table | Intra-flow branch coverage gaps |
|
|
23
|
+
| **This standard** | Multiple flows: interaction, shared state, sequential composition | Inter-flow contamination, accumulated-state bugs, regression across flows |
|
|
24
|
+
|
|
25
|
+
These are complementary, not overlapping. A project needs both.
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
## Why Cross-Flow Bugs Are Distinct
|
|
30
|
+
|
|
31
|
+
Intra-flow testing (Multi-Gate) proves that "Login" handles all 7 terminal states. But it cannot detect:
|
|
32
|
+
|
|
33
|
+
- **State contamination**: after a failed "Create Order" (FAIL_QUOTA_EXCEEDED), the quota counter is corrupted → next "Create Order" attempt fails even after quota resets
|
|
34
|
+
- **Shared resource conflicts**: "Report Generation" and "Data Export" running concurrently corrupt a shared temp directory
|
|
35
|
+
- **Sequential dependency**: "Cancel Subscription" succeeds, but the subsequent "Reactivate" flow assumes subscription still exists → NullPointerException
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## Cross-Flow Test Suite Definition
|
|
40
|
+
|
|
41
|
+
### Tier-1: Critical User Journeys (CUJ)
|
|
42
|
+
|
|
43
|
+
Critical User Journeys are end-to-end sequences spanning ≥ 2 flows that represent core business value paths. Every release must include a CUJ regression suite.
|
|
44
|
+
|
|
45
|
+
**CUJ identification**:
|
|
46
|
+
1. List all flows (from requirement-template §2.4)
|
|
47
|
+
2. Identify pairs/triples that share state or are commonly executed in sequence
|
|
48
|
+
3. Tag business-critical combinations (purchase, onboarding, authentication + downstream)
|
|
49
|
+
|
|
50
|
+
**CUJ Coverage Requirement**:
|
|
51
|
+
|
|
52
|
+
| Metric | Tier-2 Threshold (default) | Tier-1 Critical Path |
|
|
53
|
+
|--------|--------------------------|---------------------|
|
|
54
|
+
| CUJ pass rate | ≥ 95% | 100% |
|
|
55
|
+
| Business-critical flow combos | 100% | 100% |
|
|
56
|
+
|
|
57
|
+
### Tier-2: Regression on Flow Change
|
|
58
|
+
|
|
59
|
+
When any flow's §2.4 (Decision Points, Terminal States) is modified, the full CUJ suite must re-run — not just the tests for the changed flow. The triggering logic:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
# In CI: detect flow spec changes
|
|
63
|
+
changed_flows=$(git diff origin/main... --name-only | grep -E "requirement-template|SPEC.*\.md")
|
|
64
|
+
if [ -n "$changed_flows" ]; then
|
|
65
|
+
npm run test:cross-flow-regression
|
|
66
|
+
fi
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Tier-3: Concurrency and Shared Resource Tests
|
|
70
|
+
|
|
71
|
+
For projects with concurrent user operations:
|
|
72
|
+
- Two users executing the same flow simultaneously
|
|
73
|
+
- Flow A and Flow B sharing a write resource
|
|
74
|
+
- Long-running Flow (async) interacting with a short Flow result
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## Test Structure
|
|
79
|
+
|
|
80
|
+
Cross-flow regression tests use **sequential state threading** across flows (extending the `ctx` pattern from `flow-based-testing.md`):
|
|
81
|
+
|
|
82
|
+
```typescript
|
|
83
|
+
describe("CUJ: Register → Verify Email → Create First Order", () => {
|
|
84
|
+
const ctx: {
|
|
85
|
+
userId?: string
|
|
86
|
+
token?: string
|
|
87
|
+
orderId?: string
|
|
88
|
+
} = {}
|
|
89
|
+
|
|
90
|
+
// Flow 1: Register
|
|
91
|
+
it("Flow-1 Step 1: Register new user", async () => {
|
|
92
|
+
ctx.userId = await registerUser({ email: testEmail, plan: "trial" })
|
|
93
|
+
expect(ctx.userId).toBeDefined()
|
|
94
|
+
})
|
|
95
|
+
|
|
96
|
+
// Flow 2: Email Verification (depends on Flow 1 output)
|
|
97
|
+
it("Flow-2 Step 1: Verify email token", async () => {
|
|
98
|
+
const token = await getEmailToken(ctx.userId!)
|
|
99
|
+
ctx.token = await verifyEmail(token)
|
|
100
|
+
expect(ctx.token).toBeDefined()
|
|
101
|
+
})
|
|
102
|
+
|
|
103
|
+
// Flow 3: Create Order (depends on Flow 2 auth token)
|
|
104
|
+
it("Flow-3 Step 1: Create first order", async () => {
|
|
105
|
+
ctx.orderId = await createOrder(ctx.token!, orderPayload)
|
|
106
|
+
expect(ctx.orderId).toMatch(/^ord-/)
|
|
107
|
+
})
|
|
108
|
+
|
|
109
|
+
// Cross-flow verification: order state reflects trial plan limits
|
|
110
|
+
it("Cross-flow: Trial plan quota enforced on first order", async () => {
|
|
111
|
+
const order = await getOrder(ctx.token!, ctx.orderId!)
|
|
112
|
+
expect(order.quota_applied).toBe("trial")
|
|
113
|
+
})
|
|
114
|
+
})
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
### Cross-Flow Failure Isolation
|
|
118
|
+
|
|
119
|
+
When a cross-flow test fails, the failure message must identify **which flow** introduced the state corruption:
|
|
120
|
+
|
|
121
|
+
```typescript
|
|
122
|
+
// BAD: generic assertion
|
|
123
|
+
expect(result).toBe("success")
|
|
124
|
+
|
|
125
|
+
// GOOD: includes flow context
|
|
126
|
+
expect(result).toBe("success") // Flow-3 depends on Flow-2 token being valid
|
|
127
|
+
// If this fails, check Flow-2 email verification output
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
## Release Gate Integration
|
|
133
|
+
|
|
134
|
+
Cross-flow regression is **Dimension 6** in `release-readiness-gate.md` (Tier-2).
|
|
135
|
+
|
|
136
|
+
### Trigger Conditions
|
|
137
|
+
|
|
138
|
+
| Trigger | Scope |
|
|
139
|
+
|---------|-------|
|
|
140
|
+
| Every release candidate | Full CUJ suite |
|
|
141
|
+
| PR modifying any flow §2.4 | Full CUJ suite (pre-merge) |
|
|
142
|
+
| Post-deploy to staging | Smoke subset of CUJ |
|
|
143
|
+
| Post-deploy to production | Critical path CUJ only (canary) |
|
|
144
|
+
|
|
145
|
+
### Evidence for Sign-off
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
| 6 | Cross-flow Regression | PASS | CUJ suite: 47/47 passed; 0 flow-interaction failures | QA Lead |
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
### WARN Threshold
|
|
152
|
+
|
|
153
|
+
- ≥ 95% CUJ pass rate but < 100% → WARN with specific failed CUJ documented and root-caused
|
|
154
|
+
- < 95% CUJ pass rate → FAIL (release blocked)
|
|
155
|
+
- Business-critical combo fails → FAIL regardless of overall rate
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
## Anti-Patterns
|
|
160
|
+
|
|
161
|
+
- **Running cross-flow tests only when CI is slow** — they must run on every release candidate by definition
|
|
162
|
+
- **Testing each flow in isolation and calling it "regression"** — flow isolation is covered by Multi-Gate; cross-flow must have inter-flow state dependencies
|
|
163
|
+
- **Reusing the same `ctx` object across unrelated `describe` blocks** — each CUJ needs a clean, isolated `ctx`; contamination between CUJs masks bugs
|
|
164
|
+
- **No flow attribution in failure messages** — cross-flow failures are hard to debug; always indicate which upstream flow produced the corrupted state
|
|
165
|
+
- **Treating CUJ failures as flaky** — cross-flow state bugs are deterministic; "flaky" cross-flow tests are almost always a symptom of shared state corruption
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Relationship to Other Standards
|
|
170
|
+
|
|
171
|
+
- **`flow-based-testing.md`** — intra-flow gate (prerequisite for cross-flow)
|
|
172
|
+
- **`testing-standards.md`** — regression layer in the testing pyramid
|
|
173
|
+
- **`release-readiness-gate.md`** — Dimension 6 (Tier-2)
|
|
174
|
+
- **`e2e-testing.md`** — CUJ tests typically run at E2E or System Test level
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## Version History
|
|
179
|
+
|
|
180
|
+
| Version | Date | Changes |
|
|
181
|
+
|---------|------|---------|
|
|
182
|
+
| 1.0.0 | 2026-05-05 | Initial release: CUJ definition, sequential state threading, release gate criteria, Tier-1/2/3 classification |
|
|
183
|
+
|
|
184
|
+
---
|
|
185
|
+
|
|
186
|
+
## License
|
|
187
|
+
|
|
188
|
+
This standard is released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
|
|
189
|
+
|
|
190
|
+
**Source**: [universal-dev-standards](https://github.com/AsiaOstrich/universal-dev-standards)
|