universal-dev-standards 5.7.3 → 5.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -3,12 +3,86 @@
3
3
 
4
4
  id: mock-boundary
5
5
  meta:
6
- version: "1.0.0"
7
- updated: "2026-05-04"
6
+ version: "1.1.0"
7
+ updated: "2026-05-13"
8
8
  source: core/mock-boundary.md
9
9
  description: >
10
10
  Rules defining what can and cannot be mocked to prevent hollow tests —
11
11
  tests that pass while the real system is broken.
12
+ v1.1.0: Added Level 1/Level 2 mock layer distinction and external
13
+ dependency testability matrix template (XSPEC-204).
14
+
15
+ # ─────────────────────────────────────────────────────────
16
+ # Mock Levels (v1.1.0)
17
+ # ─────────────────────────────────────────────────────────
18
+ mock_levels:
19
+ description: >
20
+ Two distinct levels of test doubles with separate governance rules.
21
+ Conflating them leads to the "STUB blocked ∩ real service unavailable = untestable"
22
+ deadlock in UAT environments.
23
+
24
+ level_1:
25
+ name: Code-level Mock
26
+ description: >
27
+ Test doubles within test code — jest.mock(), Moq.Mock<T>(), vi.fn(), etc.
28
+ governance: "Regulated by STUB marker rules (full-coverage-testing)"
29
+ examples:
30
+ - "vi.mock('../../src/services/sms-service.js')"
31
+ - "new Mock<ISmsService>()"
32
+ - "httpClient.Setup(x => x.PostAsync(...)).ReturnsAsync(...)"
33
+ rules:
34
+ - "Subject to // WARNING: STUB deployment-blocking rules"
35
+ - "Not allowed in flow/integration/E2E tests (see forbidden section)"
36
+ - "STUB marker must be placed at usage site"
37
+
38
+ level_2:
39
+ name: Infrastructure-level Stub Server
40
+ description: >
41
+ Independently running test-double services — WireMock, MockSoap, stub-server, json-server, etc.
42
+ These run as separate processes alongside the application under test.
43
+ governance: "Regulated by environment stratification rules (multi-environment-e2e-testing)"
44
+ examples:
45
+ - "WireMock running on port 9999, app configured via env var to hit it"
46
+ - "MockSoap sidecar for SOAP gateway simulation"
47
+ - "json-server for external REST API simulation"
48
+ rules:
49
+ - "NOT subject to STUB deployment-blocking rules (they are infrastructure, not code)"
50
+ - "MAY be used in UAT environment when real external service is unavailable"
51
+ - "MUST be documented in the external-dependency-testability-matrix as ⚠️"
52
+ - "MUST list dimensions NOT verifiable through the stub"
53
+ - "MUST NOT be deployed to or accessible from PRD environment"
54
+ - "MUST be deployed as sidecar (not bundled in production artifact)"
55
+
56
+ level_2_rationale: >
57
+ When full-coverage-testing blocks STUB markers AND the real external service
58
+ is unavailable in UAT (e.g., SMS gateway requires billing setup), Level 2
59
+ stub servers are the correct solution. They allow the flow to execute in UAT
60
+ while clearly documenting which verification dimensions are deferred to PRD.
61
+
62
+ # ─────────────────────────────────────────────────────────
63
+ # External Dependency Testability Matrix Template
64
+ # ─────────────────────────────────────────────────────────
65
+ external_dependency_testability_matrix:
66
+ rule: >
67
+ Projects with external service dependencies MUST maintain this matrix.
68
+ It answers: "In this environment, can we fully verify this service integration?"
69
+ location: "docs/testing/dependency-testability-matrix.md"
70
+ when_required: "Any project with SMS, payment, IdP, messaging, or external SOAP/REST dependencies"
71
+
72
+ template: |
73
+ ## External Dependency Testability Matrix
74
+
75
+ | External Service | local-dev | UAT | PRD | Unverifiable Dimensions in UAT |
76
+ |-----------------|:---------:|:---:|:---:|-------------------------------|
77
+ | SMS Gateway | ⚠️ L2-stub | ⚠️ L2-stub / ❌ | ✅ | Billing correctness, DR reporting, carrier delivery |
78
+ | Payment SOAP | ⚠️ L2-stub | ⚠️/❌ | ✅ | Real debit/credit, bank reconciliation, card validation |
79
+ | LDAP / AD | ⚠️ container | ⚠️ AD LDS | ✅ | Enterprise AD tree, group sync, GPO policies |
80
+ | Push Notification | ⚠️ mock | ⚠️ sandbox | ✅ | Production delivery rates, carrier-specific behavior |
81
+
82
+ Legend:
83
+ ✅ Full verification | ⚠️ Flow passes via stub (real-world dimensions deferred) | ❌ Cannot test
84
+
85
+ note: "L2-stub = Level 2 infrastructure stub server. Refer to mock-boundary Level 2 rules."
12
86
 
13
87
  # ─────────────────────────────────────────────────────────
14
88
  # Core Problem
@@ -223,6 +297,23 @@ rules:
223
297
  If any hollow test indicator triggers, rewrite the mocking strategy.
224
298
  priority: required
225
299
 
300
+ - id: level-2-stub-server-rules
301
+ trigger: UAT environment lacks a real external service (SMS, payment, IdP)
302
+ instruction: >
303
+ Use a Level 2 infrastructure stub server (WireMock, MockSoap, etc.) deployed
304
+ as a sidecar — not Level 1 code mocks, which are blocked in UAT by STUB rules.
305
+ Document the stub server in the external-dependency-testability-matrix as ⚠️
306
+ and list which verification dimensions are deferred to PRD.
307
+ priority: required
308
+
309
+ - id: no-stub-server-in-prd
310
+ trigger: deploying to production
311
+ instruction: >
312
+ Production artifact MUST NOT include stub server code.
313
+ Stub servers MUST NOT be running or accessible in PRD environment.
314
+ See deployment-standards for CI/CD exclusion rules.
315
+ priority: required
316
+
226
317
  # ─────────────────────────────────────────────────────────
227
318
  # Quick Reference
228
319
  # ─────────────────────────────────────────────────────────
@@ -0,0 +1,250 @@
1
+ # Multi-Environment E2E Testing Standards - AI Optimized
2
+ # Source: XSPEC-204 (UDS Issue #94, #95)
3
+
4
+ id: multi-environment-e2e-testing
5
+ meta:
6
+ version: "1.0.0"
7
+ updated: "2026-05-13"
8
+ description: >
9
+ Standards for E2E test configuration across multiple deployment targets.
10
+ Core principle: The run command IS the documentation.
11
+ Each environment has one entry-point script that self-checks prerequisites
12
+ and runs the correct test subset.
13
+
14
+ # ─────────────────────────────────────────────────────────
15
+ # Core Principle
16
+ # ─────────────────────────────────────────────────────────
17
+ core_principle:
18
+ name: Executable Environment Documentation
19
+ statement: "The run command IS the documentation."
20
+ rationale: >
21
+ Developers repeatedly ask "how do I run tests against environment X?"
22
+ because the answer is scattered across README, wiki, and verbal knowledge.
23
+ When the run script self-checks prerequisites and sets the correct BASE_URL,
24
+ the script itself becomes the authoritative environment setup guide.
25
+ anti_pattern: >
26
+ Manually changing BASE_URL in .env before test runs — breaks other
27
+ developers' configs and cannot be committed without affecting everyone.
28
+
29
+ # ─────────────────────────────────────────────────────────
30
+ # Test Framework Multi-Environment Config
31
+ # ─────────────────────────────────────────────────────────
32
+ test_framework_config:
33
+ rule: "BASE_URL is baked into test project config, not read from .env"
34
+ rationale: >
35
+ If BASE_URL comes from .env, developers must modify .env before running
36
+ against a specific environment, creating race conditions in teams and
37
+ accidental commits of wrong URLs.
38
+
39
+ playwright_pattern: |
40
+ // playwright.config.ts
41
+ const ENVS = {
42
+ 'local-iis': 'http://localhost/corp',
43
+ 'local-iis-express': 'http://localhost:18080/lotest',
44
+ 'uat': 'http://portal_uat.example.com/app',
45
+ 'prd': 'https://app.example.com',
46
+ } as const;
47
+
48
+ export default defineConfig({
49
+ projects: Object.entries(ENVS).map(([name, url]) => ({
50
+ name,
51
+ use: { browserName: 'chromium', baseURL: url },
52
+ })),
53
+ });
54
+
55
+ cypress_pattern: |
56
+ // cypress.config.ts
57
+ const ENVS = {
58
+ 'local': { baseUrl: 'http://localhost:3000' },
59
+ 'uat': { baseUrl: 'http://uat.example.com' },
60
+ 'prd': { baseUrl: 'https://app.example.com' },
61
+ };
62
+ // Pass via --env target=uat; default to local
63
+
64
+ rules:
65
+ - "One test config file; environment distinction via project name or --project flag"
66
+ - "No environment-specific playwright/cypress.config.*.ts files"
67
+ - "BASE_URL never in .env for E2E test configs"
68
+
69
+ # ─────────────────────────────────────────────────────────
70
+ # Runner Script Pattern
71
+ # ─────────────────────────────────────────────────────────
72
+ runner_script_pattern:
73
+ rule: "Each environment has one entry-point script that self-checks prerequisites"
74
+ location: "scripts/run-tests-<env>.(ps1|sh)"
75
+ template_powershell: |
76
+ # scripts/run-tests-local-iis.ps1
77
+ # Self-check prerequisites before running tests
78
+
79
+ # 1. Check Docker
80
+ if (-not (Get-Process "com.docker.backend" -ErrorAction SilentlyContinue)) {
81
+ Write-Error "Docker is not running. Start Docker Desktop first."
82
+ exit 1
83
+ }
84
+
85
+ # 2. Check App is responding
86
+ try {
87
+ $resp = Invoke-WebRequest "http://localhost/corp/health" -UseBasicParsing -TimeoutSec 5
88
+ } catch {
89
+ Write-Error "App not responding at http://localhost/corp. Start IIS site first."
90
+ exit 1
91
+ }
92
+
93
+ # 3. Run tests
94
+ npx playwright test --project=local-iis @args
95
+
96
+ template_bash: |
97
+ #!/bin/bash
98
+ # scripts/run-tests-local.sh
99
+
100
+ # 1. Check Docker
101
+ if ! docker info > /dev/null 2>&1; then
102
+ echo "ERROR: Docker is not running." >&2
103
+ exit 1
104
+ fi
105
+
106
+ # 2. Check App
107
+ if ! curl -s -f http://localhost:3000/health > /dev/null 2>&1; then
108
+ echo "ERROR: App not responding at http://localhost:3000" >&2
109
+ exit 1
110
+ fi
111
+
112
+ npx playwright test --project=local "$@"
113
+
114
+ prerequisite_checks:
115
+ - "Docker / container runtime"
116
+ - "Application health endpoint"
117
+ - "Database connectivity (if separate from app)"
118
+ - "Required environment-specific services"
119
+
120
+ # ─────────────────────────────────────────────────────────
121
+ # Environment Capability Matrix
122
+ # ─────────────────────────────────────────────────────────
123
+ capability_matrix:
124
+ rule: >
125
+ Projects with external dependencies MUST maintain an environment capability
126
+ matrix committed to the repository (docs/testing/environment-capability-matrix.md
127
+ or inline in testing README).
128
+ when_required: "Any project with external HTTP services, IdP, payment, messaging"
129
+
130
+ template: |
131
+ ## Environment Capability Matrix
132
+
133
+ | Service / Feature | local-dev | local-iis | UAT | PRD |
134
+ |-------------------|:---------:|:---------:|:---:|:---:|
135
+ | Auth / SAML | ⚠️ Keycloak stub | ✅ Keycloak local | ✅ Keycloak UAT | ✅ Enterprise IdP |
136
+ | SMS Gateway | ⚠️ stub-server | ⚠️ stub-server | ⚠️ stub-server / ❌ | ✅ Real Gateway + billing |
137
+ | Payment / Finance | ⚠️ stub-server | ⚠️ stub-server | ⚠️ partial | ✅ Real + reconciliation |
138
+ | Background Jobs | ✅ in-process | ✅ in-process | ✅ | ✅ |
139
+ | File Storage | ✅ local | ✅ local | ✅ blob | ✅ blob |
140
+
141
+ Legend:
142
+ ✅ Full verification possible
143
+ ⚠️ Flow passes but through stub (real-world dimensions NOT verified)
144
+ ❌ Cannot test in this environment
145
+
146
+ ### Dimensions NOT verifiable in UAT (must defer to PRD smoke)
147
+ - SMS: Billing correctness, carrier delivery confirmation, DR reporting
148
+ - Payment: Real debit/credit, bank reconciliation, card validation
149
+
150
+ when_to_update: "Update matrix when adding any new external service dependency"
151
+
152
+ # ─────────────────────────────────────────────────────────
153
+ # CI Gate Mapping
154
+ # ─────────────────────────────────────────────────────────
155
+ ci_gate_mapping:
156
+ rule: "Map environments to CI/CD stages; document which gate must pass before each deployment stage"
157
+
158
+ pattern: |
159
+ # .github/workflows/ci.yml or .gitlab-ci.yml
160
+
161
+ e2e-smoke-gate: # Must pass → any deployment
162
+ runs-on: ubuntu-latest
163
+ script: scripts/run-tests-local.sh --grep smoke
164
+
165
+ e2e-uat-gate: # Must pass → PRD deployment
166
+ environment: uat
167
+ script: scripts/run-tests-uat.sh
168
+ only: [tags, release-branches]
169
+
170
+ e2e-prd-smoke: # Must pass → mark release as stable
171
+ environment: prd
172
+ script: scripts/run-tests-prd-smoke.sh
173
+ only: [tags]
174
+
175
+ gate_requirements:
176
+ before_staging_deploy: ["unit tests", "integration tests", "e2e-smoke-gate"]
177
+ before_uat_deploy: ["all staging gates", "e2e-uat-gate (if UAT environment available)"]
178
+ before_prd_deploy: ["e2e-uat-gate", "sign-off from capability matrix review"]
179
+ after_prd_deploy: ["e2e-prd-smoke within 10 min of deploy"]
180
+
181
+ # ─────────────────────────────────────────────────────────
182
+ # Credential Handling
183
+ # ─────────────────────────────────────────────────────────
184
+ credential_handling:
185
+ rule: "Separate what goes in git from what stays gitignored"
186
+
187
+ commit_to_git:
188
+ - "Base URLs per environment (non-secret, team needs to share)"
189
+ - "Test usernames for non-PRD environments"
190
+ - "Feature flags and test configuration"
191
+ - "Self-check scripts (no credentials embedded)"
192
+
193
+ gitignore:
194
+ - "Passwords and secrets"
195
+ - "API keys and tokens"
196
+ - ".env.test.local (personal overrides)"
197
+
198
+ ci_secrets: "Pass PRD test passwords via CI secret variables (GitHub Secrets / GitLab CI Variables)"
199
+
200
+ example_gitignore: |
201
+ # Test credentials
202
+ .env.test.local
203
+ tests/fixtures/auth-secrets.json
204
+ # Base URLs and non-secrets are committed; see playwright.config.ts
205
+
206
+ # ─────────────────────────────────────────────────────────
207
+ # Rules
208
+ # ─────────────────────────────────────────────────────────
209
+ rules:
210
+ - id: base-url-in-config
211
+ trigger: setting up E2E tests for a project with multiple environments
212
+ instruction: >
213
+ Define all environment BASE_URLs in the test framework config (playwright.config.ts /
214
+ cypress.config.ts) as named projects. Do not rely on .env for BASE_URL.
215
+ priority: required
216
+
217
+ - id: one-runner-per-env
218
+ trigger: adding a new deployment target
219
+ instruction: >
220
+ Create scripts/run-tests-<env>.(ps1|sh) with self-checking prerequisite steps.
221
+ The script must verify all required services before invoking the test runner.
222
+ priority: required
223
+
224
+ - id: capability-matrix-required
225
+ trigger: feature has external service dependencies (SMS, payment, IdP, file storage)
226
+ instruction: >
227
+ Create or update the environment capability matrix in docs/testing/.
228
+ Clearly mark ✅/⚠️/❌ for each service × environment combination.
229
+ List dimensions NOT verifiable in UAT; mark these as "PRD-only smoke" items.
230
+ priority: required
231
+
232
+ - id: ci-gate-mapping
233
+ trigger: defining CI/CD pipeline stages
234
+ instruction: >
235
+ Map each CI gate to the environments it must pass.
236
+ E2E gate must specify which environment it targets.
237
+ priority: required
238
+
239
+ # ─────────────────────────────────────────────────────────
240
+ # Relationship to Other Standards
241
+ # ─────────────────────────────────────────────────────────
242
+ related_standards:
243
+ - id: deployment-standards
244
+ relationship: "Extends — adds environment dimension to CI gates and deployment readiness"
245
+ - id: test-completeness-dimensions
246
+ relationship: "Complements — adds dimension 11: Environment Verifiability"
247
+ - id: verification-evidence
248
+ relationship: "Complements — evidence must specify which environment it was collected from"
249
+ - id: mock-boundary
250
+ relationship: "Complements — capability matrix references Level 2 stub server usage"
@@ -3,10 +3,12 @@
3
3
 
4
4
  id: test-completeness-dimensions
5
5
  meta:
6
- version: "1.2.0"
7
- updated: "2026-05-04"
6
+ version: "1.3.0"
7
+ updated: "2026-05-13"
8
8
  source: core/test-completeness-dimensions.md
9
- description: Framework for evaluating test completeness across 10 dimensions (v1.2.0 adds Flow Completeness and Branch Coverage)
9
+ description: >
10
+ Framework for evaluating test completeness across 11 dimensions.
11
+ v1.3.0 adds Dimension 11: Environment Verifiability (XSPEC-204).
10
12
 
11
13
  dimensions:
12
14
  - id: 1
@@ -114,6 +116,21 @@ dimensions:
114
116
  when_required: When flow has any conditional logic or decision points
115
117
  note: Use decision_table_expansion from flow-based-testing.ai.yaml to enumerate scenarios
116
118
 
119
+ - id: 11
120
+ name: Environment Verifiability
121
+ description: >
122
+ For each AC with external service dependencies, document which environment layer
123
+ can fully verify it, and plan PRD-smoke coverage for items that cannot be verified in UAT.
124
+ test_items:
125
+ - Environment stratification responsibility matrix exists for features with external dependencies
126
+ - Each AC is tagged with its minimum verifiable environment layer (local / UAT / PRD)
127
+ - ACs that are PRD-only are explicitly tracked and have a PRD smoke test plan
128
+ - Evidence for externally-dependent ACs includes environment_layer field
129
+ when_required: >
130
+ Any feature with external service dependencies (SMS, payment, IdP, email,
131
+ SOAP integrations, external REST APIs)
132
+ note: See multi-environment-e2e-testing.ai.yaml and deployment-standards.ai.yaml for environment capability matrix templates
133
+
117
134
  feature_type_mapping:
118
135
  note: "Dimension 8 (AI Generation Quality) applies when tests are AI-generated"
119
136
  types:
@@ -138,8 +155,9 @@ feature_type_mapping:
138
155
  with_ai: [1, 3, 5, 8]
139
156
 
140
157
  - type: External Integration
141
- dimensions: [1, 3, 7]
142
- with_ai: [1, 3, 7, 8]
158
+ dimensions: [1, 3, 7, 11]
159
+ with_ai: [1, 3, 7, 8, 11]
160
+ note: Dimension 11 required for any external service dependency
143
161
 
144
162
  - type: Workflow / Multi-step Process
145
163
  dimensions: [1, 3, 4, 5, 9, 10]
@@ -147,6 +165,11 @@ feature_type_mapping:
147
165
  required_pattern: journey-chained-test
148
166
  note: Apply flow-based-testing standard; use shared ctx; Each-Choice minimum branch coverage
149
167
 
168
+ - type: External-Dependent Workflow
169
+ dimensions: [1, 3, 4, 5, 9, 10, 11]
170
+ with_ai: [1, 3, 4, 5, 9, 10, 8, 11]
171
+ note: Combines Workflow + External Integration; Dimension 11 mandatory
172
+
150
173
  error_code_coverage:
151
174
  - code: 200
152
175
  meaning: Success
@@ -198,7 +221,7 @@ anti_patterns:
198
221
  rules:
199
222
  - id: use-checklist
200
223
  trigger: designing tests for a feature
201
- instruction: Use the 10 dimensions checklist to ensure completeness (dim 8 for AI-generated tests; dim 9/10 for multi-step workflows)
224
+ instruction: Use the 11 dimensions checklist to ensure completeness (dim 8 for AI-generated tests; dim 9/10 for multi-step workflows; dim 11 for external dependencies)
202
225
  priority: required
203
226
 
204
227
  - id: authorization-matrix
@@ -278,6 +301,12 @@ checklist_template: |
278
301
  □ Each distinct error branch has its own describe block
279
302
  □ Critical flows (auth/payment/security) use All-Combinations
280
303
 
304
+ □ Environment Verifiability (for features with external dependencies)
305
+ □ Environment stratification matrix created/updated
306
+ □ Each AC tagged with minimum verifiable environment layer
307
+ □ PRD-only ACs documented with smoke test plan
308
+ □ Evidence includes environment_layer field for externally-dependent ACs
309
+
281
310
  quick_reference:
282
311
  dimensions:
283
312
  columns: [ID, Dimension, Key Focus]
@@ -292,9 +321,10 @@ quick_reference:
292
321
  - [8, AI Generation, AI-generated test quality]
293
322
  - [9, Flow Completeness, Complete path to each terminal state]
294
323
  - [10, Branch Coverage, All decision-point branches covered]
324
+ - [11, Environment Verifiability, External-dep ACs tagged with environment layer]
295
325
 
296
326
  feature_dimensions:
297
- note: "*Dimension 8 applies when tests are AI-generated; †Dimensions 9/10 apply to multi-step workflows"
327
+ note: "*Dimension 8 applies when tests are AI-generated; †Dimensions 9/10 apply to multi-step workflows; ‡Dimension 11 applies to features with external service dependencies"
298
328
  columns: [Feature Type, Required Dimensions]
299
329
  rows:
300
330
  - [CRUD API, "1,2,3,4,6,7,8*"]
@@ -304,3 +334,5 @@ quick_reference:
304
334
  - [Background Job, "1,3,5,8*"]
305
335
  - [External Integration, "1,3,7,8*"]
306
336
  - [Workflow / Multi-step Process, "1,3,4,5,9†,10†,8*"]
337
+ - [External Integration, "1,3,7,11‡,8*"]
338
+ - [External-Dependent Workflow, "1,3,4,5,9†,10†,11‡,8*"]
@@ -8,15 +8,19 @@ standard:
8
8
  description: 驗證證據標準,強化 anti-hallucination
9
9
 
10
10
  meta:
11
- version: "1.0.0"
12
- updated: "2026-03-20"
11
+ version: "1.1.0"
12
+ updated: "2026-05-13"
13
13
  source: core/verification-evidence.md
14
- description: 驗證證據標準 — Iron Law: 無驗證證據不可聲稱完成
14
+ description: >
15
+ 驗證證據標準 — Iron Law: 無驗證證據不可聲稱完成。
16
+ v1.1.0: Evidence must specify which environment layer it was collected from (XSPEC-204).
15
17
  inspired_by: superpowers/verification-before-completion
16
18
 
17
19
  guidelines:
18
20
  - "Iron Law:無驗證證據 = 不可聲稱完成"
19
21
  - "每次驗證必須記錄 command + exitCode + output + timestamp"
22
+ - "Iron Law(Environment):驗收前必須確認「此環境層次能驗證此流程的哪些維度」"
23
+ - "驗收證據必須標明收集自哪個環境層次(local / UAT / PRD)"
20
24
  - "回歸測試必須展示 RED → GREEN 循環"
21
25
  - "代理報告 success ≠ 實際 success,需獨立驗證"
22
26
  - "驗證輸出截斷至合理長度(2000 字元)但保留關鍵資訊"
@@ -39,6 +43,13 @@ standard:
39
43
  type: string
40
44
  required: true
41
45
  description: "執行時間(ISO 8601 格式)"
46
+ - name: environment_layer
47
+ type: string
48
+ required: false
49
+ description: >
50
+ 收集證據的環境層次(local / uat / prd)。
51
+ 對有外部服務依賴的功能為必填——缺少此欄位時,該 AC 的驗收等級視為 local-only。
52
+ example: "uat"
42
53
 
43
54
  red_green_cycle:
44
55
  description: "回歸測試的 RED → GREEN 驗證"
@@ -72,13 +83,28 @@ standard:
72
83
  action: "截斷但保留錯誤訊息與摘要行"
73
84
  priority: medium
74
85
 
86
+ environment_rules:
87
+ - id: VE-005
88
+ trigger: "AC 涉及外部服務依賴(SMS、金流、IdP)"
89
+ action: >
90
+ 驗收證據必須標明 environment_layer。
91
+ local-only 證據對此類 AC 不足——需確認 UAT 或 PRD 層次的驗證計畫。
92
+ priority: required
93
+ - id: VE-006
94
+ trigger: "有外部服務依賴的 AC 標記為 done,但無 environment_layer"
95
+ action: >
96
+ 降級為 done_with_concerns。
97
+ 要求補充環境層次聲明,或在 environment-stratification-matrix 中標記為 ⚠️/❌。
98
+ priority: high
99
+
75
100
  physical_spec:
76
101
  type: checklist
77
102
  validator:
78
103
  type: ai_review
79
- rule: "檢查任務完成是否附帶驗證證據(command + exit_code + output + timestamp)"
104
+ rule: "檢查任務完成是否附帶驗證證據(command + exit_code + output + timestamp + environment_layer)"
80
105
  checks:
81
106
  - "完成聲明是否附帶 verification_evidence"
82
107
  - "evidence 是否包含所有必填欄位"
83
108
  - "exit_code 是否為 0(成功)"
84
109
  - "Bug fix 是否有 RED → GREEN 循環證據"
110
+ - "有外部服務依賴的 AC 是否標明 environment_layer"
@@ -83,28 +83,60 @@ Scenario: User login with valid credentials
83
83
  | **Covered** | ✅ | AC is fully tested | All conditions in AC have corresponding test assertions |
84
84
  | **Partial** | ⚠️ | AC is partially tested | Some conditions tested, but edge cases or paths missing |
85
85
  | **Uncovered** | ❌ | AC has no tests | No test case references this AC |
86
+ | **Not Implemented** | 🚫 | AC has no corresponding implementation | Feature code does not exist (not a test gap — a code gap) |
87
+
88
+ ### `not_implemented` vs `uncovered` Decision Tree
89
+
90
+ ```
91
+ Q1: Does the corresponding code exist in src/?
92
+ No → 🚫 not_implemented
93
+ Yes → Q2: Does any test reference this AC?
94
+ No → ❌ uncovered
95
+ Yes → Q3: Are all conditions in the AC tested?
96
+ Yes → ✅ covered
97
+ No → ⚠️ partial
98
+ ```
99
+
100
+ Typical signals of `not_implemented`: `throw new NotImplementedException()`, empty stub body, `// FEATURE_STUB:` marker.
86
101
 
87
102
  ### Coverage Calculation
88
103
 
89
104
  ```
90
- AC Coverage % = (covered_count / total_ac_count) × 100
105
+ AC Coverage % = (covered_count + partial_count × 0.5) / (total_ac_count - not_implemented_count) × 100
91
106
 
92
107
  Where:
93
108
  covered_count = count of AC with status "covered"
94
109
  total_ac_count = total number of AC in specification
110
+ not_implemented_count = count of AC with status "not_implemented" (excluded from denominator)
95
111
  partial counts as 0.5 for coverage calculation
96
112
  ```
97
113
 
98
114
  ### Example Calculation
99
115
 
100
116
  ```
101
- SPEC-001: 8 AC total
102
- - 5 covered (✅)
103
- - 2 partial (⚠️)
104
- - 1 uncovered (❌)
117
+ SPEC-001: 20 AC total
118
+ - 15 covered (✅)
119
+ - 0 partial (⚠️)
120
+ - 2 uncovered (❌)
121
+ - 3 not_implemented (🚫)
122
+
123
+ Coverage = (15 + 0) / (20 - 3) × 100 = 88.2%
124
+ Status: BLOCKED by 3 not_implemented AC(s)
125
+ ```
126
+
127
+ ### CI Gate for `not_implemented`
128
+
129
+ `not_implemented` ACs trigger a **blocking** CI gate independent of the coverage percentage gate:
105
130
 
106
- Coverage = (5 + 2×0.5) / 8 = 6/8 = 75%
107
131
  ```
132
+ [AC-NOT-IMPL] 3 AC(s) marked not_implemented:
133
+ 🚫 AC-007 OrderCancellation
134
+ 🚫 AC-012 RefundCalculation
135
+ 🚫 AC-019 ExportToPDF
136
+ All not_implemented ACs must be resolved before UAT.
137
+ ```
138
+
139
+ The gate clears only when all `not_implemented` ACs are updated to `uncovered`, `partial`, or `covered`.
108
140
 
109
141
  ---
110
142
 
@@ -273,6 +305,8 @@ Generated spec MUST include:
273
305
  | Ignoring uncovered AC | Gaps in verification | Track and plan coverage for all AC |
274
306
  | AC without testability | Cannot be verified | Ensure all AC are testable |
275
307
  | Coverage without assertions | Tests run but verify nothing | Check for meaningful assertions |
308
+ | Using `uncovered` for missing code | Hides functional completeness gap | Use `not_implemented` when code doesn't exist |
309
+ | Including `not_implemented` in denominator | Inflates coverage metric | Exclude `not_implemented` from denominator |
276
310
 
277
311
  ---
278
312
 
@@ -291,3 +325,4 @@ Generated spec MUST include:
291
325
  | Version | Date | Changes |
292
326
  |---------|------|---------|
293
327
  | 1.0.0 | 2026-03-18 | Initial version — traceability matrix, coverage calculation, spec generation rules |
328
+ | 1.1.0 | 2026-05-12 | Add `not_implemented` 4th status; update CI gate formula; add decision tree (XSPEC-199) |