universal-dev-standards 5.4.0 → 5.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/ai/options/testing/integration-testing.ai.yaml +2 -2
- package/bundled/ai/options/testing/unit-testing.ai.yaml +2 -2
- package/bundled/ai/standards/adversarial-test.ai.yaml +277 -0
- package/bundled/ai/standards/audit-trail.ai.yaml +113 -0
- package/bundled/ai/standards/browser-compatibility-standards.ai.yaml +63 -0
- package/bundled/ai/standards/chaos-injection-tests.ai.yaml +91 -0
- package/bundled/ai/standards/container-image-standards.ai.yaml +88 -0
- package/bundled/ai/standards/container-security.ai.yaml +331 -0
- package/bundled/ai/standards/contract-testing-standards.ai.yaml +62 -0
- package/bundled/ai/standards/cost-budget-test.ai.yaml +96 -0
- package/bundled/ai/standards/cross-flow-regression.ai.yaml +61 -0
- package/bundled/ai/standards/data-contract.ai.yaml +110 -0
- package/bundled/ai/standards/data-migration-testing.ai.yaml +96 -0
- package/bundled/ai/standards/data-pipeline.ai.yaml +113 -0
- package/bundled/ai/standards/disaster-recovery-drill.ai.yaml +89 -0
- package/bundled/ai/standards/flaky-test-management.ai.yaml +89 -0
- package/bundled/ai/standards/flow-based-testing.ai.yaml +240 -0
- package/bundled/ai/standards/full-coverage-testing.ai.yaml +192 -0
- package/bundled/ai/standards/iac-design-principles.ai.yaml +83 -0
- package/bundled/ai/standards/incident-response.ai.yaml +107 -0
- package/bundled/ai/standards/license-compliance.ai.yaml +106 -0
- package/bundled/ai/standards/llm-output-validation.ai.yaml +269 -0
- package/bundled/ai/standards/mock-boundary.ai.yaml +250 -0
- package/bundled/ai/standards/mutation-testing.ai.yaml +192 -0
- package/bundled/ai/standards/pii-classification.ai.yaml +109 -0
- package/bundled/ai/standards/policy-as-code-testing.ai.yaml +227 -0
- package/bundled/ai/standards/prd-standards.ai.yaml +88 -0
- package/bundled/ai/standards/product-metrics-standards.ai.yaml +111 -0
- package/bundled/ai/standards/prompt-regression.ai.yaml +94 -0
- package/bundled/ai/standards/property-based-testing.ai.yaml +105 -0
- package/bundled/ai/standards/release-quality-manifest.ai.yaml +135 -0
- package/bundled/ai/standards/release-readiness-gate.ai.yaml +77 -0
- package/bundled/ai/standards/replay-test.ai.yaml +111 -0
- package/bundled/ai/standards/runbook.ai.yaml +104 -0
- package/bundled/ai/standards/sast-advanced.ai.yaml +135 -0
- package/bundled/ai/standards/schema-evolution.ai.yaml +111 -0
- package/bundled/ai/standards/secret-management-standards.ai.yaml +105 -0
- package/bundled/ai/standards/secure-op.ai.yaml +365 -0
- package/bundled/ai/standards/security-testing.ai.yaml +171 -0
- package/bundled/ai/standards/server-ops-security.ai.yaml +274 -0
- package/bundled/ai/standards/slo-sli.ai.yaml +97 -0
- package/bundled/ai/standards/smoke-test.ai.yaml +87 -0
- package/bundled/ai/standards/supply-chain-attestation.ai.yaml +109 -0
- package/bundled/ai/standards/test-completeness-dimensions.ai.yaml +52 -5
- package/bundled/ai/standards/testing.ai.yaml +20 -13
- package/bundled/ai/standards/user-story-mapping.ai.yaml +108 -0
- package/bundled/core/accessibility-standards.md +58 -0
- package/bundled/core/adversarial-test.md +212 -0
- package/bundled/core/branch-completion.md +4 -0
- package/bundled/core/browser-compatibility-standards.md +220 -0
- package/bundled/core/chaos-injection-tests.md +116 -0
- package/bundled/core/checkin-standards.md +1 -0
- package/bundled/core/container-security.md +521 -0
- package/bundled/core/contract-testing-standards.md +182 -0
- package/bundled/core/cost-budget-test.md +69 -0
- package/bundled/core/cross-flow-regression.md +190 -0
- package/bundled/core/data-migration-testing.md +110 -0
- package/bundled/core/disaster-recovery-drill.md +73 -0
- package/bundled/core/flaky-test-management.md +73 -0
- package/bundled/core/flow-based-testing.md +275 -0
- package/bundled/core/full-coverage-testing.md +183 -0
- package/bundled/core/llm-output-validation.md +178 -0
- package/bundled/core/mock-boundary.md +100 -0
- package/bundled/core/mutation-testing.md +97 -0
- package/bundled/core/performance-standards.md +65 -0
- package/bundled/core/policy-as-code-testing.md +188 -0
- package/bundled/core/prompt-regression.md +72 -0
- package/bundled/core/property-based-testing.md +73 -0
- package/bundled/core/release-quality-manifest.md +193 -0
- package/bundled/core/release-readiness-gate.md +184 -0
- package/bundled/core/replay-test.md +86 -0
- package/bundled/core/sast-advanced.md +300 -0
- package/bundled/core/secure-op.md +314 -0
- package/bundled/core/security-testing.md +87 -0
- package/bundled/core/server-ops-security.md +493 -0
- package/bundled/core/smoke-test.md +65 -0
- package/bundled/core/supply-chain-attestation.md +117 -0
- package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
- package/bundled/locales/zh-CN/README.md +1 -1
- package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +5 -5
- package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
- package/bundled/locales/zh-TW/README.md +1 -1
- package/bundled/locales/zh-TW/core/browser-compatibility-standards.md +11 -0
- package/bundled/locales/zh-TW/core/contract-testing-standards.md +11 -0
- package/bundled/locales/zh-TW/core/cross-flow-regression.md +11 -0
- package/bundled/locales/zh-TW/core/release-readiness-gate.md +11 -0
- package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +183 -79
- package/bundled/skills/README.md +4 -3
- package/bundled/skills/SKILL_NAMING.md +94 -0
- package/bundled/skills/ai-instruction-standards/SKILL.md +181 -88
- package/bundled/skills/atdd-assistant/SKILL.md +8 -0
- package/bundled/skills/bdd-assistant/SKILL.md +7 -0
- package/bundled/skills/checkin-assistant/SKILL.md +8 -0
- package/bundled/skills/code-review-assistant/SKILL.md +7 -0
- package/bundled/skills/journey-test-assistant/SKILL.md +203 -0
- package/bundled/skills/orchestrate/SKILL.md +167 -0
- package/bundled/skills/plan/SKILL.md +234 -0
- package/bundled/skills/pr-automation-assistant/SKILL.md +8 -0
- package/bundled/skills/push/SKILL.md +49 -2
- package/bundled/skills/{process-automation → skill-builder}/SKILL.md +1 -1
- package/bundled/skills/{forward-derivation → spec-derivation}/SKILL.md +1 -1
- package/bundled/skills/spec-driven-dev/SKILL.md +7 -0
- package/bundled/skills/sweep/SKILL.md +145 -0
- package/bundled/skills/tdd-assistant/SKILL.md +7 -0
- package/package.json +6 -6
- package/src/commands/check.js +43 -0
- package/src/commands/flow.js +8 -0
- package/src/commands/init.js +2 -1
- package/src/commands/start.js +14 -0
- package/src/commands/sweep.js +8 -0
- package/src/commands/update.js +10 -0
- package/src/commands/workflow.js +8 -0
- package/standards-registry.json +483 -5
- package/bundled/locales/zh-CN/skills/ac-coverage-assistant/SKILL.md +0 -190
- package/bundled/locales/zh-CN/skills/forward-derivation/SKILL.md +0 -71
- package/bundled/locales/zh-CN/skills/forward-derivation/guide.md +0 -130
- package/bundled/locales/zh-CN/skills/methodology-system/SKILL.md +0 -88
- package/bundled/locales/zh-CN/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-CN/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-CN/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-CN/skills/process-automation/SKILL.md +0 -143
- package/bundled/locales/zh-TW/skills/ac-coverage-assistant/SKILL.md +0 -195
- package/bundled/locales/zh-TW/skills/deploy-assistant/SKILL.md +0 -178
- package/bundled/locales/zh-TW/skills/forward-derivation/SKILL.md +0 -69
- package/bundled/locales/zh-TW/skills/forward-derivation/guide.md +0 -415
- package/bundled/locales/zh-TW/skills/methodology-system/SKILL.md +0 -86
- package/bundled/locales/zh-TW/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-TW/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-TW/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-TW/skills/process-automation/SKILL.md +0 -144
- /package/bundled/skills/{ac-coverage-assistant → ac-coverage}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/create-methodology.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/guide.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/integrated-flow.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/prerequisite-check.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/runtime.md +0 -0
- /package/bundled/skills/{forward-derivation → spec-derivation}/guide.md +0 -0
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
# Mutation Testing Standards - AI Optimized
|
|
2
|
+
# Source: core/mutation-testing.md
|
|
3
|
+
|
|
4
|
+
id: mutation-testing
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-04"
|
|
8
|
+
source: core/mutation-testing.md
|
|
9
|
+
description: >
|
|
10
|
+
Mutation testing methodology to evaluate test suite effectiveness.
|
|
11
|
+
Answers "do my tests actually catch bugs?" beyond line coverage.
|
|
12
|
+
|
|
13
|
+
# ─────────────────────────────────────────────────────────
|
|
14
|
+
# Core Concepts
|
|
15
|
+
# ─────────────────────────────────────────────────────────
|
|
16
|
+
core_concepts:
|
|
17
|
+
definition: >
|
|
18
|
+
Mutation testing automatically injects small bugs (mutations) into source code,
|
|
19
|
+
then runs the test suite to see if tests detect (kill) the bug.
|
|
20
|
+
A test suite that kills most mutations is effective; one that misses them is hollow.
|
|
21
|
+
|
|
22
|
+
key_terms:
|
|
23
|
+
- term: Mutant
|
|
24
|
+
definition: A copy of source code with one small artificial bug injected
|
|
25
|
+
- term: Killed mutant
|
|
26
|
+
definition: Test suite detected the bug (test failed)
|
|
27
|
+
- term: Survived mutant
|
|
28
|
+
definition: Test suite missed the bug (all tests still pass) — indicates weak tests
|
|
29
|
+
- term: Mutation Score
|
|
30
|
+
formula: "Killed / (Killed + Survived) × 100%"
|
|
31
|
+
interpretation: Higher is better; 0% = tests prove nothing; 100% = very thorough
|
|
32
|
+
|
|
33
|
+
common_mutation_operators:
|
|
34
|
+
- category: Arithmetic operators
|
|
35
|
+
examples: ["+ → -", "* → /", "++ → --"]
|
|
36
|
+
- category: Conditional boundaries
|
|
37
|
+
examples: ["> → >=", "< → <=", "=== → !=="]
|
|
38
|
+
- category: Statement deletion
|
|
39
|
+
examples: ["Remove return statement", "Remove function call"]
|
|
40
|
+
- category: Boolean literal
|
|
41
|
+
examples: ["true → false", "false → true"]
|
|
42
|
+
|
|
43
|
+
# ─────────────────────────────────────────────────────────
|
|
44
|
+
# Tools
|
|
45
|
+
# ─────────────────────────────────────────────────────────
|
|
46
|
+
tools:
|
|
47
|
+
typescript_javascript:
|
|
48
|
+
- name: Stryker Mutator
|
|
49
|
+
packages: ["@stryker-mutator/core", "@stryker-mutator/vitest-runner"]
|
|
50
|
+
config_file: stryker.config.json
|
|
51
|
+
command: "npx stryker run"
|
|
52
|
+
strengths: [Deep vitest/jest integration, incremental mode, HTML reports]
|
|
53
|
+
note: Use incremental mode to speed up re-runs (--incremental flag)
|
|
54
|
+
|
|
55
|
+
python:
|
|
56
|
+
- name: mutmut
|
|
57
|
+
command: "mutmut run"
|
|
58
|
+
config: setup.cfg or pyproject.toml
|
|
59
|
+
- name: Cosmic Ray
|
|
60
|
+
command: "cosmic-ray init config.toml && cosmic-ray exec config.toml"
|
|
61
|
+
|
|
62
|
+
java:
|
|
63
|
+
- name: PIT (Pitest)
|
|
64
|
+
command: "mvn org.pitest:pitest-maven:mutationCoverage"
|
|
65
|
+
strengths: [Industry standard for Java, excellent IDE integration]
|
|
66
|
+
|
|
67
|
+
# ─────────────────────────────────────────────────────────
|
|
68
|
+
# Thresholds
|
|
69
|
+
# ─────────────────────────────────────────────────────────
|
|
70
|
+
thresholds:
|
|
71
|
+
description: Minimum acceptable mutation scores by module criticality
|
|
72
|
+
|
|
73
|
+
critical_modules:
|
|
74
|
+
description: Auth, payment, license validation, security controls
|
|
75
|
+
minimum_score: 80
|
|
76
|
+
enforcement: Block release if below threshold
|
|
77
|
+
examples: [auth/*, license/*, payment/*, security/*]
|
|
78
|
+
|
|
79
|
+
standard_modules:
|
|
80
|
+
description: Core business logic
|
|
81
|
+
minimum_score: 70
|
|
82
|
+
enforcement: Warning in CI; must be resolved before next release
|
|
83
|
+
|
|
84
|
+
ai_generated_tests:
|
|
85
|
+
description: Tests produced by AI tools (including this assistant)
|
|
86
|
+
minimum_score: 50
|
|
87
|
+
enforcement: Required review before accepting AI-generated test files
|
|
88
|
+
rationale: AI tends to generate hollow tests; mutation score reveals this
|
|
89
|
+
|
|
90
|
+
overall_project:
|
|
91
|
+
minimum_score: 60
|
|
92
|
+
enforcement: Advisory (track trend; alert on regression > 5%)
|
|
93
|
+
|
|
94
|
+
# ─────────────────────────────────────────────────────────
|
|
95
|
+
# Stryker Quick Start (TypeScript/Vitest)
|
|
96
|
+
# ─────────────────────────────────────────────────────────
|
|
97
|
+
stryker_quickstart:
|
|
98
|
+
install: "npm install --save-dev @stryker-mutator/core @stryker-mutator/vitest-runner"
|
|
99
|
+
|
|
100
|
+
minimal_config: |
|
|
101
|
+
{
|
|
102
|
+
"testRunner": "vitest",
|
|
103
|
+
"coverageAnalysis": "perTest",
|
|
104
|
+
"mutate": [
|
|
105
|
+
"src/license/**/*.ts",
|
|
106
|
+
"src/enterprise/quota/**/*.ts",
|
|
107
|
+
"src/runner/pipeline-runner.ts",
|
|
108
|
+
"!src/**/*.test.ts"
|
|
109
|
+
],
|
|
110
|
+
"vitest": {
|
|
111
|
+
"configFile": "vitest.config.ts"
|
|
112
|
+
},
|
|
113
|
+
"thresholds": {
|
|
114
|
+
"high": 80,
|
|
115
|
+
"low": 60,
|
|
116
|
+
"break": 50
|
|
117
|
+
},
|
|
118
|
+
"reporters": ["progress", "html", "json"],
|
|
119
|
+
"htmlReporter": {
|
|
120
|
+
"fileName": "reports/mutation/index.html"
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
incremental_mode: "npx stryker run --incremental"
|
|
125
|
+
full_run: "npx stryker run"
|
|
126
|
+
|
|
127
|
+
# ─────────────────────────────────────────────────────────
|
|
128
|
+
# When to Run
|
|
129
|
+
# ─────────────────────────────────────────────────────────
|
|
130
|
+
execution_timing:
|
|
131
|
+
- trigger: On-demand local run
|
|
132
|
+
command: "npm run test:mutation"
|
|
133
|
+
frequency: Before committing changes to critical modules
|
|
134
|
+
note: Mutation testing is slow (minutes to hours); do NOT run in every commit hook
|
|
135
|
+
|
|
136
|
+
- trigger: Pre-release quality gate
|
|
137
|
+
command: "npm run test:mutation -- --breakAt 60"
|
|
138
|
+
frequency: Before every release
|
|
139
|
+
enforcement: Break if overall score < 60%
|
|
140
|
+
|
|
141
|
+
- trigger: Critical module change
|
|
142
|
+
command: "npx stryker run --mutate 'src/license/**'"
|
|
143
|
+
frequency: Any change to auth/license/payment/security code
|
|
144
|
+
enforcement: Must maintain ≥ 80% on changed module
|
|
145
|
+
|
|
146
|
+
- trigger: AI-generated tests acceptance
|
|
147
|
+
command: "npx stryker run --mutate [module under test]"
|
|
148
|
+
frequency: Before accepting AI-generated test PRs
|
|
149
|
+
enforcement: Score < 50% → reject; require human-written tests
|
|
150
|
+
|
|
151
|
+
# ─────────────────────────────────────────────────────────
|
|
152
|
+
# Rules
|
|
153
|
+
# ─────────────────────────────────────────────────────────
|
|
154
|
+
rules:
|
|
155
|
+
- id: mutation-pre-release
|
|
156
|
+
trigger: preparing a release
|
|
157
|
+
instruction: Run mutation testing; overall score must be ≥ 60% to proceed
|
|
158
|
+
priority: required
|
|
159
|
+
|
|
160
|
+
- id: mutation-critical-modules
|
|
161
|
+
trigger: modifying auth, license, payment, or security code
|
|
162
|
+
instruction: Run module-scoped mutation testing; maintain ≥ 80% mutation score
|
|
163
|
+
priority: required
|
|
164
|
+
|
|
165
|
+
- id: mutation-ai-generated
|
|
166
|
+
trigger: accepting AI-generated test files
|
|
167
|
+
instruction: >
|
|
168
|
+
Run mutation testing on the module under test.
|
|
169
|
+
Score < 50% → reject tests; require human-authored replacements.
|
|
170
|
+
priority: required
|
|
171
|
+
|
|
172
|
+
- id: do-not-run-in-every-commit
|
|
173
|
+
trigger: planning CI pipeline
|
|
174
|
+
instruction: Do NOT add mutation testing to commit hooks or every-PR CI; it is too slow
|
|
175
|
+
priority: required
|
|
176
|
+
note: Reserve for pre-release gate and on-demand runs
|
|
177
|
+
|
|
178
|
+
anti_patterns:
|
|
179
|
+
- Treating 100% line coverage as sufficient (lines covered ≠ mutations killed)
|
|
180
|
+
- Adding mutation testing to pre-commit hooks (makes commits 10-60 minutes long)
|
|
181
|
+
- Accepting AI-generated tests without mutation score validation
|
|
182
|
+
- Killing mutations by adding trivial assertions (expect(x).toBeDefined())
|
|
183
|
+
- Targeting only happy paths in mutation testing (branches and boundaries are key)
|
|
184
|
+
|
|
185
|
+
quick_reference:
|
|
186
|
+
mutation_testing_checklist: |
|
|
187
|
+
□ Stryker configured for critical modules (license/*, auth/*, quota/*)
|
|
188
|
+
□ test:mutation script in package.json
|
|
189
|
+
□ Thresholds set: critical ≥ 80%, overall ≥ 60%, break at 50%
|
|
190
|
+
□ Pre-release: run full mutation suite before tagging version
|
|
191
|
+
□ AI-generated tests: validate with mutation score before accepting
|
|
192
|
+
□ NOT in commit hooks (too slow)
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# PII Classification and Handling Standards - AI Optimized
|
|
2
|
+
# Source: XSPEC-066 Wave 3 Compliance Pack
|
|
3
|
+
|
|
4
|
+
id: pii-classification
|
|
5
|
+
title: PII Classification and Handling Standards
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
status: Active
|
|
8
|
+
tags: [compliance, privacy, pii, gdpr, data-protection, security]
|
|
9
|
+
summary: |
|
|
10
|
+
Defines how Personally Identifiable Information (PII) and sensitive personal
|
|
11
|
+
data is classified, labeled, stored, transmitted, and disposed of. Covers
|
|
12
|
+
a three-tier data sensitivity classification, mandatory handling controls
|
|
13
|
+
per tier, data minimization principles, consent management requirements,
|
|
14
|
+
retention and deletion schedules, and cross-border transfer restrictions.
|
|
15
|
+
Aligned with GDPR Article 9, CCPA, and general privacy-by-design principles.
|
|
16
|
+
|
|
17
|
+
requirements:
|
|
18
|
+
- id: REQ-001
|
|
19
|
+
title: PII Data Sensitivity Classification
|
|
20
|
+
description: |
|
|
21
|
+
All data fields containing personal information MUST be classified into
|
|
22
|
+
one of three tiers before storage or processing. TIER-1 (Highly
|
|
23
|
+
Sensitive): health data, financial account numbers, government IDs,
|
|
24
|
+
biometrics, passwords, SSNs — requires encryption at rest and in
|
|
25
|
+
transit, access logging, no caching. TIER-2 (Sensitive): full name +
|
|
26
|
+
contact info combination, location history, behavioral profiles,
|
|
27
|
+
IP addresses — requires encryption in transit, access controls.
|
|
28
|
+
TIER-3 (General PII): first name only, country-level location, general
|
|
29
|
+
demographics — standard access controls sufficient.
|
|
30
|
+
level: MUST
|
|
31
|
+
examples:
|
|
32
|
+
- "Field: credit_card_number → TIER-1, encrypted AES-256-GCM, no logging of value"
|
|
33
|
+
- "Field: user_email + user_name together → TIER-2, TLS required, RBAC enforced"
|
|
34
|
+
- "Field: country_code → TIER-3, standard DB access controls"
|
|
35
|
+
|
|
36
|
+
- id: REQ-002
|
|
37
|
+
title: Data Minimization and Purpose Limitation
|
|
38
|
+
description: |
|
|
39
|
+
Systems MUST collect only the minimum PII necessary for the explicitly
|
|
40
|
+
stated purpose. Each PII field in the data model MUST have a documented
|
|
41
|
+
business purpose and legal basis (consent, contract, legitimate
|
|
42
|
+
interest, legal obligation). Collection of PII without documented
|
|
43
|
+
purpose is PROHIBITED. Purpose limitation MUST be enforced: data
|
|
44
|
+
collected for purpose A MUST NOT be used for unrelated purpose B
|
|
45
|
+
without separate consent.
|
|
46
|
+
level: MUST
|
|
47
|
+
examples:
|
|
48
|
+
- "Data dictionary entry: email_address, purpose: account authentication, legal_basis: contract"
|
|
49
|
+
- "Phone number collected for 2FA cannot be reused for marketing without new consent"
|
|
50
|
+
- "PR review checklist: 'Does this new field have a documented purpose in the data dictionary?'"
|
|
51
|
+
|
|
52
|
+
- id: REQ-003
|
|
53
|
+
title: PII Masking and Anonymization in Non-Production
|
|
54
|
+
description: |
|
|
55
|
+
PII MUST NOT exist in non-production environments (development, staging,
|
|
56
|
+
test) unless explicitly required and approved. Test and staging databases
|
|
57
|
+
MUST use anonymized or synthetic data. Any approved exception MUST be
|
|
58
|
+
time-limited, access-controlled, and documented. PII MUST be masked
|
|
59
|
+
in application logs: email addresses shown as u***@domain.com, phone
|
|
60
|
+
numbers as +1-XXX-XXX-1234, card numbers as ****-****-****-1234.
|
|
61
|
+
level: MUST
|
|
62
|
+
examples:
|
|
63
|
+
- "Staging DB: email stored as 'user_12345@test.invalid', not real email"
|
|
64
|
+
- "Log output: 'User u***@example.com logged in' not 'User alice@example.com logged in'"
|
|
65
|
+
- "Exception process: production data copy to staging requires security team approval + 7-day TTL"
|
|
66
|
+
|
|
67
|
+
- id: REQ-004
|
|
68
|
+
title: Data Retention and Deletion Schedule
|
|
69
|
+
description: |
|
|
70
|
+
Every data category containing PII MUST have a documented retention
|
|
71
|
+
schedule with maximum retention period aligned to legal requirements
|
|
72
|
+
and business need. Automated deletion MUST be implemented for data
|
|
73
|
+
past its retention period. Deletion MUST be verifiable (deletion
|
|
74
|
+
receipts or audit logs). Users exercising right-to-erasure MUST
|
|
75
|
+
receive deletion confirmation within 30 days (GDPR) or 45 days (CCPA).
|
|
76
|
+
level: MUST
|
|
77
|
+
examples:
|
|
78
|
+
- "Customer account data: retained 7 years after account closure (tax requirements)"
|
|
79
|
+
- "Session tokens: deleted after 24 hours of inactivity via automated cron job"
|
|
80
|
+
- "Right-to-erasure request: user data purged from all systems within 25 days, confirmation email sent"
|
|
81
|
+
|
|
82
|
+
- id: REQ-005
|
|
83
|
+
title: Cross-Border Data Transfer Controls
|
|
84
|
+
description: |
|
|
85
|
+
Transfers of TIER-1 or TIER-2 PII across national borders MUST comply
|
|
86
|
+
with applicable transfer mechanisms. EU → non-adequate country transfers
|
|
87
|
+
MUST use Standard Contractual Clauses (SCCs) or Binding Corporate Rules.
|
|
88
|
+
Data residency requirements MUST be documented in the system design.
|
|
89
|
+
Cross-border transfers MUST be logged with destination country and
|
|
90
|
+
legal basis.
|
|
91
|
+
level: MUST
|
|
92
|
+
examples:
|
|
93
|
+
- "EU user data stored in AWS eu-west-1, not replicated to us-east-1 without SCC"
|
|
94
|
+
- "Transfer log: destination=US, mechanism=SCC-2021, purpose=customer-support, timestamp=..."
|
|
95
|
+
- "Architecture doc notes: 'All PII stored in EU region per GDPR Article 46'"
|
|
96
|
+
|
|
97
|
+
- id: REQ-006
|
|
98
|
+
title: PII Impact Assessment for New Features
|
|
99
|
+
description: |
|
|
100
|
+
Any new feature or system change that introduces new PII collection or
|
|
101
|
+
processing SHOULD undergo a Privacy Impact Assessment (PIA) before
|
|
102
|
+
implementation. The PIA MUST document: what PII is collected, purpose,
|
|
103
|
+
legal basis, retention period, third-party sharing, and risk mitigations.
|
|
104
|
+
Features with TIER-1 PII require mandatory PIA; TIER-2 is recommended.
|
|
105
|
+
level: SHOULD
|
|
106
|
+
examples:
|
|
107
|
+
- "New feature: 'Save payment method' → PIA required (TIER-1 card data)"
|
|
108
|
+
- "PIA template: docs/templates/privacy-impact-assessment.md"
|
|
109
|
+
- "PIA outcome: fingerprint auth approved with biometric data stored only on-device"
|
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
# Policy as Code Testing Standards - AI Optimized
|
|
2
|
+
# Source: core/policy-as-code-testing.md
|
|
3
|
+
|
|
4
|
+
id: policy-as-code-testing
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-05"
|
|
8
|
+
source: core/policy-as-code-testing.md
|
|
9
|
+
description: >
|
|
10
|
+
Standards for unit testing Open Policy Agent (OPA) Rego policies and
|
|
11
|
+
other Policy as Code (PaC) engines. Ensures that AI agent authorization
|
|
12
|
+
policies are tested with the same rigor as application code.
|
|
13
|
+
|
|
14
|
+
# ─────────────────────────────────────────────────────────
|
|
15
|
+
# Core Concepts
|
|
16
|
+
# ─────────────────────────────────────────────────────────
|
|
17
|
+
core_concepts:
|
|
18
|
+
definition: >
|
|
19
|
+
Policy as Code (PaC) means security and authorization policies are expressed
|
|
20
|
+
as code (Rego, Cedar, CEL) rather than manual configuration. This enables
|
|
21
|
+
version control, code review, and automated testing of policies.
|
|
22
|
+
|
|
23
|
+
opa_test_framework:
|
|
24
|
+
overview: >
|
|
25
|
+
OPA's built-in test framework allows unit testing Rego policies with
|
|
26
|
+
`opa test`. Tests are Rego rules with names prefixed by `test_`.
|
|
27
|
+
Tests pass if they evaluate to `true`, fail if `false` or undefined.
|
|
28
|
+
run_command: "opa test <policy_directory> -v"
|
|
29
|
+
file_convention: "<policy_name>_test.rego in the same directory as the policy"
|
|
30
|
+
|
|
31
|
+
why_test_policies:
|
|
32
|
+
- reason: Policies encode security decisions — untested policies create silent security holes
|
|
33
|
+
- reason: Policy logic can have edge cases (reversible vs. irreversible, env-specific rules)
|
|
34
|
+
- reason: Policy changes must be validated against both allowed and denied cases
|
|
35
|
+
- reason: OPA Rego syntax errors are only caught at runtime without tests
|
|
36
|
+
|
|
37
|
+
# ─────────────────────────────────────────────────────────
|
|
38
|
+
# OPA Rego Test Structure
|
|
39
|
+
# ─────────────────────────────────────────────────────────
|
|
40
|
+
rego_test_structure:
|
|
41
|
+
file_naming: "<policy_module>_test.rego"
|
|
42
|
+
package_naming: "<policy_package>_test"
|
|
43
|
+
|
|
44
|
+
test_rule_format: |
|
|
45
|
+
# Each test is a Rego rule with `test_` prefix
|
|
46
|
+
# Test passes if rule body evaluates to true
|
|
47
|
+
test_<description> if {
|
|
48
|
+
<rule_under_test> with input as { <test_input> }
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
# Negative test (assert rule does NOT fire)
|
|
52
|
+
test_<description>_is_not_violated if {
|
|
53
|
+
not <rule_under_test> with input as { <test_input> }
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
required_test_categories:
|
|
57
|
+
- category: ALLOW cases
|
|
58
|
+
description: Inputs that must NOT trigger the policy violation
|
|
59
|
+
minimum: 2
|
|
60
|
+
example: |
|
|
61
|
+
test_safe_select_is_allowed if {
|
|
62
|
+
not data.my_pkg.has_violation with input as {
|
|
63
|
+
"plan": [{"command_type": "sql", "command": "SELECT * FROM t"}]
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
- category: DENY cases
|
|
68
|
+
description: Inputs that MUST trigger the policy violation
|
|
69
|
+
minimum: 3
|
|
70
|
+
example: |
|
|
71
|
+
test_drop_database_is_forbidden if {
|
|
72
|
+
data.my_pkg.has_forbidden_pattern with input as {
|
|
73
|
+
"plan": [{"command_type": "sql", "command": "DROP DATABASE prod"}]
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
- category: Boundary cases
|
|
78
|
+
description: Edge cases at the boundary of the policy condition
|
|
79
|
+
minimum: 1
|
|
80
|
+
example: |
|
|
81
|
+
# reversible=false triggers but reversible=true does not
|
|
82
|
+
test_irreversible_triggers if {
|
|
83
|
+
data.my_pkg.prod_violation with input as {
|
|
84
|
+
"target_env": "prod",
|
|
85
|
+
"plan": [{"reversible": false, "command": "DELETE FROM users"}]
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
test_reversible_does_not_trigger if {
|
|
89
|
+
not data.my_pkg.prod_violation with input as {
|
|
90
|
+
"target_env": "prod",
|
|
91
|
+
"plan": [{"reversible": true, "command": "SELECT * FROM users"}]
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
- category: Integration test (main policy)
|
|
96
|
+
description: Test the full policy chain via the main/root package
|
|
97
|
+
minimum: 2
|
|
98
|
+
|
|
99
|
+
# ─────────────────────────────────────────────────────────
|
|
100
|
+
# Policy Module Design Rules
|
|
101
|
+
# ─────────────────────────────────────────────────────────
|
|
102
|
+
policy_design_rules:
|
|
103
|
+
- rule: fail_closed_default
|
|
104
|
+
description: >
|
|
105
|
+
The root policy package MUST have `default allow = false`.
|
|
106
|
+
Any evaluation error or undefined result should deny, not allow.
|
|
107
|
+
example: |
|
|
108
|
+
default allow = false
|
|
109
|
+
allow if {
|
|
110
|
+
not data.my_pkg.forbidden.has_violation
|
|
111
|
+
not data.my_pkg.env.prod_violation
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
- rule: no_free_text_in_security_decisions
|
|
115
|
+
description: >
|
|
116
|
+
Policy rules MUST NOT parse user-controlled free-text fields (intent,
|
|
117
|
+
description, annotations) for security decisions. Only structured,
|
|
118
|
+
typed fields (command_type, reversible, target_env) should drive policy.
|
|
119
|
+
rationale: Free-text parsing creates prompt injection attack surface (OWASP LLM01)
|
|
120
|
+
|
|
121
|
+
- rule: set_not_array_for_violations
|
|
122
|
+
description: >
|
|
123
|
+
Use partial set rules (`violations[reason] if {...}`) to aggregate
|
|
124
|
+
violation reasons, not array rules. Arrays cannot be used with partial rules.
|
|
125
|
+
example: |
|
|
126
|
+
# CORRECT: partial set rule
|
|
127
|
+
violations[reason] if {
|
|
128
|
+
has_violation
|
|
129
|
+
reason := "VIOLATION_TYPE"
|
|
130
|
+
}
|
|
131
|
+
# INCORRECT: array.concat on sets causes type errors in OPA ≥ 0.40
|
|
132
|
+
# deny_reasons := array.concat(violations1, violations2) ← DO NOT USE
|
|
133
|
+
|
|
134
|
+
- rule: module_per_concern
|
|
135
|
+
description: >
|
|
136
|
+
Each policy concern should be a separate Rego module (file).
|
|
137
|
+
E.g., forbidden_patterns.rego / env_policy.rego / risk_gate.rego.
|
|
138
|
+
Main.rego aggregates all modules via data references.
|
|
139
|
+
benefit: Enables per-module testing and cleaner separation of concerns
|
|
140
|
+
|
|
141
|
+
# ─────────────────────────────────────────────────────────
|
|
142
|
+
# CI Integration
|
|
143
|
+
# ─────────────────────────────────────────────────────────
|
|
144
|
+
ci_integration:
|
|
145
|
+
github_actions_step: |
|
|
146
|
+
- name: Test OPA Rego Policies
|
|
147
|
+
run: |
|
|
148
|
+
docker run --rm \
|
|
149
|
+
-v "${{ github.workspace }}/src/guardian/policies:/policies:ro" \
|
|
150
|
+
openpolicyagent/opa:latest-static \
|
|
151
|
+
test /policies -v
|
|
152
|
+
|
|
153
|
+
npm_script: |
|
|
154
|
+
"test:policy": "docker run --rm -v \"$(pwd)/src/guardian/policies:/policies:ro\" openpolicyagent/opa:latest-static test /policies -v"
|
|
155
|
+
|
|
156
|
+
local_opa_binary: |
|
|
157
|
+
# If OPA binary is installed:
|
|
158
|
+
opa test src/guardian/policies/ -v
|
|
159
|
+
|
|
160
|
+
# ─────────────────────────────────────────────────────────
|
|
161
|
+
# Quality Gates
|
|
162
|
+
# ─────────────────────────────────────────────────────────
|
|
163
|
+
quality_gates:
|
|
164
|
+
- gate: OPA policy tests (CI)
|
|
165
|
+
threshold: "100% of Rego tests pass"
|
|
166
|
+
enforcement: Block merge
|
|
167
|
+
automated: true
|
|
168
|
+
note: Run via `opa test` on every PR that touches *.rego files
|
|
169
|
+
|
|
170
|
+
- gate: Policy coverage
|
|
171
|
+
threshold: "Each policy module has ≥ 2 ALLOW + ≥ 3 DENY test cases"
|
|
172
|
+
enforcement: Advisory (reviewer checklist)
|
|
173
|
+
|
|
174
|
+
- gate: Integration tests
|
|
175
|
+
threshold: "Root policy (main.rego) has tests for both allow and deny paths"
|
|
176
|
+
enforcement: Block merge
|
|
177
|
+
|
|
178
|
+
# ─────────────────────────────────────────────────────────
|
|
179
|
+
# Rules
|
|
180
|
+
# ─────────────────────────────────────────────────────────
|
|
181
|
+
rules:
|
|
182
|
+
- id: rego-unit-test-per-module
|
|
183
|
+
trigger: creating or modifying a Rego policy module
|
|
184
|
+
instruction: >
|
|
185
|
+
Every Rego module MUST have a corresponding _test.rego file with at minimum:
|
|
186
|
+
2 ALLOW cases, 3 DENY cases, and 1 boundary case.
|
|
187
|
+
priority: required
|
|
188
|
+
|
|
189
|
+
- id: fail-closed-default
|
|
190
|
+
trigger: creating a root OPA policy package
|
|
191
|
+
instruction: >
|
|
192
|
+
Root policy MUST include `default allow = false`.
|
|
193
|
+
Any undefined evaluation must result in DENY.
|
|
194
|
+
priority: required
|
|
195
|
+
|
|
196
|
+
- id: no-free-text-in-policy
|
|
197
|
+
trigger: writing Rego rules
|
|
198
|
+
instruction: >
|
|
199
|
+
Never parse intent, description, or annotation fields in Rego rules.
|
|
200
|
+
Use only structured fields: command_type, command, reversible, target_env,
|
|
201
|
+
target_resource, risk_score.
|
|
202
|
+
priority: required
|
|
203
|
+
|
|
204
|
+
- id: policy-test-on-rego-change
|
|
205
|
+
trigger: modifying any *.rego file
|
|
206
|
+
instruction: >
|
|
207
|
+
Re-run `opa test` on the entire policy directory after any change.
|
|
208
|
+
CI must block merge if OPA tests fail.
|
|
209
|
+
priority: required
|
|
210
|
+
|
|
211
|
+
anti_patterns:
|
|
212
|
+
- Using array.concat() on set-type violation rules (type error in OPA ≥ 0.40)
|
|
213
|
+
- Parsing intent/user-input fields in security policy logic
|
|
214
|
+
- Missing `default allow = false` in root policy
|
|
215
|
+
- Policy modules without corresponding _test.rego files
|
|
216
|
+
- Testing only DENY cases (no ALLOW cases means you can't tell if the policy is too restrictive)
|
|
217
|
+
- Running OPA tests only locally, not in CI
|
|
218
|
+
|
|
219
|
+
quick_reference:
|
|
220
|
+
policy_test_checklist: |
|
|
221
|
+
□ Each policy module has a _test.rego file
|
|
222
|
+
□ Tests cover: ALLOW cases (≥ 2), DENY cases (≥ 3), boundary cases (≥ 1)
|
|
223
|
+
□ main.rego / root policy tested via integration tests
|
|
224
|
+
□ `default allow = false` present in root policy
|
|
225
|
+
□ No free-text field parsing in Rego rules
|
|
226
|
+
□ `opa test <policies_dir> -v` passes locally
|
|
227
|
+
□ CI step: `opa test` runs on every PR touching *.rego
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# PRD Standards - AI Optimized
|
|
2
|
+
# Source: XSPEC-069 Wave 4 Product Layer Pack
|
|
3
|
+
|
|
4
|
+
id: prd-standards
|
|
5
|
+
title: Product Requirements Document Standards
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
status: Active
|
|
8
|
+
tags: [product, prd, requirements, user-research, planning]
|
|
9
|
+
summary: |
|
|
10
|
+
Defines the structure, content requirements, and lifecycle governance for
|
|
11
|
+
Product Requirements Documents (PRDs). Covers five mandatory PRD sections
|
|
12
|
+
(Problem Statement, Target User/Persona, Success Metrics, Scope in/out,
|
|
13
|
+
Constraints), the bridge from PRD requirements to traceable user stories,
|
|
14
|
+
and the revision policy for changes after kickoff. Designed to ensure product
|
|
15
|
+
intent is clearly communicable to engineering, design, and stakeholders with
|
|
16
|
+
measurable success criteria.
|
|
17
|
+
|
|
18
|
+
requirements:
|
|
19
|
+
- id: REQ-001
|
|
20
|
+
title: PRD Five Sections
|
|
21
|
+
description: |
|
|
22
|
+
Every PRD MUST contain five sections in the following order:
|
|
23
|
+
(1) Problem Statement — describes the user pain point or opportunity
|
|
24
|
+
in observable terms; includes quantitative data where available (e.g.,
|
|
25
|
+
support ticket volume, user research findings, funnel drop-off rates).
|
|
26
|
+
(2) Target User / Persona — identifies who is affected; references
|
|
27
|
+
named personas with role, context, and goals; at minimum one primary
|
|
28
|
+
persona and one secondary persona.
|
|
29
|
+
(3) Success Metrics — defines 2–4 measurable outcomes that indicate
|
|
30
|
+
the problem is solved; each metric must include current baseline,
|
|
31
|
+
target value, and measurement method.
|
|
32
|
+
(4) Scope In / Out — explicitly lists what is included and excluded
|
|
33
|
+
from this PRD; out-of-scope items may reference future PRDs.
|
|
34
|
+
(5) Constraints — technical, regulatory, time, budget, or dependency
|
|
35
|
+
constraints that bound the solution space.
|
|
36
|
+
level: MUST
|
|
37
|
+
examples:
|
|
38
|
+
- "Problem: 34% of users abandon checkout at payment step (Mixpanel, Q1 2026)"
|
|
39
|
+
- "Primary persona: Mid-market SaaS buyer; Secondary: IT admin approver"
|
|
40
|
+
- "Success metric: Checkout completion rate ≥ 72% (baseline 66%) measured via Amplitude"
|
|
41
|
+
- "Out of scope: saved payment methods (deferred to PRD-2026-Q3-payments)"
|
|
42
|
+
|
|
43
|
+
- id: REQ-002
|
|
44
|
+
title: PRD to User Story Bridge
|
|
45
|
+
description: |
|
|
46
|
+
Each PRD requirement MUST be broken down into one or more user stories
|
|
47
|
+
following the INVEST criteria defined in requirement-engineering.ai.yaml.
|
|
48
|
+
Every user story derived from a PRD MUST be traceable to at least one
|
|
49
|
+
PRD success metric — stories that cannot be linked to a success metric
|
|
50
|
+
MUST be flagged for PM review before inclusion in the backlog. The
|
|
51
|
+
traceability link (PRD section ID → User Story ID → Success Metric) MUST
|
|
52
|
+
be maintained in the backlog tool or as a traceability matrix in the PRD.
|
|
53
|
+
level: MUST
|
|
54
|
+
examples:
|
|
55
|
+
- "PRD-REQ-003 → US-042 'As a buyer, I want to pay with Apple Pay' → metric: checkout rate"
|
|
56
|
+
- "Story without metric link flagged with label `needs-metric-trace` in Jira"
|
|
57
|
+
- "Traceability matrix table in PRD section 6: REQ ↔ Stories ↔ Metrics"
|
|
58
|
+
- "Stories use INVEST: Independent, Negotiable, Valuable, Estimable, Small, Testable"
|
|
59
|
+
|
|
60
|
+
- id: REQ-003
|
|
61
|
+
title: Revision Policy
|
|
62
|
+
description: |
|
|
63
|
+
PRD changes requested after the development kickoff meeting MUST follow
|
|
64
|
+
a formal revision process: (1) Proposed change documented with rationale
|
|
65
|
+
and impact assessment. (2) Stakeholder sign-off obtained from PM, Tech
|
|
66
|
+
Lead, and Design Lead. (3) Scope impact assessed — if change adds scope,
|
|
67
|
+
a corresponding item must be moved to out-of-scope or timeline adjusted.
|
|
68
|
+
(4) Version history updated in the PRD with date, author, change summary,
|
|
69
|
+
and approver. PRDs without version history that have been modified after
|
|
70
|
+
kickoff are considered non-compliant.
|
|
71
|
+
level: MUST
|
|
72
|
+
examples:
|
|
73
|
+
- "PRD v1.2 (2026-04-15, @alice): Added biometric auth requirement; approved by @bob, @carol"
|
|
74
|
+
- "Scope impact: biometric auth added → saved cards feature deferred to v2"
|
|
75
|
+
- "Change log table at PRD top: Version | Date | Author | Summary | Approvers"
|
|
76
|
+
- "Minor editorial changes (typos, formatting) exempt from sign-off requirement"
|
|
77
|
+
|
|
78
|
+
anti_patterns:
|
|
79
|
+
- "PRD without measurable success metrics (qualitative goals only, e.g., 'improve UX')"
|
|
80
|
+
- "Scope creep without change log: adding requirements mid-sprint without documented approval"
|
|
81
|
+
- "Solution-first PRD: describing implementation details before establishing user problem"
|
|
82
|
+
- "PRD with no explicit out-of-scope section, causing boundary disputes during development"
|
|
83
|
+
- "Success metrics defined after development starts, making them unverifiable"
|
|
84
|
+
|
|
85
|
+
related_standards:
|
|
86
|
+
- requirement-engineering
|
|
87
|
+
- user-story-mapping
|
|
88
|
+
- product-metrics-standards
|