universal-dev-standards 5.3.2 → 5.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (122) hide show
  1. package/bundled/ai/standards/adversarial-test.ai.yaml +277 -0
  2. package/bundled/ai/standards/agent-communication-protocol.ai.yaml +32 -166
  3. package/bundled/ai/standards/agent-dispatch.ai.yaml +32 -58
  4. package/bundled/ai/standards/audit-trail.ai.yaml +113 -0
  5. package/bundled/ai/standards/branch-completion.ai.yaml +34 -70
  6. package/bundled/ai/standards/change-batching-standards.ai.yaml +31 -180
  7. package/bundled/ai/standards/chaos-injection-tests.ai.yaml +91 -0
  8. package/bundled/ai/standards/container-image-standards.ai.yaml +88 -0
  9. package/bundled/ai/standards/container-security.ai.yaml +331 -0
  10. package/bundled/ai/standards/cost-budget-test.ai.yaml +96 -0
  11. package/bundled/ai/standards/data-contract.ai.yaml +110 -0
  12. package/bundled/ai/standards/data-migration-testing.ai.yaml +96 -0
  13. package/bundled/ai/standards/data-pipeline.ai.yaml +113 -0
  14. package/bundled/ai/standards/disaster-recovery-drill.ai.yaml +89 -0
  15. package/bundled/ai/standards/execution-history.ai.yaml +30 -288
  16. package/bundled/ai/standards/flaky-test-management.ai.yaml +89 -0
  17. package/bundled/ai/standards/flow-based-testing.ai.yaml +240 -0
  18. package/bundled/ai/standards/iac-design-principles.ai.yaml +83 -0
  19. package/bundled/ai/standards/incident-response.ai.yaml +107 -0
  20. package/bundled/ai/standards/license-compliance.ai.yaml +106 -0
  21. package/bundled/ai/standards/llm-output-validation.ai.yaml +269 -0
  22. package/bundled/ai/standards/mock-boundary.ai.yaml +250 -0
  23. package/bundled/ai/standards/mutation-testing.ai.yaml +192 -0
  24. package/bundled/ai/standards/pii-classification.ai.yaml +109 -0
  25. package/bundled/ai/standards/pipeline-integration-standards.ai.yaml +28 -169
  26. package/bundled/ai/standards/policy-as-code-testing.ai.yaml +227 -0
  27. package/bundled/ai/standards/prd-standards.ai.yaml +88 -0
  28. package/bundled/ai/standards/product-metrics-standards.ai.yaml +111 -0
  29. package/bundled/ai/standards/prompt-regression.ai.yaml +94 -0
  30. package/bundled/ai/standards/property-based-testing.ai.yaml +105 -0
  31. package/bundled/ai/standards/release-quality-manifest.ai.yaml +135 -0
  32. package/bundled/ai/standards/replay-test.ai.yaml +111 -0
  33. package/bundled/ai/standards/runbook.ai.yaml +104 -0
  34. package/bundled/ai/standards/sast-advanced.ai.yaml +135 -0
  35. package/bundled/ai/standards/schema-evolution.ai.yaml +111 -0
  36. package/bundled/ai/standards/secret-management-standards.ai.yaml +105 -0
  37. package/bundled/ai/standards/secure-op.ai.yaml +365 -0
  38. package/bundled/ai/standards/security-testing.ai.yaml +171 -0
  39. package/bundled/ai/standards/server-ops-security.ai.yaml +274 -0
  40. package/bundled/ai/standards/slo-sli.ai.yaml +97 -0
  41. package/bundled/ai/standards/smoke-test.ai.yaml +87 -0
  42. package/bundled/ai/standards/supply-chain-attestation.ai.yaml +109 -0
  43. package/bundled/ai/standards/test-completeness-dimensions.ai.yaml +52 -5
  44. package/bundled/ai/standards/user-story-mapping.ai.yaml +108 -0
  45. package/bundled/ai/standards/workflow-enforcement.ai.yaml +34 -240
  46. package/bundled/ai/standards/workflow-state-protocol.ai.yaml +31 -107
  47. package/bundled/core/adversarial-test.md +212 -0
  48. package/bundled/core/chaos-injection-tests.md +116 -0
  49. package/bundled/core/container-security.md +521 -0
  50. package/bundled/core/cost-budget-test.md +69 -0
  51. package/bundled/core/data-migration-testing.md +110 -0
  52. package/bundled/core/disaster-recovery-drill.md +73 -0
  53. package/bundled/core/flaky-test-management.md +73 -0
  54. package/bundled/core/flow-based-testing.md +142 -0
  55. package/bundled/core/llm-output-validation.md +178 -0
  56. package/bundled/core/mock-boundary.md +100 -0
  57. package/bundled/core/mutation-testing.md +97 -0
  58. package/bundled/core/policy-as-code-testing.md +188 -0
  59. package/bundled/core/prompt-regression.md +72 -0
  60. package/bundled/core/property-based-testing.md +73 -0
  61. package/bundled/core/release-quality-manifest.md +147 -0
  62. package/bundled/core/replay-test.md +86 -0
  63. package/bundled/core/sast-advanced.md +300 -0
  64. package/bundled/core/secure-op.md +314 -0
  65. package/bundled/core/security-testing.md +87 -0
  66. package/bundled/core/server-ops-security.md +493 -0
  67. package/bundled/core/smoke-test.md +65 -0
  68. package/bundled/core/supply-chain-attestation.md +117 -0
  69. package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
  70. package/bundled/locales/zh-CN/README.md +1 -1
  71. package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +5 -5
  72. package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
  73. package/bundled/locales/zh-TW/README.md +1 -1
  74. package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +183 -79
  75. package/bundled/skills/README.md +4 -3
  76. package/bundled/skills/SKILL_NAMING.md +94 -0
  77. package/bundled/skills/ai-instruction-standards/SKILL.md +181 -88
  78. package/bundled/skills/atdd-assistant/SKILL.md +8 -0
  79. package/bundled/skills/bdd-assistant/SKILL.md +7 -0
  80. package/bundled/skills/checkin-assistant/SKILL.md +8 -0
  81. package/bundled/skills/code-review-assistant/SKILL.md +7 -0
  82. package/bundled/skills/journey-test-assistant/SKILL.md +203 -0
  83. package/bundled/skills/orchestrate/SKILL.md +167 -0
  84. package/bundled/skills/plan/SKILL.md +234 -0
  85. package/bundled/skills/pr-automation-assistant/SKILL.md +8 -0
  86. package/bundled/skills/push/SKILL.md +49 -2
  87. package/bundled/skills/{process-automation → skill-builder}/SKILL.md +1 -1
  88. package/bundled/skills/{forward-derivation → spec-derivation}/SKILL.md +1 -1
  89. package/bundled/skills/spec-driven-dev/SKILL.md +7 -0
  90. package/bundled/skills/sweep/SKILL.md +145 -0
  91. package/bundled/skills/tdd-assistant/SKILL.md +7 -0
  92. package/package.json +1 -1
  93. package/src/commands/flow.js +8 -0
  94. package/src/commands/start.js +14 -0
  95. package/src/commands/sweep.js +8 -0
  96. package/src/commands/workflow.js +8 -0
  97. package/standards-registry.json +474 -12
  98. package/bundled/locales/zh-CN/skills/ac-coverage-assistant/SKILL.md +0 -190
  99. package/bundled/locales/zh-CN/skills/forward-derivation/SKILL.md +0 -71
  100. package/bundled/locales/zh-CN/skills/forward-derivation/guide.md +0 -130
  101. package/bundled/locales/zh-CN/skills/methodology-system/SKILL.md +0 -88
  102. package/bundled/locales/zh-CN/skills/methodology-system/create-methodology.md +0 -350
  103. package/bundled/locales/zh-CN/skills/methodology-system/guide.md +0 -131
  104. package/bundled/locales/zh-CN/skills/methodology-system/runtime.md +0 -279
  105. package/bundled/locales/zh-CN/skills/process-automation/SKILL.md +0 -143
  106. package/bundled/locales/zh-TW/skills/ac-coverage-assistant/SKILL.md +0 -195
  107. package/bundled/locales/zh-TW/skills/deploy-assistant/SKILL.md +0 -178
  108. package/bundled/locales/zh-TW/skills/forward-derivation/SKILL.md +0 -69
  109. package/bundled/locales/zh-TW/skills/forward-derivation/guide.md +0 -415
  110. package/bundled/locales/zh-TW/skills/methodology-system/SKILL.md +0 -86
  111. package/bundled/locales/zh-TW/skills/methodology-system/create-methodology.md +0 -350
  112. package/bundled/locales/zh-TW/skills/methodology-system/guide.md +0 -131
  113. package/bundled/locales/zh-TW/skills/methodology-system/runtime.md +0 -279
  114. package/bundled/locales/zh-TW/skills/process-automation/SKILL.md +0 -144
  115. /package/bundled/skills/{ac-coverage-assistant → ac-coverage}/SKILL.md +0 -0
  116. /package/bundled/skills/{methodology-system → dev-methodology}/SKILL.md +0 -0
  117. /package/bundled/skills/{methodology-system → dev-methodology}/create-methodology.md +0 -0
  118. /package/bundled/skills/{methodology-system → dev-methodology}/guide.md +0 -0
  119. /package/bundled/skills/{methodology-system → dev-methodology}/integrated-flow.md +0 -0
  120. /package/bundled/skills/{methodology-system → dev-methodology}/prerequisite-check.md +0 -0
  121. /package/bundled/skills/{methodology-system → dev-methodology}/runtime.md +0 -0
  122. /package/bundled/skills/{forward-derivation → spec-derivation}/guide.md +0 -0
@@ -0,0 +1,331 @@
1
+ # Container Security Standard - AI Optimized
2
+ # Source: core/container-security.md
3
+
4
+ id: container-security
5
+ meta:
6
+ version: "1.0.0"
7
+ updated: "2026-05-04"
8
+ source: core/container-security.md
9
+ description: >
10
+ Container and Kubernetes security covering image hardening, registry security,
11
+ runtime protection, secrets management, network policy, and supply chain
12
+ integrity for AI Agent production environments.
13
+
14
+ # ─────────────────────────────────────────────────────────
15
+ # Core Categories
16
+ # ─────────────────────────────────────────────────────────
17
+ categories:
18
+ - id: image_hardening
19
+ name: Image Hardening
20
+ description: Container images must be minimal, non-root, and built with multi-stage patterns
21
+ base_image:
22
+ preferred:
23
+ - distroless (gcr.io/distroless/*)
24
+ - alpine (< 10 MB)
25
+ prohibited:
26
+ - ubuntu/debian as final layer with full package manager
27
+ - latest tag on base image
28
+ dockerfile_rules:
29
+ - rule: Use multi-stage build; only copy artifacts to final stage
30
+ - rule: Final stage must set non-root USER (UID 1000)
31
+ - rule: No RUN apt-get / apk add in final layer (use builder stage only)
32
+ - rule: Only COPY specific files; never COPY . . in final stage
33
+ - rule: Pin base image by digest, not tag
34
+ security_context_k8s:
35
+ required:
36
+ - runAsNonRoot: true
37
+ - runAsUser: 1000
38
+ - allowPrivilegeEscalation: false
39
+ - readOnlyRootFilesystem: true
40
+ seccomp:
41
+ profile: RuntimeDefault
42
+ capabilities:
43
+ drop: [ALL]
44
+ add: explicit allowlist only (e.g., NET_BIND_SERVICE if needed)
45
+ ai_agent_specific:
46
+ - rule: AI Agent container must run as UID 1000 (non-root)
47
+ - rule: LLM call process must not have CAP_NET_RAW or CAP_SYS_ADMIN
48
+ iso_mapping:
49
+ - "ISO/IEC 27001:2022 A.8.9 Configuration Management"
50
+ - "NIST SP 800-190 §4.1 Image Vulnerabilities"
51
+ - "CIS Docker Benchmark v1.6 §4 Container Images"
52
+
53
+ - id: registry_security
54
+ name: Registry Security
55
+ description: All container images must be stored in private registries with signing and vulnerability scanning
56
+ registry:
57
+ type: private (GHCR, ECR, GCR, Harbor)
58
+ public_registry: prohibited for production images
59
+ tagging:
60
+ immutable_tag: required (semver or git SHA)
61
+ latest_tag: prohibited in production deployments
62
+ pinned_digest: required for supply chain integrity
63
+ signing:
64
+ tools: [Cosign, Notary v2]
65
+ policy: All production images must be signed before push
66
+ verification: Admission controller (Kyverno/Gatekeeper) verifies signature on deploy
67
+ vulnerability_scan:
68
+ tool: Trivy
69
+ gate:
70
+ critical: 0 (block push if any critical CVE)
71
+ high: 0 (block push if any high CVE)
72
+ frequency: On every build + weekly re-scan of deployed images
73
+ editions:
74
+ trial: Trivy gate mandatory
75
+ community: Trivy gate mandatory
76
+ commercial: Trivy gate mandatory
77
+ ai_agent_specific:
78
+ - rule: Guardian OPA Sidecar image must also pass Trivy gate
79
+ - rule: Trial/Community edition images enforce Trivy gate to avoid supply chain compromise
80
+ iso_mapping:
81
+ - "ISO/IEC 27001:2022 A.8.8 Management of Technical Vulnerabilities"
82
+ - "NIST SP 800-190 §4.2 Image Configuration Defects"
83
+ - "CIS Docker Benchmark v1.6 §2 Docker daemon configuration"
84
+
85
+ - id: runtime_security
86
+ name: Runtime Security
87
+ description: Running containers must have minimal capabilities and access; privileged mode is prohibited
88
+ prohibitions:
89
+ - privileged: true in securityContext
90
+ - hostNetwork: true (unless explicitly required and documented)
91
+ - hostPID: true
92
+ - hostIPC: true
93
+ required_settings:
94
+ - readOnlyRootFilesystem: true
95
+ - allowPrivilegeEscalation: false
96
+ - drop: [ALL] capabilities
97
+ seccomp:
98
+ profile: RuntimeDefault (or custom restricted profile)
99
+ audit_mode: allowed for profiling only; not for production
100
+ apparmor_selinux:
101
+ apparmor: docker-default (Kubernetes annotation)
102
+ selinux: enforcing mode on RHEL/CentOS hosts
103
+ writable_volumes:
104
+ pattern: Mount named volumes or PersistentVolumeClaims for writable data
105
+ tmpfs: allowed for /tmp when truly ephemeral
106
+ ai_agent_specific:
107
+ - rule: Audit log volume must be readOnly:false but backed by append-only partition
108
+ - rule: Agent working directory may use emptyDir; audit log must not
109
+ iso_mapping:
110
+ - "ISO/IEC 27001:2022 A.8.9 Configuration Management"
111
+ - "NIST SP 800-190 §4.3 Container Runtime Vulnerabilities"
112
+ - "CIS Docker Benchmark v1.6 §5 Container Runtime"
113
+
114
+ - id: secrets_management
115
+ name: Secrets Management
116
+ description: Secrets must never be embedded in images or passed via environment variables from plain text
117
+ prohibited:
118
+ - Hardcoded secrets in Dockerfile (ENV SECRET=...)
119
+ - Secrets in docker-compose.yml env: section as plain text
120
+ - ConfigMap storing sensitive values in K8s
121
+ - Git-tracked .env files containing real credentials
122
+ allowed:
123
+ staging:
124
+ - K8s Secrets (base64) + RBAC restriction
125
+ - Sealed Secrets (Bitnami) for GitOps
126
+ production:
127
+ - HashiCorp Vault (dynamic secrets)
128
+ - AWS Secrets Manager / GCP Secret Manager
129
+ - Azure Key Vault
130
+ injection_patterns:
131
+ - pattern: Mounted volume (/run/secrets/)
132
+ preferred: true
133
+ - pattern: Init container fetches secret and writes to shared emptyDir
134
+ acceptable: true
135
+ - pattern: Environment variable from K8s Secret reference
136
+ acceptable: true (better than plain text, but volume preferred)
137
+ ai_agent_specific:
138
+ - rule: LLM API keys must be injected via mounted secret volume, not ENV
139
+ - rule: Guardian OPA policy files are not secrets; use ConfigMap
140
+ iso_mapping:
141
+ - "ISO/IEC 27001:2022 A.8.12 Data Leakage Prevention"
142
+ - "ISO/IEC 27001:2022 A.8.24 Use of Cryptography"
143
+ - "NIST SP 800-190 §4.4 Container Image and Registry Vulnerabilities"
144
+
145
+ - id: network_policy
146
+ name: Network Policy
147
+ description: K8s NetworkPolicy must default-deny; only necessary ingress/egress is permitted
148
+ default_policy:
149
+ ingress: deny-all (explicit allow per service)
150
+ egress: deny-all (explicit allow per service)
151
+ required_allow:
152
+ dns: "kube-dns (UDP/TCP 53) must be allowed for all pods"
153
+ health_checks: "kubelet health check endpoints"
154
+ ai_agent_outbound_allowlist:
155
+ - host: api.anthropic.com
156
+ port: 443
157
+ protocol: HTTPS
158
+ - host: api.openai.com
159
+ port: 443
160
+ protocol: HTTPS
161
+ - host: "bedrock.*.amazonaws.com"
162
+ port: 443
163
+ protocol: HTTPS
164
+ - host: registry.npmjs.org
165
+ port: 443
166
+ protocol: HTTPS
167
+ note: Only during build phase; not at runtime
168
+ guardian_sidecar:
169
+ placement:
170
+ - same Pod as AI Agent (sidecar container)
171
+ - same K8s namespace minimum
172
+ communication: localhost (sidecar) or ClusterIP service (same namespace)
173
+ tools:
174
+ policy_enforcement: [Calico, Cilium, Kubernetes NetworkPolicy]
175
+ validation: [kubectl describe networkpolicy, Kyverno policy report]
176
+ ai_agent_specific:
177
+ - rule: AI Agent pod egress must be restricted to outbound allowlist
178
+ - rule: Guardian OPA Sidecar must be same Pod or same namespace
179
+ - rule: Any new LLM provider endpoint must be added to allowlist and reviewed
180
+ iso_mapping:
181
+ - "ISO/IEC 27001:2022 A.8.20 Networks Security"
182
+ - "ISO/IEC 27001:2022 A.8.22 Web Filtering"
183
+ - "CIS Kubernetes Benchmark v1.8 §5.3 Network Policies and CNI"
184
+
185
+ - id: supply_chain
186
+ name: Supply Chain Security
187
+ description: Container build process must be reproducible, auditable, and signed
188
+ sbom:
189
+ formats: [CycloneDX, SPDX]
190
+ generation:
191
+ tool: Syft or trivy sbom
192
+ trigger: Every image build
193
+ storage: Attached to registry as OCI artifact
194
+ image_pinning:
195
+ rule: All FROM directives must use digest (sha256:...), not floating tag
196
+ example: "FROM gcr.io/distroless/static-debian12@sha256:<digest>"
197
+ signing:
198
+ tool: Sigstore (keyless signing via GitHub Actions OIDC)
199
+ verification: Cosign verify --certificate-identity-regexp
200
+ reproducible_build:
201
+ - Set SOURCE_DATE_EPOCH for deterministic timestamps
202
+ - Use --no-cache in CI; never rely on layer cache across builds
203
+ lockfile_verification:
204
+ nodejs:
205
+ file: package-lock.json or yarn.lock
206
+ command: "npm ci (not npm install)"
207
+ python:
208
+ file: requirements.txt with pinned versions or Pipfile.lock
209
+ command: "pip install --require-hashes -r requirements.txt"
210
+ golang:
211
+ file: go.sum
212
+ command: "go mod verify"
213
+ ai_agent_specific:
214
+ - rule: SBOM must be generated for every AI Agent image release
215
+ - rule: Third-party AI SDK (anthropic, openai) versions must be pinned in lockfile
216
+ iso_mapping:
217
+ - "ISO/IEC 27001:2022 A.8.8 Management of Technical Vulnerabilities"
218
+ - "NIST SP 800-190 §4.5 Host OS Vulnerabilities"
219
+ - "SLSA Framework Level 2+ (provenance attestation)"
220
+
221
+ # ─────────────────────────────────────────────────────────
222
+ # Quality Gates
223
+ # ─────────────────────────────────────────────────────────
224
+ quality_gates:
225
+ pre_build:
226
+ - base_image_pinned_by_digest: required
227
+ - multi_stage_build: required
228
+ - no_secrets_in_dockerfile: required
229
+ pre_push:
230
+ - trivy_critical_count: 0
231
+ - trivy_high_count: 0
232
+ - image_signed: required (cosign or notary)
233
+ - sbom_generated: required
234
+ pre_deploy:
235
+ - image_tag_immutable: required (no 'latest')
236
+ - k8s_security_context:
237
+ runAsNonRoot: true
238
+ readOnlyRootFilesystem: true
239
+ allowPrivilegeEscalation: false
240
+ - network_policy_applied: required
241
+ - secrets_from_vault_or_k8s_secret: required
242
+ ai_agent_specific:
243
+ - agent_uid: 1000
244
+ - guardian_sidecar: same_pod_or_namespace
245
+ - outbound_allowlist: configured
246
+ - audit_log_volume: append_only_partition
247
+
248
+ # ─────────────────────────────────────────────────────────
249
+ # Rules
250
+ # ─────────────────────────────────────────────────────────
251
+ rules:
252
+ - id: non-root-container
253
+ trigger: writing any Dockerfile or K8s Pod spec
254
+ instruction: >
255
+ Set USER 1000 in Dockerfile final stage.
256
+ Set runAsNonRoot: true and runAsUser: 1000 in securityContext.
257
+ Verify: docker inspect --format='{{.Config.User}}' <image>
258
+ priority: required
259
+
260
+ - id: no-latest-tag
261
+ trigger: tagging or pulling container images
262
+ instruction: >
263
+ Never use 'latest' tag in production. Use semver (v1.2.3) or git SHA.
264
+ For base images, pin by digest: FROM image@sha256:<digest>
265
+ priority: required
266
+
267
+ - id: trivy-gate-before-push
268
+ trigger: building container image for any environment
269
+ instruction: >
270
+ Run: trivy image --exit-code 1 --severity CRITICAL,HIGH <image>
271
+ Block push if exit code is non-zero. Fix all critical/high CVEs first.
272
+ priority: required
273
+
274
+ - id: drop-all-capabilities
275
+ trigger: writing K8s Pod or Deployment spec
276
+ instruction: >
277
+ Set capabilities.drop: [ALL] in securityContext.
278
+ Only add specific capabilities if explicitly required (document justification).
279
+ priority: required
280
+
281
+ - id: default-deny-network-policy
282
+ trigger: deploying any service to Kubernetes
283
+ instruction: >
284
+ Apply default-deny NetworkPolicy to namespace before deploying services.
285
+ Then add explicit allow policies per service communication requirement.
286
+ priority: required
287
+
288
+ - id: no-env-secrets
289
+ trigger: any Dockerfile ENV or K8s env[] with sensitive values
290
+ instruction: >
291
+ Never pass secrets via ENV. Use K8s Secret + volume mount or Vault agent injector.
292
+ Check: grep -r "ENV.*KEY\|ENV.*SECRET\|ENV.*TOKEN" Dockerfile
293
+ priority: required
294
+
295
+ - id: sign-and-sbom
296
+ trigger: releasing container image to production
297
+ instruction: >
298
+ Generate SBOM: syft <image> -o cyclonedx-json > sbom.json
299
+ Sign image: cosign sign <image>@<digest>
300
+ Attach SBOM: cosign attach sbom --sbom sbom.json <image>@<digest>
301
+ priority: required
302
+
303
+ anti_patterns:
304
+ - FROM ubuntu:latest (floating tag, large attack surface)
305
+ - RUN apt-get update && apt-get install in final Dockerfile stage
306
+ - Running container as root (USER root or no USER directive)
307
+ - ENV API_KEY=<real_key> in Dockerfile
308
+ - privileged: true in K8s securityContext
309
+ - No NetworkPolicy (allow all egress by default)
310
+ - Skipping Trivy scan on "quick" builds
311
+ - Using 'latest' image tag in Kubernetes Deployment
312
+ - SBOM not generated for production releases
313
+
314
+ quick_reference:
315
+ container_security_checklist: |
316
+ □ Base image pinned by digest (not tag)
317
+ □ Multi-stage build; only COPY artifacts to final stage
318
+ □ Final stage: USER 1000 (non-root)
319
+ □ securityContext: runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation:false
320
+ □ capabilities.drop: [ALL]
321
+ □ seccompProfile: RuntimeDefault
322
+ □ Trivy scan: 0 critical, 0 high before push
323
+ □ Image signed (Cosign / Notary v2)
324
+ □ SBOM generated (CycloneDX or SPDX)
325
+ □ No secrets in Dockerfile ENV or image layers
326
+ □ K8s Secrets or Vault for secret injection
327
+ □ NetworkPolicy: default-deny + explicit allow
328
+ □ AI Agent egress: allowlist (Anthropic, OpenAI, Bedrock)
329
+ □ Guardian OPA Sidecar: same Pod or same namespace
330
+ □ Audit log volume: append-only partition (not emptyDir)
331
+ □ Lockfile pinned (npm ci / pip --require-hashes / go mod verify)
@@ -0,0 +1,96 @@
1
+ # SPDX-License-Identifier: MIT
2
+ name: Cost Budget Test Standards
3
+ nameZh: 成本預算測試標準
4
+ id: cost-budget-test
5
+ version: "1.0.0"
6
+ category: testing
7
+ scope: ai-agent-systems
8
+ summary: >
9
+ Unit and integration tests that verify token budget zone classification,
10
+ pipeline cost thresholds, and runaway-loop prevention for AI agent systems.
11
+
12
+ requirements:
13
+ - id: REQ-01
14
+ title: Zone Classification Boundary Tests
15
+ titleZh: 區間分類邊界測試
16
+ level: MUST
17
+ description: >
18
+ The token budget zone classifier MUST have unit tests covering all four
19
+ zones (safe/warning/danger/blocking) and every boundary value (the exact
20
+ threshold values, one below, and one above). This ensures that threshold
21
+ drift is caught by the test suite.
22
+ implementation: |
23
+ // Test all boundaries:
24
+ classifyTokenZone(0.84) → "safe"
25
+ classifyTokenZone(0.85) → "warning" // WARNING_THRESHOLD
26
+ classifyTokenZone(0.89) → "warning"
27
+ classifyTokenZone(0.90) → "danger" // DANGER_THRESHOLD
28
+ classifyTokenZone(0.94) → "danger"
29
+ classifyTokenZone(0.95) → "blocking" // BLOCKING_THRESHOLD
30
+ classifyTokenZone(1.00) → "blocking"
31
+
32
+ - id: REQ-02
33
+ title: Pipeline Budget Config Validation Tests
34
+ titleZh: Pipeline 預算設定驗證測試
35
+ level: MUST
36
+ description: >
37
+ Tests MUST verify that PipelineBudgetConfig properties (maxCostPerRun,
38
+ maxCostPerDay, warningThreshold, autoDowngrade) have correct semantics.
39
+ At minimum: verify that exceeding maxCostPerRun triggers the expected
40
+ budget-exceeded behaviour.
41
+
42
+ - id: REQ-03
43
+ title: Overflow and Runaway Detection
44
+ titleZh: 溢位與失控偵測測試
45
+ level: MUST
46
+ description: >
47
+ Tests MUST include edge cases for budget overflow: usageRatio > 1.0
48
+ (over-budget), exactly 0.0 (no tokens used), and NaN/Infinity to verify
49
+ that the classifier does not silently return an unexpected zone.
50
+
51
+ - id: REQ-04
52
+ title: Cost Threshold Gate Integration Test
53
+ titleZh: 成本閾值閘門整合測試
54
+ level: SHOULD
55
+ description: >
56
+ Where a cost gate (e.g. CostTracker.isOverBudget) exists, integration
57
+ tests SHOULD verify that the gate blocks further LLM calls when the
58
+ daily cost limit is exceeded.
59
+
60
+ - id: REQ-05
61
+ title: Coverage of All Budget Constants
62
+ titleZh: 全預算常數覆蓋
63
+ level: MUST
64
+ description: >
65
+ All numeric constants in the budget configuration MUST appear in at least
66
+ one test. Constants that appear only in production code but never in tests
67
+ are a mutation survival risk.
68
+
69
+ examples:
70
+ - name: "Zone boundary tests in Vitest"
71
+ code: |
72
+ import { classifyTokenZone, TOKEN_BUDGET } from "../types/index.js"
73
+
74
+ it.each([
75
+ [0.0, "safe"],
76
+ [0.84, "safe"],
77
+ [TOKEN_BUDGET.WARNING_THRESHOLD, "warning"],
78
+ [TOKEN_BUDGET.DANGER_THRESHOLD, "danger"],
79
+ [TOKEN_BUDGET.BLOCKING_THRESHOLD, "blocking"],
80
+ [1.0, "blocking"],
81
+ ])("classifyTokenZone(%f) → %s", (ratio, expected) => {
82
+ expect(classifyTokenZone(ratio)).toBe(expected)
83
+ })
84
+
85
+ anti_patterns:
86
+ - description: >
87
+ Testing only happy-path ratios like 0.5 or 0.75 — boundary values
88
+ (WARNING/DANGER/BLOCKING thresholds) are where off-by-one errors hide.
89
+ - description: >
90
+ Using magic numbers instead of TOKEN_BUDGET.* constants in tests —
91
+ if the constant changes, the test should fail, not silently pass.
92
+
93
+ related_standards:
94
+ - testing
95
+ - mutation-testing
96
+ - llm-output-validation
@@ -0,0 +1,110 @@
1
+ # Data Contract Standards - AI Optimized
2
+ # Source: XSPEC-068 Wave 3 Data Engineering Pack
3
+
4
+ id: data-contract
5
+ title: Data Contract Standards
6
+ version: "1.0.0"
7
+ status: Active
8
+ tags: [data-engineering, data-contract, data-quality, api, interoperability]
9
+ summary: |
10
+ Defines how data contracts are established, versioned, and enforced between
11
+ data producers and consumers. A data contract is a formal agreement
12
+ specifying the schema, quality guarantees, SLAs, and ownership of a dataset
13
+ or data stream. Covers contract specification format, freshness and quality
14
+ SLOs, breaking-change governance, consumer notification, and automated
15
+ contract testing. Reduces data pipeline failures caused by undiscovered
16
+ upstream changes.
17
+
18
+ requirements:
19
+ - id: REQ-001
20
+ title: Data Contract Specification Format
21
+ description: |
22
+ Every shared dataset or data stream MUST have a data contract specified
23
+ in a machine-readable YAML file (data-contract.yaml) committed to
24
+ version control alongside the producer code. The contract MUST include:
25
+ contract_id, version, owner (team + contact), description, schema
26
+ (field names, types, nullability), data_quality SLOs (completeness,
27
+ freshness, uniqueness), SLA (delivery time), and consumer list.
28
+ level: MUST
29
+ examples:
30
+ - "File: datasets/orders/data-contract.yaml committed in producer repo"
31
+ - "Required fields: contract_id, version, owner.team, owner.email, schema, sla.freshness_minutes"
32
+ - "Schema entry: {field: order_amount, type: DECIMAL(12,2), nullable: false, description: 'Order total in USD'}"
33
+
34
+ - id: REQ-002
35
+ title: Data Quality SLOs
36
+ description: |
37
+ Every data contract MUST define explicit data quality SLOs for the
38
+ dataset. Required quality dimensions: completeness (% of non-null
39
+ values for required fields ≥ threshold), freshness (data updated
40
+ within N minutes/hours of source event), uniqueness (% of distinct
41
+ values for key fields), and validity (% of values conforming to
42
+ defined format/range). Quality SLO violations MUST trigger alerts
43
+ to both producer and registered consumers.
44
+ level: MUST
45
+ examples:
46
+ - "completeness_slo: {field: order_id, min_pct_non_null: 99.99}"
47
+ - "freshness_slo: {max_lag_minutes: 15, alert_on_breach: true}"
48
+ - "uniqueness_slo: {field: order_id, min_pct_unique: 100.0}"
49
+
50
+ - id: REQ-003
51
+ title: Contract Versioning and Breaking-Change Governance
52
+ description: |
53
+ Data contracts MUST be versioned using semantic versioning. PATCH
54
+ version changes (backward-compatible additions, documentation updates)
55
+ require only producer team approval. MINOR version changes (new
56
+ optional fields, relaxed constraints) require notification to consumers
57
+ with 7-day notice. MAJOR version changes (field removal, type changes,
58
+ semantic changes) require consumer approval and MUST NOT be deployed
59
+ without all registered consumers confirming readiness.
60
+ level: MUST
61
+ examples:
62
+ - "v1.0.0 → v1.1.0: added optional field shipping_method, consumers notified via email 7 days prior"
63
+ - "v1.1.0 → v2.0.0: dropped legacy_order_id field, consumer sign-off required in tracking issue"
64
+ - "Breaking change gate: CI blocks deployment until all consumers in data-contract.yaml have approved"
65
+
66
+ - id: REQ-004
67
+ title: Automated Contract Testing
68
+ description: |
69
+ Data contract compliance MUST be verified automatically. Producers
70
+ MUST run contract tests on every data pipeline execution, validating
71
+ output against the declared schema and quality SLOs. Tests MUST
72
+ check: schema conformance (no unexpected nulls, correct types),
73
+ freshness within SLO window, uniqueness of key fields, and row count
74
+ within expected range. Contract test failures MUST halt the pipeline
75
+ and page the data owner.
76
+ level: MUST
77
+ examples:
78
+ - "Contract test: `great_expectations checkpoint run orders-contract-checkpoint`"
79
+ - "Test assertion: expect_column_values_to_not_be_null('order_id', mostly=0.9999)"
80
+ - "Pipeline halt: contract test failure triggers SNS alert to #data-oncall, job aborted"
81
+
82
+ - id: REQ-005
83
+ title: Consumer Registration and Notification
84
+ description: |
85
+ Teams consuming a shared dataset MUST register as consumers in the
86
+ producer's data-contract.yaml file. Registration must include: consumer
87
+ team name, contact email/Slack, use case description, and fields
88
+ consumed. Registered consumers MUST receive automated notifications
89
+ when: the contract is modified, quality SLO breaches occur, the
90
+ dataset is deprecated, or planned maintenance will affect availability.
91
+ level: MUST
92
+ examples:
93
+ - "consumers entry: {team: analytics, contact: '#data-analytics', use_case: 'revenue reporting', fields: [order_id, amount, created_at]}"
94
+ - "Automated Slack notification: 'Data contract orders/v1 SLO breached: freshness 42min (SLO: 15min)'"
95
+ - "Deprecation notice sent 90 days before v1 end-of-life to all registered consumers"
96
+
97
+ - id: REQ-006
98
+ title: Contract Discoverability
99
+ description: |
100
+ All data contracts in an organization SHOULD be discoverable through
101
+ a central data catalog. The catalog SHOULD provide: searchable index
102
+ of all contracts by team/domain, current health status (quality SLO
103
+ compliance), schema browsing, lineage visualization (which pipelines
104
+ produce/consume each dataset), and a self-service consumer registration
105
+ workflow.
106
+ level: SHOULD
107
+ examples:
108
+ - "Data catalog: DataHub or Amundsen with all contract YAML files indexed"
109
+ - "Catalog entry includes: SLO compliance badge (green/yellow/red), last updated, consumer count"
110
+ - "Self-service: 'Subscribe to dataset' button generates PR adding consumer to data-contract.yaml"
@@ -0,0 +1,96 @@
1
+ # Data Migration Testing Standards - AI Optimized
2
+ # Source: core/data-migration-testing.md
3
+
4
+ id: data-migration-testing
5
+ meta:
6
+ version: "1.0.0"
7
+ updated: "2026-05-05"
8
+ source: core/data-migration-testing.md
9
+ description: Database migration up/down/idempotency testing standards for schema changes
10
+
11
+ requirements:
12
+ REQ-1:
13
+ id: REQ-DMT-001
14
+ title: Up Migration Test
15
+ rule: >
16
+ Every schema migration MUST have an automated test that applies it to a clean
17
+ baseline schema and verifies the expected post-state (table names, columns,
18
+ indexes, constraints).
19
+ rationale: >
20
+ Unverified migrations silently corrupt production schema; up tests catch
21
+ incompatible changes before deployment.
22
+
23
+ REQ-2:
24
+ id: REQ-DMT-002
25
+ title: Down Migration (Rollback) Test
26
+ rule: >
27
+ Every migration that has a down/rollback function MUST have a test that
28
+ applies up, then down, and verifies the schema returns to its exact pre-state.
29
+ Zero data loss is the acceptance criterion.
30
+ rationale: >
31
+ Rollback is only reliable if it is tested; untested rollbacks fail at the worst
32
+ possible moment — during a production incident.
33
+
34
+ REQ-3:
35
+ id: REQ-DMT-003
36
+ title: Idempotency Test
37
+ rule: >
38
+ Migration runners MUST be tested to ensure applying the same migration twice
39
+ does not fail or cause side effects (e.g., duplicate columns, duplicate indexes).
40
+ rationale: >
41
+ CI retries and operator mistakes can trigger double-apply; idempotent migrations
42
+ prevent partial state corruption.
43
+
44
+ REQ-4:
45
+ id: REQ-DMT-004
46
+ title: Data Preservation Test
47
+ rule: >
48
+ Migrations that ALTER or DROP columns MUST include a test that seeds representative
49
+ data before the migration and verifies data integrity or expected transformation
50
+ after the migration.
51
+ rationale: >
52
+ Schema correctness does not imply data correctness; a column rename can silently
53
+ null-out existing data if the migration omits a data transform step.
54
+
55
+ REQ-5:
56
+ id: REQ-DMT-005
57
+ title: Migration History Integrity
58
+ rule: >
59
+ The migration runner's applied-migrations table/ledger MUST be validated:
60
+ each test run uses an isolated in-memory or temporary database so migrations
61
+ do not interfere with each other or with production state.
62
+ rationale: >
63
+ Shared migration state between tests causes non-deterministic failures that are
64
+ extremely difficult to diagnose.
65
+
66
+ test_structure:
67
+ isolation: "Each migration test MUST run against an isolated in-memory or temp DB"
68
+ baseline: "Start from the schema state immediately before the migration under test"
69
+ assertions:
70
+ up: "Assert table/column/index existence and types"
71
+ down: "Assert schema returns to pre-migration state"
72
+ idempotency: "Apply migration twice; second apply must succeed without error"
73
+ data: "Seed rows before migration; assert row count and values after"
74
+
75
+ tooling_guidance:
76
+ sqlite: "Use ':memory:' DSN; apply migrations via ORM migrate() or raw SQL"
77
+ postgres: "Use pgmock or test containers; run migrate up/down in a transaction"
78
+ drizzle_orm: >
79
+ Call db.run(sql`...`) with raw DDL, or use drizzle-kit's migrate() function
80
+ against a fresh :memory: SQLite database for each test file.
81
+
82
+ anti_patterns:
83
+ - description: >
84
+ Testing migrations against a shared development database — causes cross-test
85
+ pollution and non-repeatable results.
86
+ - description: >
87
+ Skipping down migration tests because "we never rollback" — rollbacks happen
88
+ during incidents; the worst time to discover they are broken.
89
+ - description: >
90
+ Writing migration tests after the fact without seeding data — misses the
91
+ data preservation class of bugs entirely.
92
+
93
+ related_standards:
94
+ - testing
95
+ - database-standards
96
+ - verification-evidence