universal-dev-standards 5.3.2 → 5.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/ai/standards/adversarial-test.ai.yaml +277 -0
- package/bundled/ai/standards/agent-communication-protocol.ai.yaml +32 -166
- package/bundled/ai/standards/agent-dispatch.ai.yaml +32 -58
- package/bundled/ai/standards/audit-trail.ai.yaml +113 -0
- package/bundled/ai/standards/branch-completion.ai.yaml +34 -70
- package/bundled/ai/standards/change-batching-standards.ai.yaml +31 -180
- package/bundled/ai/standards/chaos-injection-tests.ai.yaml +91 -0
- package/bundled/ai/standards/container-image-standards.ai.yaml +88 -0
- package/bundled/ai/standards/container-security.ai.yaml +331 -0
- package/bundled/ai/standards/cost-budget-test.ai.yaml +96 -0
- package/bundled/ai/standards/data-contract.ai.yaml +110 -0
- package/bundled/ai/standards/data-migration-testing.ai.yaml +96 -0
- package/bundled/ai/standards/data-pipeline.ai.yaml +113 -0
- package/bundled/ai/standards/disaster-recovery-drill.ai.yaml +89 -0
- package/bundled/ai/standards/execution-history.ai.yaml +30 -288
- package/bundled/ai/standards/flaky-test-management.ai.yaml +89 -0
- package/bundled/ai/standards/flow-based-testing.ai.yaml +240 -0
- package/bundled/ai/standards/iac-design-principles.ai.yaml +83 -0
- package/bundled/ai/standards/incident-response.ai.yaml +107 -0
- package/bundled/ai/standards/license-compliance.ai.yaml +106 -0
- package/bundled/ai/standards/llm-output-validation.ai.yaml +269 -0
- package/bundled/ai/standards/mock-boundary.ai.yaml +250 -0
- package/bundled/ai/standards/mutation-testing.ai.yaml +192 -0
- package/bundled/ai/standards/pii-classification.ai.yaml +109 -0
- package/bundled/ai/standards/pipeline-integration-standards.ai.yaml +28 -169
- package/bundled/ai/standards/policy-as-code-testing.ai.yaml +227 -0
- package/bundled/ai/standards/prd-standards.ai.yaml +88 -0
- package/bundled/ai/standards/product-metrics-standards.ai.yaml +111 -0
- package/bundled/ai/standards/prompt-regression.ai.yaml +94 -0
- package/bundled/ai/standards/property-based-testing.ai.yaml +105 -0
- package/bundled/ai/standards/release-quality-manifest.ai.yaml +135 -0
- package/bundled/ai/standards/replay-test.ai.yaml +111 -0
- package/bundled/ai/standards/runbook.ai.yaml +104 -0
- package/bundled/ai/standards/sast-advanced.ai.yaml +135 -0
- package/bundled/ai/standards/schema-evolution.ai.yaml +111 -0
- package/bundled/ai/standards/secret-management-standards.ai.yaml +105 -0
- package/bundled/ai/standards/secure-op.ai.yaml +365 -0
- package/bundled/ai/standards/security-testing.ai.yaml +171 -0
- package/bundled/ai/standards/server-ops-security.ai.yaml +274 -0
- package/bundled/ai/standards/slo-sli.ai.yaml +97 -0
- package/bundled/ai/standards/smoke-test.ai.yaml +87 -0
- package/bundled/ai/standards/supply-chain-attestation.ai.yaml +109 -0
- package/bundled/ai/standards/test-completeness-dimensions.ai.yaml +52 -5
- package/bundled/ai/standards/user-story-mapping.ai.yaml +108 -0
- package/bundled/ai/standards/workflow-enforcement.ai.yaml +34 -240
- package/bundled/ai/standards/workflow-state-protocol.ai.yaml +31 -107
- package/bundled/core/adversarial-test.md +212 -0
- package/bundled/core/chaos-injection-tests.md +116 -0
- package/bundled/core/container-security.md +521 -0
- package/bundled/core/cost-budget-test.md +69 -0
- package/bundled/core/data-migration-testing.md +110 -0
- package/bundled/core/disaster-recovery-drill.md +73 -0
- package/bundled/core/flaky-test-management.md +73 -0
- package/bundled/core/flow-based-testing.md +142 -0
- package/bundled/core/llm-output-validation.md +178 -0
- package/bundled/core/mock-boundary.md +100 -0
- package/bundled/core/mutation-testing.md +97 -0
- package/bundled/core/policy-as-code-testing.md +188 -0
- package/bundled/core/prompt-regression.md +72 -0
- package/bundled/core/property-based-testing.md +73 -0
- package/bundled/core/release-quality-manifest.md +147 -0
- package/bundled/core/replay-test.md +86 -0
- package/bundled/core/sast-advanced.md +300 -0
- package/bundled/core/secure-op.md +314 -0
- package/bundled/core/security-testing.md +87 -0
- package/bundled/core/server-ops-security.md +493 -0
- package/bundled/core/smoke-test.md +65 -0
- package/bundled/core/supply-chain-attestation.md +117 -0
- package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
- package/bundled/locales/zh-CN/README.md +1 -1
- package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +5 -5
- package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
- package/bundled/locales/zh-TW/README.md +1 -1
- package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +183 -79
- package/bundled/skills/README.md +4 -3
- package/bundled/skills/SKILL_NAMING.md +94 -0
- package/bundled/skills/ai-instruction-standards/SKILL.md +181 -88
- package/bundled/skills/atdd-assistant/SKILL.md +8 -0
- package/bundled/skills/bdd-assistant/SKILL.md +7 -0
- package/bundled/skills/checkin-assistant/SKILL.md +8 -0
- package/bundled/skills/code-review-assistant/SKILL.md +7 -0
- package/bundled/skills/journey-test-assistant/SKILL.md +203 -0
- package/bundled/skills/orchestrate/SKILL.md +167 -0
- package/bundled/skills/plan/SKILL.md +234 -0
- package/bundled/skills/pr-automation-assistant/SKILL.md +8 -0
- package/bundled/skills/push/SKILL.md +49 -2
- package/bundled/skills/{process-automation → skill-builder}/SKILL.md +1 -1
- package/bundled/skills/{forward-derivation → spec-derivation}/SKILL.md +1 -1
- package/bundled/skills/spec-driven-dev/SKILL.md +7 -0
- package/bundled/skills/sweep/SKILL.md +145 -0
- package/bundled/skills/tdd-assistant/SKILL.md +7 -0
- package/package.json +1 -1
- package/src/commands/flow.js +8 -0
- package/src/commands/start.js +14 -0
- package/src/commands/sweep.js +8 -0
- package/src/commands/workflow.js +8 -0
- package/standards-registry.json +474 -12
- package/bundled/locales/zh-CN/skills/ac-coverage-assistant/SKILL.md +0 -190
- package/bundled/locales/zh-CN/skills/forward-derivation/SKILL.md +0 -71
- package/bundled/locales/zh-CN/skills/forward-derivation/guide.md +0 -130
- package/bundled/locales/zh-CN/skills/methodology-system/SKILL.md +0 -88
- package/bundled/locales/zh-CN/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-CN/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-CN/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-CN/skills/process-automation/SKILL.md +0 -143
- package/bundled/locales/zh-TW/skills/ac-coverage-assistant/SKILL.md +0 -195
- package/bundled/locales/zh-TW/skills/deploy-assistant/SKILL.md +0 -178
- package/bundled/locales/zh-TW/skills/forward-derivation/SKILL.md +0 -69
- package/bundled/locales/zh-TW/skills/forward-derivation/guide.md +0 -415
- package/bundled/locales/zh-TW/skills/methodology-system/SKILL.md +0 -86
- package/bundled/locales/zh-TW/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-TW/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-TW/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-TW/skills/process-automation/SKILL.md +0 -144
- /package/bundled/skills/{ac-coverage-assistant → ac-coverage}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/create-methodology.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/guide.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/integrated-flow.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/prerequisite-check.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/runtime.md +0 -0
- /package/bundled/skills/{forward-derivation → spec-derivation}/guide.md +0 -0
|
@@ -0,0 +1,331 @@
|
|
|
1
|
+
# Container Security Standard - AI Optimized
|
|
2
|
+
# Source: core/container-security.md
|
|
3
|
+
|
|
4
|
+
id: container-security
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-04"
|
|
8
|
+
source: core/container-security.md
|
|
9
|
+
description: >
|
|
10
|
+
Container and Kubernetes security covering image hardening, registry security,
|
|
11
|
+
runtime protection, secrets management, network policy, and supply chain
|
|
12
|
+
integrity for AI Agent production environments.
|
|
13
|
+
|
|
14
|
+
# ─────────────────────────────────────────────────────────
|
|
15
|
+
# Core Categories
|
|
16
|
+
# ─────────────────────────────────────────────────────────
|
|
17
|
+
categories:
|
|
18
|
+
- id: image_hardening
|
|
19
|
+
name: Image Hardening
|
|
20
|
+
description: Container images must be minimal, non-root, and built with multi-stage patterns
|
|
21
|
+
base_image:
|
|
22
|
+
preferred:
|
|
23
|
+
- distroless (gcr.io/distroless/*)
|
|
24
|
+
- alpine (< 10 MB)
|
|
25
|
+
prohibited:
|
|
26
|
+
- ubuntu/debian as final layer with full package manager
|
|
27
|
+
- latest tag on base image
|
|
28
|
+
dockerfile_rules:
|
|
29
|
+
- rule: Use multi-stage build; only copy artifacts to final stage
|
|
30
|
+
- rule: Final stage must set non-root USER (UID 1000)
|
|
31
|
+
- rule: No RUN apt-get / apk add in final layer (use builder stage only)
|
|
32
|
+
- rule: Only COPY specific files; never COPY . . in final stage
|
|
33
|
+
- rule: Pin base image by digest, not tag
|
|
34
|
+
security_context_k8s:
|
|
35
|
+
required:
|
|
36
|
+
- runAsNonRoot: true
|
|
37
|
+
- runAsUser: 1000
|
|
38
|
+
- allowPrivilegeEscalation: false
|
|
39
|
+
- readOnlyRootFilesystem: true
|
|
40
|
+
seccomp:
|
|
41
|
+
profile: RuntimeDefault
|
|
42
|
+
capabilities:
|
|
43
|
+
drop: [ALL]
|
|
44
|
+
add: explicit allowlist only (e.g., NET_BIND_SERVICE if needed)
|
|
45
|
+
ai_agent_specific:
|
|
46
|
+
- rule: AI Agent container must run as UID 1000 (non-root)
|
|
47
|
+
- rule: LLM call process must not have CAP_NET_RAW or CAP_SYS_ADMIN
|
|
48
|
+
iso_mapping:
|
|
49
|
+
- "ISO/IEC 27001:2022 A.8.9 Configuration Management"
|
|
50
|
+
- "NIST SP 800-190 §4.1 Image Vulnerabilities"
|
|
51
|
+
- "CIS Docker Benchmark v1.6 §4 Container Images"
|
|
52
|
+
|
|
53
|
+
- id: registry_security
|
|
54
|
+
name: Registry Security
|
|
55
|
+
description: All container images must be stored in private registries with signing and vulnerability scanning
|
|
56
|
+
registry:
|
|
57
|
+
type: private (GHCR, ECR, GCR, Harbor)
|
|
58
|
+
public_registry: prohibited for production images
|
|
59
|
+
tagging:
|
|
60
|
+
immutable_tag: required (semver or git SHA)
|
|
61
|
+
latest_tag: prohibited in production deployments
|
|
62
|
+
pinned_digest: required for supply chain integrity
|
|
63
|
+
signing:
|
|
64
|
+
tools: [Cosign, Notary v2]
|
|
65
|
+
policy: All production images must be signed before push
|
|
66
|
+
verification: Admission controller (Kyverno/Gatekeeper) verifies signature on deploy
|
|
67
|
+
vulnerability_scan:
|
|
68
|
+
tool: Trivy
|
|
69
|
+
gate:
|
|
70
|
+
critical: 0 (block push if any critical CVE)
|
|
71
|
+
high: 0 (block push if any high CVE)
|
|
72
|
+
frequency: On every build + weekly re-scan of deployed images
|
|
73
|
+
editions:
|
|
74
|
+
trial: Trivy gate mandatory
|
|
75
|
+
community: Trivy gate mandatory
|
|
76
|
+
commercial: Trivy gate mandatory
|
|
77
|
+
ai_agent_specific:
|
|
78
|
+
- rule: Guardian OPA Sidecar image must also pass Trivy gate
|
|
79
|
+
- rule: Trial/Community edition images enforce Trivy gate to avoid supply chain compromise
|
|
80
|
+
iso_mapping:
|
|
81
|
+
- "ISO/IEC 27001:2022 A.8.8 Management of Technical Vulnerabilities"
|
|
82
|
+
- "NIST SP 800-190 §4.2 Image Configuration Defects"
|
|
83
|
+
- "CIS Docker Benchmark v1.6 §2 Docker daemon configuration"
|
|
84
|
+
|
|
85
|
+
- id: runtime_security
|
|
86
|
+
name: Runtime Security
|
|
87
|
+
description: Running containers must have minimal capabilities and access; privileged mode is prohibited
|
|
88
|
+
prohibitions:
|
|
89
|
+
- privileged: true in securityContext
|
|
90
|
+
- hostNetwork: true (unless explicitly required and documented)
|
|
91
|
+
- hostPID: true
|
|
92
|
+
- hostIPC: true
|
|
93
|
+
required_settings:
|
|
94
|
+
- readOnlyRootFilesystem: true
|
|
95
|
+
- allowPrivilegeEscalation: false
|
|
96
|
+
- drop: [ALL] capabilities
|
|
97
|
+
seccomp:
|
|
98
|
+
profile: RuntimeDefault (or custom restricted profile)
|
|
99
|
+
audit_mode: allowed for profiling only; not for production
|
|
100
|
+
apparmor_selinux:
|
|
101
|
+
apparmor: docker-default (Kubernetes annotation)
|
|
102
|
+
selinux: enforcing mode on RHEL/CentOS hosts
|
|
103
|
+
writable_volumes:
|
|
104
|
+
pattern: Mount named volumes or PersistentVolumeClaims for writable data
|
|
105
|
+
tmpfs: allowed for /tmp when truly ephemeral
|
|
106
|
+
ai_agent_specific:
|
|
107
|
+
- rule: Audit log volume must be readOnly:false but backed by append-only partition
|
|
108
|
+
- rule: Agent working directory may use emptyDir; audit log must not
|
|
109
|
+
iso_mapping:
|
|
110
|
+
- "ISO/IEC 27001:2022 A.8.9 Configuration Management"
|
|
111
|
+
- "NIST SP 800-190 §4.3 Container Runtime Vulnerabilities"
|
|
112
|
+
- "CIS Docker Benchmark v1.6 §5 Container Runtime"
|
|
113
|
+
|
|
114
|
+
- id: secrets_management
|
|
115
|
+
name: Secrets Management
|
|
116
|
+
description: Secrets must never be embedded in images or passed via environment variables from plain text
|
|
117
|
+
prohibited:
|
|
118
|
+
- Hardcoded secrets in Dockerfile (ENV SECRET=...)
|
|
119
|
+
- Secrets in docker-compose.yml env: section as plain text
|
|
120
|
+
- ConfigMap storing sensitive values in K8s
|
|
121
|
+
- Git-tracked .env files containing real credentials
|
|
122
|
+
allowed:
|
|
123
|
+
staging:
|
|
124
|
+
- K8s Secrets (base64) + RBAC restriction
|
|
125
|
+
- Sealed Secrets (Bitnami) for GitOps
|
|
126
|
+
production:
|
|
127
|
+
- HashiCorp Vault (dynamic secrets)
|
|
128
|
+
- AWS Secrets Manager / GCP Secret Manager
|
|
129
|
+
- Azure Key Vault
|
|
130
|
+
injection_patterns:
|
|
131
|
+
- pattern: Mounted volume (/run/secrets/)
|
|
132
|
+
preferred: true
|
|
133
|
+
- pattern: Init container fetches secret and writes to shared emptyDir
|
|
134
|
+
acceptable: true
|
|
135
|
+
- pattern: Environment variable from K8s Secret reference
|
|
136
|
+
acceptable: true (better than plain text, but volume preferred)
|
|
137
|
+
ai_agent_specific:
|
|
138
|
+
- rule: LLM API keys must be injected via mounted secret volume, not ENV
|
|
139
|
+
- rule: Guardian OPA policy files are not secrets; use ConfigMap
|
|
140
|
+
iso_mapping:
|
|
141
|
+
- "ISO/IEC 27001:2022 A.8.12 Data Leakage Prevention"
|
|
142
|
+
- "ISO/IEC 27001:2022 A.8.24 Use of Cryptography"
|
|
143
|
+
- "NIST SP 800-190 §4.4 Container Image and Registry Vulnerabilities"
|
|
144
|
+
|
|
145
|
+
- id: network_policy
|
|
146
|
+
name: Network Policy
|
|
147
|
+
description: K8s NetworkPolicy must default-deny; only necessary ingress/egress is permitted
|
|
148
|
+
default_policy:
|
|
149
|
+
ingress: deny-all (explicit allow per service)
|
|
150
|
+
egress: deny-all (explicit allow per service)
|
|
151
|
+
required_allow:
|
|
152
|
+
dns: "kube-dns (UDP/TCP 53) must be allowed for all pods"
|
|
153
|
+
health_checks: "kubelet health check endpoints"
|
|
154
|
+
ai_agent_outbound_allowlist:
|
|
155
|
+
- host: api.anthropic.com
|
|
156
|
+
port: 443
|
|
157
|
+
protocol: HTTPS
|
|
158
|
+
- host: api.openai.com
|
|
159
|
+
port: 443
|
|
160
|
+
protocol: HTTPS
|
|
161
|
+
- host: "bedrock.*.amazonaws.com"
|
|
162
|
+
port: 443
|
|
163
|
+
protocol: HTTPS
|
|
164
|
+
- host: registry.npmjs.org
|
|
165
|
+
port: 443
|
|
166
|
+
protocol: HTTPS
|
|
167
|
+
note: Only during build phase; not at runtime
|
|
168
|
+
guardian_sidecar:
|
|
169
|
+
placement:
|
|
170
|
+
- same Pod as AI Agent (sidecar container)
|
|
171
|
+
- same K8s namespace minimum
|
|
172
|
+
communication: localhost (sidecar) or ClusterIP service (same namespace)
|
|
173
|
+
tools:
|
|
174
|
+
policy_enforcement: [Calico, Cilium, Kubernetes NetworkPolicy]
|
|
175
|
+
validation: [kubectl describe networkpolicy, Kyverno policy report]
|
|
176
|
+
ai_agent_specific:
|
|
177
|
+
- rule: AI Agent pod egress must be restricted to outbound allowlist
|
|
178
|
+
- rule: Guardian OPA Sidecar must be same Pod or same namespace
|
|
179
|
+
- rule: Any new LLM provider endpoint must be added to allowlist and reviewed
|
|
180
|
+
iso_mapping:
|
|
181
|
+
- "ISO/IEC 27001:2022 A.8.20 Networks Security"
|
|
182
|
+
- "ISO/IEC 27001:2022 A.8.22 Web Filtering"
|
|
183
|
+
- "CIS Kubernetes Benchmark v1.8 §5.3 Network Policies and CNI"
|
|
184
|
+
|
|
185
|
+
- id: supply_chain
|
|
186
|
+
name: Supply Chain Security
|
|
187
|
+
description: Container build process must be reproducible, auditable, and signed
|
|
188
|
+
sbom:
|
|
189
|
+
formats: [CycloneDX, SPDX]
|
|
190
|
+
generation:
|
|
191
|
+
tool: Syft or trivy sbom
|
|
192
|
+
trigger: Every image build
|
|
193
|
+
storage: Attached to registry as OCI artifact
|
|
194
|
+
image_pinning:
|
|
195
|
+
rule: All FROM directives must use digest (sha256:...), not floating tag
|
|
196
|
+
example: "FROM gcr.io/distroless/static-debian12@sha256:<digest>"
|
|
197
|
+
signing:
|
|
198
|
+
tool: Sigstore (keyless signing via GitHub Actions OIDC)
|
|
199
|
+
verification: Cosign verify --certificate-identity-regexp
|
|
200
|
+
reproducible_build:
|
|
201
|
+
- Set SOURCE_DATE_EPOCH for deterministic timestamps
|
|
202
|
+
- Use --no-cache in CI; never rely on layer cache across builds
|
|
203
|
+
lockfile_verification:
|
|
204
|
+
nodejs:
|
|
205
|
+
file: package-lock.json or yarn.lock
|
|
206
|
+
command: "npm ci (not npm install)"
|
|
207
|
+
python:
|
|
208
|
+
file: requirements.txt with pinned versions or Pipfile.lock
|
|
209
|
+
command: "pip install --require-hashes -r requirements.txt"
|
|
210
|
+
golang:
|
|
211
|
+
file: go.sum
|
|
212
|
+
command: "go mod verify"
|
|
213
|
+
ai_agent_specific:
|
|
214
|
+
- rule: SBOM must be generated for every AI Agent image release
|
|
215
|
+
- rule: Third-party AI SDK (anthropic, openai) versions must be pinned in lockfile
|
|
216
|
+
iso_mapping:
|
|
217
|
+
- "ISO/IEC 27001:2022 A.8.8 Management of Technical Vulnerabilities"
|
|
218
|
+
- "NIST SP 800-190 §4.5 Host OS Vulnerabilities"
|
|
219
|
+
- "SLSA Framework Level 2+ (provenance attestation)"
|
|
220
|
+
|
|
221
|
+
# ─────────────────────────────────────────────────────────
|
|
222
|
+
# Quality Gates
|
|
223
|
+
# ─────────────────────────────────────────────────────────
|
|
224
|
+
quality_gates:
|
|
225
|
+
pre_build:
|
|
226
|
+
- base_image_pinned_by_digest: required
|
|
227
|
+
- multi_stage_build: required
|
|
228
|
+
- no_secrets_in_dockerfile: required
|
|
229
|
+
pre_push:
|
|
230
|
+
- trivy_critical_count: 0
|
|
231
|
+
- trivy_high_count: 0
|
|
232
|
+
- image_signed: required (cosign or notary)
|
|
233
|
+
- sbom_generated: required
|
|
234
|
+
pre_deploy:
|
|
235
|
+
- image_tag_immutable: required (no 'latest')
|
|
236
|
+
- k8s_security_context:
|
|
237
|
+
runAsNonRoot: true
|
|
238
|
+
readOnlyRootFilesystem: true
|
|
239
|
+
allowPrivilegeEscalation: false
|
|
240
|
+
- network_policy_applied: required
|
|
241
|
+
- secrets_from_vault_or_k8s_secret: required
|
|
242
|
+
ai_agent_specific:
|
|
243
|
+
- agent_uid: 1000
|
|
244
|
+
- guardian_sidecar: same_pod_or_namespace
|
|
245
|
+
- outbound_allowlist: configured
|
|
246
|
+
- audit_log_volume: append_only_partition
|
|
247
|
+
|
|
248
|
+
# ─────────────────────────────────────────────────────────
|
|
249
|
+
# Rules
|
|
250
|
+
# ─────────────────────────────────────────────────────────
|
|
251
|
+
rules:
|
|
252
|
+
- id: non-root-container
|
|
253
|
+
trigger: writing any Dockerfile or K8s Pod spec
|
|
254
|
+
instruction: >
|
|
255
|
+
Set USER 1000 in Dockerfile final stage.
|
|
256
|
+
Set runAsNonRoot: true and runAsUser: 1000 in securityContext.
|
|
257
|
+
Verify: docker inspect --format='{{.Config.User}}' <image>
|
|
258
|
+
priority: required
|
|
259
|
+
|
|
260
|
+
- id: no-latest-tag
|
|
261
|
+
trigger: tagging or pulling container images
|
|
262
|
+
instruction: >
|
|
263
|
+
Never use 'latest' tag in production. Use semver (v1.2.3) or git SHA.
|
|
264
|
+
For base images, pin by digest: FROM image@sha256:<digest>
|
|
265
|
+
priority: required
|
|
266
|
+
|
|
267
|
+
- id: trivy-gate-before-push
|
|
268
|
+
trigger: building container image for any environment
|
|
269
|
+
instruction: >
|
|
270
|
+
Run: trivy image --exit-code 1 --severity CRITICAL,HIGH <image>
|
|
271
|
+
Block push if exit code is non-zero. Fix all critical/high CVEs first.
|
|
272
|
+
priority: required
|
|
273
|
+
|
|
274
|
+
- id: drop-all-capabilities
|
|
275
|
+
trigger: writing K8s Pod or Deployment spec
|
|
276
|
+
instruction: >
|
|
277
|
+
Set capabilities.drop: [ALL] in securityContext.
|
|
278
|
+
Only add specific capabilities if explicitly required (document justification).
|
|
279
|
+
priority: required
|
|
280
|
+
|
|
281
|
+
- id: default-deny-network-policy
|
|
282
|
+
trigger: deploying any service to Kubernetes
|
|
283
|
+
instruction: >
|
|
284
|
+
Apply default-deny NetworkPolicy to namespace before deploying services.
|
|
285
|
+
Then add explicit allow policies per service communication requirement.
|
|
286
|
+
priority: required
|
|
287
|
+
|
|
288
|
+
- id: no-env-secrets
|
|
289
|
+
trigger: any Dockerfile ENV or K8s env[] with sensitive values
|
|
290
|
+
instruction: >
|
|
291
|
+
Never pass secrets via ENV. Use K8s Secret + volume mount or Vault agent injector.
|
|
292
|
+
Check: grep -r "ENV.*KEY\|ENV.*SECRET\|ENV.*TOKEN" Dockerfile
|
|
293
|
+
priority: required
|
|
294
|
+
|
|
295
|
+
- id: sign-and-sbom
|
|
296
|
+
trigger: releasing container image to production
|
|
297
|
+
instruction: >
|
|
298
|
+
Generate SBOM: syft <image> -o cyclonedx-json > sbom.json
|
|
299
|
+
Sign image: cosign sign <image>@<digest>
|
|
300
|
+
Attach SBOM: cosign attach sbom --sbom sbom.json <image>@<digest>
|
|
301
|
+
priority: required
|
|
302
|
+
|
|
303
|
+
anti_patterns:
|
|
304
|
+
- FROM ubuntu:latest (floating tag, large attack surface)
|
|
305
|
+
- RUN apt-get update && apt-get install in final Dockerfile stage
|
|
306
|
+
- Running container as root (USER root or no USER directive)
|
|
307
|
+
- ENV API_KEY=<real_key> in Dockerfile
|
|
308
|
+
- privileged: true in K8s securityContext
|
|
309
|
+
- No NetworkPolicy (allow all egress by default)
|
|
310
|
+
- Skipping Trivy scan on "quick" builds
|
|
311
|
+
- Using 'latest' image tag in Kubernetes Deployment
|
|
312
|
+
- SBOM not generated for production releases
|
|
313
|
+
|
|
314
|
+
quick_reference:
|
|
315
|
+
container_security_checklist: |
|
|
316
|
+
□ Base image pinned by digest (not tag)
|
|
317
|
+
□ Multi-stage build; only COPY artifacts to final stage
|
|
318
|
+
□ Final stage: USER 1000 (non-root)
|
|
319
|
+
□ securityContext: runAsNonRoot, readOnlyRootFilesystem, allowPrivilegeEscalation:false
|
|
320
|
+
□ capabilities.drop: [ALL]
|
|
321
|
+
□ seccompProfile: RuntimeDefault
|
|
322
|
+
□ Trivy scan: 0 critical, 0 high before push
|
|
323
|
+
□ Image signed (Cosign / Notary v2)
|
|
324
|
+
□ SBOM generated (CycloneDX or SPDX)
|
|
325
|
+
□ No secrets in Dockerfile ENV or image layers
|
|
326
|
+
□ K8s Secrets or Vault for secret injection
|
|
327
|
+
□ NetworkPolicy: default-deny + explicit allow
|
|
328
|
+
□ AI Agent egress: allowlist (Anthropic, OpenAI, Bedrock)
|
|
329
|
+
□ Guardian OPA Sidecar: same Pod or same namespace
|
|
330
|
+
□ Audit log volume: append-only partition (not emptyDir)
|
|
331
|
+
□ Lockfile pinned (npm ci / pip --require-hashes / go mod verify)
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# SPDX-License-Identifier: MIT
|
|
2
|
+
name: Cost Budget Test Standards
|
|
3
|
+
nameZh: 成本預算測試標準
|
|
4
|
+
id: cost-budget-test
|
|
5
|
+
version: "1.0.0"
|
|
6
|
+
category: testing
|
|
7
|
+
scope: ai-agent-systems
|
|
8
|
+
summary: >
|
|
9
|
+
Unit and integration tests that verify token budget zone classification,
|
|
10
|
+
pipeline cost thresholds, and runaway-loop prevention for AI agent systems.
|
|
11
|
+
|
|
12
|
+
requirements:
|
|
13
|
+
- id: REQ-01
|
|
14
|
+
title: Zone Classification Boundary Tests
|
|
15
|
+
titleZh: 區間分類邊界測試
|
|
16
|
+
level: MUST
|
|
17
|
+
description: >
|
|
18
|
+
The token budget zone classifier MUST have unit tests covering all four
|
|
19
|
+
zones (safe/warning/danger/blocking) and every boundary value (the exact
|
|
20
|
+
threshold values, one below, and one above). This ensures that threshold
|
|
21
|
+
drift is caught by the test suite.
|
|
22
|
+
implementation: |
|
|
23
|
+
// Test all boundaries:
|
|
24
|
+
classifyTokenZone(0.84) → "safe"
|
|
25
|
+
classifyTokenZone(0.85) → "warning" // WARNING_THRESHOLD
|
|
26
|
+
classifyTokenZone(0.89) → "warning"
|
|
27
|
+
classifyTokenZone(0.90) → "danger" // DANGER_THRESHOLD
|
|
28
|
+
classifyTokenZone(0.94) → "danger"
|
|
29
|
+
classifyTokenZone(0.95) → "blocking" // BLOCKING_THRESHOLD
|
|
30
|
+
classifyTokenZone(1.00) → "blocking"
|
|
31
|
+
|
|
32
|
+
- id: REQ-02
|
|
33
|
+
title: Pipeline Budget Config Validation Tests
|
|
34
|
+
titleZh: Pipeline 預算設定驗證測試
|
|
35
|
+
level: MUST
|
|
36
|
+
description: >
|
|
37
|
+
Tests MUST verify that PipelineBudgetConfig properties (maxCostPerRun,
|
|
38
|
+
maxCostPerDay, warningThreshold, autoDowngrade) have correct semantics.
|
|
39
|
+
At minimum: verify that exceeding maxCostPerRun triggers the expected
|
|
40
|
+
budget-exceeded behaviour.
|
|
41
|
+
|
|
42
|
+
- id: REQ-03
|
|
43
|
+
title: Overflow and Runaway Detection
|
|
44
|
+
titleZh: 溢位與失控偵測測試
|
|
45
|
+
level: MUST
|
|
46
|
+
description: >
|
|
47
|
+
Tests MUST include edge cases for budget overflow: usageRatio > 1.0
|
|
48
|
+
(over-budget), exactly 0.0 (no tokens used), and NaN/Infinity to verify
|
|
49
|
+
that the classifier does not silently return an unexpected zone.
|
|
50
|
+
|
|
51
|
+
- id: REQ-04
|
|
52
|
+
title: Cost Threshold Gate Integration Test
|
|
53
|
+
titleZh: 成本閾值閘門整合測試
|
|
54
|
+
level: SHOULD
|
|
55
|
+
description: >
|
|
56
|
+
Where a cost gate (e.g. CostTracker.isOverBudget) exists, integration
|
|
57
|
+
tests SHOULD verify that the gate blocks further LLM calls when the
|
|
58
|
+
daily cost limit is exceeded.
|
|
59
|
+
|
|
60
|
+
- id: REQ-05
|
|
61
|
+
title: Coverage of All Budget Constants
|
|
62
|
+
titleZh: 全預算常數覆蓋
|
|
63
|
+
level: MUST
|
|
64
|
+
description: >
|
|
65
|
+
All numeric constants in the budget configuration MUST appear in at least
|
|
66
|
+
one test. Constants that appear only in production code but never in tests
|
|
67
|
+
are a mutation survival risk.
|
|
68
|
+
|
|
69
|
+
examples:
|
|
70
|
+
- name: "Zone boundary tests in Vitest"
|
|
71
|
+
code: |
|
|
72
|
+
import { classifyTokenZone, TOKEN_BUDGET } from "../types/index.js"
|
|
73
|
+
|
|
74
|
+
it.each([
|
|
75
|
+
[0.0, "safe"],
|
|
76
|
+
[0.84, "safe"],
|
|
77
|
+
[TOKEN_BUDGET.WARNING_THRESHOLD, "warning"],
|
|
78
|
+
[TOKEN_BUDGET.DANGER_THRESHOLD, "danger"],
|
|
79
|
+
[TOKEN_BUDGET.BLOCKING_THRESHOLD, "blocking"],
|
|
80
|
+
[1.0, "blocking"],
|
|
81
|
+
])("classifyTokenZone(%f) → %s", (ratio, expected) => {
|
|
82
|
+
expect(classifyTokenZone(ratio)).toBe(expected)
|
|
83
|
+
})
|
|
84
|
+
|
|
85
|
+
anti_patterns:
|
|
86
|
+
- description: >
|
|
87
|
+
Testing only happy-path ratios like 0.5 or 0.75 — boundary values
|
|
88
|
+
(WARNING/DANGER/BLOCKING thresholds) are where off-by-one errors hide.
|
|
89
|
+
- description: >
|
|
90
|
+
Using magic numbers instead of TOKEN_BUDGET.* constants in tests —
|
|
91
|
+
if the constant changes, the test should fail, not silently pass.
|
|
92
|
+
|
|
93
|
+
related_standards:
|
|
94
|
+
- testing
|
|
95
|
+
- mutation-testing
|
|
96
|
+
- llm-output-validation
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# Data Contract Standards - AI Optimized
|
|
2
|
+
# Source: XSPEC-068 Wave 3 Data Engineering Pack
|
|
3
|
+
|
|
4
|
+
id: data-contract
|
|
5
|
+
title: Data Contract Standards
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
status: Active
|
|
8
|
+
tags: [data-engineering, data-contract, data-quality, api, interoperability]
|
|
9
|
+
summary: |
|
|
10
|
+
Defines how data contracts are established, versioned, and enforced between
|
|
11
|
+
data producers and consumers. A data contract is a formal agreement
|
|
12
|
+
specifying the schema, quality guarantees, SLAs, and ownership of a dataset
|
|
13
|
+
or data stream. Covers contract specification format, freshness and quality
|
|
14
|
+
SLOs, breaking-change governance, consumer notification, and automated
|
|
15
|
+
contract testing. Reduces data pipeline failures caused by undiscovered
|
|
16
|
+
upstream changes.
|
|
17
|
+
|
|
18
|
+
requirements:
|
|
19
|
+
- id: REQ-001
|
|
20
|
+
title: Data Contract Specification Format
|
|
21
|
+
description: |
|
|
22
|
+
Every shared dataset or data stream MUST have a data contract specified
|
|
23
|
+
in a machine-readable YAML file (data-contract.yaml) committed to
|
|
24
|
+
version control alongside the producer code. The contract MUST include:
|
|
25
|
+
contract_id, version, owner (team + contact), description, schema
|
|
26
|
+
(field names, types, nullability), data_quality SLOs (completeness,
|
|
27
|
+
freshness, uniqueness), SLA (delivery time), and consumer list.
|
|
28
|
+
level: MUST
|
|
29
|
+
examples:
|
|
30
|
+
- "File: datasets/orders/data-contract.yaml committed in producer repo"
|
|
31
|
+
- "Required fields: contract_id, version, owner.team, owner.email, schema, sla.freshness_minutes"
|
|
32
|
+
- "Schema entry: {field: order_amount, type: DECIMAL(12,2), nullable: false, description: 'Order total in USD'}"
|
|
33
|
+
|
|
34
|
+
- id: REQ-002
|
|
35
|
+
title: Data Quality SLOs
|
|
36
|
+
description: |
|
|
37
|
+
Every data contract MUST define explicit data quality SLOs for the
|
|
38
|
+
dataset. Required quality dimensions: completeness (% of non-null
|
|
39
|
+
values for required fields ≥ threshold), freshness (data updated
|
|
40
|
+
within N minutes/hours of source event), uniqueness (% of distinct
|
|
41
|
+
values for key fields), and validity (% of values conforming to
|
|
42
|
+
defined format/range). Quality SLO violations MUST trigger alerts
|
|
43
|
+
to both producer and registered consumers.
|
|
44
|
+
level: MUST
|
|
45
|
+
examples:
|
|
46
|
+
- "completeness_slo: {field: order_id, min_pct_non_null: 99.99}"
|
|
47
|
+
- "freshness_slo: {max_lag_minutes: 15, alert_on_breach: true}"
|
|
48
|
+
- "uniqueness_slo: {field: order_id, min_pct_unique: 100.0}"
|
|
49
|
+
|
|
50
|
+
- id: REQ-003
|
|
51
|
+
title: Contract Versioning and Breaking-Change Governance
|
|
52
|
+
description: |
|
|
53
|
+
Data contracts MUST be versioned using semantic versioning. PATCH
|
|
54
|
+
version changes (backward-compatible additions, documentation updates)
|
|
55
|
+
require only producer team approval. MINOR version changes (new
|
|
56
|
+
optional fields, relaxed constraints) require notification to consumers
|
|
57
|
+
with 7-day notice. MAJOR version changes (field removal, type changes,
|
|
58
|
+
semantic changes) require consumer approval and MUST NOT be deployed
|
|
59
|
+
without all registered consumers confirming readiness.
|
|
60
|
+
level: MUST
|
|
61
|
+
examples:
|
|
62
|
+
- "v1.0.0 → v1.1.0: added optional field shipping_method, consumers notified via email 7 days prior"
|
|
63
|
+
- "v1.1.0 → v2.0.0: dropped legacy_order_id field, consumer sign-off required in tracking issue"
|
|
64
|
+
- "Breaking change gate: CI blocks deployment until all consumers in data-contract.yaml have approved"
|
|
65
|
+
|
|
66
|
+
- id: REQ-004
|
|
67
|
+
title: Automated Contract Testing
|
|
68
|
+
description: |
|
|
69
|
+
Data contract compliance MUST be verified automatically. Producers
|
|
70
|
+
MUST run contract tests on every data pipeline execution, validating
|
|
71
|
+
output against the declared schema and quality SLOs. Tests MUST
|
|
72
|
+
check: schema conformance (no unexpected nulls, correct types),
|
|
73
|
+
freshness within SLO window, uniqueness of key fields, and row count
|
|
74
|
+
within expected range. Contract test failures MUST halt the pipeline
|
|
75
|
+
and page the data owner.
|
|
76
|
+
level: MUST
|
|
77
|
+
examples:
|
|
78
|
+
- "Contract test: `great_expectations checkpoint run orders-contract-checkpoint`"
|
|
79
|
+
- "Test assertion: expect_column_values_to_not_be_null('order_id', mostly=0.9999)"
|
|
80
|
+
- "Pipeline halt: contract test failure triggers SNS alert to #data-oncall, job aborted"
|
|
81
|
+
|
|
82
|
+
- id: REQ-005
|
|
83
|
+
title: Consumer Registration and Notification
|
|
84
|
+
description: |
|
|
85
|
+
Teams consuming a shared dataset MUST register as consumers in the
|
|
86
|
+
producer's data-contract.yaml file. Registration must include: consumer
|
|
87
|
+
team name, contact email/Slack, use case description, and fields
|
|
88
|
+
consumed. Registered consumers MUST receive automated notifications
|
|
89
|
+
when: the contract is modified, quality SLO breaches occur, the
|
|
90
|
+
dataset is deprecated, or planned maintenance will affect availability.
|
|
91
|
+
level: MUST
|
|
92
|
+
examples:
|
|
93
|
+
- "consumers entry: {team: analytics, contact: '#data-analytics', use_case: 'revenue reporting', fields: [order_id, amount, created_at]}"
|
|
94
|
+
- "Automated Slack notification: 'Data contract orders/v1 SLO breached: freshness 42min (SLO: 15min)'"
|
|
95
|
+
- "Deprecation notice sent 90 days before v1 end-of-life to all registered consumers"
|
|
96
|
+
|
|
97
|
+
- id: REQ-006
|
|
98
|
+
title: Contract Discoverability
|
|
99
|
+
description: |
|
|
100
|
+
All data contracts in an organization SHOULD be discoverable through
|
|
101
|
+
a central data catalog. The catalog SHOULD provide: searchable index
|
|
102
|
+
of all contracts by team/domain, current health status (quality SLO
|
|
103
|
+
compliance), schema browsing, lineage visualization (which pipelines
|
|
104
|
+
produce/consume each dataset), and a self-service consumer registration
|
|
105
|
+
workflow.
|
|
106
|
+
level: SHOULD
|
|
107
|
+
examples:
|
|
108
|
+
- "Data catalog: DataHub or Amundsen with all contract YAML files indexed"
|
|
109
|
+
- "Catalog entry includes: SLO compliance badge (green/yellow/red), last updated, consumer count"
|
|
110
|
+
- "Self-service: 'Subscribe to dataset' button generates PR adding consumer to data-contract.yaml"
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# Data Migration Testing Standards - AI Optimized
|
|
2
|
+
# Source: core/data-migration-testing.md
|
|
3
|
+
|
|
4
|
+
id: data-migration-testing
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-05"
|
|
8
|
+
source: core/data-migration-testing.md
|
|
9
|
+
description: Database migration up/down/idempotency testing standards for schema changes
|
|
10
|
+
|
|
11
|
+
requirements:
|
|
12
|
+
REQ-1:
|
|
13
|
+
id: REQ-DMT-001
|
|
14
|
+
title: Up Migration Test
|
|
15
|
+
rule: >
|
|
16
|
+
Every schema migration MUST have an automated test that applies it to a clean
|
|
17
|
+
baseline schema and verifies the expected post-state (table names, columns,
|
|
18
|
+
indexes, constraints).
|
|
19
|
+
rationale: >
|
|
20
|
+
Unverified migrations silently corrupt production schema; up tests catch
|
|
21
|
+
incompatible changes before deployment.
|
|
22
|
+
|
|
23
|
+
REQ-2:
|
|
24
|
+
id: REQ-DMT-002
|
|
25
|
+
title: Down Migration (Rollback) Test
|
|
26
|
+
rule: >
|
|
27
|
+
Every migration that has a down/rollback function MUST have a test that
|
|
28
|
+
applies up, then down, and verifies the schema returns to its exact pre-state.
|
|
29
|
+
Zero data loss is the acceptance criterion.
|
|
30
|
+
rationale: >
|
|
31
|
+
Rollback is only reliable if it is tested; untested rollbacks fail at the worst
|
|
32
|
+
possible moment — during a production incident.
|
|
33
|
+
|
|
34
|
+
REQ-3:
|
|
35
|
+
id: REQ-DMT-003
|
|
36
|
+
title: Idempotency Test
|
|
37
|
+
rule: >
|
|
38
|
+
Migration runners MUST be tested to ensure applying the same migration twice
|
|
39
|
+
does not fail or cause side effects (e.g., duplicate columns, duplicate indexes).
|
|
40
|
+
rationale: >
|
|
41
|
+
CI retries and operator mistakes can trigger double-apply; idempotent migrations
|
|
42
|
+
prevent partial state corruption.
|
|
43
|
+
|
|
44
|
+
REQ-4:
|
|
45
|
+
id: REQ-DMT-004
|
|
46
|
+
title: Data Preservation Test
|
|
47
|
+
rule: >
|
|
48
|
+
Migrations that ALTER or DROP columns MUST include a test that seeds representative
|
|
49
|
+
data before the migration and verifies data integrity or expected transformation
|
|
50
|
+
after the migration.
|
|
51
|
+
rationale: >
|
|
52
|
+
Schema correctness does not imply data correctness; a column rename can silently
|
|
53
|
+
null-out existing data if the migration omits a data transform step.
|
|
54
|
+
|
|
55
|
+
REQ-5:
|
|
56
|
+
id: REQ-DMT-005
|
|
57
|
+
title: Migration History Integrity
|
|
58
|
+
rule: >
|
|
59
|
+
The migration runner's applied-migrations table/ledger MUST be validated:
|
|
60
|
+
each test run uses an isolated in-memory or temporary database so migrations
|
|
61
|
+
do not interfere with each other or with production state.
|
|
62
|
+
rationale: >
|
|
63
|
+
Shared migration state between tests causes non-deterministic failures that are
|
|
64
|
+
extremely difficult to diagnose.
|
|
65
|
+
|
|
66
|
+
test_structure:
|
|
67
|
+
isolation: "Each migration test MUST run against an isolated in-memory or temp DB"
|
|
68
|
+
baseline: "Start from the schema state immediately before the migration under test"
|
|
69
|
+
assertions:
|
|
70
|
+
up: "Assert table/column/index existence and types"
|
|
71
|
+
down: "Assert schema returns to pre-migration state"
|
|
72
|
+
idempotency: "Apply migration twice; second apply must succeed without error"
|
|
73
|
+
data: "Seed rows before migration; assert row count and values after"
|
|
74
|
+
|
|
75
|
+
tooling_guidance:
|
|
76
|
+
sqlite: "Use ':memory:' DSN; apply migrations via ORM migrate() or raw SQL"
|
|
77
|
+
postgres: "Use pgmock or test containers; run migrate up/down in a transaction"
|
|
78
|
+
drizzle_orm: >
|
|
79
|
+
Call db.run(sql`...`) with raw DDL, or use drizzle-kit's migrate() function
|
|
80
|
+
against a fresh :memory: SQLite database for each test file.
|
|
81
|
+
|
|
82
|
+
anti_patterns:
|
|
83
|
+
- description: >
|
|
84
|
+
Testing migrations against a shared development database — causes cross-test
|
|
85
|
+
pollution and non-repeatable results.
|
|
86
|
+
- description: >
|
|
87
|
+
Skipping down migration tests because "we never rollback" — rollbacks happen
|
|
88
|
+
during incidents; the worst time to discover they are broken.
|
|
89
|
+
- description: >
|
|
90
|
+
Writing migration tests after the fact without seeding data — misses the
|
|
91
|
+
data preservation class of bugs entirely.
|
|
92
|
+
|
|
93
|
+
related_standards:
|
|
94
|
+
- testing
|
|
95
|
+
- database-standards
|
|
96
|
+
- verification-evidence
|