universal-dev-standards 5.3.2 → 5.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/ai/standards/adversarial-test.ai.yaml +277 -0
- package/bundled/ai/standards/agent-communication-protocol.ai.yaml +32 -166
- package/bundled/ai/standards/agent-dispatch.ai.yaml +32 -58
- package/bundled/ai/standards/audit-trail.ai.yaml +113 -0
- package/bundled/ai/standards/branch-completion.ai.yaml +34 -70
- package/bundled/ai/standards/change-batching-standards.ai.yaml +31 -180
- package/bundled/ai/standards/chaos-injection-tests.ai.yaml +91 -0
- package/bundled/ai/standards/container-image-standards.ai.yaml +88 -0
- package/bundled/ai/standards/container-security.ai.yaml +331 -0
- package/bundled/ai/standards/cost-budget-test.ai.yaml +96 -0
- package/bundled/ai/standards/data-contract.ai.yaml +110 -0
- package/bundled/ai/standards/data-migration-testing.ai.yaml +96 -0
- package/bundled/ai/standards/data-pipeline.ai.yaml +113 -0
- package/bundled/ai/standards/disaster-recovery-drill.ai.yaml +89 -0
- package/bundled/ai/standards/execution-history.ai.yaml +30 -288
- package/bundled/ai/standards/flaky-test-management.ai.yaml +89 -0
- package/bundled/ai/standards/flow-based-testing.ai.yaml +240 -0
- package/bundled/ai/standards/iac-design-principles.ai.yaml +83 -0
- package/bundled/ai/standards/incident-response.ai.yaml +107 -0
- package/bundled/ai/standards/license-compliance.ai.yaml +106 -0
- package/bundled/ai/standards/llm-output-validation.ai.yaml +269 -0
- package/bundled/ai/standards/mock-boundary.ai.yaml +250 -0
- package/bundled/ai/standards/mutation-testing.ai.yaml +192 -0
- package/bundled/ai/standards/pii-classification.ai.yaml +109 -0
- package/bundled/ai/standards/pipeline-integration-standards.ai.yaml +28 -169
- package/bundled/ai/standards/policy-as-code-testing.ai.yaml +227 -0
- package/bundled/ai/standards/prd-standards.ai.yaml +88 -0
- package/bundled/ai/standards/product-metrics-standards.ai.yaml +111 -0
- package/bundled/ai/standards/prompt-regression.ai.yaml +94 -0
- package/bundled/ai/standards/property-based-testing.ai.yaml +105 -0
- package/bundled/ai/standards/release-quality-manifest.ai.yaml +135 -0
- package/bundled/ai/standards/replay-test.ai.yaml +111 -0
- package/bundled/ai/standards/runbook.ai.yaml +104 -0
- package/bundled/ai/standards/sast-advanced.ai.yaml +135 -0
- package/bundled/ai/standards/schema-evolution.ai.yaml +111 -0
- package/bundled/ai/standards/secret-management-standards.ai.yaml +105 -0
- package/bundled/ai/standards/secure-op.ai.yaml +365 -0
- package/bundled/ai/standards/security-testing.ai.yaml +171 -0
- package/bundled/ai/standards/server-ops-security.ai.yaml +274 -0
- package/bundled/ai/standards/slo-sli.ai.yaml +97 -0
- package/bundled/ai/standards/smoke-test.ai.yaml +87 -0
- package/bundled/ai/standards/supply-chain-attestation.ai.yaml +109 -0
- package/bundled/ai/standards/test-completeness-dimensions.ai.yaml +52 -5
- package/bundled/ai/standards/user-story-mapping.ai.yaml +108 -0
- package/bundled/ai/standards/workflow-enforcement.ai.yaml +34 -240
- package/bundled/ai/standards/workflow-state-protocol.ai.yaml +31 -107
- package/bundled/core/adversarial-test.md +212 -0
- package/bundled/core/chaos-injection-tests.md +116 -0
- package/bundled/core/container-security.md +521 -0
- package/bundled/core/cost-budget-test.md +69 -0
- package/bundled/core/data-migration-testing.md +110 -0
- package/bundled/core/disaster-recovery-drill.md +73 -0
- package/bundled/core/flaky-test-management.md +73 -0
- package/bundled/core/flow-based-testing.md +142 -0
- package/bundled/core/llm-output-validation.md +178 -0
- package/bundled/core/mock-boundary.md +100 -0
- package/bundled/core/mutation-testing.md +97 -0
- package/bundled/core/policy-as-code-testing.md +188 -0
- package/bundled/core/prompt-regression.md +72 -0
- package/bundled/core/property-based-testing.md +73 -0
- package/bundled/core/release-quality-manifest.md +147 -0
- package/bundled/core/replay-test.md +86 -0
- package/bundled/core/sast-advanced.md +300 -0
- package/bundled/core/secure-op.md +314 -0
- package/bundled/core/security-testing.md +87 -0
- package/bundled/core/server-ops-security.md +493 -0
- package/bundled/core/smoke-test.md +65 -0
- package/bundled/core/supply-chain-attestation.md +117 -0
- package/bundled/locales/zh-CN/CHANGELOG.md +3 -3
- package/bundled/locales/zh-CN/README.md +1 -1
- package/bundled/locales/zh-CN/skills/ai-instruction-standards/SKILL.md +5 -5
- package/bundled/locales/zh-TW/CHANGELOG.md +3 -3
- package/bundled/locales/zh-TW/README.md +1 -1
- package/bundled/locales/zh-TW/skills/ai-instruction-standards/SKILL.md +183 -79
- package/bundled/skills/README.md +4 -3
- package/bundled/skills/SKILL_NAMING.md +94 -0
- package/bundled/skills/ai-instruction-standards/SKILL.md +181 -88
- package/bundled/skills/atdd-assistant/SKILL.md +8 -0
- package/bundled/skills/bdd-assistant/SKILL.md +7 -0
- package/bundled/skills/checkin-assistant/SKILL.md +8 -0
- package/bundled/skills/code-review-assistant/SKILL.md +7 -0
- package/bundled/skills/journey-test-assistant/SKILL.md +203 -0
- package/bundled/skills/orchestrate/SKILL.md +167 -0
- package/bundled/skills/plan/SKILL.md +234 -0
- package/bundled/skills/pr-automation-assistant/SKILL.md +8 -0
- package/bundled/skills/push/SKILL.md +49 -2
- package/bundled/skills/{process-automation → skill-builder}/SKILL.md +1 -1
- package/bundled/skills/{forward-derivation → spec-derivation}/SKILL.md +1 -1
- package/bundled/skills/spec-driven-dev/SKILL.md +7 -0
- package/bundled/skills/sweep/SKILL.md +145 -0
- package/bundled/skills/tdd-assistant/SKILL.md +7 -0
- package/package.json +1 -1
- package/src/commands/flow.js +8 -0
- package/src/commands/start.js +14 -0
- package/src/commands/sweep.js +8 -0
- package/src/commands/workflow.js +8 -0
- package/standards-registry.json +474 -12
- package/bundled/locales/zh-CN/skills/ac-coverage-assistant/SKILL.md +0 -190
- package/bundled/locales/zh-CN/skills/forward-derivation/SKILL.md +0 -71
- package/bundled/locales/zh-CN/skills/forward-derivation/guide.md +0 -130
- package/bundled/locales/zh-CN/skills/methodology-system/SKILL.md +0 -88
- package/bundled/locales/zh-CN/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-CN/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-CN/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-CN/skills/process-automation/SKILL.md +0 -143
- package/bundled/locales/zh-TW/skills/ac-coverage-assistant/SKILL.md +0 -195
- package/bundled/locales/zh-TW/skills/deploy-assistant/SKILL.md +0 -178
- package/bundled/locales/zh-TW/skills/forward-derivation/SKILL.md +0 -69
- package/bundled/locales/zh-TW/skills/forward-derivation/guide.md +0 -415
- package/bundled/locales/zh-TW/skills/methodology-system/SKILL.md +0 -86
- package/bundled/locales/zh-TW/skills/methodology-system/create-methodology.md +0 -350
- package/bundled/locales/zh-TW/skills/methodology-system/guide.md +0 -131
- package/bundled/locales/zh-TW/skills/methodology-system/runtime.md +0 -279
- package/bundled/locales/zh-TW/skills/process-automation/SKILL.md +0 -144
- /package/bundled/skills/{ac-coverage-assistant → ac-coverage}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/SKILL.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/create-methodology.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/guide.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/integrated-flow.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/prerequisite-check.md +0 -0
- /package/bundled/skills/{methodology-system → dev-methodology}/runtime.md +0 -0
- /package/bundled/skills/{forward-derivation → spec-derivation}/guide.md +0 -0
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Secret Management Standards - AI Optimized
|
|
2
|
+
# Source: XSPEC-065 Wave 4 IaC Pack
|
|
3
|
+
|
|
4
|
+
id: secret-management-standards
|
|
5
|
+
title: Secret Management and Credential Hygiene Standards
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
status: Active
|
|
8
|
+
tags: [secrets, vault, kms, sops, security, rotation, credential-management]
|
|
9
|
+
summary: |
|
|
10
|
+
Defines how teams store, inject, rotate, and audit secrets and credentials
|
|
11
|
+
across development and production environments. Covers three approved secret
|
|
12
|
+
source tiers (Vault dynamic secrets, Cloud KMS, SOPS+Git), rotation policies
|
|
13
|
+
by credential type, automated hardcoded-secret prevention via pre-commit and
|
|
14
|
+
CI scanning, and safe secret injection patterns. Designed to eliminate
|
|
15
|
+
credentials from source code and CI logs while maintaining operational
|
|
16
|
+
practicality across team sizes.
|
|
17
|
+
|
|
18
|
+
requirements:
|
|
19
|
+
- id: REQ-001
|
|
20
|
+
title: Secret Source Three Options
|
|
21
|
+
description: |
|
|
22
|
+
Teams MUST use one of three approved secret source tiers based on
|
|
23
|
+
operational context. (1) HashiCorp Vault dynamic secrets (preferred for
|
|
24
|
+
production and multi-team environments) — secrets are generated on-demand
|
|
25
|
+
with short TTLs; no static credentials stored anywhere. (2) Cloud KMS
|
|
26
|
+
with native secret managers (AWS Secrets Manager / GCP Secret Manager /
|
|
27
|
+
Azure Key Vault) — suitable for cloud-native deployments; secrets fetched
|
|
28
|
+
at runtime via IAM-controlled API calls. (3) SOPS + Git encryption —
|
|
29
|
+
suitable for small teams and GitOps workflows; secrets encrypted with
|
|
30
|
+
age or KMS key before committing; decrypted only in trusted runtime
|
|
31
|
+
environments. Storing unencrypted secrets in any other location (env
|
|
32
|
+
files, wiki, chat) is PROHIBITED.
|
|
33
|
+
level: MUST
|
|
34
|
+
examples:
|
|
35
|
+
- "Vault: app requests DB credentials via Vault agent sidecar with 1h TTL lease"
|
|
36
|
+
- "AWS Secrets Manager: Lambda reads secret ARN from env var; SDK fetches at cold start"
|
|
37
|
+
- "SOPS: `secrets.yaml` encrypted with age key; decrypted in CI via SOPS_AGE_KEY env var"
|
|
38
|
+
- "Prohibited: secrets in `.env` files committed to repo, even private repos"
|
|
39
|
+
|
|
40
|
+
- id: REQ-002
|
|
41
|
+
title: Rotation Policy by Type
|
|
42
|
+
description: |
|
|
43
|
+
All secrets MUST have a defined rotation policy enforced by automated
|
|
44
|
+
tooling or calendar reminders. Minimum rotation frequencies by type:
|
|
45
|
+
Database credentials: every 90 days. API keys (third-party services):
|
|
46
|
+
every 180 days. Signing keys (JWT, code signing): every 365 days.
|
|
47
|
+
One-time tokens and session credentials: revoke immediately after use;
|
|
48
|
+
MUST NOT be reused. TLS certificates: rotate at least 30 days before
|
|
49
|
+
expiry; automate with ACME/Let's Encrypt or cert-manager where possible.
|
|
50
|
+
Rotation events MUST be logged in the audit trail.
|
|
51
|
+
level: MUST
|
|
52
|
+
examples:
|
|
53
|
+
- "DB credentials rotated via Vault dynamic secrets every 90 days automatically"
|
|
54
|
+
- "Stripe API key rotation reminder calendar event set for 180-day interval"
|
|
55
|
+
- "JWT signing key rotated annually; old key retained for 7-day grace period"
|
|
56
|
+
- "CI temporary tokens scoped to single job; revoked by runner post-job"
|
|
57
|
+
|
|
58
|
+
- id: REQ-003
|
|
59
|
+
title: Hardcoded Secret Prevention
|
|
60
|
+
description: |
|
|
61
|
+
Teams MUST implement automated scanning to detect and block hardcoded
|
|
62
|
+
secrets before they reach the repository. Two layers are REQUIRED:
|
|
63
|
+
(1) Pre-commit hook using detect-secrets, gitleaks, or truffleHog —
|
|
64
|
+
scans staged files and blocks commit if patterns are detected.
|
|
65
|
+
(2) CI pipeline scan — rescans all changed files on every PR; blocks
|
|
66
|
+
merge if secrets are found. Minimum detected patterns: AWS access key
|
|
67
|
+
format (AKIA[0-9A-Z]{16}), PEM private key headers (-----BEGIN .* PRIVATE KEY),
|
|
68
|
+
generic API token patterns (api[_-]?key\s*[:=]\s*\S{16,}), and
|
|
69
|
+
connection strings containing passwords.
|
|
70
|
+
level: MUST
|
|
71
|
+
examples:
|
|
72
|
+
- "`.pre-commit-config.yaml` includes `detect-secrets` hook; fails on AWS key pattern"
|
|
73
|
+
- "CI step: `gitleaks detect --source . --exit-code 1` blocks PR merge"
|
|
74
|
+
- "False positive whitelisted via `.secrets.baseline` with documented justification"
|
|
75
|
+
- "Developer receives pre-commit error: 'High confidence secret detected: AWS Access Key'"
|
|
76
|
+
|
|
77
|
+
- id: REQ-004
|
|
78
|
+
title: Secret Injection
|
|
79
|
+
description: |
|
|
80
|
+
Secrets MUST be injected into application processes via environment
|
|
81
|
+
variables or mounted files only. Passing secrets via command-line
|
|
82
|
+
arguments is PROHIBITED (visible in process lists). Passing secrets via
|
|
83
|
+
URL query parameters is PROHIBITED (logged by proxies and servers).
|
|
84
|
+
For environment variable injection, use the platform's native secret
|
|
85
|
+
injection (Kubernetes Secrets, ECS task definition secrets, GitHub
|
|
86
|
+
Actions secrets). For file-based injection, mount secrets as
|
|
87
|
+
read-only volumes with restrictive file permissions (0400 or 0600).
|
|
88
|
+
level: MUST
|
|
89
|
+
examples:
|
|
90
|
+
- "Kubernetes: secret mounted as env var via `secretKeyRef` in pod spec"
|
|
91
|
+
- "Prohibited: `./app --db-password=s3cr3t` (visible in `ps aux`)"
|
|
92
|
+
- "Prohibited: `https://api.example.com?token=abc123` (logged by nginx)"
|
|
93
|
+
- "File injection: secret mounted at `/run/secrets/db-password` with mode 0400"
|
|
94
|
+
|
|
95
|
+
anti_patterns:
|
|
96
|
+
- "Hardcoding credentials directly in source code or configuration files"
|
|
97
|
+
- "Storing secrets in CI/CD environment variables without encryption (plaintext in UI)"
|
|
98
|
+
- "Sharing credentials across multiple environments (dev/staging/prod use same secret)"
|
|
99
|
+
- "Long-lived static credentials without rotation schedules"
|
|
100
|
+
- "Committing .env files containing real secrets to version control"
|
|
101
|
+
|
|
102
|
+
related_standards:
|
|
103
|
+
- iac-design-principles
|
|
104
|
+
- audit-trail
|
|
105
|
+
- pii-classification
|
|
@@ -0,0 +1,365 @@
|
|
|
1
|
+
# Secure-Op: AI Agent Secure Operation Standard - AI Optimized
|
|
2
|
+
# Source: core/secure-op.md
|
|
3
|
+
|
|
4
|
+
id: secure-op
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-04"
|
|
8
|
+
source: core/secure-op.md
|
|
9
|
+
description: >
|
|
10
|
+
AI Agent secure operation methodology covering Veto-based decision pipeline,
|
|
11
|
+
SOBR multi-dimensional risk scoring, Fail-Closed principle, tamper-evident
|
|
12
|
+
audit chain, HITL escalation, and prompt injection defense.
|
|
13
|
+
|
|
14
|
+
# ─────────────────────────────────────────────────────────
|
|
15
|
+
# Core Categories
|
|
16
|
+
# ─────────────────────────────────────────────────────────
|
|
17
|
+
categories:
|
|
18
|
+
- id: veto_based_decision
|
|
19
|
+
name: Veto-Based Decision Pipeline
|
|
20
|
+
description: >
|
|
21
|
+
Security decision logic must be Veto-based, not voting-based.
|
|
22
|
+
Any single layer issuing DENY terminates the pipeline immediately.
|
|
23
|
+
principles:
|
|
24
|
+
- name: Deterministic > Probabilistic
|
|
25
|
+
rule: >
|
|
26
|
+
Deterministic systems (policy engines, rule engines) take precedence
|
|
27
|
+
over LLM inference. Policy engine DENY cannot be overridden by semantic review.
|
|
28
|
+
- name: Policy-as-Code
|
|
29
|
+
rule: >
|
|
30
|
+
All security rules must be version-controlled and regression-testable.
|
|
31
|
+
No ad-hoc runtime rule injection.
|
|
32
|
+
- name: Decision Pipeline Order
|
|
33
|
+
pipeline:
|
|
34
|
+
- step: 1
|
|
35
|
+
layer: Policy Engine (OPA or equivalent)
|
|
36
|
+
type: deterministic
|
|
37
|
+
on_deny: terminate_immediately
|
|
38
|
+
- step: 2
|
|
39
|
+
layer: Risk Score (SOBR model)
|
|
40
|
+
type: quantitative
|
|
41
|
+
on_deny: terminate_or_escalate
|
|
42
|
+
- step: 3
|
|
43
|
+
layer: Semantic Review (optional LLM)
|
|
44
|
+
type: probabilistic
|
|
45
|
+
on_deny: terminate
|
|
46
|
+
note: Only reached if steps 1 and 2 pass
|
|
47
|
+
reference_implementation: OPA (Open Policy Agent) or equivalent policy engine
|
|
48
|
+
anti_patterns:
|
|
49
|
+
- Allowing LLM to override a deterministic DENY
|
|
50
|
+
- Treating security decisions as majority-vote among layers
|
|
51
|
+
- Bypassing policy engine for "efficiency"
|
|
52
|
+
|
|
53
|
+
- id: sobr_risk_scoring
|
|
54
|
+
name: SOBR Risk Scoring Model
|
|
55
|
+
description: >
|
|
56
|
+
Four-dimension quantitative risk model: Sensitivity, Operation, BlastRadius,
|
|
57
|
+
Reversibility. Produces a 0–100 score for routing decisions.
|
|
58
|
+
formula: "RiskScore = S×0.30 + O×0.25 + B×0.25 + R×0.20"
|
|
59
|
+
dimensions:
|
|
60
|
+
- id: S
|
|
61
|
+
name: Sensitivity
|
|
62
|
+
weight: 0.30
|
|
63
|
+
description: Target resource sensitivity level
|
|
64
|
+
range: 0-100
|
|
65
|
+
reference_values:
|
|
66
|
+
user_credentials: 100
|
|
67
|
+
prod_database: 95
|
|
68
|
+
internal_api_key: 85
|
|
69
|
+
staging_database: 60
|
|
70
|
+
config_file: 45
|
|
71
|
+
dev_environment: 30
|
|
72
|
+
public_docs: 5
|
|
73
|
+
- id: O
|
|
74
|
+
name: OperationType
|
|
75
|
+
weight: 0.25
|
|
76
|
+
description: Danger level of the operation being requested
|
|
77
|
+
range: 0-100
|
|
78
|
+
reference_values:
|
|
79
|
+
delete_permanent: 95
|
|
80
|
+
execute_arbitrary_code: 100
|
|
81
|
+
modify_iam_policy: 90
|
|
82
|
+
write_production_data: 75
|
|
83
|
+
read_sensitive_data: 50
|
|
84
|
+
read_public_data: 10
|
|
85
|
+
- id: B
|
|
86
|
+
name: BlastRadius
|
|
87
|
+
weight: 0.25
|
|
88
|
+
description: Number of systems or users affected if operation goes wrong
|
|
89
|
+
range: 0-100
|
|
90
|
+
reference_values:
|
|
91
|
+
all_production_systems: 100
|
|
92
|
+
single_production_service: 70
|
|
93
|
+
staging_environment: 40
|
|
94
|
+
isolated_dev_sandbox: 10
|
|
95
|
+
- id: R
|
|
96
|
+
name: Reversibility
|
|
97
|
+
weight: 0.20
|
|
98
|
+
description: Irreversibility degree (higher = harder to undo)
|
|
99
|
+
range: 0-100
|
|
100
|
+
reference_values:
|
|
101
|
+
permanent_delete_no_backup: 100
|
|
102
|
+
overwrite_with_no_version: 85
|
|
103
|
+
delete_with_backup: 50
|
|
104
|
+
read_only_no_side_effect: 0
|
|
105
|
+
decision_thresholds:
|
|
106
|
+
- range: "0-25"
|
|
107
|
+
decision: ALLOW
|
|
108
|
+
action: Execute normally; log for audit
|
|
109
|
+
- range: "26-50"
|
|
110
|
+
decision: ALLOW_WITH_MONITORING
|
|
111
|
+
action: Execute with enhanced logging; flag for post-hoc review
|
|
112
|
+
- range: "51-75"
|
|
113
|
+
decision: REQUIRE_HITL
|
|
114
|
+
action: Escalate to human reviewer; optionally pause execution
|
|
115
|
+
- range: "76-100"
|
|
116
|
+
decision: DENY
|
|
117
|
+
action: Reject operation; log decision path and violations
|
|
118
|
+
iso_mapping:
|
|
119
|
+
- "ISO/IEC 27001:2022 Annex A.8.24 - Use of privileged utility programs"
|
|
120
|
+
- "ISO/IEC 27005 - Information security risk management"
|
|
121
|
+
|
|
122
|
+
- id: fail_closed
|
|
123
|
+
name: Fail-Closed Principle
|
|
124
|
+
description: >
|
|
125
|
+
When any security component fails or is unreachable, the default behavior
|
|
126
|
+
MUST be DENY. Silent failure or fail-open is strictly prohibited.
|
|
127
|
+
rules:
|
|
128
|
+
- condition: Policy Engine unreachable
|
|
129
|
+
response: DENY ALL
|
|
130
|
+
exception: Read-only operations with no side effects may be allowed at operator discretion
|
|
131
|
+
- condition: Risk Scoring computation fails
|
|
132
|
+
response: REQUIRE_HITL (conservative escalation)
|
|
133
|
+
rationale: Unknown risk must be treated as high risk
|
|
134
|
+
- condition: Signature verification fails
|
|
135
|
+
response: DENY ALL
|
|
136
|
+
rationale: Tampered or corrupted verdict cannot be trusted
|
|
137
|
+
- condition: Any unknown/unhandled error
|
|
138
|
+
response: DENY
|
|
139
|
+
rationale: Unknown state must default to safe state
|
|
140
|
+
anti_patterns:
|
|
141
|
+
- name: Fail-Open
|
|
142
|
+
description: Defaulting to ALLOW when an error occurs
|
|
143
|
+
severity: CRITICAL
|
|
144
|
+
- name: Partial Validation Continue
|
|
145
|
+
description: Continuing execution after a partial validation failure
|
|
146
|
+
severity: HIGH
|
|
147
|
+
- name: Silent Error Swallowing
|
|
148
|
+
description: Catching errors without changing the decision to DENY
|
|
149
|
+
severity: HIGH
|
|
150
|
+
iso_mapping:
|
|
151
|
+
- "NIST SP 800-207 Zero Trust Architecture - Section 2.1"
|
|
152
|
+
- "ISO/IEC 27001:2022 A.8.22 - Filtering of web services"
|
|
153
|
+
|
|
154
|
+
- id: audit_chain
|
|
155
|
+
name: Tamper-Evident Audit Chain
|
|
156
|
+
description: >
|
|
157
|
+
All security decisions must be recorded in a verifiable, append-only,
|
|
158
|
+
tamper-evident audit trail using cryptographic hash chaining.
|
|
159
|
+
required_fields:
|
|
160
|
+
- field: request_id
|
|
161
|
+
type: string (UUID v4)
|
|
162
|
+
description: Unique identifier for each decision event
|
|
163
|
+
- field: decision
|
|
164
|
+
type: enum (ALLOW | ALLOW_WITH_MONITORING | REQUIRE_HITL | DENY)
|
|
165
|
+
description: Final security verdict
|
|
166
|
+
- field: risk_score
|
|
167
|
+
type: number (0-100)
|
|
168
|
+
description: Computed SOBR risk score
|
|
169
|
+
- field: timestamp
|
|
170
|
+
type: ISO 8601 UTC
|
|
171
|
+
description: Decision timestamp with millisecond precision
|
|
172
|
+
- field: violations
|
|
173
|
+
type: string[]
|
|
174
|
+
description: List of policy violations (empty if ALLOW)
|
|
175
|
+
- field: signature
|
|
176
|
+
type: base64-encoded Ed25519 signature
|
|
177
|
+
description: Cryptographic signature over core fields
|
|
178
|
+
- field: prev_hash
|
|
179
|
+
type: SHA-256 hex string
|
|
180
|
+
description: Hash of the previous audit record (chain link)
|
|
181
|
+
hash_chain:
|
|
182
|
+
algorithm: SHA-256
|
|
183
|
+
chain_field: prev_hash
|
|
184
|
+
genesis_value: "0000000000000000000000000000000000000000000000000000000000000000"
|
|
185
|
+
verification: Implement verify_chain() to detect any record tampering
|
|
186
|
+
signature:
|
|
187
|
+
algorithm: Ed25519
|
|
188
|
+
signed_payload: "{request_id}:{decision}:{risk_score}:{timestamp}"
|
|
189
|
+
note: Use asymmetric cryptography; never symmetric HMAC for audit signatures
|
|
190
|
+
storage_levels:
|
|
191
|
+
- level: 0
|
|
192
|
+
name: Minimum
|
|
193
|
+
storage: Local append-only file
|
|
194
|
+
note: Acceptable for dev/test only
|
|
195
|
+
- level: 1
|
|
196
|
+
name: Recommended
|
|
197
|
+
storage: WORM storage (S3 Object Lock / Azure Immutable Blob)
|
|
198
|
+
note: Required for production
|
|
199
|
+
- level: 2
|
|
200
|
+
name: Maximum
|
|
201
|
+
storage: Immutable cloud storage + remote attestation
|
|
202
|
+
note: Required for regulated environments (SOC2, ISO 27001 certified)
|
|
203
|
+
iso_mapping:
|
|
204
|
+
- "ISO/IEC 27001:2022 A.8.15 - Logging"
|
|
205
|
+
- "ISO/IEC 27001:2022 A.5.33 - Protection of records"
|
|
206
|
+
|
|
207
|
+
- id: hitl_escalation
|
|
208
|
+
name: Human-in-the-Loop (HITL) Escalation
|
|
209
|
+
description: >
|
|
210
|
+
When Risk Score falls in the 51-75 range (REQUIRE_HITL), a defined
|
|
211
|
+
human review escalation mechanism must be triggered.
|
|
212
|
+
modes:
|
|
213
|
+
- mode: non_blocking
|
|
214
|
+
description: Send notification but allow execution to continue under enhanced monitoring
|
|
215
|
+
when_to_use: Lower-risk HITL scenarios (score 51-62), non-critical operations
|
|
216
|
+
- mode: blocking
|
|
217
|
+
description: Pause execution and wait for explicit human approval
|
|
218
|
+
when_to_use: Higher-risk HITL scenarios (score 63-75), sensitive environments
|
|
219
|
+
requirements:
|
|
220
|
+
- TTL: HITL review requests must expire (recommended 1800 seconds)
|
|
221
|
+
- on_ttl_expire: Escalate decision to DENY
|
|
222
|
+
- audit: HITL trigger event and resolution must be logged in audit chain
|
|
223
|
+
- notification_interface: Webhook (canonical); adapters for Slack, Teams, PagerDuty
|
|
224
|
+
notification_payload:
|
|
225
|
+
fields:
|
|
226
|
+
- request_id
|
|
227
|
+
- risk_score
|
|
228
|
+
- operation_summary
|
|
229
|
+
- policy_violations
|
|
230
|
+
- expires_at
|
|
231
|
+
iso_mapping:
|
|
232
|
+
- "ISO/IEC 27001:2022 A.8.2 - Privileged access rights"
|
|
233
|
+
- "NIST SP 800-53 AC-2 - Account Management"
|
|
234
|
+
|
|
235
|
+
- id: prompt_injection_defense
|
|
236
|
+
name: Prompt Injection Defense
|
|
237
|
+
description: >
|
|
238
|
+
AI Agent systems must detect and block prompt injection attacks that attempt
|
|
239
|
+
to override security controls through malicious user input.
|
|
240
|
+
detection_patterns:
|
|
241
|
+
override_commands:
|
|
242
|
+
- "ignore previous instructions"
|
|
243
|
+
- "disregard your programming"
|
|
244
|
+
- "override your instructions"
|
|
245
|
+
- "forget what you were told"
|
|
246
|
+
role_manipulation:
|
|
247
|
+
- "you are now"
|
|
248
|
+
- "act as"
|
|
249
|
+
- "pretend you are"
|
|
250
|
+
- "DAN"
|
|
251
|
+
- "jailbreak"
|
|
252
|
+
system_token_injection:
|
|
253
|
+
- "[SYSTEM]"
|
|
254
|
+
- "[INST]"
|
|
255
|
+
- "<|system|>"
|
|
256
|
+
- "<<SYS>>"
|
|
257
|
+
instruction_prefix:
|
|
258
|
+
- "New instruction:"
|
|
259
|
+
- "Updated system prompt:"
|
|
260
|
+
- "OVERRIDE:"
|
|
261
|
+
response:
|
|
262
|
+
on_detection: DENY
|
|
263
|
+
violation_code: PROMPT_INJECTION_DETECTED
|
|
264
|
+
audit: Record detected pattern and original input hash in audit chain
|
|
265
|
+
note: >
|
|
266
|
+
Do not log raw malicious input; log the detected pattern type and
|
|
267
|
+
a hash of the input to avoid storing adversarial content in logs.
|
|
268
|
+
pipeline_integration:
|
|
269
|
+
intercept_at: M1 Intake layer (earliest possible stage)
|
|
270
|
+
rationale: Reject before entering evaluation pipeline to reduce attack surface
|
|
271
|
+
maintenance:
|
|
272
|
+
- Review and update detection patterns monthly
|
|
273
|
+
- Subscribe to OWASP LLM Working Group updates
|
|
274
|
+
iso_mapping:
|
|
275
|
+
- "OWASP LLM Top 10 2025 - LLM01: Prompt Injection"
|
|
276
|
+
- "ISO/IEC 27001:2022 A.8.24 - Use of privileged utility programs"
|
|
277
|
+
|
|
278
|
+
# ─────────────────────────────────────────────────────────
|
|
279
|
+
# Quality Gates
|
|
280
|
+
# ─────────────────────────────────────────────────────────
|
|
281
|
+
quality_gates:
|
|
282
|
+
agent_operation_checklist:
|
|
283
|
+
description: AI Agent operations MUST pass all gates before execution
|
|
284
|
+
gates:
|
|
285
|
+
- id: policy_engine_check
|
|
286
|
+
check: Policy Engine returns ALLOW (not DENY)
|
|
287
|
+
on_fail: Reject immediately; log violations
|
|
288
|
+
required: true
|
|
289
|
+
- id: risk_score_check
|
|
290
|
+
check: Risk Score < 76
|
|
291
|
+
on_fail: Reject; log decision path
|
|
292
|
+
required: true
|
|
293
|
+
- id: hitl_check
|
|
294
|
+
check: If Risk Score 51-75, HITL notification sent (blocking or non-blocking per mode)
|
|
295
|
+
on_fail: Reject if blocking mode timeout; continue with monitoring if non-blocking
|
|
296
|
+
required: true
|
|
297
|
+
- id: prompt_injection_check
|
|
298
|
+
check: No prompt injection pattern detected in input
|
|
299
|
+
on_fail: Reject; log PROMPT_INJECTION_DETECTED
|
|
300
|
+
required: true
|
|
301
|
+
- id: verdict_signature_check
|
|
302
|
+
check: Verdict carries valid cryptographic signature
|
|
303
|
+
on_fail: Reject; log SIGNATURE_INVALID
|
|
304
|
+
required: true
|
|
305
|
+
- id: audit_chain_check
|
|
306
|
+
check: Decision recorded in audit chain with prev_hash linked
|
|
307
|
+
on_fail: Log failure; decision still applied but alert raised
|
|
308
|
+
required: true
|
|
309
|
+
|
|
310
|
+
# ─────────────────────────────────────────────────────────
|
|
311
|
+
# Rules
|
|
312
|
+
# ─────────────────────────────────────────────────────────
|
|
313
|
+
rules:
|
|
314
|
+
- id: veto-over-vote
|
|
315
|
+
trigger: implementing security decision logic for AI agents
|
|
316
|
+
instruction: >
|
|
317
|
+
Use Veto-based pipeline (any DENY terminates). Never use voting/consensus
|
|
318
|
+
across security layers.
|
|
319
|
+
priority: required
|
|
320
|
+
|
|
321
|
+
- id: fail-closed-default
|
|
322
|
+
trigger: handling errors in security components
|
|
323
|
+
instruction: >
|
|
324
|
+
Default to DENY on any failure. Document explicit exceptions (e.g., read-only
|
|
325
|
+
fallback) with justification in code comments.
|
|
326
|
+
priority: required
|
|
327
|
+
|
|
328
|
+
- id: sobr-score-before-execution
|
|
329
|
+
trigger: AI agent requesting an operation
|
|
330
|
+
instruction: >
|
|
331
|
+
Compute SOBR risk score before executing any operation. Route based on
|
|
332
|
+
thresholds: <26 ALLOW, 26-50 ALLOW_WITH_MONITORING, 51-75 HITL, >=76 DENY.
|
|
333
|
+
priority: required
|
|
334
|
+
|
|
335
|
+
- id: audit-every-decision
|
|
336
|
+
trigger: any security verdict (ALLOW or DENY)
|
|
337
|
+
instruction: >
|
|
338
|
+
Record all decisions in audit chain regardless of outcome. ALLOW decisions
|
|
339
|
+
are auditable too—do not only log denials.
|
|
340
|
+
priority: required
|
|
341
|
+
|
|
342
|
+
- id: prompt-injection-earliest
|
|
343
|
+
trigger: receiving user input in AI agent pipeline
|
|
344
|
+
instruction: >
|
|
345
|
+
Scan for prompt injection at M1 Intake, before any semantic evaluation.
|
|
346
|
+
Do not allow suspicious input to reach the LLM reasoning layer.
|
|
347
|
+
priority: required
|
|
348
|
+
|
|
349
|
+
anti_patterns:
|
|
350
|
+
- Defaulting to ALLOW when policy engine is unreachable (fail-open)
|
|
351
|
+
- Using LLM confidence score as the sole security gate
|
|
352
|
+
- Skipping audit chain for "low-risk" operations
|
|
353
|
+
- Storing raw malicious prompt injection content in logs
|
|
354
|
+
- Using HMAC instead of asymmetric signatures for audit records
|
|
355
|
+
- Setting HITL TTL to infinity (must have an expiry)
|
|
356
|
+
- Treating Risk Score as advisory rather than binding
|
|
357
|
+
|
|
358
|
+
quick_reference:
|
|
359
|
+
secure_op_minimum_baseline: |
|
|
360
|
+
□ Policy Engine configured and reachable (fail-closed on unavailability)
|
|
361
|
+
□ SOBR Risk Score computed for every operation
|
|
362
|
+
□ HITL mechanism implemented for score 51-75 range
|
|
363
|
+
□ Prompt injection detection active at intake layer
|
|
364
|
+
□ All decisions recorded in hash-chained audit log with signatures
|
|
365
|
+
□ Audit log stored in append-only / WORM storage in production
|
|
@@ -0,0 +1,171 @@
|
|
|
1
|
+
# Security Testing Standards - AI Optimized
|
|
2
|
+
# Source: core/security-testing.md
|
|
3
|
+
|
|
4
|
+
id: security-testing
|
|
5
|
+
meta:
|
|
6
|
+
version: "1.0.0"
|
|
7
|
+
updated: "2026-05-04"
|
|
8
|
+
source: core/security-testing.md
|
|
9
|
+
description: >
|
|
10
|
+
Security testing methodology covering SAST, dependency auditing, and DAST.
|
|
11
|
+
Complements security-standards.ai.yaml (architecture) with execution-level guidance.
|
|
12
|
+
|
|
13
|
+
# ─────────────────────────────────────────────────────────
|
|
14
|
+
# Core Categories
|
|
15
|
+
# ─────────────────────────────────────────────────────────
|
|
16
|
+
categories:
|
|
17
|
+
- id: sast
|
|
18
|
+
name: Static Application Security Testing (SAST)
|
|
19
|
+
description: Analyze source code for vulnerabilities without executing it
|
|
20
|
+
tools:
|
|
21
|
+
typescript_javascript:
|
|
22
|
+
- name: eslint-plugin-security
|
|
23
|
+
config: "eslint-plugin-security/recommended"
|
|
24
|
+
detects: [eval injection, regex DoS, path traversal, prototype pollution]
|
|
25
|
+
- name: semgrep
|
|
26
|
+
config: "p/typescript"
|
|
27
|
+
detects: [XSS, injection, hardcoded secrets]
|
|
28
|
+
python:
|
|
29
|
+
- name: bandit
|
|
30
|
+
command: "bandit -r . -ll"
|
|
31
|
+
detects: [SQL injection, hardcoded passwords, insecure deserialization]
|
|
32
|
+
java:
|
|
33
|
+
- name: SpotBugs + FindSecBugs
|
|
34
|
+
detects: [SQL injection, XSS, LDAP injection]
|
|
35
|
+
trigger: pre-commit + CI on every PR
|
|
36
|
+
severity_gate: High or Critical → block merge
|
|
37
|
+
|
|
38
|
+
- id: dependency_audit
|
|
39
|
+
name: Dependency Vulnerability Auditing
|
|
40
|
+
description: Scan third-party packages for known CVEs
|
|
41
|
+
tools:
|
|
42
|
+
nodejs:
|
|
43
|
+
- name: npm audit
|
|
44
|
+
command: "npm audit --audit-level=high"
|
|
45
|
+
audit_levels:
|
|
46
|
+
- level: critical
|
|
47
|
+
action: Block immediately — patch or remove dependency
|
|
48
|
+
- level: high
|
|
49
|
+
action: Block merge — must be resolved before shipping
|
|
50
|
+
- level: moderate
|
|
51
|
+
action: Log warning — resolve within 14 days
|
|
52
|
+
- level: low
|
|
53
|
+
action: Track — resolve in next scheduled maintenance
|
|
54
|
+
- name: snyk
|
|
55
|
+
command: "snyk test --severity-threshold=high"
|
|
56
|
+
note: Alternative to npm audit; provides remediation suggestions
|
|
57
|
+
python:
|
|
58
|
+
- name: pip-audit
|
|
59
|
+
command: "pip-audit --vulnerability-service pypi"
|
|
60
|
+
- name: safety
|
|
61
|
+
command: "safety check"
|
|
62
|
+
trigger:
|
|
63
|
+
- pre-push hook
|
|
64
|
+
- CI pipeline (every PR)
|
|
65
|
+
- Weekly scheduled scan (catch newly disclosed CVEs)
|
|
66
|
+
severity_gate: High or Critical CVE → block release
|
|
67
|
+
|
|
68
|
+
- id: dast
|
|
69
|
+
name: Dynamic Application Security Testing (DAST)
|
|
70
|
+
description: Test running application for vulnerabilities through HTTP interactions
|
|
71
|
+
when_to_use: Applications with public-facing APIs or web UIs
|
|
72
|
+
tools:
|
|
73
|
+
- name: OWASP ZAP
|
|
74
|
+
mode: [baseline scan, full scan, API scan]
|
|
75
|
+
trigger: staging deployment
|
|
76
|
+
- name: Nuclei
|
|
77
|
+
command: "nuclei -u https://staging.example.com -t cves/"
|
|
78
|
+
trigger: staging deployment
|
|
79
|
+
trigger: Post-deployment to staging — NOT in unit/integration test phase
|
|
80
|
+
severity_gate: High or Critical finding → block production deployment
|
|
81
|
+
|
|
82
|
+
- id: secret_scanning
|
|
83
|
+
name: Secret / Credential Scanning
|
|
84
|
+
description: Detect accidentally committed secrets, tokens, and credentials
|
|
85
|
+
tools:
|
|
86
|
+
- name: gitleaks
|
|
87
|
+
command: "gitleaks detect --source . --no-git"
|
|
88
|
+
detects: [API keys, JWT secrets, database URLs, private keys]
|
|
89
|
+
- name: truffleHog
|
|
90
|
+
command: "trufflehog git file://."
|
|
91
|
+
trigger: pre-commit + CI
|
|
92
|
+
severity_gate: Any detected secret → block commit/merge immediately
|
|
93
|
+
|
|
94
|
+
# ─────────────────────────────────────────────────────────
|
|
95
|
+
# Quality Gates
|
|
96
|
+
# ─────────────────────────────────────────────────────────
|
|
97
|
+
quality_gates:
|
|
98
|
+
pre_commit:
|
|
99
|
+
- sast (eslint-plugin-security or equivalent)
|
|
100
|
+
- secret_scanning (gitleaks)
|
|
101
|
+
|
|
102
|
+
pre_push:
|
|
103
|
+
- dependency_audit (npm audit --audit-level=high)
|
|
104
|
+
|
|
105
|
+
pre_merge:
|
|
106
|
+
- all pre_commit gates
|
|
107
|
+
- dependency_audit
|
|
108
|
+
|
|
109
|
+
pre_release:
|
|
110
|
+
- all pre_merge gates
|
|
111
|
+
- dast (staging scan)
|
|
112
|
+
- full dependency_audit (npm audit without --audit-level filter)
|
|
113
|
+
|
|
114
|
+
# ─────────────────────────────────────────────────────────
|
|
115
|
+
# CVE Handling Policy
|
|
116
|
+
# ─────────────────────────────────────────────────────────
|
|
117
|
+
cve_policy:
|
|
118
|
+
critical: Patch or remove within 24 hours; block all deploys until resolved
|
|
119
|
+
high: Resolve before next release; document if temporary exception needed
|
|
120
|
+
moderate: Track in backlog; resolve within 14 days
|
|
121
|
+
low: Track; resolve in scheduled maintenance window
|
|
122
|
+
|
|
123
|
+
exception_process:
|
|
124
|
+
- Document CVE ID and reason for exception
|
|
125
|
+
- Set expiry date (max 30 days)
|
|
126
|
+
- Get security lead approval
|
|
127
|
+
- Add to `.npmrc` / `audit-exceptions.json` with comment
|
|
128
|
+
|
|
129
|
+
# ─────────────────────────────────────────────────────────
|
|
130
|
+
# Rules
|
|
131
|
+
# ─────────────────────────────────────────────────────────
|
|
132
|
+
rules:
|
|
133
|
+
- id: dependency-audit-pre-push
|
|
134
|
+
trigger: pushing code to remote
|
|
135
|
+
instruction: Run npm audit --audit-level=high before push; block on High/Critical findings
|
|
136
|
+
priority: required
|
|
137
|
+
|
|
138
|
+
- id: sast-pre-commit
|
|
139
|
+
trigger: committing code changes
|
|
140
|
+
instruction: Run SAST tool (eslint-plugin-security); block on High severity findings
|
|
141
|
+
priority: required
|
|
142
|
+
|
|
143
|
+
- id: secret-scan-always
|
|
144
|
+
trigger: any code commit
|
|
145
|
+
instruction: Run secret scanning; never commit with detected secrets
|
|
146
|
+
priority: required
|
|
147
|
+
|
|
148
|
+
- id: dast-on-staging
|
|
149
|
+
trigger: deploying to staging environment
|
|
150
|
+
instruction: Run DAST baseline scan; review findings before promoting to production
|
|
151
|
+
priority: required
|
|
152
|
+
|
|
153
|
+
- id: no-mock-security-in-tests
|
|
154
|
+
trigger: writing tests involving auth or security controls
|
|
155
|
+
instruction: Never mock security controls in tests (see mock-boundary.ai.yaml)
|
|
156
|
+
priority: required
|
|
157
|
+
|
|
158
|
+
anti_patterns:
|
|
159
|
+
- Treating all CVEs as equal (Critical ≠ Low in urgency)
|
|
160
|
+
- Running DAST in CI against production (use staging only)
|
|
161
|
+
- Ignoring npm audit warnings indefinitely without documented exceptions
|
|
162
|
+
- Mocking auth middleware in security tests (defeats the purpose)
|
|
163
|
+
- Treating SAST as the only security layer (SAST + DAST + audit = defense in depth)
|
|
164
|
+
|
|
165
|
+
quick_reference:
|
|
166
|
+
minimum_security_baseline: |
|
|
167
|
+
□ npm audit --audit-level=high passes (or exceptions documented)
|
|
168
|
+
□ SAST tool configured and running in CI
|
|
169
|
+
□ Secret scanning enabled in pre-commit
|
|
170
|
+
□ No High/Critical CVEs without documented exceptions
|
|
171
|
+
□ Security controls NOT mocked in tests
|